Evaluation method and evaluation apparatus

Abstract

This technology provides a way to obtain more specific feedback from subjects when evaluating their responses. [Solution] The evaluation method includes the steps of: acquiring information about a task performed by a subject (step S10); acquiring biometric information of the subject (step S12); identifying the timing at which the biometric information satisfies predetermined conditions (step S14); outputting the portion of the information about the task that corresponds to the identified timing (step S24); and receiving input of feedback regarding the task (step S26).

Description

Technical Field

[0001] The present invention relates to evaluation of a subject's reaction using feedback from the subject.

Background Art

[0002] Conventionally, providers of products or services have used feedback from subjects (e.g., customers of the products or services) to evaluate the subjects' reactions to the products or services. Regarding the acquisition of feedback, for example, U.S. Patent Application Publication No. 2002 / 0044687 (Patent Document 1) discloses a technique that allows a customer to electronically transmit a form filled with feedback information and the customer's address, electronically processes the feedback information, and provides an incentive to the customer using the address. Various studies have also been conducted on techniques for acquiring and evaluating information related to brain activity as feedback (see Non-Patent Document 1, Non-Patent Document 2, and Non-Patent Document 3).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

[0005] Conventional technology allows for the objective evaluation of subjects' reactions to a product or service. However, there is room for improvement in the overall system when it comes to obtaining specific feedback from subjects.

[0006] This invention was conceived in view of the circumstances described herein, and its purpose is to provide a technique for obtaining more specific feedback from subjects in the evaluation of their responses. [Means for solving the problem]

[0007] An evaluation method according to a certain aspect of this disclosure comprises the steps of: obtaining information about a task performed by a subject; obtaining biometric information of the subject performing the task over time; identifying the timing at which the biometric information meets predetermined conditions; outputting the portion of the task information corresponding to the identified timing; and receiving feedback from the subject regarding the task.

[0008] An evaluation method according to other aspects of this disclosure comprises the steps of: obtaining information about a task performed by a subject; identifying the timing at which the subject received marker input while performing the task, from at least one of the subject performing the task and an observer observing the subject; outputting the portion of the task information corresponding to the identified timing; and receiving feedback input from the subject regarding the task.

[0009] An evaluation device according to a certain aspect of this disclosure comprises an interface for acquiring task-related information and biometric information of a subject performing a task, a memory for storing conditions and identifying information regarding features in the biometric information, and a processor for identifying the timing when the features in the biometric information satisfy the conditions, the processor outputting the portion of the task-related information corresponding to the identified timing, and receiving input of feedback regarding the task.

[0010] An evaluation device according to other aspects of this disclosure includes an interface for obtaining task-related and biometric information of a subject performing a task, and a processor for identifying the timing at which the subject receives marker input while performing the task, from at least one of the subject performing the task and an observer observing the subject, the processor outputs the portion of the task-related information corresponding to the identified timing, and accepts input of feedback regarding the task.

[0011] A program that follows certain aspects of this disclosure causes a computer to perform the following steps: to obtain information about a task performed by a subject; to obtain biometric information of the subject; to identify a time when the biometric information meets predetermined conditions; to provide the portion of the task information corresponding to the identified time; and to receive input of feedback regarding the task.

[0012] A program, as executed by a computer, causes the computer to perform the following steps: to obtain information about a task performed by a subject; to identify the timing at which the subject received marker input while performing the task, from at least one of the subject performing the task and an observer observing the subject; to output the portion of the information about the task corresponding to the identified timing; and to receive feedback input from the subject regarding the task. [Effects of the Invention]

[0013] In accordance with certain aspects of this disclosure, more specific feedback from the subjects is obtained in evaluating their responses. [Brief explanation of the drawing]

[0014] [Figure 1] Figure 1 is a diagram showing an overview of the evaluation system, including the evaluation device related to this disclosure. [Figure 2] Figure 2 shows an example of a scenario in the evaluation system where subject 200 inputs feedback about the task. [Figure 3] Figure 3 is a schematic diagram showing the block configuration of the evaluation system. [Figure 4] Figure 4 is a diagram illustrating the attachment sites for the first electromyography sensor 11 and the second electromyography sensor 12. [Figure 5] Figure 5 shows an overview of an example of the process performed in the evaluation system 100. [Figure 6] Figure 6 is a flowchart of the process that the computer 90 performs to obtain feedback from the subjects 200 performing the task. [Figure 7] Figure 7 shows an example of an input screen. [Figure 8] Figure 8 shows another example of an input screen. [Modes for carrying out the invention]

[0015] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The same or corresponding parts in the drawings are denoted by the same reference numerals, and the description thereof will not be repeated.

[0016] [Overview of Evaluation Method] Figure 1 is a diagram showing an overview of an evaluation system including an evaluation apparatus according to the present disclosure. In the evaluation system, a computer 90 is an example of the evaluation apparatus.

[0017] A subject 200 performs a task in order to input feedback. In the example of Figure 1, the subject 200 is assembling a shelf as an example of the task.

[0018] The computer 90 acquires information related to the task that the subject 200 is performing. In the example of Figure 1, as an example of the information related to the task, a camera 210 captures an image (overhead view image) of the space where the subject 200 is performing the task, and transmits the image to the computer 90. As an example of the information related to the task, a microphone 220 may acquire audio of the above space and transmit the audio to the computer 90.

[0019] The computer 90 also acquires biological information of the subject 200. In the example of Figure 1, as an example of the biological information, a myoelectric potential signal of the subject 200 is acquired.

[0020] More specifically, the computer 90 acquires the myoelectric potential signal of the subject 200 via the user device 10. The subject 200 is wearing the user device 10. The user device 10 includes one or more myoelectric potential sensors (a first myoelectric potential sensor 11 and a second myoelectric potential sensor 12, which will be described later with reference to Figure 3 and the like). The myoelectric potential signal is acquired by the one or more myoelectric potential sensors.

[0021] The computer 90 identifies the timing at which the feature quantities of the above-mentioned biometric information satisfy predetermined conditions, and provides the subject 200 with the portion of the information related to the task that corresponds to the identified timing. The computer 90 may also identify the timing at which one or more buttons included in the button unit 240 or the button unit 250 are operated as such timings. The button unit 240 includes two buttons 241 and 242 operated by the subject 200. The button unit 250 includes two buttons 251 and 252 operated by the observer 300.

[0022] Figure 2 shows an example of a scenario in the evaluation system where a subject 200 inputs feedback about a task. In the example in Figure 2, the subject 200 uses a smartphone 290 to obtain the above-mentioned portion of the task information provided by the computer 90. In one implementation example, this portion is a video captured by the camera 210 at the timing identified above. The subject 200 checks this portion on the smartphone 290, inputs feedback about the task, and sends this feedback to the computer 90.

[0023] [Block structure] Figure 3 is a schematic diagram showing the block configuration of the evaluation system. As shown in Figure 3, the evaluation system 100 includes a user device 10 and a computer 90. In one implementation example, the user device 10 may include a wearable terminal worn by the subject. The computer 90 may be installed in the environment surrounding the subject. The user device 10 and the computer 90 are configured to enable bidirectional communication. The respective configurations will be described below.

[0024] <User device 10> The user device 10 includes a sensor unit 1, a signal processing circuit 2, a controller 3, a communication module 4, a battery 5, and a housing 6. The housing 6 houses the signal processing circuit 2, the controller 3, the communication module 4, and the battery 5 inside.

[0025] Sensor unit 1 includes a first electromyography sensor 11 and a second electromyography sensor 12. The first electromyography sensor 11 and / or the second electromyography sensor 12 are attached to the subject's face and detect electromyographic signals at the attachment site.

[0026] Electromyographic signals refer to the weak electrical signals generated when muscles are moved. In this disclosure, the subject's "face" is not limited to the face (front or side of the face) but may include the subject's neck. For example, sensor unit 1 may be attached to the subject's throat to detect changes in electromyographic signals associated with the subject's swallowing motion.

[0027] Sensor unit 1 may include multiple sensors. The multiple sensors may be attached to different types of muscles. The type of muscle can be determined by the site where the sensors are attached. In Embodiment 1, when the type of muscle is determined, it is not necessary to limit it to the muscle's composition or structure. If the sites where the sensors are attached are different, electromyographic signals from different types of muscles may be acquired.

[0028] Figure 4 is a diagram illustrating the attachment sites for the first electromyography sensor 11 and the second electromyography sensor 12. Referring to Figure 4, one configuration and use of the first electromyography sensor 11 and the second electromyography sensor 12 will be explained.

[0029] The first electromyography sensor 11 is attached to the subject's cheek. The first electromyography sensor 11 includes an active electrode 111 and a reference electrode 112. The active electrode 111 is incorporated into a pad 151, and the reference electrode 112 is incorporated into a pad 152.

[0030] The first electromyography sensor 11 detects electromyographic signals from muscles near the cheek by attaching pad 151 directly above the risorius muscle and pad 152 directly above the zygomaticus major muscle. More specifically, the first electromyography sensor 11 detects the potential of the working electrode 111, with the potential of the reference electrode 112 as a reference, as the electromyographic signal of muscles near the cheek. The attachment sites of pads 151 and 152 (working electrode 111 and reference electrode 112) may be slightly offset from directly above the risorius muscle and zygomaticus major muscle, respectively, as long as they are near those muscles. The first electromyography sensor 11 outputs the electromyographic signals as electromyographic signals indicating the activity of the facial muscles in the cheek area to the signal processing circuit 2 of the user device 10.

[0031] The second electromyography sensor 12 is attached to the subject's eyebrow. The second electromyography sensor 12 includes an active electrode 121 and a reference electrode 122. The active electrode 121 is incorporated into a pad 171, and the reference electrode 122 is incorporated into a pad 172.

[0032] The second electromyography sensor 12 detects electromyographic signals from muscles near the eyebrows (for example, the corrugator supercilii muscle) by attaching pads 171 and 172 directly above the corrugator supercilii muscle. More specifically, the second electromyography sensor 12 detects the potential of the working electrode 121, with the potential of the reference electrode 122 as a reference, as the electromyographic signal of the muscles near the eyebrows. The attachment sites of pads 171 and 172 (working electrode 121 and reference electrode 122) may be slightly offset from directly above the corrugator supercilii muscle, as long as they are in the vicinity of the muscle. The second electromyography sensor 12 outputs the electromyographic signal as an electromyographic signal indicating the activity of the corrugator supercilii muscle to the signal processing circuit 2.

[0033] In the example in Figure 4, the risorius muscle, zygomaticus major muscle, and corrugator supercilii muscle are examples of facial muscles. More specifically, the risorius muscle and zygomaticus major muscle are examples of muscles near the cheek, while the corrugator supercilii muscle is an example of a muscle near the eyebrow.

[0034] Referring again to Figure 3, the signal processing circuit 2 applies predetermined signal processing (noise reduction, rectification, amplification, digitization, etc.) to each electromyographic signal acquired from the sensor unit 1, and outputs each processed signal to the controller 3. Although not shown in the figure, the signal processing circuit 2 includes a filter, amplifier, and A / D converter.

[0035] In Figure 3, the electromyographic signal from the first electromyographic sensor 11, processed by the signal processing circuit 2, is shown as "Electromyographic Signal MS1," and the electromyographic signal from the second electromyographic sensor 12, processed by the signal processing circuit 2, is shown as "Electromyographic Signal MS2."

[0036] Controller 3 is a computing device that includes a processor 31, memory 32, and input / output ports 33. The processor 31 is implemented, for example, by a CPU (Central Processing Unit). The memory 32 is implemented, for example, by ROM (Read Only Memory) and RAM (Random Access Memory). The input / output ports 33 are the data input / output interface in Controller 3. Controller 3 performs calculations to evaluate the subject's facial expressions based on electromyographic signals MS1 and MS2.

[0037] The communication module 4 enables the user device 10 to communicate with external equipment, which is implemented, for example, by communication equipment compliant with a short-range wireless communication standard. The controller 3 controls the exchange of information between the user device 10 and the outside (such as the computer 90) by controlling the communication module 4.

[0038] Battery 5 is a rechargeable battery such as a lithium-ion secondary battery. Battery 5 supplies operating voltage to each device in the user device 10.

[0039] The user device 10 further includes a speaker 21 and a display 22. The controller 3 outputs sound through the speaker 21 by outputting a control signal ("signal SS" in Figure 3) to each of the speaker 21 and the display 22.

[0040] The user device 10 further includes a content providing device 23. The content providing device 23 is a device that provides content. In one embodiment, the content providing device 23 may provide taste to a subject 200. Such a device is configured to electrically perceive five tastes, for example, sweet, salty, sour, bitter, and umami, and adjusts the taste provided by changing the proportion in which these tastes are perceived. More specifically, such a device incorporates five types of gels containing electrolytes that perceive each of the five tastes, as known as a sushi roll synthesizer (https: / / research.miyashita.com / papers / I42 / paper.pdf), and adjusts the taste provided by adjusting the amount of electricity applied to the five types of gels. In one embodiment, the content providing device 23 may provide touch to a subject 200. Such a device adjusts the touch provided by controlling the surface state of a given object. In one embodiment, the content providing device 23 may provide smell to a subject 200. Such devices adjust the odor provided by adjusting the respective amounts of one or more types of odors contained in the supplied air.

[0041] The evaluation system 100 acquires information about the tasks performed by the subject 200. The portion of the acquired information corresponding to the timing when the subject 200's biometric information meets predetermined conditions is then provided to the subject 200.

[0042] Task-related information may also be images. In one example, the space in which the task is performed is photographed as task-related information. In another example, images provided to the subject 200 during the performance of the task are recorded as task-related information. In this case, the above portion is provided to the subject 200. The images are provided via the display 22 (or a display device other than the user device 10).

[0043] Task-related information may also be in audio format. In one example, the audio of the space in which the task is performed is recorded as task-related information. In another example, the audio provided to subject 200 during the execution of the task is recorded as task-related information. In this case, the audio is provided to subject 200 as part of the above-mentioned portion. The audio is provided via speaker 21 (or a speaker other than user device 10).

[0044] Task-related information may include information that identifies a scent. In one example, task-related information may include the scent (air composition) of the space where the task is performed. In another example, task-related information may include the scent (air composition) provided to subject 200 during the performance of the task. In yet another example, images and / or sounds provided to subject 200 during the performance of the task, and / or substances identified from images and / or sounds recorded in the space may be recorded. In this case, subject 200 is provided with a scent reproduced based on the identified information (air composition or substance). For example, if the name of a substance, "peach," is identified from sounds recorded in the space, the scent of a peach may be reproduced and provided. The scent is provided via the content providing device 23.

[0045] Task-related information may also include information that identifies a tactile sensation. In one example, task-related information may include the tactile sensation (the state of an object's surface) provided to the subject 200 during the task. In another example, images and / or sounds provided to the subject 200 during the task, as well as / or substances identified from images and / or sounds recorded in the space, may be recorded. In this case, the subject 200 is provided with a tactile sensation reproduced based on the identified information. For example, if the name of a substance, "peach," is identified from sounds recorded in the space, a predetermined surface state may be reproduced and provided as the tactile sensation of a peach's surface. The tactile sensation is provided via the content-providing device 23.

[0046] Task-related information may also include information that identifies a taste. In one example, task-related information may include the taste provided to the subject 200 during the performance of the task (e.g., food name, information identifying the food's composition). In another example, images and / or sounds provided to the subject 200 during the performance of the task, and / or substances identified from images and / or sounds recorded in the aforementioned space, may be recorded. In this case, the subject 200 is provided with a taste reproduced based on the identified information. For example, if the name of a substance, "peach," is identified from sounds recorded in the aforementioned space, a predetermined taste of peach (a combination of five tastes: sweet, salty, sour, bitter, and umami) may be reproduced and provided. The taste is provided via the content-providing device 23.

[0047] The information related to the task may be a combination of two or more of the five types of information described above (images, sounds, smells, textures, and tastes). The content providing device 23 may include units that can reproduce information for two or more of the two or more types of smells, textures, and tastes.

[0048] <Computer 90> Computer 90 is, for example, a PC (Personal Computer) or a server. Computer 90 communicates with the user device 10 via a communication module (not shown) and receives signals indicating the calculation results of the controller 3. Computer 90 includes a controller 91 and a communication module 92.

[0049] Controller 91, like controller 3, includes a processor 95, memory 96, and input / output ports 97, and performs various arithmetic operations. Communication module 92, like communication module 4, is implemented by, for example, communication equipment compliant with short-range wireless communication standards. That is, communication module 92 enables computer 90 to communicate with other devices.

[0050] Computer 90 is connected to camera 210, microphone 220, odor sensor 230, and button units 240 and 250. Camera 210 captures images of the space where the task is performed and transmits them to computer 90. Microphone 220 acquires audio from the space where the task is performed and transmits it to computer 90. Odor sensor 230 acquires the air composition of the space where the task is performed and transmits it to computer 90. Button unit 240 detects operation on button 241 or button 242 and transmits the detection result to computer 90. Button unit 250 detects operation on button 251 or button 252 and transmits the detection result to computer 90.

[0051] [Process Overview] Figure 5 shows an overview of an example of the process performed in the evaluation system 100.

[0052] Figure 5 shows six types of data during a period in which 200 subjects performed a task. The six types of data include features of two types of biometric information (biometric information (1), (2)), the state (ON / OFF) of subject markers and observer markers, video analysis metrics, and related information.

[0053] Biological information (1) represents, for example, the electromyographic signal of muscles near the cheek. Changes in the features of biological information (1) are represented by line L1. Biological information (2) represents, for example, the electromyographic signal of muscles near the eyebrow. Changes in the features of biological information (2) are represented by line L2.

[0054] The subject markers represent, for example, the operating state of buttons 241 and 242. When button 241 is operated, the subject marker value becomes ON, and when button 242 is operated, the subject marker value becomes OFF. The change in the subject markers is represented by line L3.

[0055] The observer markers represent, for example, the operating state of buttons 251 and 252. When button 251 is operated, the observer marker value becomes ON, and when button 252 is operated, the observer marker value becomes OFF. The change in the observer markers is represented by line L4.

[0056] The video analysis metric represents features obtained from images captured by camera 210. An example of such a feature is the distance moved by at least a portion of subject 200's body within a predetermined unit of time. The video analysis metric is influenced by the movement of subject 200's body in the captured images. In this sense, the video analysis metric can be considered an example of a feature of subject 200's biometric information. Changes in the video analysis metric are represented by line L5.

[0057] Related information refers to information acquired as part of the task. An example of related information is an image captured by camera 210.

[0058] The computer 90 provides the subject 200 with the portion of the relevant information that corresponds to a given timing. In the example in Figure 5, seven timings (TM11-TM17) are shown as timings for identifying the portion to be provided.

[0059] More specifically, computer 90 identifies the moment when the subject marker value turned ON, as TM11.

[0060] Computer 90 identifies the timing when the value of the biological information (2) reaches a given threshold (Th22) as TM12.

[0061] Computer 90 identifies the timing at which the value of the biological information (1) reaches a given threshold (Th11) (decreases to the given threshold) as TM13.

[0062] Computer 90 identifies the timing when the observer marker's value turned ON, using TM14 as the identifier.

[0063] Computer 90 identifies the timing when the value of the biological information (1) reaches a given threshold (Th11) as TM15.

[0064] Computer 90 identifies the timing when the video analysis metric value reaches a given threshold (Th13) as TM16.

[0065] Computer 90 identifies the timing when the value of the biological information (2) reaches a given threshold (Th22) (when it falls to the given threshold) as TM17.

[0066] The computer 90 identifies a period, including a certain amount of time before and after each of the seven identified timings, as the period for the information to be provided to the subject 200. Thus, for each of the timings TM11-17, each of the periods PT11-17 is identified. After the task is completed, the computer 90 provides the subject 200 with the relevant information, specifically the portion corresponding to each of the periods PT11-17. The "certain amount of time" mentioned above is, for example, 2 seconds. In this case, the subject 200 is provided with a 4-second video as each portion of the periods PT11-17.

[0067] <When task-related information will be provided> In the process described with reference to Figure 6, an input screen including the above-mentioned section is generated after the completion of the task, and this input screen is provided to the subject 200. The timing of providing the input screen is not limited to after the completion of the task. The computer 90 may generate and provide the input screen to the subject 200 whenever such a timing is identified. That is, when the computer 90 identifies timing TM1, it may provide the subject 200 with relevant information for period PT11 without waiting for the next timing TM2 to be identified.

[0068] <Period during which the provided information is applicable> The period to which the above-mentioned parts correspond does not need to include a certain period of time before and after the specified timing. That is, the period to which the above-mentioned parts correspond may be a certain period of time before and after the specified timing, or it may be the specified timing and a certain period immediately preceding it, or it may be only the specified timing. As a result, for example, when the timing at which the biological information (1) reaches a given threshold is identified, video footage of a certain period of time before and after that timing, video footage of that timing and a certain period immediately preceding it, or an image of that timing is provided to the subject 200.

[0069] <Methods for identifying "timing" and types of biometric information> Computer 90 may also identify the timing described above by, in addition to, the detection of the biological information or the video analysis index reaching a given threshold, or by the saturation of the biological information detection or video analysis index (the change is less than or equal to a given value for a given amount of time or longer). In this sense, the detection of the biological information and the video analysis index are examples of biological information features. Furthermore, the saturation of a value is an example of the value being maintained within a given range.

[0070] Furthermore, the phrase "reaching the threshold" as described above may refer to either reaching the threshold from below, or reaching the threshold from above (decreasing towards the threshold).

[0071] In the example in Figure 5, specific examples of biometric features are shown, including electromyographic signals from muscles near the cheek, electromyographic signals from muscles near the eyebrow, and video analysis metrics identified from images of subject 200. Note that biometric features may also be other types of information, such as subject 200's blood pressure, electroencephalogram (EEG), electrocardiogram (ECG), respiratory rate, and / or heart rate. For example, for EEG, the agreement rate between the subject's EEG pattern and a predetermined waveform pattern could be used as an example of a feature. In other words, any information that reflects subject 200's state (e.g., emotions) can be used as a biometric feature. Furthermore, in the evaluation system 100, the computer 90 is configured to perform not only timing identification based on biometric information, but also timing identification based on button operations by subject 200 or observer 300. This allows the computer 90 to provide subject 200 with information corresponding to timings that subject 200 or observer 300 deem likely to be helpful for feedback.

[0072] <Related Information (Information related to the task)> The information acquired as related information may include information about the atmosphere of the space in which the task is performed. This information about the atmosphere of the space may include, for example, images (video) captured by the camera 210, audio acquired by the microphone 220, and / or information identifying the composition of the air acquired by the odor sensor 230. In addition, as information about the atmosphere of the space, the camera 210 may capture an image of the subject 200's field of view. In this case, the camera 210 is attached to the subject 200's head in order to capture the subject 200's field of view.

[0073] <Method of obtaining related information> The relevant information may be acquired continuously, at regular intervals, or whenever a given event occurs. In one example, as relevant information, camera 210 may continuously record video while the task is being performed. In another example, as relevant information, camera 210 may take images (photographs) at regular intervals while the task is being performed. In yet another example, as relevant information, camera 210 may take images (photographs) each time subject 200 operates button 241 and / or observer 300 operates button 251.

[0074] <Information provided as part of the related information section> The information provided as part of the related information (the part corresponding to period PT11-17) may be a part of the related information itself, or it may be information reconstructed based on information identified from a part of the related information. In other words, the information provided to subject 200 may be information secondarily generated based on the related information. For example, if the related information is an image, and it is identified that the image at a specific time contains a "peach," then subject 200 may be provided with the image containing the "peach" itself as information corresponding to that time, or with an audio representation of a "peach," or with an air (smell) of a predetermined composition corresponding to a "peach," or with a taste of a predetermined composition corresponding to a "peach," or with a predetermined texture corresponding to a "peach."

[0075] <Task> In the evaluation system 100, the subject 200 performs a task and inputs feedback about that task. The computer 90 receives the feedback from the subject 200. The task performed by the subject 200 is not limited to the task of building shelves as described with reference to Figure 1, but may be of any kind.

[0076] Another example of a task is one in which subject 200 plays a video game. When subject 200 plays a video game as part of the task, the subject 200's field of view image, as an example of relevant information, may be an image of the video game instead of one captured by camera 210. The video game image may be identified by the game's progress. Therefore, computer 90 may acquire the game's progress as relevant information corresponding to the timing when the biometric information satisfies a given condition, instead of the image itself at that time.

[0077] When computer 90 generates an input screen, as described later, with reference to Figure 7, etc., it may reconstruct the screen based on the progress of the game. Then, computer 90 may provide the reconstructed screen to subject 200 by incorporating it into the input screen.

[0078] [Process flow] Figure 6 is a flowchart of the processes performed by the computer 90 to obtain feedback from the subject 200 performing the task. For example, the computer 90 starts the processes shown in Figure 6 when it receives notification that the subject 200 has started the task. In one implementation example, the processes shown in Figure 6 are performed in the computer 90 by the processor 95 executing a given program.

[0079] In step S10, the computer 90 begins acquiring relevant information. In one implementation example, the computer 90 instructs the camera 210 to start recording.

[0080] In step S12, the computer 90 acquires biological information. In one implementation example, the computer 90 requests the user device 10 to transmit the electromyographic signal acquired by the sensor unit 1. The user device 10 transmits the electromyographic signal to the computer 90 in response to this request.

[0081] In step S14, the computer 90 determines whether a specific state has occurred. A specific state is a state in which the biological information satisfies a given condition, such as the timing TM13 or TM15 specified for the biological information (1) in Figure 5. If the computer 90 determines that a specific state has occurred (YES in step S14), it proceeds to step S16; otherwise (NO in step S14), it returns control to step S12.

[0082] In step S16, the computer 90 registers the current time as a reference point in memory 96. Instead of the current time, the elapsed time since the start of the task may be registered.

[0083] In step S18, the computer 90 determines whether the task has been completed. In one implementation example, when the task is completed, the subject 200 operates button 242. When the computer 90 detects that button 242 has been operated, it determines that the task has been completed. If the computer 90 determines that the task has been completed (YES in step S18), it proceeds to step S20; otherwise (NO in step S18), it returns control to step S12.

[0084] In step S20, the computer 90 terminates the acquisition of related information that was started in step S10. In one implementation example, the computer 90 instructs the camera 210 to stop taking pictures.

[0085] In step S22, the computer 90 generates an input screen. This input screen is provided to the subject 200 for inputting feedback.

[0086] Figure 7 shows an example of an input screen. Screen 700 includes frame 710, key 711, frames 721-725, and frame 730.

[0087] Frame 710 contains the message, "These are videos taken during this experience. Please refer to them and write down your impressions of this experience." "Videos taken during this experience" refers to the videos displayed in frames 721 to 725, which will be described later. In other words, the message in frame 710 is an example of a message prompting the subject 200 to refer to the videos provided for feedback input. The computer 90 generates the input screen by inserting the above message into frame 710. The provided videos are examples of the relevant information acquired during the execution of the task, corresponding to the "timing" explained with reference to Figure 5.

[0088] Key 711 is operated to send the information entered in the frame 730 (described later) to the computer 90.

[0089] Each of frames 721 through 725 displays a video. The displayed video is the portion of the video (an example of related information) acquired during task execution that corresponds to the "timing" explained with reference to Figure 5. This "timing" is acquired as a "reference point" in the process shown in Figure 6. In each of frames 721 through 725, video playback starts in response to touch operation.

[0090] Computer 90 generates content provided as "parts" corresponding to each of the above "timings" by extracting portions from the video (an example of related information) acquired during task execution. Then, computer 90 generates an input screen by fitting each of the generated contents into frames 721 to 725.

[0091] Note that the number of frames for displaying related information such as videos included in the input screen is not limited to "5" as shown in Figure 7. In one implementation example, the number of frames is the number of "timings" identified during task execution.

[0092] Slot 730 will accept feedback input from 200 subjects. Returning to Figure 6, when the computer 90 generates an input screen in step S22, it transmits the input screen to the subject 200 in step S24. The input screen incorporates the portion of the relevant information corresponding to the "timing" described with reference to Figure 5, as described above. In this sense, the computer 90 provides the above portion to the subject 200 by transmitting the input screen to the subject 200.

[0093] In one implementation example, at the start of the task, computer 90 obtains the email address of subject 200 along with information identifying subject 200. In step S24, computer 90 sends an email containing the input screen generated as described above to the said email address. Subject 200 receives the email on smartphone 290 (Figure 2). As a result, subject 200 views the input screen on smartphone 290. Note that the content described here is merely one aspect of sending the input screen to subject 200. The input screen can be sent to subject 200 in any manner.

[0094] In step S26, the computer 90 obtains feedback from the subject 200. In one example, the subject 200 enters feedback as text into the frame 730 on the screen 700 displayed on the smartphone 290, and then operates the key 711. In response, the smartphone 290 sends the text entered in the frame 730 to the computer 90. The computer 90 obtains the feedback by receiving the text thus sent.

[0095] In step S28, the computer 90 registers the acquired feedback in memory 96. After that, the computer 90 terminates the process shown in Figure 6.

[0096] Through the process described above, the computer 90 provides the subject 200 with the portion of the relevant information acquired during the execution of the task that corresponds to the timing identified using the subject 200's biometric information. The subject 200, referring to this portion, inputs feedback regarding the task and sends this feedback to the computer 90. In this way, the computer 90 obtains feedback from the subject 200 regarding the task.

[0097] According to the process shown in Figure 6, the subject 200 is provided with information about the task (related information) that corresponds to the timing when the subject 200's biometric information meets a given condition, and then inputs feedback. This allows the subject 200 to recall their own state at that timing more specifically. Therefore, the computer 90 can obtain more specific feedback from the subject 200, and the feedback obtained reflects the subject's true feelings with high accuracy.

[0098] [Differentiation] <Providing a "part" of the relevant information without waiting for the task to be completed.> In the process shown in Figure 6, screen 700, including frames 721-725, is provided to the subject 200 after the task is completed. Each of frames 721-725 displays a portion of the video (an example of related information) acquired during the task execution that corresponds to the "timing" described with reference to Figure 5. In other words, in the process shown in Figure 6, two or more portions are provided to the subject 200 all at once after the task is completed.

[0099] In contrast, each of the two or more parts may be provided to the subject 200 without waiting for the task to be completed. For example, when the computer 90 identifies the "timing" mentioned above, it extracts the part corresponding to that "timing" from the relevant information and sends it to the subject 200 without waiting for the task to be completed. This is expected to allow the subject 200 to continue the task while being aware of the situation when the subject 200's biometric information meets a given condition, and to easily output the content to be input as feedback.

[0100] <Number of "parts" of related information contained in a single input screen> The screen 700 in Figure 7 provides the subject 200 with multiple parts corresponding to multiple timings, collectively, as five frames 721 to 725. Alternatively, the input screen may present the subject 200 with each of the multiple parts corresponding to multiple timings, and may display one feedback input frame for each part. In other words, if five parts are provided to the subject 200 for a given task, five screens will be provided as input screens. Each of the five input screens includes one frame for displaying the aforementioned part and one frame for inputting feedback while referring to that part.

[0101] <Output of subject's status> In generating the input screen, the computer 90 may identify the state of the subject 200 by analyzing the content of the "part" provided to the subject 200, and may include the identified state in the input screen.

[0102] Figure 8 shows another example of an input screen. Screen 800 includes a frame 730 and keys 711, similar to screen 700 in Figure 7.

[0103] Screen 800 includes frame 810 instead of frame 710 of screen 700. The message displayed in frame 810 includes the message displayed in frame 710, plus the phrase "Please enter the degree of your state at each point in time." This phrase prompts subject 200 to enter the degree of their state at the time the video displayed in frames 840 and 850 (described later) was filmed.

[0104] Screen 800 includes frames 840 and 850. Each of frames 840 and 850 displays a video, similar to frames 721 to 725 in Figure 7. The displayed video is the portion of the video (an example of related information) acquired during task execution that corresponds to the "timing" explained with reference to Figure 5.

[0105] Screen 800 includes columns 841 and 851. Each of columns 841 and 851 displays the state of subject 200 identified from the video displayed in frames 840 and 850, respectively. In one implementation example, computer 90 extracts specific frames (the first frame displayed, frames displayed at intermediate points, etc.) from the video displayed in frame 840, identifies the face region from the image of that frame, and identifies the emotion corresponding to the face image by analyzing the image of the identified face region. Then, computer 90 embeds the identified emotion as the state of subject 200 in column 841. Similarly, computer 90 identifies the emotion of subject 200 from frame 850 and embeds it as the state of subject 200 in column 851.

[0106] Screen 800 includes fields 842 and 843. Field 842 is for inputting the degree of the state displayed in field 841. Field 843 is a pull-down menu that displays the degree options to be entered in field 842. In the example in Figure 8, field 841 displays the state "painful". Field 843 displays three options for the degree of the state "painful": "very painful", "somewhat painful", and "not painful". Subject 200 selects one of the options displayed in field 843. The selected option is entered in field 842.

[0107] Screen 800 includes fields 852 and 853. Field 852 is for inputting the degree of the state displayed in field 851. Field 853 is a pull-down menu that displays the degree options to be entered in field 852. In the example in Figure 8, field 851 displays the state "fun". Field 853 displays three options for the degree of the state "fun": "very fun", "somewhat fun", and "not fun". Subject 200 selects one of the options displayed in field 853. The selected option is entered in field 852.

[0108] When key 711 is transmitted on screen 800, the degree of the state entered in fields 842 and 852, in addition to the text entered in frame 730, is sent to computer 90. As a result, in step S26, computer 90 receives the degree of the state entered in fields 842 and 852, in addition to the text entered in frame 730, as feedback.

[0109] In the embodiments described above, an example was shown in which each step is performed by a single computer 90. The present invention is not limited thereto, and each step may be performed by a different computer using a plurality of computers that are connected to each other in a manner that allows for communication.

[0110] [Pattern] Those skilled in the art will understand that the above-described exemplary embodiments are specific examples of the following embodiments.

[0111] (Clause 1) An evaluation method according to one embodiment may include the steps of: acquiring information about a task performed by a subject; acquiring biometric information of the subject performing the task over time; identifying the timing at which the biometric information satisfies predetermined conditions; outputting the portion of the information about the task that corresponds to the identified timing; and receiving feedback from the subject regarding the task.

[0112] According to the evaluation method described in paragraph 1, more specific feedback from the subjects is obtained when evaluating their responses.

[0113] (Clause 2) In the evaluation method described in paragraph 1, the biological information may include at least one of the following: electromyographic signals of the subject's facial muscles, the subject's blood pressure, the subject's electroencephalogram, the subject's electrocardiogram, the subject's respiratory rate, the subject's heart rate, and an index identified from an image including the subject.

[0114] According to the evaluation method described in paragraph 2, the subject's condition is accurately reflected in the biological information.

[0115] (Clause 3) In the evaluation method described in paragraph 1 or 2, the predetermined conditions may include at least one of the following: that the feature quantity in the biological information reaches a given threshold, and that the state in which the change in the feature quantity in the biological information is less than or equal to a given value continues for a predetermined time or longer.

[0116] According to the evaluation method described in paragraph 3, it is easy to determine whether or not the characteristic features of the biological information meet predetermined conditions.

[0117] (Paragraph 4) In the evaluation method described in any one of paragraphs 1 to 3, the predetermined conditions may include the condition that the feature quantities in the biological information are maintained within a given range for a given time or longer.

[0118] According to the evaluation method described in Section 4, it is easy to determine whether or not the characteristic features of the biological information meet predetermined conditions.

[0119] (Clause 5) An evaluation method described in any one of paragraphs 1 to 4, further comprising the step of receiving marker input from at least one of the subject and / or an observer observing the subject while the subject is performing the task, wherein the output step may include providing the portion of the information relating to the task that corresponds to the time when the marker was placed.

[0120] According to the evaluation method described in Section 5, information may also be provided to the subject at a time desired by the subject or observer.

[0121] (Clause 6) An evaluation method according to one embodiment may include the steps of: obtaining information about a task performed by a subject; identifying the timing at which the subject received marker input while performing the task from at least one of the subject performing the task and an observer observing the subject; outputting the portion of the information about the task that corresponds to the identified timing; and receiving feedback input from the subject regarding the task.

[0122] According to the evaluation method described in Section 6, more specific feedback from the subjects can be obtained when evaluating their responses.

[0123] (Clause 7) In the evaluation method described in any one of paragraphs 1 to 6, the information relating to the task may include at least one of the following: information relating to the atmosphere of the space in which the task is performed, and information relating to the content provided to the subject in the task.

[0124] According to the evaluation method described in Section 7, the subject's condition is accurately reflected in the information regarding the task.

[0125] (Clause 8) In the evaluation method described in Clause 7, the information relating to the task includes information relating to the atmosphere of the space, and the information relating to the atmosphere of the space may include at least one of the following: an image of the space in which the task is performed, a recording of sound in the space in which the task is performed, and the smell of the space in which the task is performed.

[0126] According to the evaluation method described in Section 8, the subject's condition is more accurately reflected in the information regarding the task.

[0127] (Clause 9) In the evaluation method described in paragraph 7, the information relating to the task includes information relating to the atmosphere of the space, the information relating to the atmosphere of the space includes an image of the space, and the image of the space may include at least one of an image including a subject performing the task and an image of the gaze of the subject performing the task.

[0128] According to the evaluation method described in Section 9, the subject's condition is more accurately reflected in the information regarding the task.

[0129] (Clause 10) In the evaluation method described in any one of paragraphs 7 to 9, the information relating to the task includes the information relating to the content, and the output portion may include at least one of the following: images, sounds, smells, textures, and tastes.

[0130] According to the evaluation method described in Section 10, subjects are provided with information useful for providing specific feedback.

[0131] (Clause 11) In the evaluation method described in any one of paragraphs 1 to 10, the acquisition step may include acquiring information about the task at regular intervals during the execution of the task.

[0132] According to the evaluation method described in Section 11, the amount of data handled in obtaining information about the task is minimized.

[0133] (Clause 12) In the evaluation methods described in paragraphs 1 to 11, the output portion may correspond to the timing and a given period before and after the timing.

[0134] According to the evaluation method described in paragraph 12, subjects may be provided with detailed information to help them recall the situation at the aforementioned timing.

[0135] (Clause 13) In the evaluation method described in any one of paragraphs 1 to 12, the output step may include providing the portion after the completion of the task.

[0136] According to the evaluation method described in Section 13, subjects can be asked to recall the circumstances surrounding the task even after the task has been completed.

[0137] (Clause 14) In the evaluation method described in any one of paragraphs 1 to 13, the output step may include providing the portion in accordance with the timing being specified, without waiting for the completion of the task.

[0138] According to the evaluation method described in paragraph 14, it is possible to more reliably elicit from the subject the recall of their state at the aforementioned timing.

[0139] (Clause 15) In the evaluation method described in any one of paragraphs 1 to 14, the output step may include identifying the subject's condition by analyzing the output portion and outputting the subject's condition.

[0140] According to the evaluation method described in paragraph 15, subjects are provided with more specific information about themselves.

[0141] (Clause 16) In the evaluation method described in paragraph 15, the output step may include outputting information prompting input of the degree of the subject's condition, and the receiving step may include obtaining input of the degree of the subject's condition.

[0142] According to the evaluation method described in paragraph 16, the degree of the subject's condition can be managed in the computer on which the evaluation method is performed.

[0143] (Clause 17) In the evaluation method described in paragraphs 1 to 16, the output step may include providing a portion corresponding to the second timing along with a portion corresponding to the first timing, when a first timing and a second timing are identified as the timings.

[0144] According to the evaluation method described in Section 17, subjects may be provided with comprehensive information. (Clause 18) In the evaluation method described in any one of paragraphs 1 to 17, the output step may include, if a first timing and a second timing are specified as the timing, providing a portion corresponding to the first timing and then providing a portion corresponding to the second timing.

[0145] According to the evaluation method described in Section 18, subjects are provided with information regarding the first timing and the second timing individually.

[0146] (Paragraph 19) In the evaluation method described in any one of paragraphs 1 to 18, the output step may include outputting a message to the subject prompting them to refer to the portion output in the feedback input.

[0147] According to the evaluation method described in paragraph 19, subjects can be reliably made to refer to the information in the above section.

[0148] (Clause 20) An evaluation device according to one embodiment comprises an interface for acquiring information relating to the task and biometric information of a subject performing the task, a memory for storing information to identify conditions relating to features in the biometric information, and a processor for identifying the timing when the features in the biometric information satisfy the conditions, wherein the processor may output the portion of the information relating to the task that corresponds to the identified timing and accept input of feedback relating to the task.

[0149] According to the evaluation device described in Section 20, more specific feedback from the subjects can be obtained when evaluating their responses.

[0150] (Clause 21) An evaluation device according to one embodiment includes an interface for acquiring information and biometric information of a subject performing a task, and a processor for identifying the timing at which a marker input is received from at least one of the subject performing the task and an observer observing the subject while the subject is performing the task, wherein the processor may output the portion of the information regarding the task that corresponds to the identified timing, and may accept input of feedback regarding the task.

[0151] According to the evaluation device described in paragraph 21, more specific feedback from the subject is obtained when evaluating the subject's response.

[0152] (Clause 22) A program according to one embodiment may be executed by a computer to cause the computer to perform the following steps: to acquire information relating to a task performed by a subject; to acquire biometric information of the subject; to identify a timing in which the biometric information satisfies predetermined conditions; to output the portion of the information relating to the task that corresponds to the identified timing; and to receive input of feedback relating to the task.

[0153] According to the program described in Section 22, more specific feedback from the subjects is obtained when evaluating their responses.

[0154] (Clause 23) A program according to one embodiment may be executed by a computer to cause the computer to perform the following steps: to acquire information about a task performed by a subject; to identify the timing at which the subject received marker input while performing the task from at least one of the subject performing the task and an observer observing the subject; to output the portion of the information about the task that corresponds to the identified timing; and to receive feedback input from the subject regarding the task.

[0155] According to the program described in Section 23, more specific feedback from the subjects is obtained when evaluating their responses.

[0156] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this disclosure is indicated by the claims rather than by the description of the embodiments above, and all modifications within the meaning and scope equivalent to the claims are intended to be included. Furthermore, each technology in the embodiments is intended to be practiced individually or, as far as possible, in combination with other technologies in the embodiments. [Explanation of Symbols]

[0157] 1 Sensor unit, 3.91 Controller, 4.92 Communication module, 5 Battery, 6 Housing, 10 User device, 11 First electromyography sensor, 12 Second electromyography sensor, 21 Speaker, 22 Display, 23 Content delivery device, 31.95 Processor, 32.96 Memory, 33.97 Input / output ports, 90 Computer, 100 Evaluation system, 111,121 Working electrodes, 112,122 Reference electrodes, 151,152,171,172 Pads, 200 Subject, 210 Camera, 220 Microphone, 230 Odor sensor, 240,250 Button unit, 241,242,251,252 Buttons, 290 Smartphone, 300 Observer, 700,800 Screen, 710, 721, 725, 730, 810, 840, 850 Frame, 711 Key, 841, 842, 843, 851, 852, 853 Column.

Claims

1. The computer obtains information about the task the subject will perform, The computer acquires biometric information of the subject performing the task over time, The computer, based on the biometric information acquired via a camera or sensor, determines the timing at which the biometric information meets predetermined conditions. The computer outputs the portion of the information relating to the task that corresponds to the identified timing, The computer generates an input screen for the subject to input feedback regarding the task, The computer includes the step of receiving input of feedback regarding the task from the subject via the input screen, Evaluation method.

2. The aforementioned predetermined conditions are: The characteristic quantity of the aforementioned biological information has reached a given threshold, and The evaluation method according to claim 1, comprising at least one of the following: the change in the characteristic quantity of the biological information is less than or equal to a given value, and this state continues for a predetermined period of time or longer.

3. The evaluation method according to claim 1, wherein the predetermined conditions include the fact that the characteristic quantities of the biological information are maintained within a given range for a given time or longer.

4. The computer further includes the step of receiving marker input from at least one of the subject and an observer observing the subject while the subject is performing a task, The evaluation method according to claim 1, wherein the output step includes providing a portion of the information relating to the task that corresponds to the time when the marker was applied.

5. The evaluation method according to claim 1, wherein the acquisition step includes acquiring information relating to the task at regular intervals during the execution of the task.

6. The evaluation method according to claim 1, wherein the output portion corresponds to the timing and a given period before and after the timing.

7. The output step described above is: By analyzing the output portion, the subject's condition can be identified. The evaluation method according to claim 1, further comprising outputting the state of the subject.

8. The output step described above is: This includes outputting information prompting input about the degree of the subject's condition, The aforementioned acceptance step is, The evaluation method according to claim 7, comprising obtaining input of the degree of the subject's condition.

9. The evaluation method according to claim 1, wherein the output step includes, when a first timing and a second timing are identified as the timing, providing a portion corresponding to the first timing along with a portion corresponding to the second timing.

10. The evaluation method according to claim 1, wherein the output step includes, when a first timing and a second timing are identified as the timing, outputting the portion corresponding to the first timing and then providing the portion corresponding to the second timing.

11. The computer obtains information about the task the subject will perform, The computer includes the step of determining, from at least one of the subject performing the task and the observer observing the subject, the timing at which the subject received marker input while performing the task. The computer outputs the portion of the information relating to the task that corresponds to the identified timing, The computer generates an input screen for the subject to input feedback regarding the task, The computer includes the step of receiving input of feedback regarding the task from the subject via the input screen, Evaluation method.

12. The aforementioned part is Images provided to the subject during the execution of the aforementioned task, Audio provided to the subject during the execution of the aforementioned task, The odors provided to the subject during the performance of the task, or of the space during the performance of the task, The sensations provided to the subject during the performance of the aforementioned task, The tastes provided to the subject during the performance of the task, From the images and / or sounds provided to the subject during the performance of the task, or from the images and / or sounds recorded in the space, the image, sound, smell, touch, or odor of the identified substance, A combination of two or more of the five types of information mentioned above (images, sounds, smells, textures, and tastes), and At the aforementioned timing, the image of the game being played as the task or the progress of the game, The evaluation method according to claim 1 or claim 11, comprising at least one of the following.

13. An interface for acquiring information and biometric information of a subject performing a task from a camera or sensor, A memory that stores conditions and information to identify the features of the aforementioned biological information, The system includes a processor for identifying the timing when the characteristic quantities of the biological information satisfy the conditions, The aforementioned processor, The portion of the information relating to the aforementioned task that corresponds to the identified timing is output. An input screen is generated for the subject to input feedback regarding the task, The system accepts feedback from the subject regarding the task via an input screen. Evaluation device.

14. The aforementioned part is Images provided to the subject during the execution of the aforementioned task, Audio provided to the subject during the execution of the aforementioned task, The odors provided to the subject during the performance of the task, or of the space during the performance of the task, The sensations provided to the subject during the performance of the aforementioned task, The tastes provided to the subject during the performance of the task, From the images and / or sounds provided to the subject during the performance of the task, or from the images and / or sounds recorded in the space, the image, sound, smell, touch, or odor of the identified substance, A combination of two or more of the five types of information mentioned above (images, sounds, smells, textures, and tastes), and At the aforementioned timing, the image of the game being played as the task or the progress of the game, The evaluation apparatus according to claim 13, comprising at least one of the following.

Images