Selection support apparatus and selection support program

Abstract

To provide a selection support device capable of presenting other choices without troubling a user.SOLUTION: Provided is an information processing device comprising: a selection presence / absence determination unit (11) which, after a plurality of options are presented to a user, determines whether or not the user has selected any of the plurality of options; a confused state estimation unit (12) which, when the selection presence / absence determination unit (11) determines that the user has not selected any of the plurality of options, estimates whether or not the user is in a confused state on the basis of a confused state estimation condition; and an alternative option presentation unit (13) which, when the confused state estimation unit (12) estimates that the user is in the confused state, presents a plurality of alternative options which are a plurality of options different from the plurality of options currently presented.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a selection assistance device and a selection assistance program. [Background technology]

[0002] BACKGROUND ART Conventionally, there is a technique for presenting a plurality of options to a user and prompting the user to select one of the presented options (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-193520 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described above, which presents a plurality of options to a user and allows the user to select one of the options, there is a technology that allows the user to perform some operation to present other options if there is no option currently presented to the user. However, with such technology, the user must perform some operation to present other options, which is a hassle for the user.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a selection assistance device that can present other options to a user without causing the user any trouble in a technology that presents multiple options to a user and allows the user to select one of the presented options. [Means for solving the problem]

[0006] The selection assistance device disclosed herein includes a selection determination unit that determines whether or not the user has selected any of the multiple options after multiple options have been presented to the user, a confusion state estimation unit that, when the selection determination unit determines that the user has not selected any of the multiple options, estimates whether or not the user is in a confused state based on confusion estimation conditions, and an alternative option presentation unit that, when the confusion state estimation unit estimates that the user is in a confused state, presents multiple alternative options that are multiple options different from the multiple options currently presented. [Effects of the Invention]

[0007] According to the present disclosure, the selection assistance device is configured as described above, and is therefore able to present other options to the user without causing the user any trouble. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of a selection support device according to a first embodiment. [Figure 2] 2A and 2B are diagrams for explaining an example of a flow until the alternative option presenting unit presents a plurality of alternative options to the output device in the first embodiment. [Figure 3] 3A, 3B, and 3C are diagrams for explaining another example of the flow until the alternative option presentation unit presents a plurality of alternative options to the output device in the first embodiment. [Figure 4] 4A, 4B, and 4C are diagrams for explaining another example of the flow until the alternative option presentation unit presents a plurality of alternative options to the output device in the first embodiment. [Figure 5] 4 is a flowchart for explaining the operation of the selection support device according to the first embodiment. [Figure 6] 6A and 6B are diagrams illustrating an example of a hardware configuration of the selection support device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] Embodiment 1 When a selection assistance device according to embodiment 1 presents a plurality of options to a user, the selection assistance device presents a plurality of options (hereinafter referred to as "alternate options") that are different from the plurality of options currently being presented, based on the user's reaction to the options after the options have been presented. More specifically, the selection assistance device of embodiment 1 presents a plurality of options to a user, and then when it determines that the user has not selected any of the presented options and it is estimated that the user is confused, it presents a plurality of alternative options. When a plurality of options are presented to a user and there is no option to be selected among the options currently presented to the user, the user is likely to behave in a manner that indicates that he or she is confused and unable to decide which option to select. Therefore, as described above, the selection assistance device according to the first embodiment presents a plurality of alternative options when, after a plurality of options are presented to the user, it is determined that the user's selection from among the presented options has not been accepted and it is estimated that the user is confused. The selection assistance device presents a plurality of alternative options without waiting for the user to perform some operation to present other options, so the user can save time and effort in presenting a plurality of alternative options.

[0011] The selection support device may make the presentation by voice or by display, for example. The manner in which the selection support device makes the presentation is determined in advance. For example, the selection support device may make the presentation by outputting a voice in which each option is read aloud, by displaying an image representing each option, or by displaying words, icons, or the like representing each option. Furthermore, the selection assistance device may accept the user's selection of one of the multiple options presented, for example, by voice, or by inputting information specifying the option (hereinafter referred to as "option designation information"). The selection assistance device may be, for example, a voice interactive device that can have a dialogue with the user.

[0012] Furthermore, the multiple options presented to the user by the selection support device according to the first embodiment may be options relating to any matter. For example, the selection assistance device may present multiple options for cooking recipes from a display device or audio output device installed on an information terminal such as a PC (Personal Computer), smartphone, or tablet terminal owned by a user in an ordinary household. Furthermore, for example, the selection assistance device may present a plurality of menu options from a display device or audio output device provided on an information terminal such as a tablet terminal for ordering at a restaurant. Furthermore, for example, the selection assistance device may present a plurality of options for recommended products from online stores on a display device or audio output device provided on an information terminal held by the user. Furthermore, for example, the selection assistance device may present a plurality of options for restaurants located along the guide route on a display device or audio output device provided in a car navigation device in a vehicle. Furthermore, for example, the selection assistance device may be configured to present a plurality of options for recommended models of air conditioners or the like from a customer service robot installed in a home appliance retail store. Furthermore, for example, the selection assistance device may be configured to present a plurality of options for rental properties to be recommended to a customer from an information terminal installed in a real estate company. In this way, the selection assistance device according to the first embodiment can present options regarding various items by display, audio, etc. It is sufficient for the selection assistance device to be able to present a plurality of options to the user.

[0013] In the following first embodiment, as an example, the selection assistance device will be described as presenting multiple options for cooking recipes from a display device or a voice output device provided in an information terminal held by a user in an ordinary home. Also, as an example, it is assumed that the selection assistance device is a voice interactive device.

[0014] FIG. 1 is a diagram illustrating an example of the configuration of a selection support device 1 according to the first embodiment. The selection support device 1 is installed in an information terminal held by a user, and is connected to a reaction detection device 3 and an output device 2.

[0015] The reaction detection device 3 is provided in, for example, an information terminal, and detects the user's reaction to the multiple options presented by the selection support device 1. In the first embodiment, the user's reaction detected by the reaction detection device 3 includes, for example, speech, facial expression, or emotion. The reaction detection device 3 is, for example, a microphone, an imaging device, or a pulse wave sensor. The reaction detection device 3 outputs to the selection support device 1 information indicating the user's reaction to the presented multiple options (hereinafter referred to as "reaction information"). 1, only one reaction detection device 3 is connected to the selection support device 1, but this is merely an example, and multiple reaction detection devices 3 may be connected to the selection support device 1. Different types of reaction detection devices 3, such as a microphone and an imaging device, may also be connected to the selection support device 1.

[0016] The output device 2 is provided in, for example, an information terminal, and presents a plurality of options to the user. The output device 2 is, for example, a speaker or a display device. For example, if the output device 2 is a speaker, the output device 2 outputs the multiple options by voice. For example, if the output device 2 is a display device, the output device 2 displays the multiple options. The output device 2 also outputs audio or displays information for dialogue with the user. 1, only one reaction detection device 3 is connected to the selection support device 1, but this is merely an example, and multiple reaction detection devices 3 may be connected to the selection support device 1. Different types of reaction detection devices 3, such as a microphone and an imaging device, may also be connected to the selection support device 1.

[0017] When the selection support device 1 presents multiple options from the output device 2, it determines whether or not the user has selected any of the multiple options, and if it determines that the user has not selected any of the multiple options, it acquires reaction information to the presentation from the reaction detection device 3 and estimates whether or not the user is in a confused state based on the acquired reaction information. When it is estimated that the user is in a confused state, the selection support device 1 presents multiple alternative options from the output device 2.

[0018] As shown in FIG. 1, the selection support device 1 includes an initial option presenting unit 10, a selection presence / absence determining unit 11, a confused state estimating unit 12, an alternative option presenting unit 13, and a storage unit .

[0019] The initial option presentation unit 10 presents a plurality of options to the user. When an option presentation instruction is received, the initial option presentation unit 10 presents a plurality of options. For example, when the power of the selection support device 1 is turned on, a control unit (not shown) of the selection support device 1 outputs an option presentation instruction to the initial option presentation unit 10. Alternatively, for example, a user may input an option presentation instruction using an input device (not shown), and the initial option presentation unit 10 may accept this instruction. The first option presentation unit 10 presents a plurality of options only once when it receives an option presentation instruction. In the first embodiment, the options that are presented only once by the first option presentation unit 10 when it receives an option presentation instruction are referred to as "first options."

[0020] More specifically, the first option presentation unit 10 presents a plurality of first options in accordance with the first option presentation conditions. The first option presentation conditions are conditions that define how to determine the plurality of first options to be presented. The first option presentation conditions include a first presentation order determination condition that defines the order in which the plurality of first options should be presented from among the plurality of presentable options. The conditions for presenting initial options are set in advance by a developer or the like, and information indicating the conditions for presenting initial options (hereinafter referred to as "condition information for presenting initial options") is stored in an internal buffer of the initial option presentation unit 10 or in the memory unit 14 or the like. The initial option presentation condition may be set to, for example, "present four options determined in accordance with the first presentation order determining condition from among the multiple options stored in the storage unit 14." The first presentation order determining condition may be set to, for example, "follow the priority assigned to each option," "follow the order of the user's preference," or "arbitrary." If the first presentation order determining condition is set to "follow the order of the user's preference," the initial option presentation unit 10 may learn the user's preferences from, for example, a history of which options the user has selected in the past. For example, the selection presence / absence determination unit 11 may store the history of which options the user has selected in the storage unit 14 or the like. Details of the selection presence / absence determination unit 11 will be described later. The conditions for presenting the initial options may be, for example, "present four options determined in accordance with the first conditions for determining the presentation order from an external cooking site or a user-submitted recipe site (not shown) or the API (Application Programming Interface, not shown) of a recipe service," or "present four options determined in accordance with the first conditions for determining the presentation order from an image search API (not shown)."

[0021] Here, in one example of the conditions for presenting initial options as described above, the number of multiple initial options to be presented at one time (four in the above example) is set. For example, by narrowing the number of multiple initial options to be presented at one time to a predetermined number, the selection support device 1 can present multiple initial options in a manner that makes it easier for the user to select an initial option and improves convenience for the user. For example, the developer etc. may determine the number of multiple initial options to be presented at one time and set it in the initial option presentation conditions. For example, the developer etc. may set the number of multiple initial options to be presented at one time depending on the age group of the user to whom the multiple options are to be presented. For example, the developer etc. may reduce the number of multiple initial options to be presented at one time if the target is elderly users. For example, the initial option presentation conditions may include a condition (hereinafter referred to as "first number determination condition") that specifies the number of multiple initial options to be presented at one time. The first number determination condition may, for example, be set to a condition that "the number is determined according to the specifications (size, etc.) of the screen of the output device 2," or a condition that "if the options are displayed as characters, the number is determined according to the number of characters in each option." Note that the conditions for how many options should be displayed depending on the screen specifications, or how many options should be displayed depending on the number of characters in each option, are set together with the number determination condition.

[0022] When the first option presentation unit 10 presents multiple first options to the output device 2 in accordance with the first option presentation conditions, it outputs information indicating that multiple first options have been presented (hereinafter referred to as "first option presentation information") to the selection presence / absence determination unit 11. The first option presentation information includes, for example, information indicating the date and time when the first option was presented and information indicating the multiple first options that were presented. Furthermore, the initial option presentation unit 10 stores the initial option presentation information in the storage unit 14. For example, when the initial option presentation unit 10 determines multiple initial options from the multiple options stored in the storage unit 14, the initial option presentation unit 10 may assign a presented flag and information indicating the date and time when the initial option was presented to the initial option from the multiple options stored in the storage unit 14. The initial option presentation information, the presented flag, or the information indicating the date and time when the initial option was presented stored in the storage unit 14 is deleted, for example, when the selection presence / absence determination unit 11 determines that the user has selected an option, when the power to the selection support device 1 is turned off, or when a control unit (not shown) receives an instruction to end selection.

[0023] After multiple options have been presented to the user, the selection presence / absence determination unit 11 determines whether the user has selected any of the multiple options presented on the output device 2. The multiple options include multiple initial options presented by the initial option presentation unit 10 or multiple alternative options presented by the alternative option presentation unit 13. In the following first embodiment, the initial option or the alternative options are collectively referred to simply as "options." In the first embodiment, the alternative options are presented by the alternative option presentation unit 13. Details of the alternative option presentation unit 13 will be described later. For example, when a user selects one of the presented options by voice, the selection determination unit 11 acquires the user's voice from a microphone (not shown) and recognizes the acquired voice to determine whether or not a selection has been made from the multiple options. In this case, the microphone (not shown) may be a microphone shared with the reaction detection device 3. Also, for example, when a user selects one of the multiple options presented by inputting option designation information, the selection determination unit 11 receives the option designation information from an input device (not shown), and determines whether or not a selection has been made from the multiple options based on the received option designation information.

[0024] When the selection presence / absence determination unit 11 determines that the user has not selected any of the multiple options, it outputs information indicating that the user has not selected any of the multiple options (hereinafter referred to as "unselected information") to the confused state estimation unit 12, together with the initial option presented information output from the initial option presentation unit 10 or the alternative option presented information output from the alternative option presentation unit 13. The alternative option presented information will be described later. When the selection determination unit 11 determines that the user has selected one of the multiple options, it causes the output device 2 to present information based on the selected option.

[0025] When the selection determination unit 11 determines that the user has not selected any of the multiple options, i.e., when non-selection information is output from the selection determination unit 11, the confusion state estimation unit 12 estimates whether the user is in a confused state based on the confusion estimation conditions. The confusion estimation condition is a condition that defines what kind of reaction of the user when multiple options are presented will be used to estimate that the user is confused. The confusion estimation conditions are set in advance by a developer or the like, and information indicating the confusion estimation conditions (hereinafter referred to as "confusion estimation condition information") is stored in an internal buffer of the confused state estimation unit 12 or in the memory unit 14 or the like.

[0026] The confusion estimation conditions include, for example, the following <First confusion estimation condition> to <Fourth confusion estimation condition>. <Conditions for first puzzle estimation> If the user does not select an option even after a response determination time has elapsed after a plurality of options have been presented, it is assumed that the user is confused. The length of the response determination time depends on the response determination time determination conditions. <Conditions for second puzzle estimation> If the user makes an utterance indicating confusion, it is assumed that the user is in a confused state. <Conditions for estimating the third puzzle> If the user has a troubled look on their face, it is assumed that the user is in a confused state. <Conditions for estimating the fourth puzzle> If the user is having trouble, assume that the user is confused.

[0027] In the above-mentioned <first confusion estimation condition>, the response determination time determination condition is included in the confusion estimation condition. The response determination time determination condition may, for example, be set to a condition that "the response determination time is a predetermined time." In this case, the response determination time is set in advance by a developer or the like and stored in the storage unit 14 or the like together with the response determination time determination condition.

[0028] Note that this is merely an example, and the response determination time determination condition may be, for example, "set the response determination time based on the correlation between the amount of information of multiple options previously presented and the time it took the user to select one of the multiple options (hereinafter referred to as "selection time"). For example, the confusion state estimation unit 12 learns the correlation between the amount of information of multiple options previously presented and the corresponding selection time, estimates the time it will take the user to select one of the multiple options currently presented, and sets the estimated time as the response determination time. In the first embodiment, the amount of information of the options is assumed to be, for example, the number of images when the options are displayed as images, or the number of characters when the options are displayed as characters.

[0029] Furthermore, for example, the confused state estimation unit 12 may set the response determination time to the average of the selection times taken by the user for multiple options that have the same amount of information as the currently presented multiple options, among multiple options that were previously presented. Note that in the first embodiment, "same" is not limited to being exactly the same, but also includes being approximately the same within a set range. For example, when the selection presence / absence determination unit 11 determines that the user has selected one of the options, it stores information in the storage unit 14 or the like that associates the option selected by the user, the selection time, and the date and time when the option was presented (hereinafter referred to as "selection history information"). The confused state estimation unit 12 may determine, from the selection history information, the selection times taken by the user for multiple options that have the same amount of information as the currently presented multiple options, among multiple options that were previously presented.

[0030] For example, when the above-mentioned <first confusion estimation condition> is set as the confusion estimation condition, the confused state estimation unit 12 estimates that the user is in a confused state if, after multiple options are presented, the selection presence / absence determination unit 11 does not determine that the user has selected any of the multiple options even after the response determination time has elapsed. The first option presented information or the alternative option presented information output from the selection presence / absence determination unit 11 together with the unselected information includes information indicating the date and time when the first option was presented or information indicating the date and time when the alternative option was presented, respectively. Based on the first option presented information or the alternative option presented information and the current time, the confusion state estimation unit 12 can determine whether the response determination time has elapsed after multiple options were presented.

[0031] For example, when the above-mentioned <second confusion estimation condition> is set as the confusion estimation condition, the confused state estimation unit 12 acquires the user's spoken voice as reaction information from the reaction detection device 3, and when an utterance indicating confusion is recognized as a result of speech recognition of the acquired spoken voice, it estimates that the user is in a confused state. Examples of utterances that indicate confusion include "hmm," "er," "um," "that's a problem," or "what should I do," etc. The confusion state estimation unit 12 may recognize utterances that indicate confusion from the uttered voice using a known voice recognition technique. In this case, the reaction detection device 3 is, for example, a microphone.

[0032] For example, when the above-mentioned <third confusion estimation condition> is set as the confusion estimation condition, the confused state estimation unit 12 acquires an image of the user's face as reaction information, and when it recognizes a troubled expression on the user's face based on the acquired image, it estimates that the user is in a confused state. The confused state estimation unit 12 may recognize that the user has a confused expression from the captured image using a known image recognition technique. The confused state estimation unit 12 may also recognize that the user has a confused expression using a trained model (hereinafter referred to as a "machine learning model") that receives the captured image as input and outputs information indicating the facial expression of the person captured in the captured image. In this case, the reaction detection device 3 is, for example, an imaging device.

[0033] For example, when the above-mentioned <fourth confusion estimation condition> is set as the confusion estimation condition, the confused state estimation unit 12 acquires pulse wave information indicating the user's pulse wave as reaction information, and when the user's confused emotion is estimated based on the acquired pulse wave information, it estimates that the user is in a confused state. The confused state estimation unit 12 may estimate that the user is in a confused state by using a known technique for estimating a person's emotions by analyzing a person's pulse wave obtained from a pulse wave sensor that can measure a person's pulse wave non-contact. In this case, the reaction detection device 3 is, for example, a pulse wave sensor.

[0034] For example, the confused state inferring unit 12 may infer whether the user is in a confused state by combining the above-mentioned <first confusion inferring condition> to <fourth confusion inferring condition>. For example, the confused state estimation unit 12 may provisionally estimate whether the user is in a confused state based on the above <first confusion estimation condition> to <fourth confusion estimation condition>, and may estimate that the user is in a confused state when it is provisionally estimated that the user is in a confused state according to any one of the above <first confusion estimation condition> to <fourth confusion estimation condition>. Furthermore, for example, the confused state estimation unit 12 may provisionally estimate whether the user is in a confused state based on the above <first confusion estimation condition> to <fourth confusion estimation condition>, and may estimate that the user is in a confused state by a majority vote of the provisional estimation results, or may weight the provisional estimation results as "1: confused state" and "0: not confused state" and estimate whether the user is in a confused state by comparing the weighted provisional estimation result with a preset threshold.

[0035] It should be noted that the above <First confusion estimation condition> to <Fourth confusion estimation condition> are merely examples, and the confusion estimation conditions may be set to conditions other than the above <First confusion estimation condition> to <Fourth confusion estimation condition>. For example, a confusion estimation condition may be set such that "if the user's gaze moves back and forth between the displayed options, it is estimated that the user is confused." In this case, the confusion state estimation unit 12 acquires a captured image of the user's face as reaction information, detects the user's gaze for a predetermined period based on the acquired captured image, and estimates that the user is confused if it determines that the user's gaze is moving back and forth between the options. The confusion state estimation unit 12 may detect the user's gaze from the captured image using a known image recognition technology. Note that it is assumed that the installation position and angle of view of the imaging device serving as the reaction detection device 3 are known in advance, and that the positions at which the options are displayed on the display device serving as the output device 2 are known in advance. Based on the positional relationship between the installation position and angle of view of the reaction detection device 3 and the positions at which the options are displayed on the display device, the confusion state estimation unit 12 can determine whether the user's gaze detected from the captured image is moving back and forth between the options.

[0036] The confused state estimation unit 12 outputs information indicating whether or not it has been estimated that the user is in a confused state (hereinafter referred to as a “confused state estimation result”) to the alternative option presentation unit 13.

[0037] When the confused state estimation unit 12 estimates that the user is in a confused state, the alternative option presentation unit 13 presents a plurality of alternative options different from the plurality of options currently presented. More specifically, the alternative option presentation unit 13 presents multiple alternative options in accordance with the alternative option presentation conditions. The alternative option presentation conditions are conditions that define how to determine the multiple alternative options to be presented. The alternative option presentation conditions include second presentation order determination conditions that define the order in which the multiple alternative options should be presented from the multiple options that can be presented. The conditions for presenting alternative options are set in advance by a developer or the like, and information indicating the conditions for presenting alternative options (hereinafter referred to as "condition information for presenting alternative options") is stored in an internal buffer of the initial option presentation unit 10 or in the memory unit 14 or the like. The condition for presenting another option may be, for example, "present four options determined in accordance with the second condition for determining presentation order from among the multiple options stored in the storage unit 14 and not yet presented (i.e., not already presented)." The condition for determining the second condition for determining presentation order may be, for example, "follow the priority assigned to each option," "follow the order of preference of the user," or "arbitrary." The conditions for presenting alternative options may be, for example, "present four options determined in accordance with the second conditions for determining the presentation order from an external cooking site, a user-submitted recipe site, or the API of a recipe service," or "present four options determined in accordance with the second conditions for determining the presentation order from an image search API."

[0038] Furthermore, the conditions for presenting alternative options may be set to, for example, "The user is asked whether or not it is necessary to present multiple alternative options, and if the user responds that it is necessary to present multiple alternative options, four alternative options determined in accordance with the second conditions for determining presentation order are presented. The multiple alternative options may be multiple options stored in the memory unit 14 and may be options that have not been presented (i.e., not already presented), or may be options obtained from an external cooking site or a user-submitted recipe site, or may be options obtained from the API of a recipe service, or may be options obtained from an image search API." Furthermore, the conditions for presenting alternative options may be set to, for example, "When presenting multiple alternative options, the user's preference is confirmed, and four alternative options that match the preference obtained from the user are presented in accordance with the second conditions for determining the presentation order. The multiple alternative options may be multiple options stored in the memory unit 14, and may be options that have not been presented (i.e., not already presented), or may be options obtained from an external cooking site or a user-submitted recipe site, or may be options obtained from the API of a recipe service, or may be options obtained from an image search API."

[0039] Here, in one example of the conditions for presenting alternative options as described above, the number of alternative options to be presented at one time (four in the above example) is set. For example, by narrowing the number of alternative options to be presented at one time to a predetermined number, the selection support device 1 can present the alternative options in a manner that makes it easier for the user to select an alternative option and improves convenience for the user. For example, the developer etc. may determine the number of alternative options to be presented at one time and set it in the alternative option presentation conditions. For example, the developer etc. may set the number of alternative options to be presented at one time depending on the age group of the user to whom the alternative options are to be presented. For example, the developer etc. may reduce the number of alternative options to be presented at one time if the target is an elderly person. For example, a condition specifying the number of alternative options to be presented at one time (hereinafter referred to as a "second number-determining condition") may be included in the alternative option presentation condition. The second condition for determining the number may be set to, for example, a condition for adjusting the number of alternative options to be presented in accordance with the amount of information in the alternative options. For example, the second condition for determining the number may be set to "determine based on the specifications (size, etc.) of the screen of the output device 2," or "if the options are displayed in text, determine based on the number of characters in each option." It is assumed that conditions such as how many options should be displayed depending on the screen specifications, or how many options should be displayed depending on the number of characters in each option, are set together with the number determination conditions. In this way, by making it possible for the alternative option presentation unit 13 to adjust the number of multiple alternative options to be presented depending on the amount of information in the multiple alternative options, the selection support device 1 can present multiple alternative options in a manner that makes it easier for the user to select an alternative option and improves convenience for the user.

[0040] When the alternative option presentation unit 13 presents multiple alternative options to the output device 2 in accordance with the conditions for presenting alternative options, it outputs information to the selection presence / absence determination unit 11 that multiple alternative options have been presented. In the first embodiment, the information that multiple alternative options have been presented is referred to as "alternative option presentation information." The alternative option presentation information includes, for example, information indicating the date and time when the alternative option was presented, and information indicating the multiple alternative options that were presented. Furthermore, the alternative option presentation unit 13 stores the alternative option presentation information in the storage unit 14. For example, when the alternative option presentation unit 13 determines multiple alternative options from the multiple options stored in the storage unit 14, the alternative option presentation unit 13 may assign a presented flag and information indicating the date and time when the alternative option was presented to the alternative options from the multiple options stored in the storage unit 14. The alternative option presentation information, the presented flag, or the information indicating the date and time when the alternative option was presented stored in the storage unit 14 is deleted, for example, when the selection presence / absence determination unit 11 determines that the user has selected an option, when the power to the selection support device 1 is turned off, or when a control unit (not shown) receives a selection end instruction.

[0041] 2A and 2B are diagrams for explaining an example of a flow until the alternative option presenting unit 13 presents a plurality of alternative options to the output device 2 in the first embodiment. 2A and 2B, the initial option presentation condition is set to "present four options determined in accordance with the first presentation order determining condition from among the multiple options stored in the storage unit 14," and the first presentation order determining condition is set to "arbitrary." The other option presentation condition is set to "present four options determined in accordance with the second presentation order determining condition from among the multiple options stored in the storage unit 14 and that have not yet been presented (i.e., have not already been presented)," and the second presentation order determining condition is set to "arbitrary." The confusion estimation condition is set to the above-mentioned <second confusion estimation condition>. Figure 2A shows the user uttering "Hmm" after four initial options are presented on the output device 2 by the initial option presentation unit 10, and Figure 2B shows the user uttering multiple alternative options by the alternative option presentation unit 13. In addition, in FIGS. 2A and 2B, the output device 2 is a display device, and multiple options are presented on a display.

[0042] For example, as shown in Fig. 2A, if the user utters "Hmm" after being presented with a plurality of initial options, the confusion state estimation unit 12 acquires the user's speech as reaction information from the reaction detection device 3, recognizes the utterance indicating confusion, and estimates that the user is in a confused state. In response to the confusion state estimation result indicating that the user is estimated to be in a confused state, the alternative option presentation unit 13 presents four alternative options as shown in Fig. 2B. The user can display other options without having to take the trouble of operating a button to display other options.

[0043] 3A, 3B, and 3C are diagrams for explaining another example of the flow until the alternative option presenting unit 13 presents a plurality of alternative options to the output device 2 in the first embodiment. In the example shown in FIGS. 3A, 3B, and 3C, the initial option presentation condition is set to "present four options determined according to the first presentation order determining condition from among the multiple options stored in the storage unit 14," and the first presentation order determining condition is set to "optional." The alternative option presentation condition is set to "ask the user whether multiple alternative options need to be presented, and if the user responds that multiple alternative options need to be presented, present four alternative options determined according to the second presentation order determining condition. The multiple alternative options may be multiple options stored in the storage unit 14 and may be unpresented (i.e., not already presented) options, options acquired from an external cooking website or a user-submitted recipe website, options acquired from the API of a recipe service, or options acquired from an image search API." The second presentation order determining condition is set to "optional." The confusion estimation condition is set to the above-mentioned <second confusion estimation condition>. Figure 3A shows the user looking confused after four initial options are presented on the output device 2 by the initial option presentation unit 10, Figure 3B shows the alternative option presentation unit 13 checking whether or not multiple alternative options need to be presented to the user from the output device 2, and receiving a response from the user indicating that multiple alternative options need to be presented, and Figure 3C shows the alternative option presentation unit 13 presenting multiple alternative options. In addition, in FIGS. 3A, 3B, and 3C, the output device 2 is a display device, and multiple options are presented on a display.

[0044] For example, as shown in FIG. 3A, if the user makes a confused face after multiple initial options are presented, the confusion state estimation unit 12 acquires an image of the user as reaction information from the reaction detection device 3, recognizes the confused expression, and estimates that the user is in a confused state. As shown in FIG. 3B, the alternative option presentation unit 13 asks the user, "Would you like to see other recommended dishes?" to confirm whether multiple alternative options need to be presented. Note that the message used to confirm whether multiple alternative options need to be presented to the user is determined in advance. Then, when the user responds "Yes," indicating that multiple alternative options need to be presented, the alternative option presentation unit 13 presents four alternative options as shown in FIG. 3C. The user can display other options without having to take the trouble of operating a button to display other options.

[0045] 4A, 4B, and 4C are diagrams for explaining another example of the flow until the alternative option presenting unit 13 presents a plurality of alternative options to the output device 2 in the first embodiment. In the example shown in FIGS. 4A, 4B, and 4C, the initial option presentation condition is set to "present four options determined according to the first presentation order determining condition from among the multiple options stored in the storage unit 14," and the first presentation order determining condition is set to "optional." The alternative option presentation condition is set to "when presenting multiple alternative options, confirm the user's preference, and present four alternative options that meet the user's preference according to the second presentation order determining condition. The multiple alternative options may be multiple options stored in the storage unit 14 and may be unpresented (i.e., not already presented), options acquired from an external cooking site or a user-submitted recipe site, options acquired from the API of a recipe service, or options acquired from an image search API." The second presentation order determining condition is set to "optional." The confusion estimation condition is set to the above-mentioned <second confusion estimation condition>. Figure 4A shows the user looking confused after four initial options are presented on the output device 2 by the initial option presentation unit 10, Figure 4B shows the alternative option presentation unit 13 confirming with the user via the output device 2 whether they would like multiple alternative options, resulting in the user responding that they would prefer a fish recipe rather than a meat recipe, and Figure 4C shows the alternative options presented by the alternative option presentation unit 13. In addition, in FIGS. 4A, 4B, and 4C, the output device 2 is a display device, and multiple options are presented on a display.

[0046] For example, as shown in FIG. 4A, if the user makes a confused face after being presented with multiple initial options, the confusion state estimation unit 12 acquires an image of the user as reaction information from the reaction detection device 3, recognizes the confused expression, and estimates that the user is in a confused state. As shown in FIG. 4B, the alternative option presentation unit 13 asks the user, "Do you have any preferences?" to confirm multiple alternative options. Note that the message used to confirm the user's preferences for multiple alternative options is determined in advance. Then, when the user responds that they prefer "fish dishes over meat," the alternative option presentation unit 13 presents four alternative options that meet the user's preferences, as shown in FIG. 3C. The user can display other options without having to take the trouble of operating a button to display other options.

[0047] Returning to the description of the configuration example of the selection support device 1 according to the first embodiment. The storage unit 14 stores various types of information. 1, the storage unit 14 is provided in the selection support device 1, but this is merely an example. For example, the storage unit 14 may be provided in a location outside the selection support device 1 that can be referenced by the selection support device 1.

[0048] The operation of the selection support device 1 according to the first embodiment will be described. FIG. 5 is a flowchart for explaining the operation of the selection support device 1 according to the first embodiment. For example, when the selection support device 1 receives an instruction to present options, it starts the operation shown in the flowchart of Fig. 5. For example, when the power of the selection support device 1 is turned off or when an instruction to end the operation is received, the selection support device 1 ends the operation shown in the flowchart of Fig. 5. The instruction to end the operation is input by the user using an input device (not shown), for example.

[0049] The first option presentation unit 10 presents a plurality of options to the user. More specifically, the first option presentation unit 10 presents a plurality of first options in accordance with the first option presentation conditions (step ST1). When the first option presentation unit 10 presents a plurality of first options on the output device 2 in accordance with the first option presentation conditions, it outputs first option presentation completion information to the selection presence / absence determination unit 11. Furthermore, the first option presentation unit 10 stores the first option presentation completion information in the storage unit 14.

[0050] The selection presence / absence determination unit 11 determines whether or not the user has selected any of the multiple options presented on the output device 2. When it is determined that the user has selected one of the multiple options ("YES" in step ST2), the selection presence / absence determination unit 11 presents information based on the selected option to the output device 2. Then, the selection support device 1 ends the operation shown in the flowchart of FIG.

[0051] On the other hand, if it is determined that the user has not selected any of the multiple options ("NO" in step ST2), the selection determination unit 11 outputs unselected information to the confused state estimation unit 12 together with the initial option presentation information output from the initial option presentation unit 10 or the alternative option presentation information output from the alternative option presentation unit 13.

[0052] The confused state estimation unit 12 estimates whether the user is in a confused state based on the confusion estimation conditions (step ST3). The confused state estimation unit 12 outputs the confused state estimation result to the alternative option presentation unit 13.

[0053] In step ST3, if the confusion state estimation unit 12 estimates that the user is in a confused state (if "YES" in step ST4), the alternative option presentation unit 13 presents multiple alternative options that are different from the multiple options currently presented (step ST5). When the alternative option presentation unit 13 presents a plurality of alternative options to the output device 2 in accordance with the alternative option presentation conditions, it outputs alternative option presentation information to the selection presence / absence determination unit 11. In addition, the alternative option presentation unit 13 stores the alternative option presentation information in the storage unit 14. Then, the operation of the selection support device 1 returns to the processing of step ST2. In the processing of step ST2 executed for the second time, the selection presence / absence determination unit 11 determines whether or not the user has selected any one of the multiple alternative options presented on the output device 2.

[0054] If the confusion state estimation unit 12 does not estimate in step ST3 that the user is in a confused state (if "NO" in step ST4), the operation of the selection support device 1 skips the processing of step ST5 and returns to the processing of step ST2.

[0055] In this way, after multiple options are presented to the user, the selection support device 1 determines whether the user has selected any of the multiple options, and if it determines that the user has not selected any of the multiple options, it estimates whether the user is in a confused state based on the confusion estimation condition. Then, if it estimates that the user is in a confused state, the selection support device 1 presents multiple alternative options that are multiple options different from the multiple options currently presented. This allows the selection support device 1 to present other options to the user without causing the user any trouble. The selection support device 1 can observe the user's reaction and present other options without causing the user any trouble, just like a store clerk (a real person) who can observe the customer's subtle reactions and respond accordingly, and therefore can give the user a good impression of being considerate and easy to use.

[0056] In the above-described first embodiment, as shown in FIG. 1, the initial option presentation unit 10 is provided in the selection support device 1, but this is merely an example, and the selection support device 1 does not necessarily have to be provided with the initial option presentation unit 10. For example, the initial option presenting unit 10 may be provided in a device separate from the selection support device 1, which is external to the selection support device 1 and connected to the selection support device 1. If the selection support device 1 is configured not to include the initial option presentation unit 10, the selection support device 1 can omit the processing of step ST1 in the operation described using the flowchart of FIG.

[0057] In the first embodiment, the selection support device 1 is mounted on an information terminal, but this is merely an example. The selection support device 1 may be mounted on a server, for example. Alternatively, some of the initial option presentation unit 10, selection presence / absence determination unit 11, confusion state estimation unit 12, and alternative option presentation unit 13 provided in the selection support device 1, or a control unit (not shown), may be provided in an information terminal held by a user, and the rest may be provided in the server, so that the information terminal and the server constitute a system.

[0058] An example of the hardware configuration of the selection support device 1 according to the first embodiment will be described. 6A and 6B are diagrams illustrating an example of a hardware configuration of the selection support device 1 according to the first embodiment. In the first embodiment, the functions of the initial option presentation unit 10, the selection presence / absence determination unit 11, the confusion state estimation unit 12, the alternative option presentation unit 13, and a control unit (not shown) are realized by a processing circuit 101. That is, the selection support device 1 includes the processing circuit 101 for performing control to present alternative options when it is estimated that the user is in a confused state. The processing circuit 101 may be dedicated hardware as shown in FIG. 6A, or may be a processor 104 that executes a program stored in a memory 105 as shown in FIG. 6B.

[0059] When the processing circuitry 101 is dedicated hardware, the processing circuitry 101 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0060] When the processing circuit is a processor 104, the functions of the initial choice presentation unit 10, the selection determination unit 11, the confusion state estimation unit 12, the alternative choice presentation unit 13, and a control unit (not shown) are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in memory 105. The processor 104 reads and executes the program stored in memory 105 to perform the functions of the initial choice presentation unit 10, the selection determination unit 11, the confusion state estimation unit 12, the alternative choice presentation unit 13, and a control unit (not shown). In other words, the selection support device 1 includes memory 105 for storing a program that, when executed by the processor 104, results in the execution of steps ST1 to ST5 of FIG. 5 described above. It can also be said that the program stored in memory 105 causes a computer to execute the processing procedures or methods of the initial choice presentation unit 10, the selection determination unit 11, the confusion state estimation unit 12, the alternative choice presentation unit 13, and a control unit (not shown). Here, memory 105 refers to, for example, non-volatile or volatile semiconductor memory such as RAM, ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disc), etc.

[0061] It should be noted that the functions of the initial option presentation unit 10, the selection presence / absence determination unit 11, the confusion state estimation unit 12, the alternative option presentation unit 13, and the control unit (not shown) may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the functions of the initial option presentation unit 10 and the selection presence / absence determination unit 11 may be realized by a processing circuit 101 as dedicated hardware, and the functions of the confusion state estimation unit 12, the alternative option presentation unit 13, and the control unit (not shown) may be realized by a processor 104 reading and executing a program stored in a memory 105. The selection support device 1 also includes an input interface device 102 and an output interface device 103 that perform wired or wireless communication with devices such as the output device 2 or the reaction detection device 3.

[0062] As described above, according to the first embodiment, the selection support device 1 is configured to include a selection determination unit 11 that determines whether or not the user has selected any of the multiple options after multiple options have been presented to the user, a confusion state estimation unit 12 that, when the selection determination unit 11 determines that the user has not selected any of the multiple options, estimates whether or not the user is in a confused state based on a confusion estimation condition, and an alternative option presentation unit 13 that, when the confusion state estimation unit 12 estimates that the user is in a confused state, presents multiple alternative options that are multiple options different from the multiple options currently presented. Therefore, the selection support device 1 can present other options to the user without causing the user any trouble.

[0063] Any of the components of the embodiments may be modified or omitted.

[0064] Various aspects of the present disclosure are summarized below as appendices.

[0065] (Appendix 1) a selection determination unit that determines whether or not a selection has been made by the user from among a plurality of options presented to the user; a confusion state estimation unit that, when the selection presence / absence determination unit determines that none of the options has been selected by the user, estimates whether the user is in a confused state based on a confusion estimation condition; and an alternative option presentation unit that presents a plurality of alternative options that are different from the plurality of options currently presented when the confusion state estimation unit estimates that the user is in a confused state; A selection assistance device comprising: (Appendix 2) The confusion estimation conditions include: a first confusion inference condition is set such that, if the user does not select one of the options even after a response determination time has elapsed after the multiple options have been presented, it is inferred that the user is confused; The confused state estimation unit After the plurality of options are presented, if the selection determination unit does not determine that the user has selected any of the plurality of options even after the response determination time has elapsed, it is estimated that the user is confused. 2. The selection support device according to claim 1, (Appendix 3) The confused state estimation unit The response determination time is set based on a correlation between an amount of information of the plurality of options previously presented and a time required for the user to select one of the plurality of options. 3. The selection support device according to claim 2. (Appendix 4) The confusion estimation conditions include: a second confusion estimation condition is set such that, when the user makes an utterance indicating confusion, it is estimated that the user is in a confused state; The confused state estimation unit The speech of the user is acquired as reaction information indicating the user's reaction to the presented plurality of options, and when an utterance indicating confusion is recognized as a result of speech recognition of the acquired speech, it is estimated that the user is in a confused state. 2. The selection support device according to claim 1, (Appendix 5) The confusion estimation conditions include: a third confusion estimation condition is set such that, if the user has a troubled face, it is estimated that the user is in a confused state; The confused state estimation unit A captured image of the user's face is acquired as reaction information indicating the user's reaction to the presented plurality of options, and when a troubled expression of the user is recognized based on the acquired captured image, it is estimated that the user is in a confused state. 2. The selection support device according to claim 1, (Appendix 6) The confusion estimation conditions include: a fourth confusion estimation condition is set, which is that if the user is in trouble, it is estimated that the user is in a confused state; The confused state estimation unit Pulse wave information showing the pulse wave of the user is acquired as reaction information showing the user's reaction to the presented plurality of options, and when the user's troubled feelings are estimated based on the acquired pulse wave information, it is estimated that the user is in a confused state. 2. The selection support device according to claim 1, (Appendix 7) The confusion estimation conditions include: a first confusion estimating condition that, if the user does not select one of the options even after a response determination time has elapsed after the multiple options have been presented, it is estimated that the user is in a confused state; a second confusion estimating condition that, if the user makes an utterance indicating confusion, it is estimated that the user is in a confused state; a third confusion estimating condition that, if the user has a troubled face, it is estimated that the user is in a confused state; or a fourth confusion estimating condition that, if the user is troubled, it is estimated that the user is in a confused state; The confused state estimation unit If the selection determination unit does not determine that the user has selected any of the options even after the response determination time has elapsed after the options have been presented, A speech of the user is acquired as reaction information indicating a reaction of the user to the presented plurality of options, and as a result of speech recognition of the acquired speech, if a speech indicating confusion is recognized, A captured image of the face of the user is acquired as the reaction information, and when a troubled expression of the user is recognized based on the acquired captured image, or Pulse wave information indicating a pulse wave of the user is acquired as the reaction information, and when a distressed feeling of the user is estimated based on the acquired pulse wave information, It is assumed that the user is confused. 2. The selection support device according to claim 1, (Appendix 8) It is a voice interactive device, The alternative option presentation unit When the confusion state estimation unit estimates that the user is in a confused state, the unit confirms with the user whether or not it is necessary to present the multiple alternative options, and when a response is received from the user indicating that it is necessary to present the multiple alternative options, the unit presents the multiple alternative options. 8. The selection support device according to any one of claims 1 to 7, (Appendix 9) The alternative option presentation unit The number of the alternative options to be presented is adjusted according to the amount of information of the alternative options. 9. The selection support device according to any one of appendices 1 to 8, (Appendix 10) The amount of information is the number of characters or the number of images. 10. The selection support device according to claim 9, (Appendix 11) The plurality of options are presented by display or sound. 2. The selection support device according to claim 1, (Appendix 12) Computer, a selection determination unit that determines whether or not a selection has been made by the user from among a plurality of options presented to the user; a confusion state estimation unit that, when the selection presence / absence determination unit determines that none of the options has been selected by the user, estimates whether the user is in a confused state based on a confusion estimation condition; and an alternative option presentation unit that presents a plurality of alternative options that are different from the plurality of options currently presented when the confusion state estimation unit estimates that the user is in a confused state; Selection support program to function as. [Explanation of symbols]

[0066] 1 Selection support device, 10 Initial option presentation unit, 11 Selection presence / absence determination unit, 12 Confusion state estimation unit, 13 Alternative option presentation unit, 14 Memory unit, 2 Output device, 3 Response detection device, 101 Processing circuit, 102 Input interface device, 103 Output interface device, 104 Processor, 105 Memory.

Claims

1. a selection determination unit that determines whether or not a selection has been made by the user from among a plurality of options presented to the user; a confusion state estimation unit that, when the selection presence / absence determination unit determines that none of the options has been selected by the user, estimates whether the user is in a confused state based on a confusion estimation condition; and an alternative option presentation unit that presents a plurality of alternative options that are different from the plurality of options currently presented when the confusion state estimation unit estimates that the user is in a confused state; A selection assistance device comprising:

2. The confusion estimation conditions include: a first confusion inference condition is set such that, if the user does not select one of the options even after a response determination time has elapsed after the multiple options have been presented, it is inferred that the user is confused; The confused state estimation unit After the plurality of options are presented, if the selection determination unit does not determine that the user has selected any of the plurality of options even after the response determination time has elapsed, it is estimated that the user is confused.

2. The selection support device according to claim 1.

3. The confused state estimation unit The response determination time is set based on a correlation between an amount of information of the plurality of options previously presented and a time required for the user to select one of the plurality of options.

3. The selection support device according to claim 2.

4. The confusion estimation conditions include: a second confusion estimation condition is set such that, when the user makes an utterance indicating confusion, it is estimated that the user is in a confused state; The confused state estimation unit The speech of the user is acquired as reaction information indicating the user's reaction to the presented plurality of options, and when an utterance indicating confusion is recognized as a result of speech recognition of the acquired speech, it is estimated that the user is in a confused state.

2. The selection support device according to claim 1.

5. The confusion estimation conditions include: a third confusion estimation condition is set such that, if the user has a troubled face, it is estimated that the user is in a confused state; The confused state estimation unit A captured image of the user's face is acquired as reaction information indicating the user's reaction to the presented plurality of options, and when a troubled expression of the user is recognized based on the acquired captured image, it is estimated that the user is in a confused state.

2. The selection support device according to claim 1.

6. The confusion estimation conditions include: a fourth confusion estimation condition is set such that, if the user is in trouble, it is estimated that the user is in a confused state; The confused state estimation unit Pulse wave information showing the pulse wave of the user is acquired as reaction information showing the user's reaction to the presented plurality of options, and when the user's troubled feelings are estimated based on the acquired pulse wave information, it is estimated that the user is in a confused state.

2. The selection support device according to claim 1.

7. The confusion estimation conditions include: a first confusion estimating condition that, if the user does not select one of the options even after a response determination time has elapsed after the multiple options have been presented, it is estimated that the user is in a confused state; a second confusion estimating condition that, if the user makes an utterance indicating confusion, it is estimated that the user is in a confused state; a third confusion estimating condition that, if the user has a troubled face, it is estimated that the user is in a confused state; or a fourth confusion estimating condition that, if the user is troubled, it is estimated that the user is in a confused state; The confused state estimation unit If the selection determination unit does not determine that the user has selected any of the options even after the response determination time has elapsed after the options have been presented, A speech of the user is acquired as reaction information indicating a reaction of the user to the presented plurality of options, and as a result of speech recognition of the acquired speech, if a speech indicating confusion is recognized, A captured image of the face of the user is acquired as the reaction information, and when a troubled expression of the user is recognized based on the acquired captured image, or Pulse wave information indicating a pulse wave of the user is acquired as the reaction information, and when a distressed feeling of the user is estimated based on the acquired pulse wave information, It is assumed that the user is confused.

2. The selection support device according to claim 1.

8. It is a voice interactive device, The alternative option presentation unit When the confusion state estimation unit estimates that the user is in a confused state, the unit confirms with the user whether or not it is necessary to present the multiple alternative options, and when a response is received from the user indicating that it is necessary to present the multiple alternative options, the unit presents the multiple alternative options.

8. The selection support device according to claim 1, wherein the selection support device is a device for selecting a selection target.

9. The alternative option presentation unit The number of the alternative options to be presented is adjusted according to the amount of information of the alternative options.

2. The selection support device according to claim 1.

10. The amount of information is the number of characters or the number of images.

10. The selection support device according to claim 9.

11. The plurality of options are presented by display or sound.

2. The selection support device according to claim 1.

12. Computer, a selection determination unit that determines whether or not a selection has been made by the user from among a plurality of options presented to the user; a confusion state estimation unit that, when the selection presence / absence determination unit determines that none of the options has been selected by the user, estimates whether the user is in a confused state based on a confusion estimation condition; and an alternative option presentation unit that presents a plurality of alternative options that are different from the plurality of options currently presented when the confusion state estimation unit estimates that the user is in a confused state; Selection support program to function as.

Images