Support device, computer program, and support method

Abstract

The present invention provides a support device that assists users in using user-related devices stored in a storage device in association with the user. [Solution] The support device, Server 1, comprises an input unit, a control unit, and an output unit. The control unit receives input of first information C1 from the user terminal 3 via the input unit, which explicitly or implicitly represents the user's needs. Based on the first information, it generates a prompt P1 to give to a trained model, which identifies a target device from among the user's devices that has a function to meet the user's needs, and obtains second information from the trained model as a response to support the user regarding the use of the target device. The control unit then outputs the second information obtained from the trained model that provided the prompt to the user terminal or user device via the output unit.

Description

Technical Field

[0001] The present disclosure relates to a support device, a computer program, and a support method.

Background Art

[0002] The performance of devices used by users, such as home appliances, is improving. Therefore, some users may not fully understand the functions and may not know how to use them appropriately according to the situation or may not be able to use them appropriately.

[0003] For example, Japanese Patent Application Laid-Open No. 2007-43254 (hereinafter, Patent Document 1) discloses a remote control device that sends a control code telegram that causes an operation corresponding to the operation to occur on a device to be controlled by operating an operator such as a button or a switch in a predetermined operation procedure, and has a display screen that displays the state of the device, a dedicated explanation display operator, and memory means that stores handling information of each operator. When the explanation display operator is operated, a telegram for causing an operation to occur on the device to be controlled is not transmitted, and immediately after or immediately before the operation of the explanation display operator, the handling explanation corresponding to the operator operated immediately before or after is read from the memory means and displayed on the display screen.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

[0005] In the remote control device described in Patent Document 1, an operation is required to display instructions for each button. For complex functions, multiple button operations may be required. To understand how to operate such complex functions, the user has to repeatedly display and confirm the instructions for each of the multiple buttons. Furthermore, the user must operate the button themselves after understanding the instructions displayed for each button. As a result, the more complex the function, the more difficult it becomes for the user to understand and use it effectively. One of the objectives of this disclosure is to provide an assistive device, a computer program, and a method for assisting the use of user devices stored in a storage device in association with the user.

[0006] According to one embodiment, the support device comprises an input unit, a control unit, and an output unit. The control unit receives input of first information from a user terminal via the input unit, which explicitly or implicitly represents the user's needs, and generates a prompt to give to a trained model based on the first information, which is a prompt to identify a target device from among the user's devices that has a function to meet the user's needs, and to obtain second information from the trained model as a response to support the user regarding the use of the target device, and outputs the second information obtained from the trained model that provided the prompt to the user terminal or user device via the output unit.

[0007] According to one embodiment, the computer program is a computer program that causes the computer to function as an assistive device, and via the computer's input device, it receives input of first information from a user terminal that explicitly or implicitly represents the user's needs, generates a prompt to give to a trained model based on the first information, which identifies a target device from among the user's devices that has a function to meet the user's needs, generates a prompt to obtain second information from the trained model as an answer to assist the user regarding the use of the target device, and outputs the second information obtained from the trained model that provided the prompt to the user terminal or user device via the computer's output unit.

[0008] According to one embodiment, the support method comprises: a support device receiving input of first information from a user terminal via an input unit that explicitly or implicitly represents the user's needs; the support device generating a prompt to give to a trained model based on the first information, which identifies a target device from among the user's devices that has a function to meet the user's needs, and obtains second information from the trained model as a response to support the user regarding the use of the target device; and the support device outputting the second information obtained from the trained model that provided the prompt to the user terminal or user device via an output unit.

[0009] Further details will be described in the embodiments below. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is an overview of the support method according to the first embodiment, and is a diagram showing an overview of the support method based on the first embodiment. [Figure 2] Figure 2 is a schematic diagram of the support devices used in the support method, including the server, user terminal, and user equipment. [Figure 3] Figure 3 is a flowchart illustrating an example of server operation. [Figure 4] Figure 4 is a flowchart illustrating an example of user terminal operation. [Figure 5] Figure 5 is a schematic diagram of the user terminal's display screen. [Figure 6] Figure 6 is a flowchart illustrating an example of the operation of a user terminal in the support method according to the third embodiment. [Figure 7] Figure 7 is a diagram illustrating the overview of the support method based on the second embodiment. [Figure 8] Figure 8 is a diagram illustrating the overview of the support method based on the third embodiment. [Figure 9] Figure 9 is a diagram illustrating an overview of the support method based on the fourth embodiment. [Figure 10]Figure 10 is a diagram illustrating the overview of the support method based on the fifth embodiment. [Modes for carrying out the invention]

[0011] <1. Overview of support devices, computer programs, and support methods> (1) Support device according to the embodiment, It comprises an input section, a control section, and an output section. The control unit, The input unit receives input from the user terminal of first information that explicitly or implicitly represents the user's needs. Based on the first information, a prompt is generated to give to the trained model, which identifies a target device from among the user's devices that has a function to meet the user's needs, and obtains a second information from the trained model as a response to support the user regarding the use of the target device. The second information obtained from the trained model to which the prompt was given is output to the user terminal or user device via the output unit.

[0012] The support device receives input of first information from the user terminal that explicitly or implicitly represents the user's needs, and generates prompts to give to the trained model. These prompts identify a target device among the user's devices that has the function to meet the user's needs, and obtain second information from the trained model as a response to assist the user regarding the use of that target device. The support device also outputs the second information obtained from the trained model as a response to the prompt to the user terminal or user device. Therefore, even if the user does not have detailed knowledge of the functions of their user devices, they can find out which user devices can be used to meet their needs simply by inputting first information that explicitly or implicitly represents their needs.

[0013] (2) The support device of (1), preferably the control unit, further generates a prompt using manual data of the user's equipment or access information to the manual data. The support device provides this prompt to a trained model, from which the trained model obtains second information, which includes information about the function in the target equipment that best suits the user's needs by referring to the manual. The support device outputs this second information to the user terminal or user equipment, so that the user can find out which function of which user equipment is available to meet their needs, as obtained by referring to the manual.

[0014] (3)(1)(2) The support device, preferably the control unit, further generates a prompt using at least one of information relating to the user's attributes and information relating to the status of the user's equipment. User attributes include, for example, gender, age, and language used. By providing the prompt created by the support device using the information relating to the user's attributes to a trained model, the trained model can obtain second information that is appropriate for the user's attributes in terms of expression and level of detail. By outputting this second information to the user terminal or user equipment, the user can more easily understand which functions of which user equipment are available. Information relating to the status of the user's equipment includes, for example, operating status (in use, idle, etc.), mode in use, status of replacement of consumables such as filters, and status of cleaning. By providing the prompt created by the support device using the status of the user's equipment to the trained model, the trained model can obtain second information that also takes into account the status of the user's equipment. The support device outputs this second piece of information to the user terminal or user device, allowing the user to know which functions of which user devices are available, taking into account the status of the user devices.

[0015] (4) The support device according to (1) to (3), preferably, the second information includes a control signal for executing a function that meets the user's needs in the target device or access information to the control signal, and the control unit further outputs the control signal or access information to the control signal to the user device specified as the target device. As a result, the user device can be in a state where it can execute a function that meets the user's needs.

[0016] (5) The computer program according to the embodiment is a computer program that causes a computer to function as a support device, receives an input of first information that explicitly or implicitly represents the user's needs from a user terminal via an input device of the computer, generates a prompt to be given to a learned model based on the first information, the prompt being for identifying a target device having a function that meets the user's needs from user devices and obtaining, as an answer from the learned model, second information for assisting the user regarding the use of the target device, causes the computer to output the second information obtained from the learned model to which the prompt has been given to the user terminal or the user device via an output unit of the computer.

[0017] With this computer program, the computer receives an input of first information that explicitly or implicitly represents the user's needs from the user terminal and generates a prompt to be given to the learned model. This prompt is for identifying a target device having a function that meets the user's needs from user devices and obtaining, as an answer from the learned model, second information for assisting the user regarding the use of the target device. Also, the computer outputs the second information obtained from the learned model as an answer to the prompt to the user terminal or the user device. For this reason, even if the user has no detailed knowledge about the functions of the user devices, the user can know which user device can be used to meet their needs by simply inputting the first information that explicitly or implicitly represents their needs.

[0018] (6) The support method according to the embodiment is as follows. The support device receives, via an input unit, an input of first information that explicitly or potentially represents a user's needs from a user terminal. Based on the first information, the support device generates a prompt to be given to a learned model, which identifies a target device having a function that meets the user's needs from user devices and generates a prompt for obtaining, as an answer from the learned model, second information for supporting the user regarding the use of the target device. The support device outputs the second information obtained from the learned model that was given the prompt, via an output unit, to the user terminal or the user device. This support method includes the above steps.

[0019] In this support method, the support device receives an input of first information that explicitly or potentially represents a user's needs from the user terminal and generates a prompt to be given to a learned model. This prompt is for identifying a target device having a function that meets the user's needs from user devices and obtaining, as an answer from the learned model, second information for supporting the user regarding the use of the target device. Also, the support device outputs the second information obtained from the learned model as an answer to the prompt to the user terminal or the user device. Therefore, even if the user has no detailed knowledge about the functions of the user devices, the user can know which user device can be used to meet their needs by simply inputting the first information that explicitly or potentially represents their needs.

[0020] <2. Examples of Support Devices, Computer Programs, and Support Methods> [First Embodiment]

[0021] FIG. 1 is a diagram showing an overview of the support method according to the first embodiment. FIG. 2 is a schematic configuration diagram of a server 1, a user terminal 3, and a user device 5, which are support devices used in the support method according to the first embodiment. The user terminal 3 is an example of an input device for inputting first information to the server 1, which is a support device.

[0022] (Overview of support methods) The support method according to this embodiment is a method of supporting the use of devices 5, 5A, 5B, ... (hereinafter collectively referred to as user devices 5) stored in a storage device in association with the user, by a support device. The support device according to the first embodiment is a server 1.

[0023] User device 5 can be any device owned or usable by the user. Examples of user devices include household appliances such as washing machines, air conditioners, air purifiers, electric cookers, refrigerators, lighting fixtures, cleaning robots, water heaters, and their remote controls, as well as smartphones, tablets, personal computers, televisions, audio equipment, video / playback devices, landline telephones, and facsimile machines. User devices are not limited to devices used by individuals or in homes; they may also be commercial equipment used in offices or factories, such as transport robots, photocopiers, facsimile machines, and lighting equipment. In Figure 1, user device 5 is shown as being able to communicate with server 1, but user device 5 may not have communication capabilities.

[0024] The user inputs information to the support device, Server 1, using the user terminal 3. For example, the user registers their owned or available devices with Server 1 using the user terminal 3. The user terminal 3 can be any device that can communicate with Server 1 and has a display device such as a screen. Examples of user terminals 3 include smartphones, tablets, and personal computers.

[0025] The first type of information is information that explicitly or implicitly represents the user's needs. Explicitly expressed information is information where the object (stuffed animal) and the user's request (to clean the dirt) are clearly stated, such as "I want to clean the dirt off the stuffed animal." In contrast, implicitly expressed information is information where at least one of the object and the request is not explicitly stated, such as an image of a dirty stuffed animal. The first type of information includes content generated by the user terminal 3 in response to user operations, information input into the user terminal 3 in response to user operations, and information obtained using the user terminal 3. Information obtained using the user terminal 3 includes information acquired by various sensors equipped in the user terminal 3. The data format of the first type of information is arbitrary.

[0026] Server 1 generates a prompt using the first information and inputs it to the generating AI 7. The prompt instructs the generating AI 7 to identify a device with functions that would be beneficial to the user (referred to as the "target device") from among the user devices available to the user, in order to respond to the user's situation inferred from the first information, and to generate and output information (referred to as the "second information") that will assist the user in using that target device. The prompt includes one or more sentences in natural language. The prompt may be text data or may include non-text data.

[0027] (Generation AI) The Generating AI 7 is implemented by one or more external devices separate from Server 1. The Generating AI 7 generates and outputs content in response to prompts. As an example, the Generating AI 7 is an interactive Generating AI that utilizes LLM (Large Language Models). The output from the Generating AI 7 (second information) can take any format and may be text, HTML data for display in a browser, images or audio, or control programs and control parameters for user devices. In this embodiment and the embodiments described later, examples are shown using an interactive Generating AI 7 as a trained model, but the trained models (AI) available in this disclosure are not limited to Generating AIs and may be, for example, LLMs that output text in response to prompts. Furthermore, the trained model does not have to be interactive. In this embodiment, the Generating AI 7 may be, for example, a trained model specifically for Server 1 that has been trained to support the use of user devices.

[0028] The Generative AI7 is trained to generate and output content (secondary information) related to solutions and suggestions using user devices for the user's daily or business needs. In other words, the Generative AI7 has learned from a large amount of training data how user devices and their functions are involved in solving certain events. Furthermore, during training, it may be trained to output more accurate answers based on correct data (training data) indicating whether each function of each user device is effective in solving a certain event. For example, if the user's need is to remove dirt or odor from an object, the Generative AI7 will generate a response as secondary information suggesting to the user the use of a user device with a cleaning or deodorizing function suitable for that object. Therefore, the Generative AI7 has learned at least a general relationship between the user's daily or business needs and the functions of user devices from a variety of training data. For example, as a result of its training, the Generative AI7 can derive answers such as being able to use the cleaning function of a washing machine to remove dirt from fabric products, or being able to use an air purifier or an air conditioner with an air purification function to remove odors from a room. Furthermore, the generating AI7 can refer to additional information provided by prompts in order to generate more accurate or detailed conclusions as secondary information. For example, if the generating AI7 can derive from its learning results that a washing machine is the user equipment that can be used to remove stains from fabric products, it can refer to the operating manual of a specific washing machine model and generate secondary information that includes information on the most suitable course (operating mode) for washing that fabric product on that specific washing machine by prompting the user with information on the model of the washing machine and access information to the operating manual.

[0029] Furthermore, based on its learning results, Generative AI7 can infer the characteristics and properties of the object included in the first information. For example, if the first information includes the keyword "judo uniform" or an image of a judo uniform, Generative AI7 can infer that the judo uniform has characteristics such as being heavy, made of thick fabric, and containing sweat and odor. This is because, among the various content used as training data for Generative AI7, the frequency of such characteristic words being included in contexts containing the word "judo uniform" was high, making it highly likely that Generative AI7 has learned that judo uniforms have these characteristics. As a result of this learning, Generative AI7 can generate an answer to the effect that, in order to remove dirt from a judo uniform, it is appropriate to run the washing machine on a heavy, thick fabric cycle that can also effectively remove sweat and odor.

[0030] The generation AI 7 in this embodiment may be a general-purpose generation AI that is made available to the public unconditionally or conditionally. In this case, the prompt preferably includes one or more example answers, known as one-shot learning or few-shot learning, and instructs the generation of an answer corresponding to the example answers.

[0031] Server 1 receives second information from the generating AI 7 as a response to the prompt, and transmits the obtained second information to user terminal 3 or user device 5 according to its content. For example, if the second information is something to be presented to the user, such as an explanation of how to operate user device 5 which has been identified as the target device, Server 1 presents this second information to the user via the user interface of user terminal 3 or user device 5. The presentation of information is not limited to display; it may also be presented audibly, or both audibly.

[0032] Furthermore, the prompts that instruct the generating AI 7 to generate the second information are not limited to a single input, but may be input multiple times in a dialogue format with the AI. In other words, Server 1 may cause the generating AI 7 to generate the second information by repeatedly exchanging information with it. Examples of prompts that may be input multiple times include asking whether control of the user device 5 is necessary for the first information, asking what action to perform if it is determined that control of the user device 5 is necessary, asking for items related to the first information from the table of contents information of the manual data, having the AI ​​distinguish between user needs and other information from the first information, asking for a response based on the manual data regarding the user needs obtained from the first information, and asking for a response regarding information other than user needs obtained from the first information.

[0033] (Server configuration) Server 1 is, for example, a computer having a control unit 11 and memory 12, or a single computer and its peripheral devices. Server 1 may also be realized by multiple computers working together. The control unit 11 is, for example, a CPU (Central Processing Unit). The memory 12 is, for example, ROM (Read Only Memory), RAM (Random Access Memory), etc. The memory 12 stores the program 121 executed by the control unit 11. The program 121 may be provided by recording it on a recording medium or via a communication line such as the Internet.

[0034] Server 1 further has a data storage device 14 for storing information about user devices 5 (device information). The device information for user devices 5 is information about devices owned or usable by each user, and is stored in the data storage device 14 in association with information that identifies each user (user identification information). Since the device information is used to extract information about how to use each user device 5 (e.g., operation manuals), the same device information may be assigned to a group of devices that share common handling information. In this case, the model number of the user device 5 can be used as the device information. Alternatively, unique device information may be assigned to each individual user device 5. Note that the data storage device 14 may be another device different from Server 1 that is accessible from Server 1.

[0035] Server 1 has a communication unit 13. The communication unit 13 is a communication module for communicating with other devices. Server 1 receives first information from user terminal 3 via the communication unit 13. Therefore, the communication unit 13 also serves as an input unit that receives the first information for the support device.

[0036] Server 1 inputs a prompt to the generating AI 7 via the communication unit 13 and receives second information generated by the generating AI 7 in response to the prompt. Server 1 also transmits control signals to the user terminal 3 and user equipment 5 via the communication unit 13.

[0037] Server 1 accesses Manual DB 8 via the communication unit 13. Manual DB 8 contains manual data for each device, associated with device information. Manual data is data that describes how to handle and operate each device, and may include text data or HTML data. Access information for each device's manual data (for example, the URL of the manual data) is stored in the data storage device 14, associated with the device information.

[0038] Server 1 accesses the control information DB9 via the communication unit 13. The control information DB9 stores control programs and parameters required for each operation of the equipment, associated with the equipment information. In Figure 2, the manual DB8 and the control information DB9 are shown as separate blocks, but these DBs may be physically located on the same storage device. Furthermore, if the control programs and parameters stored in the control information DB9 are related to a specific function in the manual data of the manual DB8, that association is also stored there.

[0039] (User terminal device configuration) User terminal 3 is, for example, a computer having a control unit 31 such as a CPU, memory 32 such as ROM or RAM, and a communication unit 36 ​​capable of communicating with server 1, or a single computer and its peripheral devices. Memory 32 stores a program 321 executed by the control unit 31. The program 321 may be provided by recording it on a recording medium or via a communication line such as the Internet.

[0040] The user terminal 3 has a touch panel 33, a camera 34, and a microphone 35. The touch panel 33 is an example of an input device that accepts user input, as well as an example of a display device.

[0041] (User equipment configuration) User device 5 is equipped with a control board 50 that includes, for example, a control unit 51 such as a CPU, a memory 52 such as ROM or RAM, and a communication unit 53. User device 5 communicates with server 1 via the communication unit 53.

[0042] The user device 5 includes an operating unit 54, an operation unit 55, and a notification unit 56. The operation unit 55 receives user input and inputs an operation signal to the control unit 51. The control unit 51 reads and executes a program stored in the memory 52 according to the operation signal from the operation unit 55, and controls the operation of the operating unit 54. The control unit 51 executes the program and controls the notification by the notification unit 56.

[0043] (Server functions) <<Prompt generation process>> The control unit 11 executes the prompt generation process 111 by executing the program 121. The control unit 11 uses the user identification information input from the user terminal 3 along with the first information to read the device information of user devices 5, 5A, 5B, ... owned or usable by the user from the data storage device 14. The control unit 11 also reads access information for the manual data for each device from the data storage device 14 based on the device information of user devices 5, 5A, 5B, ....

[0044] As an example, program 121 defines the prompt format. The prompt format is, for example, the string "Please select and explain how to deal with the situation X1 using one of the devices X2 from among X3." The control unit 11 generates the prompt by formatting the obtained information in a format suitable for the prompt and placing it in X1, X2, and X3 in the format. X1 contains text representing the user's situation and needs based on the first information from the user terminal 3. X2 contains a list of user devices as options for the target devices. X3 contains access information to the manuals of the user devices included in the list of X2.

[0045] The control unit 11 converts the first information into text and places it in format X1. When the first information is audio data, the control unit 11 may perform speech recognition processing. When the first information is an image representing characters, such as handwritten characters on the touch panel 33 or a copy of a document, the control unit 11 may perform optical character recognition processing. When the first information is a photograph, the control unit 11 may perform image analysis to extract the included objects and convert them into text. The control unit 11 may also obtain the textualized information by inputting the first information into an AI (an AI different from the generating AI 7) that has been trained to convert non-text content such as audio and images into text. Furthermore, if the generating AI 7 accepts non-text content as a prompt, the control unit 11 may omit the process of converting the first information into text and include the first information input from the user terminal 3 directly into the prompt.

[0046] The control unit 11 places the text representing the device information of user devices 5, 5A, 5B, ... read from the data storage device 14 into X2 of the format. The control unit 11 places the manual data for each user device 5, 5A, 5B, ... read from the manual DB8 into X3 of the format. Note that the information placed in X3 of the format may be access information such as the URL (Uniform Resource Locator) of the manual DB8 that stores the manual data, instead of the manual data itself.

[0047] Here, assume that the device information for user devices 5, 5A, and 5B is XX5, XX5A, and XX5B respectively, and the access information for the manual data is YY5, YY5A, and YY5B. If the prompt format is the string above, the control unit 11 generates a prompt that reads, "Please select and explain how to deal with the situation X1 using one of the devices XX5, XX5A, or XX5B from YY5, YY5A, or YY5B."

[0048] When a prompt is entered, for example, the generating AI7 identifies the user device (target device) corresponding to the content of the text placed in X1 (i.e., the user's needs), and then determines the functions in that target device that match the user's needs, referring to the manual data placed in X3. Specifically, for example, based on the first information placed in X1 and the user device manual information placed in X2, the corresponding manual data from the user device's manual data is selected, and the functions corresponding to the first information are determined from the selected manual data. The generating AI7 generates and outputs content representing the determined content as second information.

[0049] When the generating AI7 determines that action YY51 in manual data YY5 corresponds to the content of the text placed in X1, it generates second information using action YY51 in manual data YY5. The second information is, for example, the string "To deal with the situation X1, it is appropriate to execute YY51 in YY5 using the XX5 device."

[0050] Furthermore, when generating a prompt, the control unit 11 may add any sentence in addition to the sentence containing X1, X2, and X3 described above. For example, it may include user attribute information (gender, age, language used, etc.) in the prompt and instruct the control unit 11 to generate a response according to the appropriate expression and level of detail for the user's attributes. Such user attribute information is stored in the data storage device 14 in association with each user's user identification information.

[0051] <<Support Information Provision Processing>> The control unit 11 of server 1 executes a process (support information provision process 112) to provide the second information received from the generating AI 7 to the user terminal 3 or user device 5 by executing program 121. For example, the control unit 11 sends the second information (for example, the above string "To deal with the situation X1, it is appropriate to execute function YY51 in manual YY5 on device XX5.") to the user terminal 3 for display. At this time, the control unit 11 may convert the format of the second information so that it is displayed in accordance with the display manner of program 321 on the user terminal 3. Alternatively, if user device XX5 selected as the target device can communicate with server 1, the second information from the generating AI 7 may be sent to user device XX5 and presented to the user via the user interface of user device XX5.

[0052] (Server operation) Figure 3 is a flowchart illustrating an example of the operation of Server 1. When the control unit 11 receives input of first information from the user terminal 3 (YES in step S101), it executes the following processes. Specifically, the control unit 11 reads the device information of user devices 5, 5A, 5B, ... associated with the user of user terminal 3 from the data storage device 14 (step S103). Next, the control unit 11 reads the access information for the manual data of user devices 5, 5A, 5B, ... from the data storage device 14 (step S105).

[0053] The control unit 11 generates a prompt using the first information, the device information of user devices 5, 5A, 5B, ... and the access information of the manual data (step S107), and inputs it to the generated AI 7 (step S109).

[0054] In step S109, the generation AI7 generates second information according to the prompt entered. The control unit 11 acquires the second information output from the generation AI7 (step S111).

[0055] The control unit 11 provides the second information to the user terminal 3 or user device 5 (step S113).

[0056] (User terminal operation) Figure 4 is a flowchart illustrating an example of the operation of user terminal 3. The operation shown in Figure 4 begins when the control unit 31 of user terminal 3 launches the application (program 321). Figure 5 is a schematic diagram of the display screen of user terminal 3.

[0057] First, the control unit 31 displays the home screen on the touch panel 33 (step S200). The screen 331 in Figure 5 is a schematic diagram of the home screen. The screen 331 includes a button 311 that instructs the user to switch to a mode for inputting first information to the server 1. The screen 331 may also include a button 312 that instructs the user to switch to a mode for registering user devices 5, a button 313 that displays a list of registered user devices 5 for selection, a button 314 that displays information distributed from the server 1 in chronological order, and a button 315 that displays the service history used over a predetermined period.

[0058] When inputting the first information to server 1, the user operates button 311 on screen 331. This causes screen 331 to transition to screen 332. Screen 332, for example, includes an input field 320 for entering information and a button 324 for sending the entered information to server 1 as the first information. Screen 332 further includes a button 322 for instructing the input of voice C11 and a button 323 for instructing the input of a captured image C12.

[0059] When the user operates button 324 on screen 332, the control unit 31 of the user terminal 3 receives an instruction to send the first information (YES in step S201). Upon receiving the instruction to send the first information, the control unit 31 sends the information entered by the user operation, along with the user identification information stored in the user terminal 3, to the server 1 as the first information (step S203).

[0060] The first information is not limited to information generated in accordance with user operations. For example, the first information can be various pieces of information obtained on the user terminal 3, such as the user's web browsing history, registered information in the schedule app, content of sent and received emails, call history, call content, audio input from the microphone 35, photo data stored in association with the user, and the user's age, gender, family structure, presence and type of pets, etc., which are registered as user information. In this case, preferably, the control unit 31 presents candidate information for the first information on the screen 332 and sends it to the server 1 as the first information when the send button 324 is pressed.

[0061] Subsequently, when the control unit 31 receives the second information from the server 1 (step S205), it displays the received second information on the touch panel 33 as a response to the user (step S207).

[0062] In the first embodiment, even if a user is unaware of how to use or even the existence of the functions of user device 5, they can learn which user device can be used to meet their needs simply by inputting first information that explicitly or implicitly represents their needs into user terminal 3, and the generating AI 7 will provide a response. Furthermore, by adding user device manual data or access information to it to the prompts given to the generating AI 7, the generating AI 7 can refer to the manual data to generate a response, thereby providing more accurate and detailed support regarding the use of the user device.

[0063] [Second Embodiment] In the second embodiment, the second information from the generating AI7 includes access information (e.g., a URL for downloading the control signals) for causing the target device to execute functions that meet the user's needs. These control signals are a concept that includes the control program and control parameters of the user device 5, which is the target device. The server 1 further prompts the generating AI7 to identify the programs and parameters necessary to bring the target device into a state where it can perform operations corresponding to the first information, and to generate second information that includes access information (download URL) for downloading those programs and parameters.

[0064] In this case, Server 1 may further input additional prompts, as defined by Program 121, to the Generating AI 7, along with a prompt instructing it to generate the second information. The additional prompts may be, for example, the string "Identify the programs and parameters necessary to make the device perform the operation, and provide the URL for downloading those programs and parameters." Alternatively, Server 1 may input the above additional prompts after receiving the second information from Generating AI 7.

[0065] The generating AI7 generates second information, including a download URL, in response to additional prompts. The control unit 11 of the server 1 reads the program or parameters based on the download URL contained in the second information received from the generating AI7, and sends them to the target device for configuration.

[0066] As described above, according to the second embodiment, access information to a control program or control parameters for executing a function that meets the user's needs on the target device is provided as second information from the generated AI 7 and transmitted to the user device 5, which is the target device, by the server 1 and set. This makes it possible to execute the function that meets the user's needs more reliably on the user device 5. The control signal transmitted to the target device may, in addition to or instead of putting the target device into a state where it can perform an operation corresponding to the first information, also instruct the target device to perform an operation. As a result, the operation corresponding to the first information is automatically executed on the target device.

[0067] In the above explanation, the second information is assumed to include access information for the control signal, but it is also possible to receive the control program or control parameters themselves from the generating AI7 instead of access information.

[0068] Alternatively, the AI ​​may prompt for program or parameter options, or it may not be necessary to prompt for options if the generating AI7 has already learned the control programs and parameters for each user device.

[0069] [Third Embodiment] The support device may be implemented through the cooperation of the control unit 11 of the server 1 and the control unit 21 of the user terminal 3. As an example, the user terminal 3 according to the third embodiment executes a part of the prompt generation process 111. Figure 6 is a flowchart showing an example of the operation of the user terminal 3 in the support method according to the third embodiment. The operation in Figure 6 differs from the operation in Figure 4 in that it further executes steps S202 and S203-1.

[0070] In this embodiment, when the control unit 31 receives an instruction to transmit the first information in step S201, it converts the first information into text (step S202) and transmits it to the server 1 (step S203-1). The control unit 11 of the server 1 generates a prompt using the text received from the user terminal 3.

[0071] The control unit 31 of user terminal 3 may execute all of the prompt generation process 111. Furthermore, the control unit 31 of user terminal 3 may also execute the support information provision process 112. In other words, the control unit 31 of user terminal 3 may execute all of the processes of the control unit 11 of server 1. In this case, the support device becomes user terminal 3.

[0072] [Fourth Embodiment] The user device 5 may transmit information indicating its usage status, such as operating status (in use, idle, etc.), mode being used, status of replacement of consumables such as filters, and status of cleaning, to the server 1 via the communication unit 53. This information may also be transmitted from the user terminal 3 to the server 1. The server 1 may register information indicating the usage status of each user device 5 in the data storage device 14. Furthermore, in this embodiment, by including information indicating the usage status of each user device 5 in the prompt, the generating AI 7 is made to create a response that also takes into account the usage status of each user device 5. As an example, if there are multiple user devices with similar functions, the user device with the highest frequency of use will be prioritized and identified as the target device.

[0073] In this embodiment, the server 1 generates a prompt in the prompt generation process 111, further using information indicating usage status. The prompt format in this embodiment is, for example, the string "In the event X1, select a method from among the devices X2 to deal with it from X3, refer to X4, select the method using the device with the longest usage time, and explain it."

[0074] The control unit 11 places the text indicating the usage time of user devices 5, 5A, 5B, ... read from the data storage device 14 into the X4 format. Thus, in the support method according to this embodiment, the target device is determined taking into account the user's usage status.

[0075] [Fifth Embodiment] In the prompt generation process 111, the control unit 11 may generate prompts using AI. For example, the control unit 11 may generate prompts using a technique called HyDE (Hypothetical Document Embeddings).

[0076] For example, when generating a prompt that includes one or more example answers as described above, the control unit 11 may input the first information as the first prompt to the generation AI 7 to generate an example answer (a temporary document) L. The first prompt is, for example, the string "Please create an answer to explain the situation X1." The control unit 11 may input the first prompt multiple times to the generation AI 7 to generate multiple example answers. The control unit 11 generates a prompt using the example answer L obtained from the generation AI 7. The control unit 11 adds the example answer L to the string in the above prompt format and generates a prompt using, for example, the string "Please select and explain how to deal with the situation X1 using one of the devices X2 from among X3. Example answer L." Similarly, the control unit 11 places the first information in text form in X1, the text representing the device information of user devices 5, 5A, 5B, ... in X2, and the manual data for each of user devices 5, 5A, 5B, ... in X3.

[0077] As another example, in the prompt generation process 111, the control unit 11 may use an AI that has been trained to output a prompt to be input to the generation AI 7 when content is input. In this case, the server 1 obtains a prompt to be input to the generation AI 7 by inputting the first information input from the user terminal 3 to the above AI. As yet another example, the control unit 31 of the user terminal 3 may obtain a prompt to be input to the generation AI 7 by inputting information input by user operation as the first information to the AI, and send the prompt to the server 1.

[0078] [Sixth Embodiment] Communication between Server 1 and user equipment may also be conducted via User Terminal 3. In this case, data to be transmitted to the target equipment is sent from Server 1 to User Terminal 3, and then transferred from User Terminal 3 to the target equipment (user equipment) via short-range wireless communication such as Bluetooth®. This allows user equipment that does not have a communication unit capable of communicating with Server 1 to be controlled by Server 1 if it is capable of short-range wireless communication with User Terminal 3, thereby supporting use by users with support devices.

[0079] [Seventh Embodiment] The generating AI7 may be a generating AI trained specifically for server 1. In that case, the generating AI7 may be included in server 1.

[0080] Furthermore, the generating AI7 may be trained using the device manual data. In that case, server 1 only needs to provide the generating AI7 with the device information of user devices 5, 5A, 5B, ... and does not need to provide the manual data for each device. For example, the prompt format specified by program 121 may be the string "Explain how to deal with situation X1 using one of the devices X2." This makes prompt generation easier.

[0081] [Eighth Embodiment] The generating AI 7 may be included in the user terminal 3. For example, the application (program 321) of the user terminal 3 may be a so-called generating AI application based on the generating AI. In this case, the user terminal 3 may function as a support device. That is, the user terminal 3 may generate a prompt using the first information and obtain the second information by inputting the prompt to the generating AI.

[0082] [Example 1] Figure 1 shows an overview of the support method based on the first embodiment. The first embodiment assumes a scenario in which the user writes the string "I got the stuffed animal dirty" into the input field 320 on the screen 332 of the user terminal 3. That is, the first information C1 in the first embodiment is the string "I got the stuffed animal dirty".

[0083] Various scenarios are envisioned for inputting the first information C1 to the user terminal 3. For example, such input may be made from an application with a user interface that allows input of problems or situations that can be handled by the user's device. Alternatively, strings related to problems or situations that can be handled by the user's device may be extracted from posts on social media, etc. Or, words or conversational phrases related to problems or situations that can be handled by the user's device may be extracted from conversations picked up by the microphone of the user terminal 3.

[0084] In the following embodiments, it is assumed that user devices such as a washing machine (device information XX5), an air conditioner (device information XX5A), an air purifier (device information XX5B), and a smartphone (device information XX5C) are stored in the data storage device 14 in association with user identification information, and that their respective manual data YY5, YY5A, YY5B, and YY5C are registered in the manual DB8.

[0085] Server 1 generates prompt P1, which is the string "In the situation where you have soiled a stuffed animal, please explain how to deal with the situation using the devices with device information XX5, XX5A, XX5B, and XX5C, by selecting from the manual data YY5, YY5A, YY5B, and YY5C." Server 1 inputs prompt P1 into generation AI7.

[0086] The generating AI7, referring to manual data YY5, YY5A, YY5B, and YY5C, determines the action corresponding to the first piece of information C1, "The stuffed animal got dirty," to be the string "Stuffed animal-only course" based on the device information XX5. Then, it outputs the string "To deal with the fact that the stuffed animal got dirty, run the stuffed animal-only course on the device specified in device information XX5" as the second piece of information.

[0087] Server 1 then inputs an additional prompt to Generator AI7, which is the string "Identify the program and parameters necessary to make the device perform that operation, and provide the download URL for that program and parameters." As second information from Generator AI7, Server 1 obtains the download URL for the program to run the "stuffed animal-only course" on the target device. This second information also includes the message "The stuffed animal-only course has been downloaded to device XX5." Based on the download URL obtained from Generator AI7, Server 1 reads the program from Control Information DB9.

[0088] Furthermore, Server 1 sends the program for executing the stuffed animal-specific course, which it read from Control Information DB9, to User Device 5 (Washing Machine), along with a control signal to install it on the target device (User Device 5 (Washing Machine)) which is Device Information XX5. As a result, the program for the stuffed animal-specific course is downloaded to User Device 5, and the system becomes ready to operate the stuffed animal-specific course. Server 1 also sends the message included in the second information to User Terminal 3. As a result, User Terminal 3 displays the response A1 "The stuffed animal-specific course has been downloaded to the device with Device Information XX5."

[0089] When a stuffed animal gets dirty, users may not know whether they can use their washing machine, or which function of the washing machine to use. In such cases, they may not know which manual to consult or how to find the information. In the first embodiment, the user can ask for help by typing the string "I got my stuffed animal dirty" into the input field 320 of the screen 332 of the user terminal 3, and receive assistance such as running a special cycle for stuffed animals on their washing machine.

[0090] [Example 2] Figure 7 is a diagram illustrating the overview of the support method based on the second embodiment. The second embodiment assumes a scenario in which the user casually says, "I'm getting a cat starting today." The first piece of information C2 in the second embodiment is the voice saying, "I'm getting a cat starting today."

[0091] Server 1 analyzes the audio and converts it to text. It then arranges the converted text (First Information C2), the user's device information, and their respective manual data into a prompt format to generate prompt P2, which is the string "In the situation where you're getting a cat starting today, please explain how to deal with the situation using the devices with device information XX5, XX5A, XX5B, and XX5C, by selecting from the manual data YY5, YY5A, YY5B, and YY5C." Server 1 inputs prompt P2 into the generator AI7.

[0092] When the generating AI7 determines that the action corresponding to the first information C2 "I'm going to keep a cat starting today" is "pet mode" according to device information XX5A from the manual data YY5, YY5A, YY5B, and YY5C, it outputs the string "To keep a cat, activate pet mode on the device with device information XX5A when you go out" as the second information. The generating AI7 also generates the message "Pet mode on the device with device information XX5A is available" for user terminal 3 and includes it in the second information.

[0093] Server 1 then inputs an additional prompt to Generator AI7, which is the string "Identify the programs and parameters necessary to make the device perform that action, and provide the download URL for those programs and parameters," to obtain a download URL for the parameters necessary to execute "pet mode" on the target device from Generator AI7. Based on the download URL obtained from Generator AI7, Server 1 reads the parameters from Control Information DB9.

[0094] Server 1 extracts the message "Pet mode is available for device with device information XX5A" from the second information received from generating AI 7 and sends it to user terminal 3. As a result, user terminal 3 displays the response A2 "Pet mode is available for device with device information XX5A".

[0095] Furthermore, Server 1 generates a control signal to set pet mode when the target device (User Device 5A (Air Conditioner)), which is device information XX5A, is set to go away, and sends it to User Device 5A along with the parameters read from Control Information DB9. As a result, when User Device 5A is set to go away, the parameters are set so that it automatically enters pet mode, creating an environment suitable for keeping a cat.

[0096] Users who are not familiar with the functions of their device may not realize when it would be beneficial to use each function. In such cases, users will not bother to look up how to use their device or read the manual. In contrast, in the second embodiment, a casual voice message from the user, such as "I'm getting a cat starting today," can provide assistance such as suggesting that the air conditioner be set to pet mode when going out.

[0097] [Example 3] Figure 8 is a diagram illustrating the overview of the support method based on the third embodiment. The third embodiment assumes a scenario in which the user casually says, "We're going to have yakiniku (Japanese barbecue) at 6pm today." The first piece of information C3 in the third embodiment is the voice saying, "We're going to have yakiniku at 6pm today."

[0098] Server 1 analyzes the audio and converts it to text. It then arranges the converted first information C3, the user's equipment information, and their respective manual data into a prompt format to generate prompt P3, which is the string "In the situation where we're having a barbecue at 6 PM today, please explain how to deal with the situation using equipment with equipment information XX5, XX5A, XX5B, and XX5C, by selecting from manual data YY5, YY5A, YY5B, and YY5C." Server 1 inputs prompt P3 into generation AI7.

[0099] When the generating AI7 determines that the action corresponding to the first information C3 "We're going to have a barbecue at 6 PM today" is the "deodorizing mode" for equipment information XX5A and equipment information XX5B, respectively, from the manual data YY5, YY5A, YY5B, and YY5C, it outputs the string "To have a barbecue at 6 PM today, the equipment with equipment information XX5A and the equipment with equipment information XX5B will run deodorizing mode from 6 PM today" as the second information. The generating AI7 also generates the string "Let's start deodorizing mode at 6 PM" as a message for user terminal 3 and includes it in the second information.

[0100] Server 1 extracts the message "Deodorizing mode from 6 PM" from the second piece of information received from Generating AI 7 and sends it to User Terminal 3. As a result, User Terminal 3 displays the response A3 "Let's start deodorizing mode from 6 PM."

[0101] Furthermore, Server 1 generates a control signal to set the target device with device information XX5A (User device 5A (air conditioner)) and the target device with device information XX5B (User device 5B (air purifier)) to deodorizing mode at 6 PM today, and sends it to User device 5A and User device 5B along with the parameters read from Control Information DB9. As a result, the parameters are set so that User device 5A and User device 5B automatically enter deodorizing mode at 6 PM today.

[0102] Users who are not familiar with the functions of their equipment may not think to use it at the appropriate time. As a result, the appropriate mode of operation may not be activated until the necessary conditions are reached, such as when an odor is detected by a sensor installed in the air conditioner or air purifier. In this case, by the time the air conditioner or air purifier switches to deodorizing mode, the odor has already filled the room. In contrast, in the third embodiment, the user's casual voice statement, "We're having a barbecue at 6 PM today," provides support suggesting that the air conditioner and air purifier should be switched to deodorizing mode at 6 PM. Therefore, by using the support device according to this embodiment, appropriate use of the user's equipment is supported before the barbecue begins and the odor fills the room, that is, before it is detected by the sensor.

[0103] [Example 4] Figure 9 is a diagram illustrating the overview of the support method based on the fourth embodiment. The fourth embodiment assumes a scenario in which the user writes the purchased items in a free format in the input field 320 on the screen 332 of the user terminal 3 after shopping. The first piece of information C4 in the fourth embodiment is the string "I bought carrots, pork, and radishes."

[0104] Server 1 generates prompt P4 by arranging the string C4 (the first piece of information), the device information of the user's device, and the respective manual data in the prompt format, resulting in the string "In the situation where you have bought carrots, pork, and radishes, please explain how to deal with the situation using the devices with device information XX5, XX5A, XX5B, and XX5C, by selecting from the manual data YY5, YY5A, YY5B, and YY5C." Server 1 inputs prompt P4 into generation AI7.

[0105] When the generating AI7 determines from the manual data YY5, YY5A, YY5B, and YY5C that the action corresponding to the first piece of information C4, "I bought carrots, pork, and radishes," is to register them in the "Ingredients Memo" application using the device information XX5C, it outputs the string "To deal with the fact that I bought carrots, pork, and radishes, I will register them in the Ingredients Memo of the device with device information XX5C" as the second piece of information. The generating AI7 also generates the string "Registered in Ingredients Memo" as a message for user terminal 3 and includes it in the second piece of information.

[0106] Server 1 extracts the message "Added to ingredient memo" from the second piece of information generated by AI7 and sends it to user terminal 3. As a result, user terminal 3 displays the response A4 "Added to ingredient memo".

[0107] Furthermore, Server 1 generates a control signal to register carrots, pork, and radishes in the ingredient memo of the target device (User Device 5C (smartphone)), which is identified as Device Information XX5C, and sends it to User Device 5C. As a result, carrots, pork, and radishes are registered in the ingredient memo of User Device 5C.

[0108] Users unfamiliar with smartphone functions may not think to register groceries in their shopping list after shopping. In contrast, in the fourth embodiment, the user can register groceries in the shopping list of the user terminal 3 by writing the string "bought carrots, pork, and radish" in the input field 320 of the screen 332 of the user terminal 3, just like a memo. Therefore, by using the support device according to this embodiment, appropriate use of the user device is supported even if the user does not understand the existence or content of the function.

[0109] [Example 5] Figure 10 is a diagram illustrating the overview of the support method based on the fifth embodiment. The fifth embodiment assumes a scenario in which the user tells the user terminal 3 by voice, "I'm going home at 6 PM today." The first piece of information C5 in the fifth embodiment is the voice message, "I'm going home at 6 PM today."

[0110] Server 1 analyzes the audio and converts it to text. It then arranges the converted first information C5, the user's equipment information, and their respective manual data into a prompt format to generate prompt P5, which is the string "In the situation where you have to go home at 6 PM today, please explain how to deal with the situation using equipment information XX5, XX5A, XX5B, and XX5C, by selecting from manual data YY5, YY5A, YY5B, and YY5C." Server 1 inputs prompt P5 into generation AI7.

[0111] When the generating AI7 determines that the action corresponding to the first information C5 "I will be home at 6 PM today" is to change the laundry reservation time for machine information XX5A from the manual data YY5, YY5A, YY5B, and YY5C, it outputs the string "I will be home at 6 PM today, so I will change the laundry completion time reservation for machine information XX5A" as the second information. In addition, the generating AI7 generates the string "The laundry completion time reservation has been changed" as a message for user terminal 3 and includes it in the second information.

[0112] Server 1 extracts the message "The laundry completion time reservation has been changed" from the second information received from Generating AI 7 and sends it to User Terminal 3. As a result, User Terminal 3 displays the response A5 "The laundry completion time reservation has been changed".

[0113] Furthermore, Server 1 generates a control signal to change the washing completion time of the target device (User Device 5 (Washing Machine)), which is device information XX5A, to 18:00 today, and sends it to User Device 5. As a result, the parameters are changed so that the washing completion time is 18:00 today.

[0114] Users who are not familiar with the functions of their user equipment may find it difficult or cumbersome to remotely change their reservations. In contrast, in the fifth embodiment, users can receive assistance in changing the necessary settings on their user equipment simply by informing the user terminal 3 of their schedule changes. Therefore, by using the support device according to this embodiment, appropriate use of user equipment is supported even for users who are not familiar with the functions of their user equipment.

[0115] <3. Addendum> The present invention is not limited to the above embodiments, and various modifications are possible. In the above explanation, an example was given where user device 5 and user terminal 3 are different devices, but user device 5 may also function as user terminal 3.

[0116] Furthermore, in the above explanation, the target device was assumed to be selected from devices owned or available to the user. However, the target device may be selected from devices other than those owned or available to the user. This allows for the suggestion of new device functions and uses to the user, and can generate interest in new devices. [Explanation of Symbols]

[0117] 1: Server, 3: User terminal, 5, 5A, 5B, 5C: User equipment, 7: Generation AI, 8: Manual DB, 14: Data storage device, 100: User, 111: Prompt generation process, 112: Support information provision process, 121, 321: Program

Claims

1. A support device comprising an input unit, a control unit, and an output unit, The control unit, The input unit receives input from the user terminal of first information that explicitly or implicitly represents the user's needs. Based on the first information, a prompt is generated to give to the trained model, which identifies a target device from among the user's devices that has a function to meet the user's needs, and obtains a second information from the trained model as a response to support the user regarding the use of the target device. The second information obtained from the trained model to which the aforementioned prompt was given is output to the user terminal or user device via the output unit. Support equipment.

2. The control unit further generates the prompt using the manual data of the user device or access information to the manual data. The support device according to claim 1.

3. The control unit further uses at least one of the user attribute information and the user device status information to create the prompt. The support device according to claim 1.

4. The second information includes control signals for executing functions that meet the user's needs in the target device, or access information to the control signals. The control unit further outputs the control signal or access information to the control signal to the user device identified as the target device. The support device according to claim 1.

5. A computer program that makes a computer function as an assistive device, The computer's input device receives input from a user terminal of first information that explicitly or implicitly represents the user's needs. Based on the first information, a prompt is generated to give to the trained model, which identifies a target device from among the user's devices that has a function to meet the user's needs, and obtains a second information from the trained model as a response to support the user regarding the use of the target device. The computer is instructed to output the second information obtained from the trained model that provided the prompt to the user terminal or user device via the computer's output unit. Computer program.

6. The support device receives input from the user terminal via the input unit, which is first information that explicitly or implicitly represents the user's needs. Based on the first information, the support device generates a prompt to give to the trained model, which identifies a target device from among the user's devices that has a function to meet the user's needs, and generates a prompt to obtain a second information from the trained model as a response to support the user regarding the use of the target device. The support device includes outputting the second information obtained from the trained model that provided the prompt to the user terminal or user device via an output unit. How to help.

Images