Task processing device, task processing method, task processing program, task processing system, information processing device, and program

Abstract

The present disclosure provides a task processing device that receives, from a user, task instruction information providing an instruction for a task and causes an information processing device to execute the task corresponding to the task instruction information.　The task processing device is connected to the Internet and communicably connected to an information processing device owned by a user and comprises: a reception unit that receives, from the user, task instruction information providing an instruction for a task; a generation unit that generates, on the basis of the task instruction information, XML data defining a plurality of operations for the information processing device, which are performed on an operation interface image for receiving an operation by the user; and an execution unit that outputs the XML data to the information processing device and causes the information processing device to execute the plurality of operations on the operation interface image, thereby executing the task corresponding to the task instruction information.

Description

Task processing device, task processing method, task processing program, task processing system, information processing device, and program

[0001] The present disclosure relates to a task processing device, a task processing method, a task processing program, a task processing system, an information processing device, and a program.

[0002] Patent Document 1 discloses a technique for displaying a response message on a server device using a generative AI in response to a task instruction for searching for required human resources in an information processing system in the field of human resource recruitment.

[0003] Japanese Patent Publication No. 7474376

[0004] The present disclosure provides a task processing device, a task processing method, a task processing program, a task processing system, an information processing device, and a program that receive task instruction information for instructing a task from a user and cause an information processing device to execute a task corresponding to the task instruction information.

[0005] This specification includes all the contents of Japanese Patent Application No. 2024-231875 filed on December 27, 2024. One aspect of the present disclosure is a task processing device connected to the Internet and communicably connected to an information processing device owned by a user, the task processing device including a reception unit that receives task instruction information for instructing a task from the user, a generation unit that generates XML data defining a plurality of operations on the information processing device in an operation interface image for receiving a user operation based on the task instruction information, and an execution unit that outputs the XML data to the information processing device and causes the information processing device to execute the plurality of operations in the operation interface image, thereby executing a task corresponding to the task instruction information.

[0006] According to the task processing device, task processing method, task processing program, task processing system, information processing device, and program of the present disclosure, it is possible to receive task instruction information for instructing a task from a user and cause an information processing device to execute a task corresponding to the task instruction information.

[0007] Figure 1 shows an example of the configuration of the task processing system according to the embodiment. Figure 2 is a block diagram showing an example of the configuration of each device in the task processing system. Figure 3 shows an example of the configuration of the task processing device and the smartphone. Figure 4 is a sequence diagram showing an example of the processing of the task processing device. Figure 5 is a flowchart showing an example of the processing of the task processing device.

[0008] (Knowledge and other information forming the basis of this disclosure) At the time the inventors conceived of this disclosure, as shown in Patent Document 1, there was a technology that used generating AI to display a response message on a server device in response to a task instruction to search for personnel. However, with the above technology, it was difficult to receive task instruction information from a user and have the information processing device execute the task corresponding to the task instruction information. For example, it was difficult to have a smartphone perform multiple operations on an operation interface image displayed on the smartphone. The inventors discovered this problem and came to constitute the subject matter of this disclosure in order to solve it. Therefore, this disclosure provides a task processing device, a task processing method, a task processing program, a task processing system, an information processing device, and a program that can receive task instruction information from a user and have the information processing device execute the task corresponding to the task instruction information.

[0009] The embodiments will be described in detail below with reference to the drawings. However, some descriptions may be omitted to avoid unnecessary detail. For example, detailed explanations of already well-known matters or redundant explanations of substantially identical configurations may be omitted. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter described in the claims.

[0010] (Embodiment 1) [1-1. Configuration of the Task Processing System] Figure 1 is a diagram showing an example of the configuration of the task processing system 1. The task processing system 1 receives task instruction information SD from the user to instruct a task TS, and causes the smartphone 6 to execute the task TS corresponding to the task instruction information SD. The space in which the sensor group 4 constituting the task processing system 1 is installed is the target space for sensing by the task processing system 1. The shape and use of the target space are not limited. The target space is, for example, a closed space surrounded by walls and a ceiling. Also, for example, the target space is a living space, a living space, an office or other work space. The smartphone 6 will be further explained with reference to Figure 3.

[0011] Task processing system 1 detects voice from the user in the indoor space 9. Task processing system 1 also detects the user's location in the indoor space 9.

[0012] The interior space 9 illustrated in Figure 1 is a space enclosed by a floor 91, a ceiling 92, and four walls 93, 94, 95, and 96. The interior space 9 is provided with a door, and when the door is closed, the interior space 9 becomes a closed space. Figure 1 shows the X, Y, and Z axes. The X, Y, and Z axes are virtual axes that are orthogonal to each other, with the Z axis corresponding to the height direction of the interior space 9. The X axis corresponds to the width direction of the interior space 9, i.e., the left-right direction. The Y axis corresponds to the depth direction of the interior space 9, i.e., the front-back direction.

[0013] The sensor group 4, which constitutes the task processing system 1, is arranged in the indoor space 9. The sensors that make up the sensor group 4 can be installed on the floor 91, ceiling 92, and the four walls 93, 94, 95, 96, etc., of the indoor space 9. In this embodiment, each of the sensors that make up the sensor group 4 is incorporated into the "electrical equipment" installed in the indoor space 9.

[0014] "Electrical equipment" includes components, equipment, machinery, and appliances that are connected to the commercial power supply and permanently installed in the indoor space 9. More specifically, "electrical equipment" includes switches, outlets, and other power wiring components. It also includes ventilation fans, lighting fixtures, intercoms, and other power-consuming equipment. Furthermore, "electrical equipment" includes distribution boards, building energy management devices, and other power control equipment.

[0015] For example, the indoor space 9 shown in Figure 1 includes a lighting device 10, a switch 20, and outlets 30A and 30B as "electrical equipment." The "electrical equipment" may also include a distribution board 50 installed inside or outside the indoor space 9.

[0016] Power lines 5A, 5B, and 5C are drawn into the indoor space 9 from the distribution board 50. Power lines 5A, 5B, and 5C are, for example, wires laid concealed within the walls of the indoor space 9, and are power cables such as VVF cables and CV cables. In the following description, when power lines 5A, 5B, and 5C are not distinguished from each other, they may be referred to as power line 5.

[0017] The lighting device 10 is mounted on the ceiling 92. The lighting device 10 may be directly fixed to the ceiling 92, or it may be mounted on a lighting duct rail or the like fixed to the ceiling 92. In this embodiment, a ceiling socket 112 is fixed to the ceiling 92, and the lighting device 10 is mounted on the ceiling socket 112. The ceiling socket 112 may be of a form called a lighting rosette. The ceiling socket 112 corresponds to an example of a fixing member. The ceiling socket 112 is connected to a power line 5A, and the lighting device 10 is connected to the power line 5A via the ceiling socket 112.

[0018] The lighting device 10 comprises a light source 11, which will be described later with reference to Figure 2, and a cover 110 that diffuses the light emitted by the light source 11. The light source 11 is provided in the circuit body 111 together with a driver 12, etc., which will be described later with reference to Figure 2, and is covered by the cover 110.

[0019] The switch 20 has a switch body 21 that is connected to the power line 5A. The switch 20 connects and disconnects the power line 5A between the lighting device 10 and the distribution board 50. When the switch 20 is ON, power is supplied to the lighting device 10. When the switch 20 is OFF, the power supply to the lighting device 10 is cut off, and the lighting device 10 turns off.

[0020] Outlets 30A and 30B are power outlets (also called outlets) to which load equipment that operates using power supplied from the commercial power supply can be connected. For example, outlets 30A and 30B are single-phase 100V power outlets, with outlet 30A connected to the distribution board 50 by power line 5B, and outlet 30B connected to the distribution board 50 by power line 5C.

[0021] The distribution board 50 branches off the service drop line 58, which is connected to the commercial power system, and connects it to the power line 5. The distribution board 50 is fixed to a wall inside or outside the indoor space 9.

[0022] Each piece of electrical equipment installed in the indoor space 9 is equipped with a sensor. The lighting device 10 is equipped with a sensor 14. The sensor 14 is positioned so as to be exposed to the indoor space 9 from the cover 110. The sensor 14 is mounted on the lighting device 10 facing downwards so that the area below the lighting device 10 is the sensing area.

[0023] The sensor 14 is attached to the circuit body 111 and connected to the power line 5A via the circuit body 111. Therefore, by attaching the lighting device 10 to the ceiling 92, power is supplied to the sensor 14 from the power line 5A.

[0024] The switch 20 is equipped with a sensor 24. The sensor 24 is positioned on the switch 20 so as to face the room space 9 from the wall 94.

[0025] Outlet 30A is equipped with sensor 34A, and outlet 30B is equipped with sensor 34B. Sensor 34A is positioned to face the interior space 9 from wall 95, and sensor 34B is positioned to face the interior space 9 from wall 94.

[0026] Sensors 14, 24, 34A, and 34B may be of the same type or of different types. Sensors 14, 24, 34A, and 34B may be, for example, a position detection sensor, an image output sensor, and a state detection sensor.

[0027] The position detection sensor outputs detection results regarding the position of a human body P in the indoor space 9. Specifically, the position detection sensor may be a PIR (Passive Infrared Ray) sensor, a distance measuring sensor, or a radar sensor. The distance measuring sensor may be an ultrasonic distance measuring sensor or a LiDAR (Light Detection and Ranging) sensor. Since sensors 14, 24A, 24B, 34A, and 34B are fixed in the indoor space 9, if sensors 14, 24A, 24B, 34A, and 34B include a position detection sensor, information regarding the position of the human body P can be obtained based on the position of the position detection sensor and its detection result. The human body P corresponds to an example of a user.

[0028] The image output sensor is a sensor that outputs an image as a detection result in the indoor space 9. The image output sensor performs two-dimensional or three-dimensional detection on the indoor space 9 and outputs an image as a detection result. Specifically, examples of image output sensors include infrared array sensors, image sensors that capture images using light outside the visible range including infrared light, and digital cameras that capture images using visible light. The infrared array sensor comprises multiple infrared sensors arranged in a grid, and outputs a thermal image of the area to be detected based on the detection values ​​of these multiple infrared sensors. Furthermore, by generating a depth image that includes LiDAR measurements, LiDAR can also be used as an image output sensor.

[0029] If cameras are used as sensors 14, 24, 34A, and 34B, the user corresponding to the human body P in the indoor space 9 may become aware of the camera, potentially causing psychological distress. Therefore, if the indoor space 9 is a private space, or if the comfort of the user corresponding to the human body P in the indoor space 9 is important, it is appropriate not to perform visible light photography or high-resolution photography. For example, it is preferable to use an infrared array sensor as sensor 14.

[0030] It is not ruled out to use cameras as sensors 14, 24, 34A, and 34B. For example, if the task processing system 1 has a function to edit the camera's captured images so that individuals cannot be identified, it is possible to avoid psychological impact on the user corresponding to the human body P.

[0031] A state detection sensor is a sensor that detects the state of the indoor space 9. The state detection sensor outputs a detected value that indicates the state in the indoor space 9. Examples of state detection sensors include microphones, vibration sensors, temperature sensors, humidity sensors, pressure sensors that detect atmospheric pressure, and Doppler sensors.

[0032] Each of the sensors 14, 24, 34A, and 34B is selected from the sensors described above. For example, the type of sensor 14, 24, 34A, and 34B is selected according to the position of the sensors 14, 24, 34A, and 34B in the indoor space 9.

[0033] Sensors 14, 24, 34A, and 34B each have different installation heights in the indoor space 9. Height H1 indicates the height of the installation position of sensor 14, height H2 indicates the height of sensor 24, height H3 indicates the height of sensor 34A, and height H4 indicates the height of sensor 34B. Heights H1 to H4 are all relative to the floor 91.

[0034] Since sensor 14 is installed on the ceiling 92, its height H1 is the height of the entire room space 9. The height H2 of sensor 24, and the heights H3 and H4 of sensors 34A and 34B are lower than height H1. Heights H3 and H4 may be the same or different. Sensors 34A and 34B are installed close to the floor 91, and their heights H3 and H4 are lower than height H2.

[0035] Since the sensor 14 is located at the highest position and detects from the ceiling 92 downwards, it can detect a wide area, and detection is less likely to be obstructed by furniture and fixtures installed in the room 9. Therefore, by using an image output sensor or a position detection sensor as the sensor 14, a lot of information can be obtained about a wide area of ​​the room 9.

[0036] When using an image output sensor or a position detection sensor as sensor 14, it is preferable to use an image output sensor or a position detection sensor as at least one of sensors 24, 34A, and 34B. Sensors 24, 34A, and 34B can detect the indoor space 9 from the walls 94 and 95. For this reason, for example, if at least one of sensors 24, 34A, and 34B and sensor 14 are composed of the same type of sensor and are image output sensors, a three-dimensional detection result of the indoor space 9 can be obtained.

[0037] In Embodiment 1, for example, sensor 14 has a position detection sensor. Also, for example, sensors 24, 34A, and 34B each have a microphone. Sensor 14 detects the user's position in the indoor space 9. Also, sensors 24, 34A, and 34B each detect voice from the user in the indoor space 9. Based on the user's position in the indoor space 9 detected by sensor 14, the task processing system 1 determines which of sensors 24, 34A, and 34B will detect the voice from the user. For example, the task processing system 1 causes the sensor closest to the user among sensors 24, 34A, and 34B to detect the voice from the user.

[0038] [1-2. Configuration of Each Device] Figure 2 is a block diagram showing an example of the configuration of each device in the task processing system 1, and shows the configuration of the lighting device 10, switch 20, outlets 30A and 30B, and distribution board 50. Figure 2 is a schematic diagram showing the configuration of each device and does not limit the detailed circuit configuration. For example, each device in Figure 2 may have circuits not shown in Figure 2.

[0039] The task processing system 1 includes a task processing device 40 that processes the detection results of sensors 14, 24, 34A, and 34B. The lighting device 10 transmits the detection result of sensor 14 to the task processing device 40. Similarly, the switch 20 transmits the detection result of sensor 24, the outlet 30A transmits the detection result of sensor 34A, and the outlet 30B transmits the detection result of sensor 34B to the task processing device 40.

[0040] The task processing unit 40 can be installed in any of the following devices: the lighting device 10, the switch 20, the outlets 30A and 30B, and the distribution board 50. Alternatively, the task processing unit 40 may be implemented in a device other than these. In this embodiment, the configuration in which the lighting device 10 is equipped with the task processing unit 40 will be described as an example.

[0041] A service drop 58 is connected to the distribution board 50. The service drop 58 is a power line brought in from outside the building, including the interior space 9, or is the main wiring of the building, including the interior space 9, and supplies commercial AC power to the distribution board 50.

[0042] As shown in Figure 2, the distribution board 50 includes a main breaker 51, branch wiring 52, and circuit breakers 53A, 53B, and 53C. The main breaker 51 is connected to the service drop 58. The branch wiring 52 connects the main breaker 51 to each of the circuit breakers 53A, 53B, and 53C. Embodiment 1 shows an example in which a single-phase three-wire 200V AC power supply is supplied to the distribution board 50 from the service drop 58. In this example, the branch wiring 52 includes branch wiring 52R, 52N, and 52T corresponding to the R phase, N phase, and T phase, respectively. The branch wiring 52R, 52N, and 52T are made of stranded wire, single-core wire, or metal components such as busbars. The distribution board 50 may also be provided with a grounding terminal (not shown).

[0043] The main breaker 51 interrupts the incoming line 58 and the branch wiring 52 when the current flowing from the incoming line 58 to the branch wiring 52 exceeds a specified capacity. In addition to the main breaker 51, the distribution board 50 may include a leakage breaker and a neutral-line open-phase protection circuit (not shown).

[0044] The breaker 53A is connected to the branch wiring 52R and the branch wiring 52N and supplies a single-phase 100V alternating current to the power line 5A. The breaker 53A interrupts the power line 5A from the branch wiring 52 when the current flowing through the power line 5A exceeds a predetermined capacity. The breaker 53B is connected to the branch wiring 52N and the branch wiring 52T and supplies a single-phase 100V alternating current to the power line 5B. The breaker 53C is connected to the branch wiring 52N and the branch wiring 52T and supplies a single-phase 100V alternating current to the power line 5C. The breaker 53B interrupts the power line 5B from the branch wiring 52 when the current flowing through the power line 5B exceeds a predetermined capacity. The breaker 53C interrupts the power line 5C from the branch wiring 52 when the current flowing through the power line 5C exceeds a predetermined capacity.

[0045] The branch wiring 52 can be referred to as secondary-side wiring with respect to the main breaker 51. That is, the main breaker 51 functions as a breaker that disconnects the commercial power supply and the secondary-side wiring. Also, the power lines 5 can be referred to as secondary-side wiring with respect to the breakers 53A, 53B, and 53C, and the breakers 53A, 53B, and 53C function as breakers that disconnect the commercial power supply and the secondary-side wiring.

[0046] In the task processing system 1, the lighting device 10 and the switch 20 are connected to the power line 5A, the outlet 30A is connected to the power line 5B, and the outlet 30B is connected to the power line 5C. The power lines 5A, 5B, and 5C are all connected to the branch wiring 52 of the distribution board 50. That is, the lighting device 10, the switch 20, the outlets 30A and 30B are electrically connected to each other via the branch wiring 52 and the power lines 5.

[0047] In the task processing system 1, either wired communication or wireless communication can be adopted as the communication method when communicating between devices such as the lighting device 10, the switch 20, and the power outlets 30A and 30B. As the wired communication, power line communication (PLC: Power Line Communication) through the power line 5 can be executed. As the wireless communication method, Wi-Fi (registered trademark), Bluetooth (registered trademark), or other wireless communication methods can be used. In the present embodiment, an example of using power line communication by the power line 5 in the task processing system 1 will be described.

[0048] The lighting device 10 includes a light source 11 and a driver 12 that lights the light source 11. The light emitted by the light source 11 is diffused by the cover 110 shown in FIG. 1 to illuminate the indoor space 9. The light source 11 is a solid-state light source such as an LED (Light Emitting Diode), or a lamp such as an incandescent lamp or a fluorescent lamp. A driver 12 is connected to the light source 11, and power is supplied from the driver 12 to the light source 11.

[0049] The driver 12 is connected to the power line 5A and receives the supply of commercial alternating current through the power line 5A. The driver 12 includes a circuit that supplies the power required for the light source 11 to light. For example, when the light source 11 is an LED, the driver 12 includes an inverter circuit and outputs a direct current with a predetermined voltage to the light source 11 by converting the commercial alternating current. The driver 12 may have a function of adjusting the light amount of the light source 11. For example, the driver 12 may be configured to be able to adjust the light amount of the light source 11 by PWM (Pulse Width Modulation) control.

[0050] The sensor 14 is incorporated into the circuit main body 111 shown in FIG. 1 of the lighting device 10 as a sensor unit 13 together with the control unit 15, the communication unit 16, and the power supply unit 17. The power supply unit 17 is connected to the power line 5A through the ceiling socket 112, converts the alternating current supplied from the power line 5A into a direct current with a predetermined voltage, and outputs it to the sensor 14, the control unit 15, and the communication unit 16.

[0051] The control unit 15 includes a processor that processes data by executing a program. The processor is composed of an integrated circuit such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a microcontroller. The control unit 15 may also include a storage device that non-volatilely stores programs and data using flash memory, magnetic recording media, optical recording media, etc. The control unit 15 may be composed of an integrated circuit that integrates the processor and the storage device, or it may be composed of a SoC (System on a Chip) that further integrates the sensor 14.

[0052] The sensor unit 13 is equipped with a task processing unit 40. The task processing unit 40 comprises a first processor 41 and a first memory 42.

[0053] The first processor 41 processes data by executing a program. The first processor 41 is composed of an integrated circuit such as a CPU, MPU, or microcontroller. The first processor 41 may also be programmed hardware. The first memory 42 includes a storage device that non-volatilely stores programs and data using flash memory, a magnetic recording medium, an optical recording medium, or the like. The first memory 42 may be composed of an integrated circuit integrated with the first processor 41.

[0054] The first processor 41 acquires detection data including the detection results from sensors 14, 24, 34A, and 34B, performs various processes on the detection data, and stores the processing results in the first memory 42. The processes executed by the first processor 41 will be described later with reference to Figures 3-5.

[0055] The control unit 15 operates the sensor 14 to perform detection of the indoor space 9. The control unit 15 acquires the detection result from the sensor 14 and generates detection data including the detection result. The control unit 15 outputs the detection data to the task processing unit 40.

[0056] The communication unit 16 is a communication device that performs communication with other devices constituting the task processing system 1 in accordance with the control of the control unit 15, and includes a transmitter, receiver, interface circuit, etc.

[0057] The communication unit 16 is, for example, a communication modem connected to two cables constituting the power line 5A, and performs power line communication via the power line 5A. The communication unit 16 includes, for example, an encoding circuit for encoding data, a transmitting circuit for superimposing the encoded signal onto a carrier wave, a receiving circuit equipped with a filter for extracting the signal superimposed on the carrier wave, and a decoding circuit for decoding the extracted signal to generate received data and outputting the received data to the control unit 15. Alternatively, the communication unit 16 may be a wireless communication device that performs wireless communication. The communication unit 16 receives detection data transmitted by the switch 20, outlets 30A and 30B. The detection data received by the communication unit 16 is output to the task processing device 40.

[0058] The switch 20 comprises a switch body 21 and a sensor unit 23. Figure 2 shows a single-pole switch as the switch body 21, which disconnects only one of the pair of cables constituting the power line 5A, but the switch body 21 may also be a double-pole switch.

[0059] The sensor unit 23 comprises a sensor 24, a control unit 25, a communication unit 26, and a power supply unit 27. The power supply unit 27 is connected to the power line 5A at a position closer to the task processing unit 40 than the switch body 21. The power supply unit 27 converts the alternating current supplied from the power line 5A into a direct current of a predetermined voltage and outputs it to the sensor 24, the control unit 25, and the communication unit 26.

[0060] The control unit 25 includes a processor that processes data by executing a program. The processor is composed of an integrated circuit such as a CPU, MPU, or microcontroller. The control unit 25 may also include a storage device that non-volatilely stores programs and data using flash memory, magnetic recording media, optical recording media, etc. The control unit 25 may be composed of an integrated circuit that integrates the processor and the storage device, or it may be composed of an SoC that further integrates the sensor 24.

[0061] The control unit 25 operates the sensor 24 to perform detection of the indoor space 9. The control unit 25 acquires the detection result from the sensor 24 and generates detection data including the detection result. The control unit 25 transmits the detection data to the task processing unit 40 via the communication unit 26.

[0062] The communication unit 26 communicates with the task processing unit 40 according to the control of the control unit 25. For example, the communication unit 26 transmits detection data generated by the control unit 25 to the task processing unit 40. The communication unit 26 is a communication modem connected to two cables that make up the power line 5A and performs power line communication via the power line 5A. The communication unit 26 includes, for example, an encoding circuit for encoding data, a transmitting circuit for superimposing the encoded signal onto a carrier wave, a receiving circuit equipped with a filter for extracting the signal superimposed on the carrier wave, and a decoding circuit for decoding the extracted signal to generate received data and outputting the received data to the control unit 25.

[0063] The outlet 30A comprises an outlet body 31A and a sensor unit 33A. The outlet body 31A has a pair of terminals 32A that are connected to the power line 5B. Load equipment is connected to terminals 32A. Terminals 32A correspond to an example of power terminals. Load equipment is, for example, an electrical appliance used in the indoor space 9.

[0064] The sensor unit 33A comprises a sensor 34A, a control unit 35A, a communication unit 36A, and a power supply unit 37A. The power supply unit 37A is connected to the power line 5B and converts the alternating current supplied from the power line 5B into a direct current of a predetermined voltage, which is then output to the sensor 34A, the control unit 35A, and the communication unit 36A.

[0065] The control unit 35A includes a processor that processes data by executing a program. The processor is composed of an integrated circuit such as a CPU, MPU, or microcontroller. The control unit 35A may also include a storage device that non-volatilely stores programs and data using flash memory, magnetic recording media, optical recording media, etc. The control unit 35A may be composed of an integrated circuit that integrates the processor and the storage device, or it may be composed of an SoC that further integrates the sensor 34A.

[0066] The control unit 35A operates the sensor 34A to perform detection of the indoor space 9. The control unit 35A acquires the detection result from the sensor 34A and generates detection data including the detection result. The control unit 35A transmits the detection data to the task processing unit 40 via the communication unit 36A.

[0067] The communication unit 36A communicates with the task processing unit 40 in accordance with the control of the control unit 35A. For example, the communication unit 36A transmits detection data generated by the control unit 35A to the task processing unit 40. The communication unit 36A is a communication modem connected to two cables that constitute the power line 5B and performs power line communication via the power line 5B. The communication unit 36A includes, for example, an encoding circuit for encoding data, a transmitting circuit for superimposing the encoded signal onto a carrier wave, a receiving circuit equipped with a filter for extracting the signal superimposed on the carrier wave, and a decoding circuit for decoding the extracted signal to generate received data and outputting the received data to the control unit 35A.

[0068] Outlet 30B can have the same configuration as outlet 30A. That is, outlet 30B comprises an outlet body 31B and a sensor unit 33B, and the outlet body 31B has a pair of terminals 32B connected to the power line 5B. Terminals 32B correspond to an example of power terminals. Load equipment is connected to terminals 32B.

[0069] The sensor unit 33B comprises a sensor 34B, a control unit 35B, a communication unit 36B, and a power supply unit 37B. The power supply unit 37B is connected to the power line 5C and converts the alternating current supplied from the power line 5C into a direct current of a predetermined voltage, which it outputs to the sensor 34B, the control unit 35B, and the communication unit 36B.

[0070] The control unit 35B includes a processor that processes data by executing a program. The processor is composed of an integrated circuit such as a CPU, MPU, or microcontroller. The control unit 35B may also include a storage device that non-volatilely stores programs and data using flash memory, magnetic recording media, optical recording media, etc. The control unit 35B may be composed of an integrated circuit that integrates the processor and the storage device, or it may be composed of an SoC that further integrates the sensor 34B.

[0071] The control unit 35B operates the sensor 34B to perform detection of the indoor space 9. The control unit 35B acquires the detection result from the sensor 34B and generates detection data including the detection result. The control unit 35B transmits the detection data to the task processing unit 40 via the communication unit 36B.

[0072] The communication unit 36B communicates with the task processing unit 40 according to the control of the control unit 35B. For example, the communication unit 36B transmits detection data generated by the control unit 35B to the task processing unit 40. The communication unit 36B is a communication modem connected to two cables that constitute the power line 5B and performs power line communication via the power line 5B. The communication unit 36B includes, for example, an encoding circuit for encoding data, a transmitting circuit for superimposing the encoded signal onto a carrier wave, a receiving circuit equipped with a filter for extracting the signal superimposed on the carrier wave, and a decoding circuit for decoding the extracted signal to generate received data and outputting the received data to the control unit 35B.

[0073] The task processing unit 40, provided in the lighting device 10, operates by receiving power from the power supply unit 17. The first processor 41 also directly or indirectly controls the communication unit 16 when communicating with the switch 20 and the outlets 30A and 30B. The first processor 41 acquires detection data including the detection results of sensor 24, detection data including the detection results of sensor 34A, and detection data including the detection results of sensor 34B from the communication unit 16. The first processor 41 also acquires detection data including the detection results of sensor 14 from the control unit 15.

[0074] [1-3. Configuration of Task Processing Device and Smartphone] Figure 3 shows an example of the configuration of the task processing device 40 and the smartphone 6. The task processing device 40 is configured to communicate wirelessly with the smartphone 6. The task processing device 40 is configured to communicate wirelessly with the smartphone 6, for example, via Wi-Fi®. The smartphone 6 corresponds to an example of an "information processing device".

[0075] The task processing unit 40 receives task instruction information SD from the sensor group 4, which is a user instructing a task TS to be performed by a human body P. The task processing unit 40 also causes the smartphone 6 to execute the task TS corresponding to the task instruction information SD. The task TS, for example, indicates purchasing the food desired by the user from a store of the user's choice via a delivery service and having it delivered to the user's address or residence. The task instruction information SD is, for example, voice information spoken by the user.

[0076] Embodiment 1 describes a case where Task TS is a task that utilizes a delivery service, but the embodiments are not limited to this. Task TS may be a task related to a service that the smartphone 6 can use. For example, the service may be a goods purchase service. Alternatively, the service may be a ride-hailing service or a reservation service for restaurants, etc.

[0077] As explained with reference to Figure 2, the task processing device 40 comprises a first processor 41 and a first memory 42. The first memory 42 stores the first program 421. The first processor 41 functions as a receiving unit 411, a generating unit 412, and an execution unit 413 by reading the first program 421 from the first memory 42 and executing it. The first processor 41 causes the first memory 42 to function as a trained model 422 and a Large Language Model (LLM) 423 by reading the first program 421 from the first memory 42 and executing it. The first processor 41 corresponds to an example of a "processor". The first program 421 corresponds to an example of a "task processing program".

[0078] The trained model 422 is generated, for example, by generative artificial intelligence (generative AI). The trained model 422 outputs XML (Extensible Markup Language) data XMD, for example, which causes the smartphone 6 to execute task TS.

[0079] The trained model 422 is generated, for example, by performing machine learning (e.g., supervised learning) on ​​a neural network using a training dataset. The training dataset is composed of an input dataset and an output dataset. The input dataset includes, for example, an operation interface image QJP displayed on the touch panel 61 of a smartphone 6, and a plurality of operations QP for the operation interface image QJP. The output dataset is XML data XMD that defines the plurality of operations QP for the operation interface image QJP.

[0080] Task TS includes, for example, a first task TS1, a second task TS2, and a third task TS3. The first task TS1 is, for example, the process of launching an application program AP corresponding to the delivery service included in the task instruction information SD from among various application programs installed on the smartphone 6. The second task TS2 is, for example, the process of adding the food items desired by the user to the shopping cart at the store of the user's choice, as included in the task instruction information SD, using multiple operations QP on the operation interface image QJP of the smartphone 6. The third task TS3 is a payment execution task that executes the payment. That is, it is the process of executing the payment for task TS, which is to "purchase the food items desired by the user at the store of the user's choice via the delivery service and deliver them to the user's address or residence." The third task TS3 corresponds to an example of a "payment execution task."

[0081] The trained model 422 outputs the first XML data XMD1 corresponding to the first task TS1, the second XML data XMD2 corresponding to the second task TS2, and the third XML data XMD3 corresponding to the third task TS3.

[0082] The large-scale language model 423 outputs a task TS from the task instruction information SD. The task instruction information SD is voice information spoken by the user. The task instruction information SD is received by the reception unit 411 from the sensor group 4. The reception unit 411 inputs the task instruction information SD to the large-scale language model 423 and causes it to output a task TS.

[0083] Embodiment 1 describes a case where the first memory 42 stores the trained model 422 and the large-scale language model 423, but the embodiment is not limited to this. At least one of the trained model 422 and the large-scale language model 423 may be stored in a server device not shown in the figure. The trained model 422 and the large-scale language model 423 may be stored in a server device not shown in the figure. In this case, the configuration of the task processing unit 40 can be simplified.

[0084] The reception unit 411 receives task instruction information SD from the user, which instructs the user to perform a task TS. The reception unit 411 acquires sensing data from the sensor group 4 located in the indoor space 9 where the user is located, and based on the acquired sensing data, it receives the task instruction information SD.

[0085] The reception unit 411 determines, for example, which sensor among sensors 24, 34A, and 34B will detect the user's voice based on the user's position in the indoor space 9 detected by sensor 14. The reception unit 411 then causes the sensor closest to the user among sensors 24, 34A, and 34B to detect the user's voice. Here, for example, we will describe the case where sensor 24 is the sensor closest to the user among sensors 24, 34A, and 34B. That is, the reception unit 411 receives voice information emitted by the user from sensor 24 as task instruction information SD. The voice information emitted by the user corresponds to an example of "sensing data".

[0086] The reception unit 411 interprets the task instruction information SD via the large-scale language model 423 and extracts the task TS. In this case, the task TS is "to purchase the food requested by the user from the store of the user's choice via a delivery service and deliver it to the user's address or residence."

[0087] "Delivery service" is, for example, a delivery service provided by "Company A1". "The store desired by the user" is, for example, "Store B1". "The food desired by the user" is, for example, "Food C1". "The user's address or residence" is, for example, stored on smartphone 6. "Company A1" corresponds to an example of a company that provides delivery services. "Store B1" is an example of a store that provides delivery services and corresponds to an example of a store that provides food. "Food C1" corresponds to an example of food that Store B1 provides delivery services for.

[0088] In other words, Task TS is "Purchase food item C1 from store B1 via company A1's delivery service and deliver it to the user's address or residence." The reception unit 411 extracts "company A1," "store B1," and "food item C1" that constitute Task TS from the task instruction information SD via the large-scale language model 423.

[0089] The generation unit 412 generates XML data XMD that defines multiple operations for the smartphone 6 in the operation interface image QJP that accepts user input, based on the task instruction information SD. The generation unit 412 generates XML data XMD that causes the smartphone 6 to execute the task TS corresponding to the task instruction information SD, for example, via the trained model 422. In other words, the generation unit 412 causes the trained model 422 to generate XML data XMD that causes the smartphone 6 to execute the task TS.

[0090] Task TS consists of, for example, a first task TS1, a second task TS2, and a third task TS3. The first task TS1 indicates launching the application program AP for "Company A1's" delivery service on the smartphone 6. The second task TS2 indicates selecting food item C1 from store B1. The third task TS3 indicates executing the payment process.

[0091] The trained model 422 outputs the first XML data XMD1 corresponding to the first task TS1, the second XML data XMD2 corresponding to the second task TS2, and the third XML data XMD3 corresponding to the third task TS3. In other words, the generation unit 412 causes the trained model 422 to generate the first XML data XMD1, the second XML data XMD2, and the third XML data XMD3.

[0092] The execution unit 413 outputs XML data XMD to the smartphone 6 and executes task TS corresponding to task instruction information SD by causing the smartphone 6 to perform multiple operations on the operation interface image QJP. For example, the execution unit 413 causes the smartphone 6 to execute task TS by sequentially outputting the first XML data XMD1, the second XML data XMD2, and the third XML data XMD3 to the smartphone 6.

[0093] The reception unit 411 receives the user's approval SCD before the execution unit 413 executes the third task TS3 on the smartphone 6, that is, before the execution unit 413 executes the third XML data XMD3 on the smartphone 6. The reception unit 411 outputs a voice message from the smartphone 6's speaker saying, "The payment screen is displayed on the touch panel. Please indicate whether you want to proceed with the payment." The reception unit 411 receives a voice message indicating the user's approval SCD from the smartphone 6's microphone or sensor 24, for example. If the user's approval SCD is not obtained, the execution unit 413 terminates the execution of task TS without executing the third task TS3. If the user's approval SCD is obtained, the execution unit 413 executes the third task TS3 on the smartphone 6.

[0094] Next, the configuration of the smartphone 6 will be described with reference to Figure 3. The smartphone 6 includes a touch panel 61, a second processor 62, and a second memory 63. The smartphone 6 is communicated with the task processing unit 40.

[0095] The touch panel 61 includes a display such as an LCD (Liquid Crystal Display) and a touch sensor. The touch panel 61 displays various images, including an operation interface image QJP, on the display according to instructions from the second processor 62. The touch sensor receives touch operations from the user on the operation interface image QJP. When the touch sensor receives a touch operation from the user, it outputs the detection result of the touch sensor to the second processor 62.

[0096] The second processor 62 processes data by executing a program. The second processor 62 is composed of an integrated circuit such as a CPU, MPU, or microcontroller. The second processor 62 may also be programmed hardware. The second memory 63 includes a storage device that non-volatilely stores programs and data using flash memory, a magnetic recording medium, an optical recording medium, or the like. The second memory 63 may be composed of an integrated circuit integrated with the second processor 62.

[0097] The second memory 63 stores the second program 631 and the application program AP. The second processor 62 functions as a control unit 621 by reading the second program 631 from the second memory 63 and executing it. The control unit 621 controls the operation of each part of the smartphone 6.

[0098] The application program AP is an application program that provides delivery services corresponding to task TS. The application program AP provides delivery services to users by communicating with the server device (not shown in the diagram).

[0099] The control unit 621 receives XML data XMD from the task processing unit 40 and executes task TS. The control unit 621 displays the operation interface image QJP on the touch panel 61 according to the XML data XMD. The control unit 621 also accepts touch operations on the operation interface image QJP according to the XML data XMD. Then, the control unit 621 performs a transition in the operation interface image QJP or executes a predetermined process according to the accepted touch operation. The control unit 621 also transmits instruction information CM to a server device (not shown) via the application program AP according to the accepted touch operation. The server device (not shown) is, for example, a server device owned by a company that provides delivery services to users. The instruction information CM includes, for example, instructions for selecting a store, instructions for selecting food items, and instructions for executing payment.

[0100] [1-4. Processing of the Task Processing Unit] Next, the processing of the task processing unit 40 will be described with reference to Figures 4 and 5. Figure 4 is a sequence diagram 700 showing an example of the processing of the task processing unit 40. Sequence diagram 700 shows, from left to right, user instruction 710, situation understanding and judgment 720, and smartphone operation 730. User instruction 710 shows voice instructions from the user. Situation understanding and judgment 720 shows the process by which the reception unit 411 extracts the task TS from the task instruction information SD via the large-scale language model 423. Smartphone operation 730 shows the XML data XMD generated by the generation unit 412. The XML data XMD generated by the generation unit 412 is output to the smartphone 6 by the execution unit 413 and executed by the smartphone 6.

[0101] User instruction 710 includes a first user instruction 711 and a second user instruction 712. The first user instruction 711 corresponds to an example of task instruction information SD. The task TS indicated by the task instruction information SD is "to purchase the food desired by the user from a store of the user's choice via a delivery service and deliver it to the user's address or residence." The first user instruction 711 represents, for example, the user's voice saying, "Buy food C1 from store B1 at company A1!" That is, the delivery service is, for example, the delivery service provided by "company A1." The "store of the user's choice" is, for example, "store B1." The "food desired by the user" is, for example, "food C1."

[0102] The reception unit 411 extracts task TS from task instruction information SD via the large-scale language model 423. Task TS is, for example, "Purchase food C1 from store B1 via company A1's delivery service and deliver it to the user's address or residence." Task TS consists of, for example, a first task TS1, a second task TS2, and a third task TS3.

[0103] The first task TS1 indicates, for example, launching the delivery service application program AP of "Company A1" on the smartphone 6. The generation unit 412 causes the trained model 422 to generate the first XML data XMD1 corresponding to the first task TS1.

[0104] To execute the first task TS1 on the smartphone 6, first, the touch panel 61 of the smartphone 6 is scrolled to display the operation interface image QJP, which includes the icon JC corresponding to the application program AP of "Company A1"'s delivery service. Next, the icon JC corresponding to the application program AP of "Company A1"'s delivery service is touched to launch the application program AP. The first XML data XMD1 defines the above two operations in association with the operation interface image QJP.

[0105] The second task TS2 indicates, for example, selecting food item C1 from store B1. The generation unit 412 causes the trained model 422 to generate the second XML data XMD2 corresponding to the second task TS2.

[0106] To execute the second task TS2 on the smartphone 6, first, the touch panel 61 of the smartphone 6 is operated to display the operation interface image QJP, which includes store B1, in the application program AP for the home delivery service of "Company A1," and a touch operation is performed to select store B1. Button BT1 is touched, for example, when selecting store B1. Next, the touch panel 61 of the smartphone 6 is operated to display the operation interface image QJP, which includes food item C1, in the application program AP for the home delivery service of "Company A1," and a touch operation is performed to select food item C1. These operations are implemented, for example, by a search function within the application program AP. The second XML data XMD2 defines the above operations in association with the operation interface image QJP.

[0107] When the second task TS2 is completed, the reception unit 411 requests an approval SCD from the user. The reception unit 411 outputs a voice message from the speaker of the smartphone 6, for example, saying, "The payment screen is displayed on the touch panel. Please indicate whether you want to proceed with the payment." The reception unit 411 receives a voice message indicating the user's approval SCD from the microphone of the smartphone 6 or from the sensor 24, for example. If the user's approval SCD is not obtained, the execution unit 413 terminates the execution of task TS without executing the third task TS3. If the user's approval SCD is obtained, the execution unit 413 causes the smartphone 6 to execute the third task TS3.

[0108] The third task TS3 indicates, for example, the execution of a payment process. The generation unit 412 causes the trained model 422 to generate the third XML data XMD3 corresponding to the third task TS3.

[0109] To execute the third task TS3 on the smartphone 6, first, a payment screen is displayed on the touch panel 61 of the smartphone 6. The payment screen corresponds to an example of the operation interface image QJP. The payment screen is displayed by the application program AP. The operation interface image QJP corresponding to the payment screen includes a purchase button BT2 that instructs the user to execute the payment. By touching the purchase button BT2, the payment process is executed on the server device (not shown in the figure). The third XML data XMD3 defines the above operation in association with the operation interface image QJP.

[0110] Embodiment 1 describes a case where task TS consists of a first task TS1, a second task TS2, and a third task TS3, but the embodiment is not limited to this. Task TS may consist of four or more tasks. Task TS may also consist of one task or two tasks.

[0111] Next, the processing of the task processing device 40 will be described with reference to Figure 5. Figure 5 is a flowchart showing an example of the processing of the task processing device 40. As shown in Figure 5, in step S101, the reception unit 411 determines whether or not it has received task instruction information SD from the user to instruct a task TS. If the reception unit 411 determines that it has not received task instruction information SD from the user (step S101; NO), the processing enters a waiting state. If the reception unit 411 determines that it has received task instruction information SD from the user (step S101; YES), the processing proceeds to step S103.

[0112] Then, in step S103, the reception unit 411 interprets the task instruction information SD via the large-scale language model 423 and extracts the task TS. Next, in step S105, the generation unit 412 instructs the trained model 422 to generate XML data XMD that will cause the smartphone 6 to execute task TS. Next, in step S107, the execution unit 413 outputs the XML data XMD to the smartphone 6 and causes the smartphone 6 to execute task TS. Here, the execution unit 413 outputs the first XML data XMD1 and the second XML data XMD2 to the smartphone 6 and causes the smartphone 6 to execute the first task TS1 and the second task TS2.

[0113] Next, in step S109, the reception unit 411 determines whether the task to be performed by the smartphone 6 next is the third task TS3, that is, the payment execution task. In other words, the reception unit 411 determines whether the smartphone 6 has completed the execution of the first task TS1 and the second task TS2. If the reception unit 411 determines that the task to be performed by the smartphone 6 next is not the payment execution task (step S109; NO), the process proceeds to step S111. Then, in step S111, the execution unit 413 determines whether the smartphone 6 has finished executing task TS. If the execution unit 413 determines that the smartphone 6 has finished executing task TS (step S111; YES), the process then ends. If the execution unit 413 determines that the smartphone 6 has not finished executing task TS (step S111; NO), the process returns to step S107.

[0114] Next, if the reception unit 411 determines that the task to be executed by the smartphone 6 is a payment execution task (step S109; YES), the process proceeds to step S113. In step S113, the reception unit 411 requests the user to consent to the execution of the payment process. Next, in step S115, the reception unit 411 determines whether or not it has received an approval SCD from the user for the execution of the payment process. If the reception unit 411 determines that it has not received an approval SCD from the user for the execution of the payment process (step S115; NO), the process proceeds to step S117. In step S117, the reception unit 411 determines whether or not it has received a refusal to execute the payment process from the user. If the reception unit 411 determines that it has not received a refusal to execute the payment process from the user (step S117; NO), the process returns to step S115. If the reception unit 411 determines that it has received a request from the user to deny permission to execute the payment process (step S117; YES), the process then terminates.

[0115] If the reception unit 411 determines that it has received an approval SCD from the user for the execution of the payment process (step S115; YES), the process proceeds to step S119. In step S119, the execution unit 413 outputs the third XML data XMD3 to the smartphone 6 and causes the smartphone 6 to execute the third task TS3, i.e., the payment execution task. After that, the process ends.

[0116] Step S101 corresponds to an example of a "reception step". Step S105 corresponds to an example of a "generation step". Steps S107 and S119 correspond to an example of an "execution step".

[0117] [1-5. Effects, etc.] As described above, in Embodiment 1, the task processing device 40 is connected to the Internet and is communicatively connected to a smartphone 6 owned by the user, and comprises: a receiving unit 411 that receives task instruction information SD that instructs the user to perform a task TS; a generating unit 412 that generates XML data XMD which defines a plurality of operations to be performed on the smartphone 6 in the operation interface image QJP that accepts user operations, based on the task instruction information SD; and an execution unit 413 that outputs the XML data XMD to the smartphone 6 and executes the task TS corresponding to the task instruction information SD by causing the smartphone 6 to perform the plurality of operations in the operation interface image QJP.

[0118] In this configuration, XML data XMD is output to the smartphone 6, and by having the smartphone 6 perform multiple operations on the operation interface image QJP, the task TS corresponding to the task instruction information SD received from the user is executed. Therefore, the smartphone 6 can be made to execute the task TS corresponding to the task instruction information SD.

[0119] The task processing device 40 described above includes a trained model 422 which is trained using an operation interface image QJP and multiple operations on the smartphone 6 in the operation interface image QJP as input datasets and XML data XMD as output datasets. The trained model 422 is generated by a generation AI, and the generation unit 412 generates XML data XMD using the trained model 422.

[0120] With this configuration, the generation unit 412 generates XML data XMD using the trained model 422. By properly training the trained model 422, it is possible to generate appropriate XML data XMD. Therefore, the smartphone 6 can properly execute the task TS corresponding to the task instruction information SD.

[0121] In the task processing device 40 described above, the reception unit 411 acquires sensing data from the sensor group 4 arranged in the indoor space 9 where the user is located, and receives task instruction information SD based on the sensing data.

[0122] With this configuration, sensing data is acquired from a group of sensors 4 arranged in the indoor space 9, and task instruction information SD is received based on the sensing data, thus enabling the reception of task instruction information SD with a simple configuration.

[0123] In the task processing device 40 described above, each of the sensors constituting the sensor group 4 is incorporated into electrical equipment installed in the indoor space 9.

[0124] With this configuration, each of the sensors constituting the sensor group 4 is incorporated into electrical equipment installed in the indoor space 9, making it easy to place the sensors constituting the sensor group 4 in the indoor space 9. Furthermore, power can be easily supplied to each of the sensors constituting the sensor group 4. In addition, the detection signals from each of the sensors constituting the sensor group 4 can be easily transmitted to the task processing unit 40.

[0125] In the task processing device 40 described above, the sensor group 4 includes a microphone, and the reception unit 411 receives the task TS via the large-scale language model 423 based on the user's voice data detected by the microphone.

[0126] With this configuration, the task TS is received via the large-scale language model 423 based on the user's voice data detected by the microphone, thus enabling the task TS to be received appropriately.

[0127] In the task processing device 40 described above, task TS includes a third task TS3 that performs settlement, and the reception unit 411 receives the user's approval SCD before the third task TS3.

[0128] With this configuration, user approval SCD is received before the third task TS3 (the payment execution task), allowing the user to confidently execute task TS on the smartphone 6. Therefore, user convenience can be improved.

[0129] As described above, in Embodiment 1, the task processing method is a task processing method for a task processing device 40 that is connected to the Internet and is communicatively connected to a smartphone 6 owned by a user, and includes: a reception step of receiving task instruction information SD that instructs a task TS from a user; a generation step of generating XML data XMD that defines a plurality of operations to be performed on the smartphone 6 in an operation interface image QJP that accepts user operations, based on the task instruction information SD; and an execution step of outputting the XML data XMD to the smartphone 6 and executing the task TS corresponding to the task instruction information SD by causing the smartphone 6 to perform the plurality of operations in the operation interface image QJP. With this configuration, the task processing method has the same effects as the task processing device 40 described above.

[0130] As described above, in Embodiment 1, the first program 421 is the first program 421 of a task processing device 40 that is connected to the Internet and is communicatively connected to a smartphone 6 owned by the user. The first processor 41 of the task processing device 40 functions as a receiving unit 411 that receives task instruction information SD from the user instructing a task TS, a generating unit 412 that generates XML data XMD that defines a plurality of operations for the smartphone 6 in the operation interface image QJP that accepts user operations based on the task instruction information SD, and an execution unit 413 that outputs the XML data XMD to the smartphone 6 and executes the task TS corresponding to the task instruction information SD by causing the smartphone 6 to execute the plurality of operations in the operation interface image QJP. With this configuration, the first program 421 has the same effect as the task processing device 40 described above.

[0131] As described above, in Embodiment 1, the task processing system 1 comprises a smartphone 6 owned by the user and a task processing device 40 connected to the internet and capable of communicating with the smartphone 6. The task processing device 40 comprises a receiving unit 411 that receives task instruction information SD from the user instructing a task TS, a generating unit 412 that generates XML data XMD which defines a plurality of operations for the smartphone 6 in an operation interface image QJP that accepts user operations based on the task instruction information SD, and an execution unit 413 that outputs the XML data XMD to the smartphone 6 and executes the task TS corresponding to the task instruction information SD by causing the smartphone 6 to execute the plurality of operations in the operation interface image QJP. With this configuration, the task processing system 1 achieves the same effects as the task processing device 40 described above. (Embodiment 2) [2-1. Smartphone Configuration]

[0132] Next, the configuration of the smartphone 6 according to Embodiment 2 will be described. Below, the differences between the configuration of the smartphone 6 according to Embodiment 2 and the configuration of the smartphone 6 according to Embodiment 1, which was described with reference to Figure 3, will be mainly explained. The smartphone 6 includes a touch panel 61, a second processor 62, and a second memory 63. The smartphone 6 is communicated with the task processing device 40. The smartphone 6 corresponds to an example of an "information processing device". The second processor 62 and the second memory 63 correspond to an example of a "computer".

[0133] The second memory 63 stores the second program 631 and the application program AP. The second processor 62 reads the second program 631 from the second memory 63 and executes it, thereby causing the second memory 63 to function as a storage unit 62D. The storage unit 62D pre-stores XML data XMD. The XML data XMD is pre-generated, for example, by the generation unit 412 of the task processing device 40 according to Embodiment 1, and pre-stored in the second memory 63. The second program 631 corresponds to an example of a "program".

[0134] The second processor 62 functions as a receiving unit 62A, an extraction unit 62B, and an execution unit 62C by reading the second program 631 from the second memory 63 and executing it.

[0135] The reception unit 62A receives task instruction information SD from the user, which instructs a task TS. The reception unit 62A receives task instruction information SD from the user, for example, via the microphone of the smartphone 6. The extraction unit 62B extracts XML data XMD from the XML data XMD stored in the storage unit 62D based on the task instruction information SD. The extraction unit 62B extracts XML data XMD that executes the task TS corresponding to the task instruction information SD from the XML data XMD stored in the second memory 63, for example. The execution unit 62C executes the task TS corresponding to the task instruction information SD by executing the XML data XMD extracted by the extraction unit 62B.

[0136] The execution unit 62C displays the operation interface image QJP on the touch panel 61 according to the XML data XMD. The execution unit 62C also receives touch operations on the operation interface image QJP according to the XML data XMD. The execution unit 62C then performs a transition in the operation interface image QJP or executes a predetermined process according to the received touch operation. The execution unit 62C also transmits instruction information CM to a server device (not shown) via the application program AP according to the received touch operation. The server device (not shown) is, for example, a server device owned by a company that provides delivery services to users. The instruction information CM includes, for example, instructions for selecting a store, instructions for selecting food items, and instructions for executing payment.

[0137] [2-2. Effects, etc.] As described above, Embodiment 2 is a smartphone 6 connected to the internet, comprising: a reception unit 62A that receives task instruction information SD from a user instructing a task TS; a second memory 63 that pre-stores XML data XMD that defines a plurality of operations in an operation interface image QJP that receives user operations; an extraction unit 62B that extracts XML data XMD from the XML data XMD stored in the second memory 63 based on the task instruction information SD; and an execution unit 62C that executes the task TS corresponding to the task instruction information SD by executing the XML data XMD extracted by the extraction unit 62B.

[0138] In this configuration, the smartphone 6 extracts XML data XMD from the XML data XMD stored in the second memory 63 based on the task instruction information SD, and executes the task TS corresponding to the task instruction information SD by executing the extracted XML data XMD. Therefore, the smartphone 6 can execute the task TS corresponding to the task instruction information SD.

[0139] As described above, in Embodiment 2, the second program 631 causes the computer (second processor 62 and second memory 63) of the smartphone 6 connected to the internet to function as a receiving unit 62A that receives task instruction information SD that instructs a task TS from a user, a storage unit 62D that pre-stores XML data XMD that defines multiple operations in an operation interface image QJP that accepts user operations, an extraction unit 62B that extracts XML data XMD from the XML data XMD stored in the storage unit 62D based on the task instruction information SD, and an execution unit 62C that executes the task TS corresponding to the task instruction information SD by executing the XML data XMD extracted by the extraction unit 62B.

[0140] This configuration produces the same effect as the smartphone 6 described above.

[0141] (Other Embodiments) As described above, the above embodiments have been explained as examples disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments. Therefore, other embodiments are described below as examples.

[0142] In the above embodiment, the lighting device 10, the switch 20, and the outlets 30A and 30B are examples of electrical equipment to which sensors are provided. Electrical equipment to which sensors constituting the task processing system 1 are incorporated can be, for example, outlets with two or more sockets, outlets with ground terminals, switches with pilot lamps, switches with timers, switches with lighting, ventilation fans, etc. It is also possible to incorporate sensors into other electrical equipment.

[0143] In the above embodiment, the case where the "information processing device" is a smartphone 6 has been described, but the embodiment is not limited to this. The "information processing device" may be, for example, a tablet computer. Also, the "information processing device" may be, for example, a notebook personal computer or a desktop personal computer.

[0144] Each of the first processor 41 and the second processor 62 may consist of a single processor or multiple processors. These processors may also be hardware programmed to implement the corresponding functional units. That is, these processors may consist of, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0145] The configuration of each part of the task processing system 1 shown in Figure 2 is an example, and the specific implementation form is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each part in Figure 2 be implemented individually; for example, it is certainly possible to configure a single integrated circuit to realize the functions of each part. Furthermore, in the embodiment described above, the functions of the first processor 41 may be realized by software or by hardware.

[0146] The configuration of each part of the task processing device 40 shown in Figure 3 is an example, and the specific implementation form is not particularly limited. In other words, it is not necessarily required that hardware corresponding to each part in Figure 3 be implemented individually; for example, it is certainly possible to configure a single integrated circuit to realize the functions of each part. Furthermore, in the embodiment described above, the functions of the first processor 41 may be realized by software or by hardware.

[0147] The operation steps shown in Figure 5 are divided according to the main processing content to facilitate understanding of the operation, and the operation is not limited by the way the processing units are divided or the names of the processing units. Depending on the processing content, it may be further divided into more steps. Alternatively, it may be divided so that one step unit includes even more processing. Furthermore, the order of the steps may be changed as appropriate, as long as it does not hinder the intent of this disclosure.

[0148] Since the embodiments described above are for illustrative purposes of the technology described herein, various modifications, substitutions, additions, omissions, etc., can be made within the claims or their equivalents.

[0149] (Note) The above description of embodiments discloses the following technology.

[0150] (Technology 1) A task processing device connected to the Internet and communicatively connected to an information processing device owned by a user, comprising: a receiving unit that receives task instruction information from a user that instructs a task; a generating unit that generates XML data defining a plurality of operations for the information processing device in an operation interface image that accepts user operations, based on the task instruction information; and an execution unit that outputs the XML data to the information processing device and executes the task corresponding to the task instruction information by causing the information processing device to execute the plurality of operations in the operation interface image.

[0151] According to this, XML data is output to the information processing device, and by having the information processing device perform multiple operations on the operation interface image, tasks corresponding to task instruction information received from the user are executed. Therefore, the information processing device can be made to execute tasks corresponding to task instruction information.

[0152] (Technology 2) A task processing device according to Technology 1, comprising a trained model trained with the operation interface image and a plurality of operations on the information processing device in the operation interface image as input datasets and the XML data as output datasets, wherein the trained model is generated by a generation AI, and the generation unit generates the XML data using the trained model. According to this, since the generation unit generates XML data using the trained model, by properly training the trained model, appropriate XML data can be generated. Therefore, the information processing device can be made to properly execute the task corresponding to the task instruction information.

[0153] (Technology 3) The receiving unit is a task processing device according to Technology 1 or Technology 2, wherein the receiving unit acquires sensing data from a group of sensors arranged in the indoor space where the user is located, and receives the task instruction information based on the sensing data. With this, since sensing data is acquired from a group of sensors arranged in the indoor space and task instruction information is received based on the sensing data, task instruction information can be received with a simple configuration.

[0154] (Technology 4) The task processing device according to Technology 3, wherein each of the sensors constituting the sensor group is incorporated into electrical equipment installed in the indoor space. With this, since each of the sensors constituting the sensor group is incorporated into electrical equipment installed in the indoor space, the sensors constituting the sensor group can be easily placed in the indoor space. Power can also be easily supplied to each of the sensors constituting the sensor group. Furthermore, the detection signals of each of the sensors constituting the sensor group can be easily transmitted to the task processing device.

[0155] (Technical 5) The task processing device according to Technical 3 or Technical 4, wherein the sensor group includes a microphone, and the receiving unit accepts the task via a large-scale language model based on the user's voice data detected by the microphone. With this, since the task is accepted via a large-scale language model based on the user's voice data detected by the microphone, the task can be accepted appropriately.

[0156] (Technical 6) The task processing device according to any one of Technical 1 to Technical 5, wherein the task includes a settlement execution task that performs settlement, and the receiving unit accepts the user's approval before the settlement execution task. With this, since the user's approval is accepted before the settlement execution task, the user can confidently have the information processing device perform the task. Therefore, user convenience can be improved.

[0157] (Technical 7) A task processing method for a task processing device connected to the Internet and capable of communicating with an information processing device owned by a user, comprising: a receiving step of receiving task instruction information from a user instructing a task; a generating step of generating XML data that defines a plurality of operations to be performed on the information processing device in an operation interface image that accepts user operations, based on the task instruction information; and an execution step of outputting the XML data to the information processing device and causing the information processing device to perform the plurality of operations on the operation interface image, thereby executing the task corresponding to the task instruction information. This method provides the same effects as the task processing device described in Technical 1.

[0158] (Technology 8) A task processing program for a task processing device connected to the Internet and capable of communicating with an information processing device owned by a user, wherein the processor of the task processing device functions as a receiving unit that receives task instruction information from a user, a generating unit that generates XML data defining a plurality of operations for the information processing device in an operation interface image that accepts user operations based on the task instruction information, and an execution unit that outputs the XML data to the information processing device and executes the task corresponding to the task instruction information by causing the information processing device to execute the plurality of operations in the operation interface image. This provides the same effects as the task processing device described in Technology 1.

[0159] (Technology 9) A task processing system comprising an information processing device owned by a user and a task processing device connected to the Internet and capable of communicating with the information processing device, wherein the task processing device comprises: a receiving unit that receives task instruction information from a user that instructs a task; a generating unit that generates XML data defining a plurality of operations for the information processing device in an operation interface image that accepts user operations, based on the task instruction information; and an execution unit that outputs the XML data to the information processing device and executes the task corresponding to the task instruction information by causing the information processing device to execute the plurality of operations in the operation interface image. This provides the same effects as the task processing device described in Technology 1.

[0160] (Technical 10) An information processing device connected to the Internet, comprising: a receiving unit that receives task instruction information from a user that instructs a task; a storage unit that pre-stores XML data defining a plurality of operations in an operation interface image that accepts user operations; an extraction unit that extracts XML data from the XML data stored in the storage unit based on the task instruction information; and an execution unit that executes a task corresponding to the task instruction information by executing the XML data extracted by the extraction unit.

[0161] According to this, the information processing device extracts XML data from the XML data stored in the memory unit based on task instruction information, and executes the task corresponding to the task instruction information by executing the extracted XML data. Therefore, the information processing device can execute the task corresponding to the task instruction information.

[0162] (Technical 11) A program that causes a computer in an information processing device connected to the Internet to function as a receiving unit that receives task instruction information from a user, a storage unit that pre-stores XML data defining multiple operations in an operation interface image that accepts user operations, an extraction unit that extracts XML data from the XML data stored in the storage unit based on the task instruction information, and an execution unit that executes the task corresponding to the task instruction information by executing the XML data extracted by the extraction unit.

[0163] According to this, the same effects as those of the information processing device described in Technology 10 can be achieved.

[0164] As described above, the task processing device, task processing method, task processing program, task processing system, information processing device, and program related to this disclosure can be used to receive task instruction information from a user and to cause the information processing device to execute the task corresponding to the task instruction information.

[0165] 1 Task Processing System 4 Sensor Group 5, 5A, 5B, 5C Power Line 6 Smartphone (Information Processing Device) 61 Touch Panel 62 Second Processor (Part of Computer) 62A Reception Unit 62B Extraction Unit 62C Execution Unit 62D Storage Unit 63 Second Memory (Part of Computer) 9 Indoor Space 10 Lighting Device 14 Sensor 20 Switch 24 Sensor 30A, 30B Outlet 34A, 34B Sensor 40 Task Processing Device 41 First Processor (Processor) 411 Reception Unit 412 Generation Unit 413 Execution Unit 42 First Memory 421 First Program (Task Processing Program) 422 Trained Model 423 Scale Language Model 50 Distribution Board AP Application Program QJP Operation Interface Image SCD Approval SD Task Instruction Information TS Task TS1 First Task TS2 Second Task TS3 Third Task (Settlement Execution Task) XMD XML Data XMD1 First XML Data XMD2 Second XML Data XMD3 Third XML Data

Claims

1. A task processing device connected to the Internet and capable of communicating with an information processing device owned by a user, comprising: a receiving unit that receives task instruction information from a user that instructs a task; a generating unit that generates XML data defining a plurality of operations for the information processing device in an operation interface image that accepts user operations, based on the task instruction information; and an execution unit that outputs the XML data to the information processing device and executes the task corresponding to the task instruction information by causing the information processing device to execute the plurality of operations in the operation interface image.

2. The task processing device according to claim 1, comprising a trained model trained with the operation interface image and a plurality of operations on the information processing device in the operation interface image as input datasets and the XML data as output datasets, wherein the trained model is generated by a generation AI, and the generation unit generates the XML data using the trained model.

3. The task processing device according to claim 1 or 2, wherein the receiving unit acquires sensing data from a group of sensors arranged in the indoor space where the user is located, and receives the task instruction information based on the sensing data.

4. The task processing device according to claim 3, wherein each of the sensors constituting the sensor group is incorporated into electrical equipment installed in the indoor space.

5. The task processing apparatus according to claim 3, wherein the sensor group includes a microphone, and the receiving unit accepts the task via a large-scale language model based on the user's voice data detected by the microphone.

6. The task processing apparatus according to claim 1 or claim 2, wherein the task includes a settlement execution task that performs settlement, and the receiving unit receives approval from the user before the settlement execution task.

7. A task processing method for a task processing device that is connected to the Internet and is communicatively connected to an information processing device owned by a user, comprising: a receiving step of receiving task instruction information that instructs a user to perform a task; a generation step of generating XML data that defines a plurality of operations to be performed on the information processing device in an operation interface image that accepts user operations, based on the task instruction information; and an execution step of outputting the XML data to the information processing device and causing the information processing device to perform the plurality of operations on the operation interface image, thereby executing a task corresponding to the task instruction information.

8. A task processing program for a task processing device that is connected to the Internet and is communicatively connected to an information processing device owned by a user, wherein the processor of the task processing device functions as: a receiving unit that receives task instruction information from a user; a generating unit that generates XML data defining a plurality of operations for the information processing device in an operation interface image that accepts user operations based on the task instruction information; and an execution unit that outputs the XML data to the information processing device and executes the task corresponding to the task instruction information by causing the information processing device to execute the plurality of operations in the operation interface image.

9. A task processing system comprising: an information processing device owned by a user; and a task processing device connected to the Internet and capable of communicating with the information processing device, wherein the task processing device comprises: a receiving unit that receives task instruction information from a user; a generating unit that generates XML data defining a plurality of operations for the information processing device in an operation interface image that accepts user operations, based on the task instruction information; and an execution unit that outputs the XML data to the information processing device and executes a task corresponding to the task instruction information by causing the information processing device to execute the plurality of operations in the operation interface image.

10. An information processing device connected to the Internet, comprising: a receiving unit that receives task instruction information from a user that instructs a task; a storage unit that pre-stores XML data defining a plurality of operations in an operation interface image that accepts user operations; an extraction unit that extracts XML data from the XML data stored in the storage unit based on the task instruction information; and an execution unit that executes a task corresponding to the task instruction information by executing the XML data extracted by the extraction unit.

11. A program that causes a computer in an information processing device connected to the Internet to function as: a receiving unit that receives task instruction information from a user; a storage unit that pre-stores XML data defining multiple operations in an operation interface image that accepts user operations; an extraction unit that extracts XML data from the XML data stored in the storage unit based on the task instruction information; and an execution unit that executes the task corresponding to the task instruction information by executing the XML data extracted by the extraction unit.

Images