Display method, information processing device, and program

Abstract

In a device control system, a user device, a sequence generation device, and an information processing device (400) are connected via a network. An information processing device (400) is provided with a main processing unit (422). The main processing unit (422) acquires an evaluation formula in which one or more functions indicating a process executed by a device or a person and having a start condition, an execution parameter, and an end condition as input variables are arranged. The main processing unit (422) displays information relating to the process for each process on the basis of one or a plurality of functions included in the acquired evaluation formula.

Description

Technical Field

[0001] The present disclosure relates to information processing technology, and more particularly to a display method for indicating functions and actions of a device, an information processing apparatus, and a program. Background Art

[0002] In order to improve the convenience of the user of the cooking device through communication technology, the cooking device sends a request for providing recipe information related to the steps of making dishes using the cooking device. After receiving the request, the information providing server provides the recipe information to the terminal device, and the terminal device prompts the recipe information (for example, refer to patent document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-48431 Summary of the invention

[0006] Technical problem to be solved by the invention

[0007] Devices such as cooking machines are controlled by software. By updating the software, the device can perform various functions and actions. If a user generates or updates such software, the device can perform functions and actions corresponding to the user's wishes. However, most users do not fully understand the relationship between the functions and actions of the device and the software, so it is difficult for the user to generate or update the software. In order to make it easy for users to generate or update software, it is expected that users can easily know the relationship between the functions and actions of the device and the software.

[0008] The present disclosure has been made in view of such circumstances, and an object of the present disclosure is to provide a technology for displaying the functions and operations of a device in an easily understandable manner.

[0009] Methods used to solve technical problems

[0010] In order to solve the above-mentioned technical problems, a display method of one scheme disclosed in the present invention includes: a step of obtaining an evaluation formula, wherein the evaluation formula arranges one or more functions representing processes performed by a device or a person, with start conditions, execution parameters, and end conditions as input variables; and a step of displaying process-related information according to each process based on one or more functions contained in the obtained evaluation formula.

[0011] Another scheme of the present disclosure is an information processing device. The device includes: a first processing unit, which obtains an evaluation formula, wherein the evaluation formula arranges one or more functions representing the process executed by a device or a person, and takes a start condition, an execution parameter, and an end condition as input variables; and a second processing unit, which displays information related to the process according to each process based on the one or more functions included in the obtained evaluation formula.

[0012] Furthermore, arbitrary combinations of the above-described constituting elements and modes in which the expression of the present disclosure is converted into methods, apparatuses, systems, recording media, computer programs, etc. may also be practiced as modes of the present disclosure.

[0013] Effects of the Invention

[0014] According to the present disclosure, the functions and operations of a device can be displayed in an easily understandable manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a diagram showing the configuration of the device control system according to the embodiment.

[0016] Figure 2 (a)-(e) are shown in Figure 1 A diagram showing the configuration of the functional blocks used in the device control system.

[0017] Figure 3 (a)-(c) are shown in Figure 1 A diagram showing the structure of a sequence of function blocks used in a device control system.

[0018] Figure 4 It is shown Figure 1 A diagram showing an overview of the operation of the device.

[0019] Figure 5 (a)-(d) show Figure 1 Diagram of various representations of the function block sequence in a plant control system.

[0020] Figure 6 It is shown Figure 5 (b) is a graph of a specific example of a function.

[0021] Figure 7 It is shown Figure 5 A diagram of another specific example of the function of (b).

[0022] Figure 8 It shows the change Figure 5 (b) is a diagram of the evaluation formula after a part of the evaluation formula.

[0023] Fig. 9 It is shown Figure 1 A diagram of a specific example of a functional block sequence in a device control system.

[0024] Fig.10 It shows Fig. 9 Diagram of the evaluation formula for the function block sequence.

[0025] Fig.11 It is shown Figure 1 Diagram of the composition of the device control system.

[0026] Fig.12 It is shown Fig.11 Diagram of the composition of the device.

[0027] Fig.13 It is shown Fig.11 A diagram showing the structure of a sequence generating device.

[0028] Fig.14 It is shown Fig.11 A diagram showing a configuration of an information processing device.

[0029] Fig.15 It is shown Fig.11 FIG. 1 is a diagram showing a configuration of a user device.

[0030] Fig.16 (a)-(c) show Fig.13 The display section displays a diagram of the sequence generated IF.

[0031] Fig.17 It is shown based on Figure 2 A timing diagram showing the steps of the device control system.

[0032] Fig.18 (a)-(c) show Fig.15 The sequence displayed in the display section shows an IF diagram.

[0033] Fig.19 (a)-(b) shows Fig.18 (a) is a diagram showing a material display area.

[0034] Fig. 20 It is shown based on Figure 2 A timing diagram showing the steps of the device control system.

[0035] Fig.21 (a)-(c) show Fig.15 The sequence displayed in the display section shows an IF diagram.

[0036] Fig. 22 It is shown Fig.15 The sequence displayed in the display section shows an IF diagram.

[0037] Fig.23 (a)-(c) show Fig.15 A diagram showing an example of an image displayed on a display unit.

[0038] Fig.24 (a)-(e) show Fig.15 A diagram showing an example of an image displayed on a display unit.

[0039] Fig.25 (a)-(e) show Fig.15 A diagram showing an example of an image displayed on a display unit.

[0040] Fig.26 (a)-(b) shows Fig.15 A diagram showing an example of an image displayed on a display unit.

[0041] Fig. 27 (a)-(b) shows Fig.15 FIG. 1 is a diagram of the appearance of a user device. DETAILED DESCRIPTION

[0042] The embodiments described below all represent a preferred specific example of the present disclosure. Therefore, the numerical values, shapes, materials, constituent elements, configuration positions and connection methods of constituent elements, as well as steps (processes) and the order of steps shown in the following embodiments are examples, and the main purpose is not to limit the present disclosure. Therefore, among the constituent elements in the following embodiments, the constituent elements not recorded in the independent claims representing the top-level concepts of the present disclosure are described as arbitrary constituent elements. In addition, in each figure, the same figure mark is marked for substantially the same structure, and repeated descriptions are omitted or simplified. In the following, the embodiments are described in the order of (1) an overview of functional blocks and functional block sequences, (2) evaluation formulas and image representations of functional block sequences, (3) the composition of the device control system, (4) the production of functional block sequences, (5) the updating of functional block sequences, (6) the execution of target processing, and (7) modified examples.

[0043] (1) Overview of function blocks and function block sequences

[0044] In household electrical appliances such as rice cookers, washing machines, and microwave ovens (hereinafter referred to as "apparatus"), the functions and actions of the hardware are controlled by software for realizing specific functions. In this embodiment, an apparatus control system is introduced as a structure capable of generating or updating software for controlling the apparatus.

[0045] Figure 1 The structure of the device control system 1000 is shown. In the device control system 1000, a four-layer model consisting of the first layer to the fourth layer is defined. In the first layer, the structure of the device 100 is specified. The device 100 is, for example, a rice cooker (device 100a), a washing machine (device 100b), and a microwave oven (device 100c). The device 100 is not limited to this. Each device 100 includes a plurality of components 102, a plurality of drivers 104, and a plurality of functional blocks 110.

[0046] The so-called component 102 is a hardware element that is a unit in which the actions (actions, sensing) of the device 100 are divided, and includes actuators and sensors that perform the functions of the device 100. The actuator is an output device, and the sensor is an input device. The actuator is, for example, the bottom outer IH (Induction Heating) coil (component 102a), the main body IH coil (component 102b), the stepping motor (component 102c), the water container IH coil (component 102d), the cooling fan (component 102e), and the piezoelectric buzzer (component 102f) in the rice cooker (device 100a). The sensor is, for example, the temperature sensor (component 102g) in the rice cooker (device 100a). The component 102 included in the rice cooker (device 100a) is not limited to this, and the washing machine (device 100b) and the microwave oven (device 100c) are also constructed in the same way.

[0047] The so-called driver 104 is software for directly controlling the component 102. The IH control (driver 104) in the rice cooker (device 100a) controls the bottom outer IH coil (component 102a). In addition, the IH control (driver 104b) controls the main body IH coil (component 102b), the pressure valve control (driver 104c) controls the stepping motor (component 102c), and the IH control (driver 104d) controls the water container IH coil (component 102d). In addition, the fan control (driver 104e) controls the cooling fan (component 102e), the buzzer control (driver 104f) controls the piezoelectric buzzer (component 102f), and the sensor control (driver 104g) controls the temperature sensor (component 102g). The driver 104 included in the rice cooker (device 100a) is not limited to this, and the washing machine (device 100b) and the microwave oven (device 100c) are also configured in the same manner.

[0048] The function block 110 corresponds to one or more drivers 104 and is a software interface (API: Application Programming Interface) for operating one or more components 102. The function block 110 can receive one or more parameters for controlling the operation of (each) component 102. The details of the function block 110 will be described later.

[0049] In the second layer, in order for the device 100 to execute the target processing, a function block sequence 120 is defined in which one or more function blocks 110 are arranged in the order of operation. In other words, the function block sequence 120 defines the execution order of one or more function blocks 110. The target processing is determined according to the device 100, for example, cooking for the rice cooker (device 100a) and the microwave oven (device 100c), and washing for the washing machine (device 100b). The function block sequence 120a is used in the rice cooker (device 100a), the function block sequence 120b is used in the washing machine (device 100b), and the function block sequence 120c is used in the microwave oven (device 100c). The device 100 executes the operation in the order of the function blocks 110 arranged in the function block sequence 120. Therefore, by changing the arrangement of the function blocks 110 or changing the parameters set in the function blocks 110, the function and operation of the device 100 can be updated. The details of the functional block sequence 120 will be described later.

[0050] In the third layer, a platform server 130 is configured to manage various information in the device control system 1000. The platform server 130 includes a sequence manager, a device manager, and various databases. The sequence manager manages the function block sequence 120, the device manager manages the registration information of the device 100 that can use the function block sequence 120, and the various databases manage the user information that can use the function block sequence 120.

[0051] In the fourth layer, a user application server 132 is arranged to publish each function block sequence 120 as a user application. The function block sequence 120 published by the user application server 132 is downloaded to the device 100. The downloaded function block sequence 120 can be used in the device 100. After other function block sequences 120 are downloaded to the device 100, other function block sequences 120 can be used in the device 100.

[0052] The third layer and the fourth layer in the device control system 1000 may be integrated. In this case, the platform server 130 and the user application server 132 are integrally configured. In addition, the third layer and the fourth layer in the device control system 1000 may be arranged in the same layer. Furthermore, the third layer and the fourth layer in the device control system 1000 may be omitted. In this case, the function block sequence 120 is downloaded from a user device (not shown) owned by the user to the device 100.

[0053] Figure 2 (a) to (e) show the configuration of the functional block 110 used in the device control system 1000 . Figure 2(a) shows the basic structure of the function block 110. The function block 110 is defined as a functional unit that can be executed by the device 100, and has a "box name" corresponding to the content of the function. A plurality of parameters corresponding to the function can be set for the function block 110. Each parameter set for the function block 110 is output to the driver 104. After receiving the parameters from the function block 110, the driver 104 controls the action of the component 102 according to the parameters.

[0054] Figure 2 (b) shows Figure 1 The function block 110a of "pre-cooking" in the electric rice cooker (device 100a). For the function block 110a of "pre-cooking", the parameters of the pot bottom temperature, duration, convection mode, bottom (outer) IH time, and bottom (inner) IH time can be set. Figure 2 (c) shows Figure 1 The "boiling" function block 110b in the rice cooker (device 100a) Figure 2 (d) shows Figure 1 The "stew and steam" function block 110c in the electric rice cooker (device 100a), Figure 2 (e) shows a "keep warm" function block 110d. A plurality of parameters can also be set in each of the function blocks 110b to 110d. Figure 1 The same is true for the functional block 110 in the washing machine (device 100b) and the microwave oven (device 100c).

[0055] Figure 3 (a) to (c) show a functional block sequence 120 used in the device control system 1000, especially in Figure 1 The composition of the function block sequence 120a used in the rice cooker (device 100a). Figure 3 (a) shows the sequence for "steaming rice", Figure 3 (b) shows the sequence for "cooking", Figure 3 (c) shows the sequence for "Roast beef (cooked at low temperature)".

[0056] exist Figure 3 In the sequence for "steaming rice" shown in (a), three "pre-cooking" function blocks 110a, "cooking completion" function blocks 110n, "boiling" function blocks 110b, "stewed steaming" function blocks 110c, and "keeping warm" function blocks 110d are arranged in order. Different parameters are set in the three "pre-cooking" function blocks 110a. By arranging the three "pre-cooking" function blocks 110a with different parameters in this way, three-stage pre-cooking can be performed.

[0057] exist Figure 3In the sequence for “cooking” shown in (b), a function block 110a of “pre-cooking”, a function block 110n of “cooking completion”, a function block 110b of “boiling”, and a function block 110d of “keeping warm” are arranged in order. Figure 3 The sequence for "roast beef (low temperature cooking)" shown in (c) includes a "keep warm" function block 110d. By changing the type, arrangement and parameters of the function blocks 110 used in this way, it is possible to perform processing for different purposes of "steaming rice", "boiling cooking" and "roast beef (low temperature cooking)". Figure 1 The same is true for the functional block sequence 120 in the washing machine (device 100b) and the microwave oven (device 100c).

[0058] Figure 4 The device 100 is shown, and in particular Figure 1 This indicates the operation of the rice cooker (device 100a) according to the Figure 3 (a) The operation of the device 100a in the sequence of "steaming rice". In the soaking process, three "pre-cooking" function blocks 110a with different parameters are sequentially executed, and the corresponding components 102 operate according to the parameters. As a result, the temperature of the pot increases in a step-like manner over time. Then, by sequentially executing the "cooking completion" function block 110n, the "boiling" function block 110b, the "stewing" function block 110c, and the "keeping warm" function block 110d, the corresponding components 102 operate according to the parameters. That is, by sequentially executing multiple function blocks 110, rice is steamed in the device 100a.

[0059] (2) Function block sequence evaluation formula and image-based display

[0060] In the above description, the function block 110 is defined as the functional unit that the device 100 can perform, and the function block sequence 120 in which one or more function blocks 110 are arranged in the order of action is defined. Such a function block 110 does not include the action that a person should perform, and the function block sequence 120 does not include information other than the function block 110, such as information on materials. On the other hand, in actual target processing, these are sometimes necessary. In order to correspond to this, the above function block 110 and function block sequence 120 are expanded below. Therefore, the function block 110 is also defined for the action unit that a person should perform, and information on materials, etc. is also arranged in the function block sequence 120. By expressing it as a function block 110, the action of a person can be treated as equivalent to the action of the device 100. Therefore, when the action of a person that cannot be replaced by the device 100 at present can be replaced by the device 100 in the future, dishes can be prepared using the same function block sequence 120.

[0061] Figure 5(a) to (d) show various expressions of the function block sequence 120 in the device control system 1000 . Figure 5 (a) represents a function block sequence 120 for cooking "fried chicken". As shown in the figure, next to the material 700a "chicken", the function block 110p of "cutting" is arranged, and the function block 110p of "cutting" and the material 700b "starch" are input into the function block 110q of "coating". In this way, the function block 110 can accept multiple inputs (the output of the function block 110p and the material 700b). In addition, the function block 110q of "coating", the function block 110r of "frying", the function block 110s of "cooling", and the function block 110t of "frying" are arranged in sequence. At this time, the parameter of "bite-sized" is set in the function block 110p of "cutting", the parameter of "160 degrees" is set in the function block 110r of "frying", and the parameter of "180 degrees" is set in the function block 110t of "frying".

[0062] In the function block sequence 120, a first process, a second process, etc. are defined in the order in which the function blocks 110 are arranged. Figure 5 In the function block sequence 120 of (a), the function block 110p of "cutting" corresponds to the "first process", the function block 110q of "coating" corresponds to the "second process", the function block 110r of "frying" corresponds to the "third process", the function block 110s of "cooling" corresponds to the "fourth process", and the function block 110t of "frying" corresponds to the "fifth process".

[0063] The function blocks 110r of "frying" and 110t of "frying" are function blocks 110 defined in functional units that can be executed by the device 100. In addition, the function blocks 110p of "cutting", the function blocks 110q of "coating", and the function blocks 110s of "cooling" are function blocks 110 defined in action units that people should perform. When the function blocks 110 defined in functional units that can be executed by the device 100 are referred to as "first function blocks 110", the function blocks 110 defined in action units that people should perform are referred to as "second function blocks 110". In order to distinguish between the first function blocks 110 and the second function blocks 110, marks are set for the function blocks 110. Material 700a "chicken" and material 700b "starch" are information on the materials.

[0064] As described above, by arranging one or more function blocks 110, a function block sequence 120 can be created, and by changing the function blocks 110 included in the function block sequence 120, the function block sequence 120 can be updated. By using such a function block sequence 120, various recipes can be presented. However, when the relationship between the functions and actions of the device 100 and the function blocks 110 is not fully understood, the user will have questions about which function block 110 to use and what parameters to set for the function block 110. For example, it is difficult for the user of the recipe to imagine what kind of dishes can be made by the function block sequence 120. In addition, for example, it is difficult for the maker of the recipe to imagine which function blocks should be arranged in what order to make the desired recipe. For such users, images or texts are easier to understand than function block sequences 120.

[0065] In order to be able to convert between the function block sequence 120, the image and the article, in this embodiment, an evaluation formula that can be converted with each of the function block sequence 120, the image and the article is introduced. The evaluation formula is a formula that represents the steps for causing the device 100 or the person to perform the processing of the target. The evaluation formula is a nested structure of one or more functions. One function corresponds to one function block. As the input of the function, materials are included, so that the materials can also be processed as parameters of the function block 110.

[0066] Figure 5 (b) represents the evaluation formula 800 introduced in this embodiment. Figure 5 The function block sequence 120 of (a) similarly represents the evaluation formula 800 for making "fried chicken". The evaluation formula 800 includes a function 802. Function 802, like the function block 110, represents the process (function, action) performed by the device 100. Function 802a "cut", function 802b "spread", function 802c "fry", function 802d "cool", function 802e "fry" correspond to Figure 5 (a) includes a function block 110p for “cutting”, a function block 110q for “coating”, a function block 110r for “frying”, a function block 110s for “cooling”, and a function block 110t for “frying”.

[0067] In addition, in the evaluation formula 800, a nested structure is provided in order to express the steps in the function block sequence 120. The evaluation formula 800a of the first process includes the function 802a "cut". The evaluation formula 800b of the second process includes the function 802b "apply all over", and the independent variable of the function 802b "apply all over" includes the evaluation formula 800a of the first process. This means that the result of the evaluation formula 800a of the first process is input into the evaluation formula 800b of the second process, and the relationship between the evaluation formulas 800 is clearly shown. The same is true for the evaluation formula 800c of the third process, the evaluation formula 800d of the fourth process, and the evaluation formula 800e of the fifth process. That is, in the evaluation formula 800, one or more functions 802 are arranged.

[0068] Here, in order to explain the independent variable of function 802, Figure 6 . Figure 6 Specific examples of function 802 are shown. Function 802a "cut" is shown on the left, and function 802c "explode" and function 802e "explode" are shown on the right. Operation content, external factors, start conditions, execution parameters, and end conditions are set as input variables as independent variables of these functions 802.

[0069] In the case of function 802a "cut", the operation content shows the cutting method such as cutting into quarters and chopping, and the external factor shows the props such as kitchen knife and knife. In addition, the start condition indicates the material, and the execution parameter indicates the direction, position, size (numerical value), size (enumeration type). As an example, Figure 5 In the function 802a "cut" in the evaluation formula 800a of the first step of (b), the material "chicken" and the size (enumeration type) "bite-sized" are set.

[0070] In the case of function 802e "frying", the operation content shows the frying method such as deep frying, dry frying, etc., and the external factor shows the props such as the pot and deep fryer. In addition, the starting condition indicates the material, and the execution parameter indicates preheating, no preheating, temperature, flipping time, and secondary frying. As an example, in Figure 5 In the function 802c "frying" in the evaluation formula 800c of the third step of (b), the material "evaluation formula 800b of the second step" and the temperature "160 degrees" are set. Here, the material "evaluation formula 800b of the second step" shows the result of the evaluation formula 800b of the second step, that is, "coating the bite-sized chicken with starch".

[0071] Figure 7 Another specific example of function 802 is shown. This shows another example of the operation content, start condition, execution parameter, and end condition of function 802a "cut". In addition, regarding function 802 "burn", the operation content and the like are shown. In addition, the start condition is not only the material, but also the prop shown as an external factor.

[0072] Figure 5 (c) indicates that Figure 5 The sequence of images 620 converted by the evaluation formula 800 of (b). The sequence of images 620 is a plurality of images 600 arranged in the order of actions for causing the device 100 or a person to perform a target process. The image 600a "chicken chunks" and the image 600c "starch" correspond to Figure 5 The independent variable of function 802a "cut" in (b), and the independent variable of function 802b "coat". In addition, image 600b "chicken cut into bite-sized pieces", image 600d "bite-sized chicken thinly coated with starch", image 600e "chicken fried at 160°C", image 600f "cooled chicken", and image 600g "chicken fried at 180°C" correspond to Figure 5 (b) The result of function 802a "cut", the result of function 802b "spread", the result of function 802c "fry", the result of function 802d "cool", and the result of function 802e "fry". Each image 600 represents the result of a process, but can also represent the state of an executable process, that is, it can represent the start state of a process, or it can represent the middle of a process.

[0073] Figure 5 (d) indicates that Figure 5 The sequence article 860 converted by the evaluation formula 800 of (b). The sequence article 860 is a plurality of articles 850 arranged in the order of actions in order to make the device 100 or the person perform the target processing. Article 850a "Cut the chicken into bite-sized pieces", article 850b "Coat the cut chicken with starch", article 850c "Fry the chicken coated with starch at 160°C", article 850d "Cool the fried chicken", and article 850e "Fry the cooled chicken at 180°C" correspond to Figure 5 Function 802a of (b) indicates the process content of "cutting", function 802b indicates the process content of "spreading", function 802c indicates the process content of "frying", function 802d indicates the process content of "cooling", and function 802e indicates the process content of "frying". Article 850 indicates the process content.

[0074] like Figure 5As shown in (a) to (d) of FIG. 8 , the function block sequence 120, the evaluation formula 800, the sequence image 620, and the sequence article 860 correspond to each other. Therefore, the correspondence between the function 802 and the function block 110 (hereinafter referred to as the “first correspondence”), the correspondence between the function 802 and the image 600 (hereinafter referred to as the “second correspondence”), and the correspondence between the function 802 and the article 850 (hereinafter referred to as the “third correspondence”) are defined, so that the function block sequence 120, the sequence image 620, and the sequence article 860 can be converted via the evaluation formula 800.

[0075] Figure 8 Show changes Figure 5 The evaluation formula 800f of the first step is the evaluation formula 800f of the part of the evaluation formula 800 of (b). Figure 5 The evaluation formula 800a of the first process in (b) of FIG. 1 similarly includes the "cut" function 802a. However, the independent variable of the "cut" function 802a is changed from "chicken" to "carrot". In addition, the evaluation formulas 800g to 800j of the second process respectively correspond to Figure 5 The evaluation formula 800b of the second process to the evaluation formula 800d of the fifth process in (b) are changed from "chicken" to "carrot" as described above. In this way, by changing the independent variable of the function 802, the evaluation formula 800 can be updated. The update of the evaluation formula 800 can also be reflected in the function block sequence 120, the sequence image 620, and the sequence article 860 after the evaluation formula 800 is converted.

[0076] Fig. 9 and Fig.10 Shown with Figure 5 (a)-(b) Examples of processing with different objectives. Fig. 9 A specific example of the function block sequence 120 in the device control system 1000 is shown. Fig.10 Show Fig. 9 The evaluation formula 800 of the function block sequence 120. Fig. 9 In the function block sequence 120, material 700c "radish (leaves, roots)" is input into the function block 110u of "cut". The function block 110u of "cut" outputs material 700d "radish (leaves)" and material 700g "radish (roots)". Material 700d "radish (leaves)" is input into the function block 110v of "cut into long strips", and the function block 110v of "cut into long strips" outputs material 700e "5 cm radish (leaves)". Material 700e "5 cm radish (leaves)" is input into the function block 110w of "steaming", and the function block 110w of "steaming" outputs material 700f "5 cm radish (leaves) steamed at 60°C for 10 minutes".

[0077] Material 700g "radish (root)" is input into the "cut into quarters" function block 110x, and the "cut into quarters" function block 110x outputs material 700h "2 cm thick, 90 degree quarter-cut radish (root)". Material 700h "2 cm thick, 90 degree quarter-cut radish (root)" is input into the "burn" function block 110y, and the "burn" function block 110y outputs material 700i "2 cm thick, 90 degree quarter-cut radish (root) with a burnt color on the surface".

[0078] exist Fig.10 In the evaluation formula 800, material 810, the evaluation formula 800k of the first process, the evaluation formula 800l of the second process, and the evaluation formula 800m of the third process are arranged in sequence. The evaluation formula 800k of the first process includes function 802f "cut" and function 802g "cut", the evaluation formula 800l of the second process includes function 802h "cut into long strips" and function 802i "cut into quarters", and the evaluation formula 800m of the third process includes function 802j "steam" and function 802k "roast". Function 802f "cut" and function 802g "cut" correspond to Fig. 9 In addition, function 802h "cut into strips", function 802i "cut into quarters", function 802j "steam", and function 802k "burn" correspond to Fig. 9 The function block 110v for "cut into strips", the function block 110x for "cut into quarters", the function block 110w for "steaming", and the function block 110y for "roasting". Fig. 9 , Fig.10 In the case of , the function block sequence 120 and the evaluation formula 800 can be switched. In addition, the evaluation formula 800 can also be switched between the sequence image 620 (not shown) or the sequence text 860 (not shown).

[0079] (3) Structure of equipment control system

[0080] Fig.11 1 shows the configuration of a device control system 1000. The device control system 1000 includes devices 100a to 100c, a user device 150, a sequence generating device 200, a test environment 250, a network 300, an information processing device 400, and a storage device 450. The storage device 450 also includes a sequence DB 440.

[0081] Devices 100a to 100c are, for example, Figure 1 The rice cooker, washing machine, and microwave oven of each device 100 execute Figure 1The first layer and the second layer of the processing are connected to the network 300. In addition to the device 100, the network 300 is also connected with a user device 150, a sequence generation device 200, a test environment 250, an information processing device 400, and a storage device 450. In the network 300, any of wired communication, wireless communication, and a combination of wired communication and wireless communication is performed between these devices.

[0082] The sequence generation device 200 is a device used by a user (hereinafter also referred to as a “creator”) who creates the function block sequence 120 , and is, for example, a computer, a mobile phone, or a tablet terminal.

[0083] The information processing device 400 cooperates with the sequence generation device 200 to create or update the function block sequence 120. In addition, the information processing device 400 performs conversion between the evaluation formula 800, the function block sequence 120, the sequence image 620, and the sequence article 860. The information processing device 400 is, for example, a computer such as a server or a cloud server equipped with a processor, a memory, etc. In addition, the information processing device 400 is also used to perform Figure 1 The fourth layer processes the user application server 132 .

[0084] The storage device 450 is, for example, a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and can store electronic information. The sequence DB 440 stores the function block sequence 120 created or generated by the information processing device 400 .

[0085] The test environment 250 is a device for operating the function block sequence 120 produced by the producer. The test environment 250 receives the function block sequence 120 from the information processing device 400 and executes operations according to the function block sequence 120. The test environment 250 is used to confirm whether the function block sequence 120 produced by the producer has a malfunction.

[0086] The user device 150 is a device used by a user who performs a target process, such as cooking, and is, for example, a computer, a mobile phone, or a tablet terminal. The user device 150 receives a sequence image 620 or a sequence text 860 of a target process from the information processing device 400, and displays the received sequence image 620 or sequence text 860. The user performs the target process by confirming the sequence image 620 or sequence text 860 displayed by the user device 150.

[0087] Furthermore, information processing apparatus 400 transmits a portion of function block sequence 120 executed by device 100 for target processing (hereinafter also referred to as "function block sequence 120") to device 100. Device 100 executes processing according to received function block sequence 120.

[0088] Fig.12 1 shows the structure of the device 100. The device 100 includes a component 102, a communication unit 140, a display unit 142, an operation unit 144, a processing unit 146, and a storage unit 148. The processing unit 146 includes a functional block 110 and a driver 104. As mentioned above, the device 100 is a household electrical appliance such as a rice cooker, a washing machine, or a microwave oven. Each of the components 102, the driver 104, and the functional block 110 is as follows: Figure 1 Although multiple ones are shown, only one is shown here for the sake of clarity of the drawing.

[0089] The communication unit 140 is connected to the network 300, and communicates with the information processing device 400 via the network 300. For example, the communication unit 140 receives information of the function block sequence 120 from the information processing device 400. The function block sequence 120 includes the function block 110 that the device 100 should execute among the function block sequence 120 for the target process. In addition, the storage unit 148 stores the function block 110 executable in the device 100. The processing unit 146 reads the function block 110 stored in the storage unit 148 based on the information of the function block sequence 120 received from the communication unit 140, and causes the component 102 to execute the process according to the function block sequence 120 via the driver 104.

[0090] The display unit 142 displays information from the processing unit 146. The operation unit 144 is an interface capable of receiving input from the user, such as a button. In addition, the display unit 142 and the operation unit 144 may be integrated as a touch panel. The operation unit 144 outputs the received input to the processing unit 146.

[0091] Fig.13 The structure of the sequence generation device 200 is shown. The sequence generation device 200 includes a display unit 202, an operation unit 204, a processing unit 206, a storage unit 208, and a communication unit 210. As mentioned above, the sequence generation device 200 is a computer, a mobile phone, or a tablet terminal. The display unit 202 displays information received from the processing unit 206. The operation unit 204 is an interface capable of receiving input from the producer, such as a button. In addition, the display unit 202 and the operation unit 204 can also be integrated as a touch panel. The operation unit 204 outputs the received input to the processing unit 206.

[0092] The processing unit 206 outputs information to be displayed to the display unit 202, and receives input from the operation unit 204. In addition, the processing unit 206 causes the storage unit 208 to store information, and reads information from the storage unit 208. Furthermore, the processing unit 206 is connected to the network 300 via the communication unit 210, and communicates with the information processing device 400 via the network 300. With such a configuration, the processing unit 206 performs a creation process or an update process of the function block sequence 120 based on the input from the creator while communicating with the information processing device 400.

[0093] Fig.14 4 shows the configuration of the information processing device 400. The information processing device 400 includes a processing unit 406, a storage unit 408, and a communication unit 410. The processing unit 406 includes a receiving unit 420, a main processing unit 422, an evaluation formula conversion unit 424, an image conversion unit 426, an article conversion unit 428, and an inventory management unit 430.

[0094] The processing unit 406 performs processing in the information processing device 400, such as the processing of creating or updating the function block sequence 120, and the conversion processing between the evaluation formula 800, the function block sequence 120, the sequence image 620, and the sequence article 860. The details of the processing in the processing unit 406 will be described later. The storage unit 408 stores information used in the processing unit 406. The communication unit 410 is connected to the network 300, and communicates with the device 100, the user device 150, the sequence generation device 200, the test environment 250, and the storage device 450 via the network 300.

[0095] Fig.15 The configuration of the user device 150 is shown. The user device 150 includes a display unit 152, an operation unit 154, a processing unit 156, a storage unit 158, and a communication unit 160. As described above, the user device 150 is a computer, a mobile phone, or a tablet terminal. The display unit 152 displays information received from the processing unit 156. The operation unit 154 is an interface capable of receiving input from a user, such as a button. In addition, the display unit 152 and the operation unit 154 may be integrated as a touch panel. The operation unit 154 outputs the received input to the processing unit 156.

[0096] The processing unit 156 outputs information to be displayed to the display unit 152, and receives input from the operation unit 154. In addition, the processing unit 156 stores information in the storage unit 158, or reads information from the storage unit 158. Furthermore, the processing unit 156 is connected to the network 300 via the communication unit 160, and communicates with the information processing device 400 via the network 300.

[0097] According to such a configuration, after the operation unit 154 receives the execution instruction of the target process, the processing unit 156 causes the communication unit 160 to transmit the execution instruction to the information processing device 400. In addition, after the communication unit 160 receives information for displaying a screen (hereinafter referred to as "sequence display IF") for the sequence image 620 or the sequence article 860 from the information processing device 400, the processing unit 156 causes the display unit 152 to display the sequence display IF.

[0098] (4) Production of function block sequence

[0099] As described above, the function block sequence 120 communicates with the information processing device 400 through the sequence generation device 200, and is generated in the information processing device 400 according to the operation of the creator on the sequence generation device 200. The processing unit 206 of the sequence generation device 200 causes the display unit 202 to display a screen for generating the function block sequence 120 (hereinafter referred to as "sequence generation IF"). Fig.16 (a) to (c) of FIG. 2 show the sequence generation IF 220 displayed on the display unit 202. Fig.16 In the sequence generation IF 220 of (a), a material display area 222, a function block display area 224, a sequence creation area 226, and an image display area 228 are arranged.

[0100] In the material display area 222, the material 700a "chicken" and the material 700b "starch" are arranged. Other materials 700 may also be arranged in the material display area 222. The material display area 222 may also be arranged as follows. Fig.16 In this case, the material 700 is displayed as a group by type. For example, pork and chicken as meat are displayed as a group, and cabbage and lettuce as vegetables are displayed as a group.

[0101] In the function block display area 224, there are arranged a function block 110p for "cut", a function block 110q for "spread", and a function block 110r for "fry". Other function blocks 110 may also be arranged in the function block display area 224. The function blocks 110 may also be arranged as follows: Fig.16 At this time, the function blocks 110 related to automatic operation and the function blocks 110 related to manual operation are displayed separately. The function blocks 110 related to automatic operation are equivalent to the first type of function blocks 110, and the function blocks 110 related to manual operation are equivalent to the second type of function blocks 110.

[0102] The sequence creation area 226 is an area for creating a function block sequence 120 using the material 700 and the function block 110. The creator operates the operation unit 204 to move the material 700 in the material display area 222 to the sequence creation area 226, and to move the function block 110 in the function block display area 224 to the sequence creation area 226. In addition, the creator operates the operation unit 204 to arrange the function block 110 and the material 700 in the order of action in the sequence creation area 226. At this time, the parameters are input to the function block 110. Fig.15 The processing unit 206 of the sequence generating device 200 transmits information of the functional blocks 110 and the material 700 arranged in the sequence creating area 226 from the communication unit 210 to the information processing device 400 as input information.

[0103] Fig.14 The communication unit 410 of the information processing device 400 receives input information from the sequence generating device 200. The receiving unit 420 receives the information of the function block 110 and the material 700 included in the input information. The main processing unit 422 outputs an update request including the received information of the function block 110 and the material 700 to the sequence DB 440, so that the function block 110 and the material 700 are stored in the sequence DB 440 as the function block sequence 120. The main processing unit 422 receives the stored function block sequence 120 from the sequence DB 440.

[0104] The main processing unit 422 outputs the function block sequence 120 to the evaluation formula conversion unit 424, and requests the evaluation formula conversion unit 424 to convert the function block sequence 120 into the evaluation formula 800. The evaluation formula conversion unit 424 converts the function block sequence 120 into the evaluation formula 800 by converting each function block 110 included in the function block sequence 120 into a function 802 according to the aforementioned first correspondence relationship previously maintained. The evaluation formula conversion unit 424 outputs the evaluation formula 800 to the main processing unit 422.

[0105] The main processing unit 422 receives the evaluation formula 800 from the evaluation formula conversion unit 424. This is equivalent to obtaining the evaluation formula 800 in which one or more functions 802 representing the process executed by the device 100 or a person and having the start condition, execution parameter, and end condition as input variables are arranged. The main processing unit 422 outputs the evaluation formula 800 to the image conversion unit 426, and requests the image conversion unit 426 to convert the evaluation formula 800 into a sequence of images 620. The image conversion unit 426 converts the evaluation formula 800 into a sequence of images 620 by converting each function 802 included in the evaluation formula 800 into an image 600 based on the second correspondence relationship stored in advance. Here, the image conversion unit 426 may also obtain the image 600 corresponding to each function 802 included in the received evaluation formula 800 through Internet search without using the second correspondence relationship, and generate the sequence of images 620 from the obtained images 600. The image conversion unit 426 outputs the sequence of images 620 to the main processing unit 422. The main processing unit 422 causes the communication unit 410 to transmit a display instruction for displaying the sequence image 620 to the sequence generating device 200. Transmitting the display instruction corresponds to displaying information related to each process according to one or more functions 802 included in the acquired evaluation formula 800.

[0106] The communication unit 210 of the sequence generation device 200 receives the display instruction from the information processing device 400. The processing unit 206 receives the display instruction in accordance with the display instruction. Fig.16 The sequence generation IF 220 is updated so that the sequence image 620 is arranged in the image display area 228. The processing unit 206 displays the updated sequence generation IF 220 on the display unit 202. Fig.16 The image display area 228 of FIG. 228 is provided with a sequence of images 620. Here, the images 600 following the image 600e "chicken fried at 160°C" are omitted. In addition, in the image display area 228, auxiliary images 610 indicating the contents of the process of the function 802 may be provided. For example, the function 802a "cut", the function 802b "spread", and the function 802c "fry" are provided.

[0107] The creator can change at least one of the function block 110 and the material 700 shown in the sequence creation area 226 by operating the operation unit 204 of the sequence generation device 200 while observing the sequence image 620 arranged in the image display area 228. After receiving the change content from the operation unit 204, the processing unit 206 transmits the input information including the change content from the communication unit 210 to the information processing device 400.

[0108] Fig.14The communication unit 410 of the information processing device 400 receives input information from the sequence generating device 200. The receiving unit 420 obtains the change content included in the input information. The main processing unit 422 updates the function block sequence 120 stored in the sequence DB 440 by outputting the received change content, that is, the update request including the difference of the function block sequence 120 to the sequence DB 440. The main processing unit 422 receives the updated function block sequence 120 from the sequence DB 440. The subsequent processing is the same as the previous one.

[0109] As a result, the sequence image 620 arranged in the image display area 228 of the sequence generation device 200 is sequentially updated according to the change of the functional block sequence 120. In this way, the functional block sequence 120 is converted and displayed as the sequence image 620, so that the function and operation of the device 100 are displayed in an easily understandable manner. As a result, the creator can easily understand the content of the functional block sequence 120 by checking the sequence image 620.

[0110] The main processing unit 422 may include the function block sequence 120 in the display instruction. In this case, the user device 150 updates the display of the sequence creation area 226 in the sequence generation IF 220 based on the function block sequence 120 included in the display instruction.

[0111] This structure can be realized in hardware by a CPU (Central Processing Unit), memory, or other LSI (Large Scale Integration) of any computer, and in software by a program loaded into a memory, etc., but here, a block of functions realized by their cooperation is drawn. Therefore, it should be understood by those skilled in the art that these functional blocks can be realized in various forms only by hardware or by a combination of hardware and software.

[0112] Fig.17 1 is a timing diagram showing the display steps of the device control system 1000. The main processing unit 422 receives a display request of the sequence generation IF 220 from the sequence generation device 200 (S10). The main processing unit 422 sends the display instruction of the sequence generation IF 220 to the sequence generation device 200 (S12). The sequence generation device 200 displays the sequence generation IF 220 (S14). The sequence generation device 200 receives an input from the producer (S16). The input here is set as an update of the function block sequence 120. The sequence generation device 200 sends the input information to the main processing unit 422 (S18). The main processing unit 422 generates an update request for the function block sequence 120 (Sq) (S20), and sends the Sq difference to the sequence DB 440 (S22). The sequence DB 440 updates Sq (S24) and sends Sq to the main processing unit 422 (S26).

[0113] The main processing unit 422 generates a conversion request for the evaluation formula 800 (S28), and outputs Sq to the evaluation formula conversion unit 424 (S30). The evaluation formula conversion unit 424 performs the conversion to the evaluation formula 800 (S32), and outputs the evaluation formula 800 to the main processing unit 422 (S34). The main processing unit 422 generates a conversion request to the sequence image 620 (S36), and outputs the evaluation formula 800 to the image conversion unit 426 (S38). The image conversion unit 426 performs the conversion to the sequence image 620 (S40), and outputs the sequence image 620 to the main processing unit 422 (S42). The main processing unit 422 sends a display instruction to the sequence generation device 200 (S44). The sequence generation device 200 updates and displays the sequence generation IF220 (S46).

[0114] (5) Update of function block sequence

[0115] Here, it is assumed that a user who uses the created function block sequence 120 updates the function block sequence 120 according to the usage environment at home by using the user device 150 . Fig.11 The sequence DB 440 in the storage device 450 stores the produced function block sequence 120. In response to a request from the user device 150, the main processing unit 422 of the information processing device 400 obtains the function block sequence 120 from the sequence DB 440. The main processing unit 422 receives the evaluation formula 800 corresponding to the function block sequence 120 from the evaluation formula conversion unit 424 and receives the sequence image 620 corresponding to the evaluation formula 800 from the image conversion unit 426 by performing the same processing as before.

[0116] The main processing unit 422 outputs the evaluation formula 800 to the article conversion unit 428, and requests the article conversion unit 428 to convert the evaluation formula 800 into a sequence article 860. The article conversion unit 428 converts the evaluation formula 800 into a sequence article 860 by converting each function 802 included in the evaluation formula 800 into an article 850 according to the third correspondence relationship previously maintained. Here, the article conversion unit 428 may also convert the article 850 corresponding to each function 802 included in the received evaluation formula 800 to an external server via the Internet without using the third correspondence relationship, and generate a sequence article 860 from the converted article 850. The article conversion unit 428 outputs the sequence article 860 to the main processing unit 422.

[0117] The inventory management unit 430 communicates with the device 100 owned by the user, such as a refrigerator, via the network 300. The refrigerator is provided with a sensor (camera) capable of photographing the interior, for example, and the sensor photographs the materials stored in the refrigerator. The refrigerator sends the photographed image (hereinafter referred to as "photographed image") to the information processing device 400 via the network 300. The inventory management unit 430 identifies the material 700 stored in the refrigerator by performing image recognition processing on the photographed image. The inventory information can also be input by the user via the user device 150. The inventory management unit 430 obtains the image 600 corresponding to the identified material 700. The image 600 can be stored in the inventory management unit 430 or retrieved through the Internet. In addition, an image generated by an image generation AI (Artificial Intelligence) can also be used as the image 600. The inventory management unit 430 outputs the image 600 corresponding to the material 700 to the main processing unit 422 as inventory information. The main processing unit 422 causes the communication unit 410 to transmit a display instruction for displaying the sequence images 620 , the sequence texts 860 , and the stock information to the user device 150 .

[0118] Fig.15 The communication unit 160 of the user device 150 receives the display instruction from the information processing device 400. The processing unit 156 generates the sequence display IF 900 according to the display instruction, and displays the sequence display IF 900 on the display unit 152. Fig.18 (a) to (c) show the sequence display IF 900 displayed on the display unit 152 . Fig.18 In the sequence display IF 900 of (a), a material display area 902, an auxiliary image display area 904, an image display area 906, and an article display area 90 are arranged.

[0119] In the material display area 902, an image 600a of "chicken", an image 600c of "starch", and an image 600h of "flour" are arranged. When multiple types of materials 700 can be selected, if an image 600 in the material display area 902 is selected, Fig.18 (b) of the screen. Selectable ingredients 700, such as "chicken" and "beef", and "chicken" of the selected ingredient 700 are displayed. In addition, when multiple ingredients 700 are selectable, images 600 of the ingredients 700 selected based on the criteria of "ingredients in the refrigerator", "ingredients used recently", "in season", etc. may also be displayed. When "ingredients in the refrigerator" is selected, images 600 of the ingredients 700 included in the inventory information are displayed.

[0120] The amount of the material 700 used in the function block sequence 120 or the amount of the material 700 included in the stock information may be reflected in the image 600 of the material 700 displayed in the material display area 902. This process is performed in the user device 150 or the information processing device 400. Fig.19 (a) to (b) show the display of the material display area 902 . Fig.19 (a) shows a display of a case where the amount of the material 700 "chicken" included in the stock information is greater than the amount of the material 700 "chicken" used in the functional block sequence 120. For example, it is assumed that the amount of the material 700 "chicken" included in the stock information is "500g", and the amount of the material 700 "chicken" used in the functional block sequence 120 is "200g". In the material display area 902, an image 600i "chicken" corresponding to the "500g" of the material 700 "chicken" included in the stock information is arranged. In addition, the used portion 602 for displaying the portion corresponding to the "200g" of the material 700 "chicken" used in the functional block sequence 120 is overlapped and arranged on the image 600i "chicken".

[0121] Fig.19 (b) shows the display when the amount of material 700 "chicken" included in the inventory information is the same as the amount of material 700 "chicken" used in the functional block sequence 120. For example, assume that the amount of material 700 "chicken" included in the inventory information is also "200g". In the material display area 902, an image 600j "chicken" corresponding to the material 700 "chicken" of "200g" included in the inventory information is configured. Return Fig.18 (a).

[0122] In the auxiliary image display area 904, auxiliary image 610d "fry" and auxiliary image 610e "use oil" are arranged as available auxiliary images 610. The auxiliary images 610 displayed in the auxiliary image display area 904 are not limited to these. As described above, the auxiliary image 610 is an image that represents the content of the process of the function 802.

[0123] The sequence of images 620 is arranged in the image display area 906. Here, the image 600 after the image 600e "chicken fried at 160°C" is omitted. In addition, the auxiliary image 610 may also be arranged in the image display area 228. The auxiliary image 610a "cut" may also be, for example, Fig.18 A sequence article 860 is arranged in the article display area 908. The sequence article 860 includes a plurality of articles 850. Such an article 850 indicates the content of a process.

[0124] The user confirms the sequence images 620 arranged in the image display area 906 or the sequence articles 860 arranged in the article display area 908. When the user wants to update the sequence images 620, he / she operates the operation unit 154 to change the arrangement of the images 600 or the auxiliary images 610 in the image display area 906. For example, the image 600h "flour" arranged in the material display area 902 can be replaced with the image 600c "starch" arranged in the image display area 906. In addition, the user selects the image 600 in the image display area 906 by operating the operation unit 154 and changes the parameters of the image 600.

[0125] The processing unit 156 receives the configuration change of the image 600 or the auxiliary image 610, or the parameter change of the image 600 from the operation unit 154. The processing unit 156 transmits the sequence image 620 reflecting these changes (hereinafter referred to as "updated sequence image 620") from the communication unit 160 to the information processing device 400 as update information.

[0126] The communication unit 410 of the information processing device 400 receives the update information from the user device 150. The receiving unit 420 obtains the updated sequence image 620 from the update information. The main processing unit 422 outputs the updated sequence image 620 to the image conversion unit 426. The image conversion unit 426 converts the updated sequence image 620 into the evaluation formula 800 using the second correspondence. The evaluation formula 800 is equivalent to the evaluation formula 800 updated in accordance with the update of the sequence image 620. The image conversion unit 426 outputs the evaluation formula 800 to the main processing unit 422.

[0127] The main processing unit 422 outputs the received evaluation formula 800 to the evaluation formula conversion unit 424. The evaluation formula conversion unit 424 converts the evaluation formula 800 into a function block sequence 120 using the first correspondence. The function block sequence 120 is equivalent to the function block sequence 120 updated in accordance with the update of the sequence image 620. The evaluation formula conversion unit 424 outputs the function block sequence 120 to the main processing unit 422. The main processing unit 422 updates the function block sequence 120 stored in the sequence DB 440 by outputting the received function block sequence 120 to the sequence DB 440.

[0128] The main processing unit 422 outputs the received evaluation formula 800 to the article conversion unit 428. The article conversion unit 428 converts the evaluation formula 800 into a sequence article 860 using the third correspondence. The sequence article 860 is equivalent to the sequence article 860 updated in accordance with the update of the sequence image 620. The article conversion unit 428 outputs the sequence article 860 to the main processing unit 422. In addition, the main processing unit 422 receives inventory information from the inventory management unit 430. The main processing unit 422 sends the sequence image 620, the sequence article 860, and a display instruction for displaying the inventory information from the communication unit 410 to the user device 150.

[0129] The communication unit 160 of the user device 150 receives the display instruction from the information processing device 400. The processing unit 156 generates the sequence display IF 900 according to the display instruction, and displays the sequence display IF 900 on the display unit 152. As a result, the change is reflected in the sequence display IF 900.

[0130] The main processing unit 422 may include the function block sequence 120 in the display instruction. In this case, the user device 150 arranges the function block sequence 120 in the sequence display IF 900, so that the function block sequence 120 is displayed on the display unit 152.

[0131] Fig. 20 1 is a timing chart showing the display steps of the device control system 1000. The main processing unit 422 sends a display instruction of the sequence display IF900 to the user device 150 (S100). The user device 150 displays the sequence display IF900 (S102). The user device 150 receives an update made by the user (S104). The user device 150 sends the update information to the main processing unit 422 (S106).

[0132] The main processing unit 422 generates a conversion request to the evaluation formula 800 (S108), and outputs the sequence image 620 to the image conversion unit 426 (S110). The image conversion unit 426 performs the conversion to the evaluation formula 800 (S112), and outputs the evaluation formula 800 to the main processing unit 422 (S114). The main processing unit 422 generates a conversion request to the function block sequence 120 (S116), and outputs the evaluation formula 800 to the evaluation formula conversion unit 424 (S118). The evaluation formula conversion unit 424 performs the conversion to the function block sequence 120 (S120), and outputs the function block sequence 120 (Sq) to the main processing unit 422 (S122).

[0133] The main processing unit 422 generates an update request for the function block sequence 120 (S124), and sends Sq to the sequence DB 440 (S126). The sequence DB 440 updates Sq (S128), and sends Sq to the main processing unit 422 (S130). The main processing unit 422 generates a conversion request to the sequence article 860 (S132), and outputs the evaluation formula 800 to the article conversion unit 428 (S134). The article conversion unit 428 performs the conversion to the sequence article 860 (S136), and outputs the sequence article 860 to the main processing unit 422 (S138). The main processing unit 422 sends a display instruction to the user device 150 (S140). The user device 150 updates the sequence display IF 900 and displays it (S142).

[0134] (6) Execution of target processing

[0135] Here, it is assumed that a user executes a target process while using the user device 150. An example of a target process is Figure 5 The cooking of "fried chicken" shown in (a) to (d) of the present invention. The user inputs the execution of the target process into the user device 150 by operating the operation unit 154 of the user device 150. The user device 150 transmits an instruction signal for instructing the execution of the target process to the information processing device 400 based on the input from the user. The instruction signal includes, for example, information indicating the content of the target process.

[0136] The communication unit 410 of the information processing device 400 receives the instruction signal from the user device 150. The receiving unit 420 receives the instruction signal and extracts information indicating the content of the target processing from the instruction signal. The main processing unit 422 sends the output requirement of the function block sequence 120 corresponding to the content of the target processing to the sequence DB 440. The sequence DB 440 sends the function block sequence 120 that meets the output requirement to the main processing unit 422. The main processing unit 422 receives the function block sequence 120 from the sequence DB 440. The function block sequence 120 is, for example, Figure 5 1. Function block sequence 120 for a "fried chicken" dish as shown in (a).

[0137] The main processing unit 422 checks the flags of the function blocks 110 included in the function block sequence 120 received from the sequence DB 440, and selects the function block 110 having the flag indicating the first type of function block 110. The main processing unit 422 transmits information related to the selected function block 110 from the communication unit 410 to the device 100 as information of the function block sequence 120 for the device 100.

[0138] On the other hand, the main processing unit 422 outputs the function block sequence 120 received from the sequence DB 440 to the evaluation formula conversion unit 424, and requests the evaluation formula conversion unit 424 to convert the function block sequence 120 into the evaluation formula 800. The evaluation formula conversion unit 424 converts the function block sequence 120 into the evaluation formula 800. The evaluation formula conversion unit 424 outputs the evaluation formula 800 to the main processing unit 422.

[0139] The main processing unit 422 outputs the evaluation formula 800 to the image conversion unit 426, and requests the image conversion unit 426 to convert each function 802 included in the evaluation formula 800 into an image 600, and generate a dynamic image corresponding to the evaluation formula 800. The image conversion unit 426 converts each function 802 included in the evaluation formula 800 into an image 600 based on the second correspondence relationship maintained in advance. In addition, the image conversion unit 426 can also obtain the image 600 corresponding to each function 802 included in the received evaluation formula 800 through Internet search without using the second correspondence relationship. In addition, the image conversion unit 426 generates a dynamic image by connecting multiple images 600. Such a dynamic image, for example, represents the order of actions of multiple processes. The image conversion unit 426 can also perform interpolation processing between the images 600 when connecting the images 600. The image conversion unit 426 outputs each image 600 and the dynamic image to the main processing unit 422.

[0140] The main processing unit 422 outputs the evaluation formula 800 to the article conversion unit 428, thereby requesting the article conversion unit 428 to convert each function 802 included in the evaluation formula 800 into an article 850. The article conversion unit 428 converts each function 802 included in the evaluation formula 800 into an article 850 based on the third correspondence relationship previously maintained. The article conversion unit 428 outputs each article 850 to the main processing unit 422. Through these processes, the main processing unit 422 obtains a dynamic image and a plurality of images 600, and a plurality of articles 850. The main processing unit 422 causes the display instruction for displaying the dynamic image to be sent from the communication unit 410 to the user device 150. The display instruction includes the dynamic image, but may also include the article 850.

[0141] The communication unit 160 of the user device 150 receives the display instruction from the information processing device 400. The processing unit 156 displays the sequence display IF 950a on the display unit 152 according to the display instruction. Fig.21 (a) to (c) show the sequence display IF 950 displayed on the display unit 152 . Fig.21(a) shows a sequence display IF950a. In the sequence display IF950a, an image 600k (dynamic image), an article 850f, and an article 850g are arranged. Articles 850f and 850g are, for example, article 850 of the first process, but they can also be omitted. The user can confirm the summary of the steps of the actions of multiple processes by replaying the image 600k (dynamic image).

[0142] When a certain period of time has passed since the sequence display IF 950 a was displayed, or when the operation unit 154 receives a subsequent instruction from the user, the communication unit 160 of the user device 150 transmits an update signal for requesting update of the sequence display IF 950 to the information processing device 400 .

[0143] The communication unit 410 of the information processing device 400 receives the update signal from the user device 150. After the receiving unit 420 receives the update signal, the main processing unit 422 selects the image 600 and the article 850 of the first process from the acquired plurality of images 600 and the plurality of articles 850. The main processing unit 422 transmits a display instruction for displaying the image 600 and the article 850 of the first process from the communication unit 410 to the user device 150.

[0144] The communication unit 160 of the user device 150 receives the display instruction from the information processing device 400. The processing unit 156 displays the sequence display IF 950b on the display unit 152 according to the display instruction. Fig.21 (b) represents the sequence display IF950b. The sequence display IF950b is configured with the image 6001 of the first process, the article 850f of the first process, and the article 850g of the first process. In addition, the sequence display IF950b can also be configured with a one point advice 852a such as "cut into large pieces, the appearance is closer to the product". This one point advice 852a is a suggestion from the perspective of "support for implementing the recipe to the minimum extent possible" when the user is a novice. For example, it can also be the mixing method of seasonings. In addition, when the user is a skilled person, the one point advice 852a can also be a suggestion from the perspective of "further improving the level". For example, adding a small amount of special seasoning, etc.

[0145] In order to display such one suggestion 852a, the storage unit 408 of the information processing device 400 stores the user's level, and the main processing unit 422 selects the content of the one suggestion 852a by referring to the level stored in the storage unit 408. The main processing unit 422 includes the one suggestion 852a of the selected content in the display instruction.

[0146] When a certain period of time has passed since the sequence display IF 950 b was displayed, or when the operation unit 154 receives a subsequent instruction from the user, the communication unit 160 of the user device 150 transmits an update signal to the information processing device 400 .

[0147] The communication unit 410 of the information processing device 400 receives the update signal from the user device 150. After the receiving unit 420 receives the update signal, the main processing unit 422 selects the image 600 and the article 850 of the second process from the acquired plurality of images 600 and the plurality of articles 850. The subsequent processing is as described above, and the image 600 and the article 850 of the second process are arranged in the sequence display IF950 displayed on the display unit 152 of the user device 150. And by repeating the same processing, the sequence display IF950 in which the image 600 and the article 850 of the third process are arranged, and the sequence display IF950 in which the image 600 and the article 850 of the fourth process are arranged are sequentially displayed on the display unit 152 of the user device 150.

[0148] exist Fig.21 In the sequence display IF950c shown in (c), an image 600m and an article 850h of the third process are configured. In addition, in the sequence display IF950c, a suggestion 852b corresponding to the third process is also configured. In addition, in the sequence display IF950c, a cooking process 854a for notifying the progress of the third process is also displayed. The cooking process 854a, as an example, includes a progress bar. The progress bar is a horizontal bar that is gradually filled with color as the third process progresses. That is, the remaining time of the third process is displayed by the progress bar. In addition, the cooking process 854a can also display the remaining time of the third process as text.

[0149] The main processing unit 422 of the information processing device 400 monitors the remaining time of the third step by measuring the time after the start of the third step, and transmits the remaining time to the user device 150 via the communication unit 410. The processing unit 156 of the user device 150 updates the display of the cooking process 854a based on the remaining time received by the communication unit 160. The update of the display of the cooking process 854a includes, for example, filling the color of the progress bar or changing the text "5min". The update of the display of the cooking process 854a can also be performed in the main processing unit 422.

[0150] The cooking process 854a may also display "the changing conditions during cooking" such as the temperature in the pot. In this case, the temperature change is displayed in color, or the process of water boiling is represented in an image. In addition, the numerical value of the changing parameter may be displayed. The main processing unit 422 of the information processing device 400 sends the temperature received from the temperature sensor provided in the device 100 to the user device 150 via the communication unit 410. The processing unit 156 of the user device 150 updates the display of the cooking process 854a based on the temperature received in the communication unit 160. The cooking process 854a may also display the change of parameters other than temperature.

[0151] Up to this point, the IF 950 is displayed in the order of each process display. However, the display on the display unit 152 is not limited to this. Fig. 22 The sequence display IF960 displayed on the display unit 152 is shown. In the sequence display IF960, images 600 and articles 850 corresponding to each of a plurality of processes, for example, three processes are arranged. In the horizontal center portion of the sequence display IF960, an image 600o and an article 850j corresponding to the current process are arranged. In addition, a suggestion 852c corresponding to the current process is arranged below the article 850j. In the left portion of the sequence display IF960, an image 600n and an article 850i corresponding to the previous process are arranged. In the right portion of the sequence display IF960, an image 600p and an article 850k corresponding to the next process are arranged. In the sequence display IF960, the cooking process 854b, the cooking process 854c and Fig.21 The cooking process 854a of (c) is displayed in the same manner. Such a sequence display IF 960 is also generated in the main processing unit 422 of the information processing device 400 and transmitted to the user device 150.

[0152] (7) Modification

[0153] When the user executes a target process, the image or moving image displayed on the display unit 152 of the user device 150 may be as follows in order to improve the convenience of the user.

[0154] Fig.23(a) to (c) show an example of an image 630 displayed by the display unit 152. Images 630a to 630c are dynamic images of cooking. Images 630a to 630c can be pre-stored in the storage unit 408 of the information processing device 400 in a manner corresponding to the function 802, or can be retrieved from the Internet. In addition, the main processing unit 422 or the image conversion unit 426 can also obtain the image of the previous process and the image of the subsequent process from the storage unit 408 or the Internet, and generate a dynamic image by interpolating between the image of the previous process and the image of the subsequent process. The information processing device 400 sends a display instruction including the image 630 to the user device 150.

[0155] The user can understand the outline of the dish, confirm the cooking utensils to be prepared, and confirm the steps in cooking by displaying the image 630 on the display unit 152 of the user device 150. The outline of the dish includes, for example, whether it is boiled or roasted, and the difficulty of the manual part: can I do it / easy / time-consuming, etc. The cooking utensils to be prepared include, for example, the type and size of the cutting board, bowl, plate, etc.

[0156] Image 630 can be any of "a short animation showing only the prominent parts", "an animation showing the whole cooking process in real time", or "an animation showing only the complicated parts of cooking". In addition, image 630 can also have playback functions such as "repeated playback of important parts" and "synchronous playback with the automatic cooking device (explaining the manual part before the manual section, etc.)". Image 630 can also be processed to make it look delicious and easy to understand the production method. For example, "brightness and color tone adjustment" and "overlapping additional instructions".

[0157] In the images 630a and 630c, the cooking scene is generated from an objective viewpoint. In this case, "CG (Computer Graphics) models and motion data", "combination of existing photos", "visualization of tableware and cooking utensils held by the user" can be used. In the second image 630b, the cooking scene is generated from a subjective viewpoint. In this case, "generating a VR (Virtual Reality)-style user viewpoint image" can be used.

[0158] The user's preferences are registered in the storage unit 408 of the information processing device 400, and the main processing unit 422 can also change the image 600 and the article 850 according to the preferences. For example, if the user is a fan of a star, the image 630 of the idol cooking or the article 850 explaining in the idol's voice tone can be displayed. In addition, the display ratio of the image 600 or the article 850 can be changed according to the user's preference. Such a service can be only for the users of the fan club.

[0159] Fig.24 (a) to (e) of FIG. 1 show an example of an image 630 displayed on the display unit 152. When the image 630 is expressed using CG, the direction of the image 630 is changed according to the situation. Fig.24 As shown in (a), in order to make the user see the pressing method of the food and the holding method of the knife, the image 630d from the marked position is used. Fig.24 (b) Fig.24 As shown in (c), when the cutting angle is to be seen, the enlarged images 630e and 630f are displayed. Fig.24 In order to see the cooking process as shown in (d), use the image 630g from above. Fig.24 In the case of a subjective viewpoint such as (e) of FIG. 8 , an image 630h that also shows the hand of the creator is used. The image conversion unit 426 obtains these images 630 by converting the function 802 included in the evaluation formula 800 .

[0160] Fig.25 (a) to (e) show an example of an image 630 displayed on the display unit 152. Fig.25 In the image 630i of (a), the state of the cooking appliance is shown by CG-style expression such as semi-transparency and partial cutout.

[0161] exist Fig.25 In the image 630j shown in (b), it is difficult to know which part of the displayed process is the change point. Therefore, the main processing unit 422 may Fig.25 As in the image 630k of (c), the contour display 632 is shown at the change point, and the annotation is shown at the same time. The change point in the new process can also be highlighted. The main processing unit 422 generates the image 630k by superimposing the contour display 632 on the image 630e converted by the evaluation formula conversion unit 424 according to the function 802.

[0162] The information processing device 400 may also change the display in the portion where the manual mark is turned on. Fig.25 The image 6301 of (d) is a display when automatic cooking is performed, for example, showing an image of cooking in a pot. Fig.25 Image 630m of (e) is displayed when manual cooking is performed, and a tutorial is also shown at the bottom of the screen. In order to switch image 630 according to automatic cooking or manual cooking, a mark set in function block 110 is used. When evaluation formula conversion unit 424 converts function block 110 into function 802, the mark information is retained. When image conversion unit 426 converts function 802 of evaluation formula 800 into image 630, image 630 is determined with reference to the mark information.

[0163] Fig.26(a) to (b) show an example of an image 630 displayed on the display unit 152. In the image 630n and the image 630o, cooking performed by the cooking device is anthropomorphized and displayed. The main processing unit 422 or the image conversion unit 426 generates an animation such that the pot does not cook automatically, but a character anthropomorphized by the device cooks as the image 630n and the image 630o. Since CG is used for the character, the character can be replaced with arbitrary data.

[0164] Fig. 27 (a) to (b) show the appearance of the user device 150 . Fig. 27 The user device 150a shown in (a) is, for example, a smart phone. Fig. 27 The user device 150b shown in (b) is, for example, a television receiver. Since a television receiver is larger than a mobile phone, the size of the display unit 152b of the user device 150b is larger than the size of the display unit 152a of the user device 150a.

[0165] The main processing unit 422 acquires information related to the size of the display unit 152 in the user device 150. The information related to the size shows the size of the display unit 152. In addition, the classification such as large, medium, and small is predefined according to the size, and the information related to the size may also show any one of large, medium, and small. The classification of the size is not limited to large, medium, and small. The main processing unit 422 outputs the information related to the size to the image conversion unit 426. The image conversion unit 426 changes the image 630 according to the information related to the size.

[0166] Fig. 27 (a) shows an image 630p with a small size. The image 630p is a detailed image 630 of only the target portion. The image 630p is displayed on the display unit 152a of the user device 150a. Fig. 27 (b) shows an image 630q when the size is medium or large. The image 630q is an image 630 that also includes the surroundings of the object portion. The image 630q is displayed on the display unit 152b of the user device 150b.

[0167] According to the present embodiment, information related to a process is displayed for each process according to one or more functions 802 included in the evaluation formula 800, so that the information related to the process can be displayed in various forms. In addition, the information related to the process is displayed in various forms, so that the function and action of the device 100 can be displayed easily and clearly. In addition, conversion can be performed between each of the function block sequence 120, the sequence image 620, and the sequence article 860 and the evaluation formula 800, so the conversion between the function block sequence 120, the sequence image 620, and the sequence article 860 can be performed via the evaluation formula 800. In addition, conversion between the function block sequence 120, the sequence image 620, and the sequence article 860 is performed via the evaluation formula 800, so the change of form can be easily performed.

[0168] In addition, the format conversion is easy, so the sequence image 620 or sequence article 860 corresponding to the function block sequence 120 can be displayed. In addition, the sequence image 620 or sequence article 860 corresponding to the function block sequence 120 is displayed, so the function and action of the device 100 can be easily displayed. In addition, as information related to the process, the image 600 showing the result of the process is displayed, so the function and action of the device 100 can be easily displayed. In addition, as information related to the process, the image 600 showing the middle of the process is displayed, so the function and action of the device 100 can be easily displayed. In addition, as information related to the process, the image 600 showing the state in which the process can be executed is displayed, so the function and action of the device 100 can be easily displayed. In addition, as information related to the process, the article 850 showing the content of the process is displayed, so the function and action of the device 100 can be easily displayed. In addition, a dynamic image showing the steps of the actions of multiple processes is displayed, so the steps of the entire multiple processes can be easily notified. Furthermore, the function block sequence 120 converted by the evaluation formula 800 is displayed, and thus various forms of display can be performed.

[0169] The summary of one scheme of the present disclosure is as follows. The display method of one scheme of the present disclosure includes: the step of obtaining an evaluation formula (800) of a function (802) with a start condition, an execution parameter, and an end condition as input variables, which is arranged with one or more functions (802) representing processes performed by a device (100) or a person; and the step of displaying information related to each process according to one or more functions (802) included in the obtained evaluation formula (800).

[0170] The information related to the process may be an image (600) or a dynamic image showing the result of the process.

[0171] The information related to the process may be an image (600) or a dynamic image showing the intermediate process of the process.

[0172] The information related to the process may be an image (600) or a dynamic image showing a state in which the process can be executed.

[0173] The information related to the process may be a text (850) indicating the content of the process.

[0174] In the step of displaying information related to each process, a dynamic image showing actions related to a plurality of processes may be displayed.

[0175] The step of displaying process-related information for each process may also display a block sequence (120) in which blocks (110) specified as functional units executable by the device (100) or blocks (110) specified as action units to be performed by a person are arranged in order of action.

[0176] Another embodiment of the present disclosure is an information processing device. The device includes: a first processing unit that obtains an evaluation formula (800) that arranges one or more functions (802) representing processes executed by a device (100) or a person and takes a start condition, an execution parameter, and an end condition as input variables; and a second processing unit that displays information related to each process according to the one or more functions (802) included in the obtained evaluation formula (800).

[0177] The present disclosure has been described above based on the embodiments. These embodiments are for illustration only, and those skilled in the art should understand that various modifications can be implemented for the combination of these components or processing steps, and such modifications are also within the scope of the present disclosure.

[0178] The image 600 of this embodiment represents a state in which a process can be performed, or an intermediate process of a process, or a result of a process. However, this is not limited to this, and for example, the image 600 may also be a dynamic image. The dynamic image may also represent a state in which a process can be performed, or an intermediate process of a process, or a result of a process. In the processing of the image 600 described above, the image 600 may be replaced by a dynamic image. According to this modified example, the degree of freedom of the configuration may be increased.

[0179] In the updating of the function block sequence (5) of the present embodiment, the user operates the operation unit 154 of the user device 150 to change the blocks arranged in the image display area 906 ( Fig.18) of the sequence image 620, thereby updating the sequence image 620. The updated sequence image 620 is sent from the user device 150 to the information processing device 400 as update information. The image conversion unit 426 of the information processing device 400 converts the updated sequence image 620 into the evaluation formula 800 using the second correspondence relationship, and the evaluation formula conversion unit 424 converts the evaluation formula 800 into the function block sequence 120 using the first correspondence relationship.

[0180] However, the present invention is not limited to this. For example, in the stage (4) of creating a function block sequence, the information processing device 400 may receive image data, the image conversion unit 426 may convert the image data into the evaluation formula 800, and the evaluation formula conversion unit 424 may convert the evaluation formula 800 into the function block sequence 120. Here, the image data is read into an application of the user device 150 as a QR code (registered trademark), for example, and is sent from the user device 150 to the information processing device 400. According to this modification, the function block sequence 120 is created based on the image data, so that the convenience of the user can be improved.

[0181] In addition, in the stage (4) of creating a function block sequence, after the information processing device 400 obtains the information of the usable device 100, the main processing unit 422 creates a function block sequence 120 corresponding to the usable device 100. In this case, the storage unit 408 stores the correspondence between the usable device 100 and the function block sequence 120 in advance. In addition, when there are multiple function block sequences 120 corresponding to the usable device 100, the user can select one function block sequence 120. According to this modification, the function block sequence 120 is created based on the information of the usable device 100, so that the convenience of the user can be improved.

[0182] Industrial Availability

[0183] According to the present disclosure, the functions and operations of a device can be displayed in an easily understandable manner.

[0184] Description of Reference Numerals

[0185] 100 device, 102 component, 104 driver, 110 function block, 120 function block sequence, 130 platform server, 132 user application server, 140 communication unit, 142 display unit, 144 operation unit, 146 processing unit, 148 storage unit, 150 user device, 152 display unit, 154 operation unit, 156 processing unit, 158 storage unit, 160 communication unit, 200 sequence generation device, 202 display unit, 204 operation unit, 206 processing unit, 208 storage unit, 210 communication unit, 220 sequence generation IF, 222 material display area, 224 function block display area, 226 sequence production area, 228 image display area, 250 test environment, 300 network, 400 information processing device, 406 processing unit, 40 8 storage unit, 410 communication unit, 420 receiving unit, 422 main processing unit (first processing unit, second processing unit), 424 evaluation formula conversion unit, 426 image conversion unit, 428 article conversion unit, 430 inventory management unit, 440 sequence DB, 450 storage device, 600 image, 602 usage part, 610 auxiliary image, 620 sequence image, 630 image, 632 contour display, 700 material, 800 evaluation formula, 802 function, 810 material, 850 article, 852 a suggestion, 860 sequence article, 900 sequence display IF, 902 material display area, 904 auxiliary image display area, 906 image display area, 908 article display area, 950, 960 sequence display IF, 1000 equipment control system.

Claims

1. A display method, comprising: A step of obtaining an evaluation formula, wherein the evaluation formula is an evaluation formula in which one or more functions representing processes performed by equipment or people are arranged and which have a start condition, an execution parameter, and an end condition as input variables; and A step of displaying information related to the process for each of the processes based on one or more functions included in the acquired evaluation formula.

2. The display method according to claim 1, The information related to the process is an image or a moving image showing a result of the process.

3. The display method according to claim 1, The information related to the process is an image or a moving image showing the progress of the process.

4. The display method according to claim 1, The information related to the process is an image or a moving image showing a state in which the process can be executed.

5. The display method according to claim 1, The information related to the process is a text indicating the content of the process.

6. The display method according to claim 1, In the step of displaying information related to the steps for each of the steps, a moving image showing a sequence of operations of a plurality of the steps is displayed.

7. The display method according to claim 1, In the step of displaying information related to the process for each of the processes, a block sequence is displayed, wherein blocks specified in functional units executable by the device or blocks specified in action units to be performed by the person are arranged in action order.

8. An information processing device, comprising: a first processing unit that obtains an evaluation formula, wherein the evaluation formula is an evaluation formula in which one or more functions representing a process performed by a device or a person are arranged and which have a start condition, an execution parameter, and an end condition as input variables; as well as The second processing unit displays information related to the process for each of the processes based on one or more functions included in the acquired evaluation formula.

9. A program for causing a computer to execute: A step of obtaining an evaluation formula, wherein the evaluation formula is an evaluation formula in which one or more functions representing processes performed by equipment or people are arranged and which have a start condition, an execution parameter, and an end condition as input variables; and A step of displaying information related to the process for each of the processes based on one or more functions included in the acquired evaluation formula.

Images