Execution method, program and robot control system

Abstract

To easily create a program for allowing a robot to execute predetermined work.SOLUTION: An execution method for allowing a robot 1 to execute predetermined work, includes the steps of: allowing an artificial intelligence model to learn an execution command sentence example EI for commanding execution of the predetermined work and a combination example of execution units EP for executing the predetermined work corresponding to the execution command sentence example EI; inputting to the artificial intelligence model an execution command sentence to be commanded to the robot 1 so as to execute the predetermined work; acquiring an application program AP configured by combination of the execution units generated by the artificial intelligence model after input of the execution command sentence; and allowing the robot 1 to execute the predetermined work by allowing the robot 1 to execute the application program AP.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a method for causing a robot to execute a predetermined operation, a program for causing a computer to execute this method, and a robot control system for causing a robot to execute a predetermined operation.

Background Art

[0002] In recent years, in order to solve the labor shortage, etc., it has been proposed to have a robot perform a predetermined operation that has hitherto been carried out manually. For example, a robot for guiding visitors in a store has been proposed (see Patent Document 1). This robot can acquire the information of the visiting user and perform corresponding customer service so as to be able to grasp the information of the visiting user and respond to customer service.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional robot as described above, a user has to devise and create a program (a combination of execution units such as instructions, functions, and subroutines) for causing the robot to execute a predetermined operation. For this reason, in the conventional robot, only a user having advanced knowledge about robots and programming can cause the robot to perform a desired operation. That is, a user who does not have advanced knowledge about robots and programming cannot cause the robot to perform a desired operation.

[0005] In addition, devising and creating a program for causing a robot to execute a predetermined operation imposes a great workload and cost burden on the user.

[0006] Therefore, an object of the present invention is to easily create a program for causing a robot to execute a predetermined operation and cause the robot to execute the program.

Means for Solving the Problems

[0007] An execution method according to an aspect of the present invention is a method for causing a robot to execute a predetermined operation. The execution method includes the following steps. (a) A step of causing an artificial intelligence model to learn an execution command sentence example for instructing the execution of a predetermined operation and a combination example of execution units for executing the predetermined operation corresponding to the execution command sentence example. (b) A step of inputting an execution command sentence for instructing the robot to execute a predetermined operation into the artificial intelligence model. (c) A step of obtaining a program constituted by a combination of execution units generated by the artificial intelligence model in response to the input of the execution command sentence. (d) A step of causing the robot to execute the program to cause the robot to execute a predetermined operation.

[0008] In the above execution method for causing a robot to execute a predetermined operation, an artificial intelligence model is caused to learn an example of an execution command sentence (execution command sentence example) for executing a predetermined operation and an example of a combination of execution units (combination example) for executing the predetermined operation corresponding to this execution command sentence example, and an execution command sentence for executing a desired operation to be executed by the robot is input to the artificial intelligence model that has undergone the learning, whereby a program constituted by a combination of execution units necessary for executing this operation is generated.

[0009] In this way, in the above execution method, just by inputting an execution command statement for the operation desired by the user into the artificial intelligence model, a program for executing the operation is generated. Therefore, even a user who does not have advanced knowledge about robots and programming can generate a program for causing the robot to execute the desired operation and cause the robot to execute this operation. As a result, it is possible to reduce the working burden and cost burden of the user for creating a program for causing the robot to execute the desired operation.

[0010] Also, by having the artificial intelligence model learn an example of an execution command statement (execution command statement example) for instructing the execution of a predetermined operation and an example of a combination of execution units corresponding thereto, even if there is a linguistic difference between the input execution command statement and the execution command statement example, if their contents are similar to each other, it is possible to obtain an appropriate combination of execution units corresponding to the input execution command statement. As a result, the burden of devising an appropriate execution command statement to input into the artificial intelligence model can be reduced.

[0011] The above execution method may further include a step of having the artificial intelligence model learn a keyword representing a work unit executed by an execution unit and an execution unit corresponding to the keyword. According to this configuration, even if there is a linguistic difference between the input execution command statement and the execution command statement example, the execution command statement can be interpreted more flexibly and an appropriate combination of execution units can be output.

[0012] In the above execution method, the keyword may be composed of a noun and a verb that represent a work unit. According to this configuration, since the association between the execution unit and the keyword representing the work unit can be clarified, it is possible to output a more appropriate combination of execution units for the execution command statement.

[0013] The above execution method may instruct the artificial intelligence model to generate a program for the input of an execution instruction statement. According to this configuration, it is possible to notify the artificial intelligence model that the input of the execution instruction statement means that a command to generate a program composed of a corresponding combination of execution units has been given. As a result, it is possible to prevent an unintended response from being returned for the input of the execution instruction statement.

[0014] In the above execution method, the execution instruction statement may be input by voice. According to this configuration, it is possible to facilitate the instruction of the operation to be performed by the robot. As a result, the work burden and cost burden on the user can be reduced.

[0015] A program according to another aspect of the present invention is a program for causing a computer to execute the above execution method.

[0016] A robot control system according to still another aspect of the present invention includes a robot and a server that causes the robot to execute a predetermined operation. The server causes an artificial intelligence model to learn an execution instruction statement example for instructing the execution of a predetermined operation and a combination example of execution units for executing the predetermined operation corresponding to the execution instruction statement example, inputs an execution instruction statement for instructing the robot to execute the predetermined operation to the artificial intelligence model, obtains a program constituted by a combination of execution units generated by the artificial intelligence model in response to the input of the execution instruction statement, and causes the robot to execute the program, thereby causing the robot to execute the predetermined operation.

[0017] In the above robot control system, an artificial intelligence model is made to learn an example of an execution instruction statement (execution instruction statement example) for executing a predetermined operation and an example of a combination of execution units (combination example) for executing the predetermined operation corresponding to this execution instruction statement example, and by inputting an execution instruction statement for the desired operation to be executed by the robot to the artificial intelligence model that has undergone such learning, a program constituted by a combination of execution units necessary for executing this operation is generated.

[0018] In this way, in the above robot control system, just by inputting an execution command sentence for a desired operation into the artificial intelligence model, a program for executing the operation is generated. Therefore, even a user who does not have advanced knowledge about robots and programming can generate a program for causing the robot to execute a desired operation and cause the robot to execute this operation. As a result, the workload and cost burden on the user for creating a program for causing the robot to execute a desired operation can be reduced.

[0019] Also, by having the artificial intelligence model learn an example of an execution command sentence (execution command sentence example) for instructing the execution of a predetermined operation and an example of a combination of execution units corresponding thereto, even if there is a linguistic difference between the input execution command sentence and the execution command sentence example, if these contents are similar to each other, an appropriate combination of execution units corresponding to the input execution command sentence can be obtained. As a result, the burden of devising an appropriate execution command sentence to input into the artificial intelligence model can be reduced.

Advantages of the Invention

[0020] According to the present invention, even a user who does not have advanced knowledge about robots and programming can generate a program for causing the robot to execute a desired operation and cause the robot to execute this operation. Even if there is a linguistic difference between the input execution command sentence and the execution command sentence example, if these contents are similar to each other, an appropriate combination of execution units corresponding to the input execution command sentence can be obtained. As a result, the workload and cost burden on the user for creating a program for causing the robot to execute a desired operation can be reduced.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0022] Hereinafter, the robot control system according to the present embodiment will be described with reference to the drawings.

[0023] <Robot control system> Hereinafter, the robot control system 100 will be described with reference to FIG. 1. FIG. 1 is a diagram showing the robot control system 100. As shown in FIG. 1, the robot control system 100 includes a robot 1 and a control server 2. The robot 1 and the control server 2 are communicably connected to each other via a network 9. The network 9 is realized by, for example, the Internet, a wired / wireless LAN, a communication network provided by a communication carrier, or the like.

[0024] Robot 1 executes a predetermined operation. Robot 1 can independently execute a plurality of functions for executing the predetermined operation. The functions that Robot 1 can execute include, for example, an autonomous driving function for autonomously driving to a predetermined position, a voice output function for outputting a predetermined voice, a voice input function for converting the input voice into a predetermined format (for example, text data, etc.), an object discrimination function for discriminating an object detected by a camera or the like, a face authentication function for performing face authentication of a face detected by a camera or the like, and the like. In addition, Robot 1 may have a remote control function for remotely controlling Robot 1.

[0025] The control server 2 is a computer system constituted by a CPU, a storage device (ROM, RAM, SSD, hard disk, etc.), various interfaces (for example, a communication interface), and the like. The control server 2 may be a virtual computer that software-realizes the CPU, the storage device, various interfaces, and the like.

[0026] The control server 2 commands Robot 1 to execute a predetermined operation. When the control server 2 causes Robot 1 to execute a predetermined operation, it causes Robot 1 to execute, in order, combinations of a plurality of execution units. That is, the predetermined operation executed by Robot 1 is configured as a combination of a plurality of execution units.

[0027] An execution unit is a program unit (for example, a function) that causes Robot 1 to perform a specific operation. Examples of the execution unit include an execution unit for autonomously driving Robot 1 to a predetermined position, an execution unit for outputting a predetermined voice to Robot 1, an execution unit for inputting a voice to Robot 1, an execution unit for acquiring an image (still image / moving image) of a person or the like existing in front of Robot 1, an execution unit for identifying an object existing in front of Robot 1, an execution unit for causing Robot 1 to drive following a person or the like, and the like. By an easy method of combining the plurality of execution units as described above, an arbitrary operation can be caused to be executed by Robot 1.

[0028] For example, by combining the above-described execution units, the robot 1 can be made to perform tasks such as a food serving operation of transporting the served food to a designated table, a warning operation of warning a suspicious person when a suspicious person is found, security, item search, reception, advertising, etc. In security, when the robot 1 finds a suspicious person, it performs actions such as warning the suspicious person and calling for support from other robots 1. In item search, the robot 1 autonomously travels to search for a specific item and notifies its location. In reception, when the robot 1 detects a visitor, it greets the visitor with "Hello", guides the visitor to a predetermined position, counts the number of visitors, etc. In advertising, the robot 1 displays advertisements according to various situations and / or outputs voices related to the advertisements.

[0029] The execution unit for making the robot 1 perform a specific operation is composed of at least one API (Application Programming Interface) request, definition of predetermined parameters, and / or various other commands, etc. For example, the execution unit for making the robot 1 autonomously travel is composed of an API request for autonomous travel that requests the robot 1 to autonomously travel to a predetermined position, definition of various parameters, and various other commands. The execution unit is publicly available in advance as source code, as an executable function, and / or as a subroutine. The API request is for requesting the robot 1 to execute a predetermined function.

[0030] In the robot control system 100, by describing a combination of a plurality of execution units in the application program AP and the control server 2 executing this application program AP, the robot 1 can be made to perform a predetermined task. For this reason, the control server 2 has a function (API function) of interpreting the API requests included in the execution unit, instructing the robot 1 to execute the operations requested by the API requests, and obtaining execution results from the robot 1 as necessary. Examples of the API functions of the control server 2 include a status request API, an autonomous travel API, a map request API, a follow-up travel API, a voice output API, a voice input API, an object discrimination API, a video shooting API, a face authentication API, etc.

[0031] The status request API is an API for obtaining the status of Robot 1. The autonomous driving API is an API for autonomously driving Robot 1 to a specified position. The map request API is an API for obtaining map data representing the operating area of Robot 1 and the current position of Robot 1. The follow-up driving API is an API for making Robot 1 drive following a person or the like.

[0032] The voice output API is an API that converts specified voice content (for example, voice content described in text) into predetermined voice data (for example, MP3 data) and outputs it to Robot 1. The voice input API is an API that acquires the voice around Robot 1 and converts the acquired voice into a predetermined format (text data) and outputs it.

[0033] The object discrimination API is an API for discriminating objects existing around Robot 1 and outputting information on the objects. The video shooting API is an API for making Robot 1 shoot still images and / or moving images of people or the like existing around it. The face authentication API is an API that instructs Robot 1 to perform face authentication using a pre-created learned model and outputs information on the detected person.

[0034] As described above, in the robot control system 100, a predetermined operation to be executed by Robot 1 is configured as a combination of execution units, and each execution unit is defined as an API, a function, a subroutine, or the like. That is, the correspondence between the execution unit and the API, function, subroutine, source code, etc. that execute it is clear. Therefore, the application program AP for executing a predetermined operation can be created simply by arranging these APIs, functions, subroutines, and source codes according to the execution units that constitute the predetermined operation.

[0035] In the robot control system 100, an execution command sentence example for instructing the execution of a predetermined operation and a combination example of execution units for executing a predetermined operation corresponding to the execution command sentence example are input into an artificial intelligence model that has been learned. By inputting an execution command sentence (referred to as an execution command sentence) for a desired operation, an application program AP composed of a combination of execution units for causing the robot 1 to execute the desired operation is generated.

[0036] Specifically, the control server 2 inputs the execution command sentence received by the robot 1 by voice or the execution command sentence input by the user terminal 4, and transmits the execution command sentence to the artificial intelligence server 3 equipped with the artificial intelligence model. The control server 2 then acquires the application program AP generated by the artificial intelligence model from this artificial intelligence server 3.

[0037] The artificial intelligence server 3 is communicably connected to the robot 1 and the control server 2 via the network 9. The artificial intelligence server 3 provides services using the artificial intelligence model. The artificial intelligence server 3 inputs a sentence (for example, a sentence instructing to generate something, a sentence asking a question, etc.) input from an external device or the like into the artificial intelligence model, and transmits the answer output by the artificial intelligence model for the input sentence to the device or the like that input the sentence. The artificial intelligence model is, for example, an artificial intelligence model called a generative AI. Examples of the artificial intelligence model include large language models such as ChatGPT and Bard.

[0038] The user terminal 4 is communicably connected to the robot 1 and the control server 2 via the network 9. The user terminal 4 is a terminal used by the user of the robot control system 100. The user can issue various commands to the robot 1 and the control server 2 using the user terminal 4. In addition, the user can refer to various information about the robot 1 and the control server 2 displayed on the user terminal 4.

[0039] The user terminal 4 is a device composed of an information processing circuit (e.g., CPU) that executes various information processes, a storage device (ROM, RAM, SSD, hard disk, etc.), various interfaces, and the like. The user terminal 4 is, for example, a computer such as a tablet terminal, a smartphone, a personal computer, or an IoT device such as a smart speaker.

[0040] <Configuration of the robot> Using FIG. 2, the specific configuration of the robot 1 will be described. FIG. 2 is a diagram showing the hardware configuration of the robot 1. As shown in FIG. 2, the robot 1 includes a first control device 11, a traveling unit 12, a first communication unit 13, a sensor 14, a camera 15, a voice output unit 16, and a voice input unit 17.

[0041] The first control device 11 is a computer system composed of a CPU, a storage device (ROM, RAM, SSD, hard disk, etc.), various interfaces, and the like, and controls the robot 1. The first control device 11 includes a first control unit 111 and a first storage unit 113. The first control unit 111 is composed of the CPU of the first control device 11 and the like, and performs various information processes in the first control device 11.

[0042] The first control unit 111 realizes various functions of the robot 1 by executing each function program stored in the first storage unit 113.

[0043] The first storage unit 113 is the data storage area of the storage device of the first control device 11, and stores the above-mentioned various function programs, various setting parameters, etc. Specifically, for example, the first storage unit 113 stores an autonomous driving program P1 that realizes an autonomous driving function for autonomously driving the robot 1 to a predetermined position, a voice output program P2 that realizes a voice output function for outputting a predetermined voice to the robot 1, a voice input program P3 that realizes a voice input function for converting the input voice into a predetermined format, an object discrimination program P4 that realizes an object discrimination function for discriminating an object detected by the camera 15 or the like, a face authentication program P5 that realizes a face authentication function for performing face authentication on a face detected by the camera 15 or the like, and so on. In addition, the first storage unit 113 may store a remote operation program that realizes a remote operation function for remotely operating the robot 1.

[0044] As described above, the first storage unit 113 of the robot 1 stores programs for executing each function required for a predetermined operation, rather than a program for executing the entire predetermined operation to be executed by the robot 1. The first control unit 111 executes any of the above programs to cause the robot 1 to execute a predetermined function.

[0045] The traveling unit 12 causes the robot 1 to travel. The traveling unit 12 is composed of, for example, wheels, a motor for rotating the wheels, and the like. The first control unit 111 controls the rotation of the motor of the traveling unit 12 to control the traveling of the robot 1.

[0046] The first communication unit 13 is connected to the network 9 and executes communication with other devices (such as the control server 2, the artificial intelligence server 3, the user terminal 4, etc.). The first communication unit 13 is, for example, a communication circuit such as an Ethernet (registered trademark) card.

[0047] Sensor 14 is configured to detect the distance to an object around robot 1. Sensor 14 outputs data regarding the detected distance to the first control device 11. Sensor 14 is, for example, a distance measuring sensor such as LiDAR (Light Detection and Ranging). Sensor 14 outputs the detected data (for example, point cloud data) to the first control device 11. The first control unit 111 of the first control device 11 performs self-position estimation and environmental map creation based on the data acquired from sensor 14. The environmental map represents the existence positions of walls, obstacles, etc. in the place where robot 1 operates. The created environmental map is stored in the first storage unit 113. The control server 2 and the user terminal 4 can acquire the environmental map stored in the first storage unit 113.

[0048] Camera 15 is configured to image the periphery of robot 1. The video data acquired by camera 15 is output to the first control device 11. The first control unit 111 stores the acquired video data in the first storage unit 113 as a moving image or a still image.

[0049] The voice output unit 16 is configured to output a predetermined voice. The voice output unit 16 is, for example, a speaker. The first control unit 111 can output a predetermined voice from the voice output unit 16 by executing the voice output program P2 with the content of the voice to be output from the voice output unit 16 as input data.

[0050] The voice input unit 17 is configured to input the voice emitted around robot 1. The voice input unit 17 is, for example, a microphone. The first control unit 111 can acquire data (for example, text data of the voice content) representing the content of the input voice by executing the voice input program P3 with the voice input by the voice input unit 17 as input data.

[0051] Robot 1 may have a display unit. The display unit displays various information such as advertisements. The display unit is, for example, a display device such as a liquid crystal display, an organic EL display, or a CRT display.

[0052] <Configuration of the Control Server> Using FIG. 3, the specific configuration of the control server 2 will be described. FIG. 3 is a diagram showing the hardware configuration of the control server 2. As shown in FIG. 3, the control server 2 has a second control unit 21, a second storage unit 22, and a second communication unit 23. The second control unit 21 is a CPU or the like provided in the control server 2, and performs information processing in the control server 2. The second control unit 21 executes information processing in the control server 2 by executing a program stored in the second storage unit 22. Specifically, the second control unit 21 executes information processing for generating the application program AP, information processing for executing the application program AP, information processing for interpreting the content of the execution unit included therein, information processing for realizing the above API function, and the like.

[0053] The second storage unit 22 is a data storage area of the storage device of the control server 2, and stores various programs, setting parameters, and the like. Specifically, the second storage unit 22 stores an API program IP for realizing the API function, a control program CP for interpreting the content of the execution unit, a generation program MP for generating the application program AP, the application program AP, and the like.

[0054] The second communication unit 23 is connected to the network 9 and executes communication with other devices (such as the robot 1, the artificial intelligence server 3, the user terminal 4, etc.). The second communication unit 23 is, for example, a communication circuit such as an Ethernet (registered trademark) card.

[0055] <Method for Executing a Predetermined Task by the Robot> Next, using FIG. 4, in the above-described robot control system 100, an execution method for causing the robot 1 to execute a predetermined task will be described. FIG. 4 is a flowchart showing the execution method of the predetermined task by the robot 1. The execution method described below is performed by the control server 2 executing the generation program MP. Alternatively, the following commands may be sequentially input and executed using the user terminal 4 or the like.

[0056] First, the second control unit 21 of the control server 2 trains the artificial intelligence model of the artificial intelligence server 3 to generate the application program AP (step S1). The training of the artificial intelligence model is executed according to the flowchart shown in FIG. 5. FIG. 5 is a flowchart showing a method for training an artificial intelligence model.

[0057] First, when there is an input of an execution command sentence, the second control unit 21 commands the artificial intelligence model to generate an application program AP for the input execution command sentence (step S11). Specifically, for example, the second control unit 21 outputs a prompt (command sentence) such as "You are an assistant for creating js code. Refer to the following code, read the content to be executed from the given text, and create js code" to the artificial intelligence server 3. The artificial intelligence server 3 inputs this command sentence to the artificial intelligence model.

[0058] In the above prompt, "js code" means that the application program AP is described in the JavaScript (registered trademark) language. Therefore, in the above prompt, "js code" can be changed according to the language for generating the application program AP.

[0059] By inputting the above prompt to the artificial intelligence model, it is possible to notify the artificial intelligence model that the input of an execution command sentence means that a command to generate an application program AP composed of a combination of corresponding execution units has been given. As a result, it is possible to prevent an unintended response from being returned from the artificial intelligence model for the input of the execution command sentence.

[0060] Next, the second control unit 21 causes the artificial intelligence model to learn which application program AP should be generated for an execution command sentence for executing a predetermined operation. For this purpose, first, the second control unit 21 causes the artificial intelligence model to learn the correspondence between a keyword representing an operation unit executed by an execution unit constituting a predetermined operation and the execution unit corresponding to this keyword (step S12).

[0061] Specifically, the second control unit 21 inputs a plurality of sets of a keyword KW representing an operation unit and an execution unit EU corresponding thereto, as shown in FIG. 6, to the artificial intelligence server 3 as a prompt. The artificial intelligence server 3 inputs this prompt to the artificial intelligence model. FIG. 6 is a diagram showing an example of a prompt for learning the correspondence between the keyword KW and the execution unit EU.

[0062] Note that the set of the correspondence between the keyword KW and the execution unit EU may include not only the set shown in FIG. 5 but also other sets of the keyword KW and the execution unit EU. Further, the above prompt may include more sets of the correspondence between the keyword KW and the execution unit EU.

[0063] In the prompt for learning the above correspondence, the keyword KW is composed of a noun and a gerund (a verb made into a noun) representing an operation unit. Specifically, for example, the keyword KW of "camera shooting" is composed of the noun "camera" and the gerund "shooting", and represents an operation unit of "shooting with a camera". The execution unit EU corresponding to the keyword KW of "camera shooting" is "await get_image(id,c)". This execution unit is a function for causing the robot 1 having the identification information of the argument "id" to take an image.

[0064] For example, the keyword KW "voice output" is composed of the noun "voice" and the verb-noun "output", and represents the work unit of "outputting voice". The execution unit EU corresponding to the keyword KW "voice output" is "await speak(id, speak_text)". This execution unit is a function that causes the robot 1 with the identification information of the argument "id" to output the characters specified in the argument "speak_text" as voice.

[0065] For example, the keyword KW "music playback" is composed of the noun "music" and the verb-noun "playback", and represents the work unit of "playing music", that is, outputting music as sound. The execution unit EU corresponding to the keyword KW "music playback" is "await play_media(id, media_id)". This execution unit is a function that causes the robot 1 with the identification information of the argument "id" to output the sound data with the identification information specified by the argument "media_id".

[0066] As shown in FIG. 6, the prompt for learning the correspondence between the above keyword KW and the corresponding execution unit EU further includes a parameter definition DP that defines the values, identification information, etc. to be specified by the arguments of the above execution unit EU. For example, the parameter definition "fun music media_id:aaaaaaa" means that when you want to play "fun music", you should specify the identification information "aaaaaaa" for the argument "media_id" of the function play_media(id, media_id).

[0067] With a prompt as shown in FIG. 6, the artificial intelligence model can learn which execution unit EU should be assigned to the keyword recognized from the content of the execution command sentence, or conversely, which keyword recognized from the content of the execution command sentence the execution unit EU corresponds to.

[0068] Next, the second control unit 21 causes the artificial intelligence model to learn an execution command sentence example for instructing the execution of a predetermined operation and a combination example of execution units for executing the predetermined operation corresponding to the execution command sentence example (step S13).

[0069] Specifically, the second control unit 21 inputs a plurality of sets of an execution command sentence example EI and a combination example EP of execution units (instructions, functions, subroutines, etc.) for executing the operation commanded by this execution command sentence example EI, as shown in FIG. 7, into the artificial intelligence server 3 as a prompt. The artificial intelligence server 3 inputs this prompt into the artificial intelligence model. FIG. 7 is a diagram showing an example of a prompt for learning the correspondence between the execution command sentence example EI and the combination example EP of execution units.

[0070] In the prompt shown in FIG. 7, the execution command sentence example EI is arranged beside the description of "input text". The combination example EP of execution units is arranged directly below "output code". This arrangement can be any arrangement as long as the positions of the execution command sentence example EI and the combination example EP of execution units in the prompt can be recognized.

[0071] Note that the set of the correspondence between the execution command sentence example EI and the combination example EP of execution units described in the above prompt is not limited to the set shown in FIG. 7, and there may also be other sets of the execution command sentence example EI and the combination example EP of execution units. Further, the above prompt may include more sets of the correspondence between the execution command sentence example EI and the combination example EP of execution units.

[0072] Based on the above prompts, the artificial intelligence model can learn how the content shown in the execution command example EI is combined into execution units. For example, the execution command example EI of "Speak 'Hello'" can be learned as an execution unit where the argument "speak_text" of the execution unit "await speak(id, speak_text)" is substituted with "Hello". That is, the content of the above execution command example EI can be represented by a keyword such as "voice output", and the corresponding execution unit is the execution unit of "await speak(id, speak_text)", and it can be learned that the content to be spoken is substituted into the argument of "speak_text".

[0073] Also, for example, the execution command example EI of "Take a photo and then speak 'Hello'" can be learned as a combination of the execution unit "await get_image(id, c)" and the subsequent execution unit "await speak(id, speak_text)". That is, the content of the above execution command example EI is represented by a combination of a keyword such as "camera shooting" and a keyword such as "voice output". The execution unit associated with the keyword "camera shooting" is "await get_image(id, c)", and the execution unit associated with the keyword "voice output" is the execution unit of "await speak(id, speak_text)", and it can be learned that the content to be spoken is substituted into the argument of "speak_text".

[0074] For example, the execution command example EI of "Go to the desk" can be learned as an execution unit of "await navigate(id, 10, 20, 40)". That is, the content of the above execution command example EI is represented by a keyword such as "autonomous driving", and the corresponding execution unit is "await navigate(id, 10, 20, 40)", and it can be learned that the position of the desk ((X - Y coordinate values of (10, 20)) and the orientation of robot 1 when it reaches the desk (angle of 40°) are substituted as the arguments after the argument of "id".

[0075] For example, it can be learned that an execution command sentence example EI such as "Perform face authentication and state the result" can be represented by a combination of an instruction (execution unit constituted thereby) and an execution unit as shown in FIG. 7.

[0076] As shown in FIG. 7, at the head of the prompt for learning the correspondence between the execution command sentence example EI and the combination example EP of the execution units (the part surrounded by the two-dot chain line in FIG. 7), when there is an input of an execution command sentence, according to the correspondence between the subsequent execution command sentence example EI and the combination example EP of the execution units, there may be a command sentence for instructing the artificial intelligence model to determine the corresponding combination of execution units for the input execution command sentence and use it as the application program AP. This command sentence can have the content such as "Please create js code following the following example. Please return only the code."

[0077] After learning the correspondence between the execution command sentence example EI and the combination example EP of the execution units in the artificial intelligence model of the artificial intelligence server 3 as described above, the control server 2 and the artificial intelligence server 3 (artificial intelligence model) receive an input of an execution command sentence (step S2). The execution command sentence can be input, for example, by uttering the execution command sentence by voice to the robot 1 or the user terminal 4. Since the execution command sentence can be input by voice, it is possible to facilitate the command of the work to be executed by the robot 1. As a result, the work burden and cost burden on the user can be reduced.

[0078] The execution command sentence can also be input by entering the text of the execution command sentence using the user terminal 4.

[0079] When there is an input of an execution command sentence as described above (Yes in step S2), the second control unit 21 of the control server 2 causes the robot 1 to execute the work indicated by the execution command sentence (step S3). The execution of the work according to the execution command sentence is executed according to the flowchart shown in FIG. 8. FIG. 8 is a flowchart showing a method of executing the work according to the execution command sentence.

[0080] First, in step S2, when an execution command sentence is input from the robot 1 or the user terminal 4, the second control unit 21 of the control server 2 outputs the input execution command sentence to the artificial intelligence server 3. The artificial intelligence server 3 inputs the received execution command sentence into the artificial intelligence model (step S31).

[0081] When the execution command sentence is input, the artificial intelligence model determines a combination of execution units corresponding to the input execution command sentence, and generates an application program AP composed of the determined combination of execution units. The artificial intelligence server 3 outputs the application program AP generated by the artificial intelligence model to the control server 2. The second control unit 21 of the control server 2 receives the application program AP from the artificial intelligence server 3 and stores it in the second storage unit 22 (step S32).

[0082] For example, when an execution command sentence such as "Carry it to table A, say 'Sorry to keep you waiting,' and then come back to the original position after 10 seconds" is input, an application program AP composed of a combination of execution units as shown in FIG. 9 is generated. FIG. 9 is a diagram showing an example of the generated application program AP. Thus, in the robot control system 100, by simply inputting an execution command sentence with the content of requesting a person or the like to perform work as described above, an application program AP corresponding to the content of the execution command sentence can be generated. That is, even a user who does not have advanced knowledge about robots and programming can generate an application program AP that causes the robot to execute a desired operation.

[0083] Note that the user or the like may check the application program AP generated by the artificial intelligence model, correct any errors, and let the artificial intelligence model learn the correction results. The confirmation of the application program AP can be performed by, for example, visually checking the source code of the application program AP, actually running the application program AP on the robot 1, or simulating the operation of the robot 1 according to the application program AP. The execution frequency of this operation can be reduced as the learning of the artificial intelligence model progresses.

[0084] After obtaining the application program AP, the second control unit 21 of the control server 2 causes the robot 1 to execute the work according to the application program AP by executing the application program AP (step S33). Specifically, the second control unit 21 sequentially interprets the API requests included in the execution units included in the application program AP, and commands the robot 1 to sequentially execute the operations requested by the API requests, and obtains the execution results from the robot 1 as necessary, thereby causing the robot 1 to execute the work according to the application program AP.

[0085] In the robot control system 100, when the learning of the artificial intelligence model reaches a certain level, an appropriate application program AP can be generated by an execution command sentence indicating a combination of the contents of specific execution units, such as "Carry it to the A table, say 'I'm sorry to keep you waiting,' and then come back to the original position after 10 seconds." On the other hand, as the learning of the artificial intelligence model progresses, the robot 1 can appropriately interpret the work content to be executed from an execution command sentence with more abstract content, and generate an appropriate application program AP accordingly.

[0086] From an artificial intelligence model with advanced learning, for example, an application program AP as shown in FIG. 10 can be generated from an execution command sentence with the content of "serve food to children". FIG. 10 is a diagram showing another example of the generated application program AP. That is, the artificial intelligence model with advanced learning interprets the execution command sentence of "serve food to children" as, for example, "autonomously drive to the position (of the table where the child is sitting)", "output the voice of 'I'm sorry to keep you waiting'", "play pleasant music", and "return to the original position" and execute these operations on the robot 1, and determines that keywords such as "autonomous driving", "voice output", "music playback", and "autonomous driving" are assigned to each of them, and an application program AP can be generated by assigning corresponding execution units to each keyword.

[0087] Also, for example, an application program AP as shown in FIG. 11 can be generated from an execution command sentence with the content of "receive the people coming to the company". FIG. 11 is a diagram showing still another example of the generated application program AP. That is, the artificial intelligence model with advanced learning interprets the execution command sentence of "receive the people coming to the company" as, for example, "autonomously drive to the reception position of the company", "perform face authentication on the people at the reception", and "output the voice of 'Hello' to the people at the reception" and execute these operations on the robot 1, and determines that keywords such as "autonomous driving", "face authentication", and "voice output" are assigned to each of them, and an application program AP can be generated by assigning corresponding execution units to each keyword.

[0088] When generating the application program AP from the execution command sentence with the above abstract content, the artificial intelligence model may be trained using a prompt as shown in FIG. 7 with an abstract execution command sentence example EI and a combination example EP of execution units corresponding to this execution command sentence example EI. Thereby, the accuracy of the application program AP generated for the abstract execution command sentence is improved.

[0089] In this way, in the robot control system 100, just by inputting an execution command sentence for a task desired by the user into the artificial intelligence model, an application program AP for executing the task is generated. Therefore, even a user who does not have advanced knowledge about robots and programming can generate a program for causing the robot 1 to execute a desired task and cause the robot 1 to execute this task. As a result, the working burden and cost burden on the user for creating the application program AP for causing the robot 1 to execute a desired task can be reduced.

[0090] Also, in the robot control system 100, by having the artificial intelligence model learn an example of an execution command sentence (execution command sentence example EI) for instructing the execution of a predetermined task and an example of a combination of execution units EP corresponding thereto, even if there is a linguistic difference between the input execution command sentence and the execution command sentence example, if their contents are similar to each other, an appropriate combination of execution units corresponding to the input execution command sentence can be obtained. As a result, the burden of devising an appropriate execution command sentence to input into the artificial intelligence model can be reduced.

[0091] For example, for the above execution command sentence of "Carry it to Table A, say 'I'm sorry to have kept you waiting,' and then come back to the original position after 10 seconds", the artificial intelligence model interprets "Carry it to Table A" as "Move to Table A (autonomously drive)", interprets "say 'I'm sorry to have kept you waiting'" as "Speak 'I'm sorry to have kept you waiting,'", interprets "after 10 seconds" as "Wait for 10 seconds", interprets "come back to the original position" as "Move back to the original position (autonomously drive)", and determines that keywords such as "autonomous driving", "voice output", "time waiting", and "autonomous movement" are assigned to each of them, and an application program AP can be generated by assigning corresponding execution units to each keyword.

[0092] [Modification Example] As described above, the embodiments of the present invention have been explained, but the present invention is not limited to these, and various modifications are possible without departing from the spirit of the present invention.

[0093] (a) The control server 2 may provide APIs other than the above-described APIs.

[0094] (b) The application program AP generated by the control server 2 may be stored in the user terminal 4 and executed by the user terminal 4. Alternatively, the application program AP may be stored in a server other than the control server 2 and executed on this server by an operation of the user terminal 4. Additionally, the application program AP may be stored in the robot 1 and executed by the robot 1.

[0095] (c) The API functions provided by the control server 2 and each function (program for realizing the function) of the robot 1 may be automatically updatable. Also, when new devices or the like are attached to the robot 1, the API (API program IP) for operating the devices or the like and the program on the robot 1 side may be automatically updatable / installable.

[0096] (d) The artificial intelligence model may be in a closed state for each robot control system 100. That is, the artificial intelligence model of a specific robot control system 100 may be made inaccessible from other robot control systems 100, other devices, etc. This can improve the security of the robot control system 100.

[0097] (e) Conversely, the artificial intelligence model may be in an open state with respect to other robot control systems 100, other devices, etc. That is, the artificial intelligence model of a specific robot control system 100 may also be accessible from other robot control systems 100, other devices, etc. In this case, the learning of the artificial intelligence model can be further advanced. Whether to make the artificial intelligence model in an open state or a closed state can be switched according to the settings of the artificial intelligence model, etc.

[0098] (f) The artificial intelligence model may be incorporated into the control server 2. In this case, the artificial intelligence server 3 can be made unnecessary.

[0099] (g) A generation program MP for generating the above application program AP by the artificial intelligence model may be stored in the user terminal 4, and the method for generating the application program AP described above may be executed on the user terminal 4.

[0100] <Features of the Embodiment> The above embodiment can also be described as follows.

[0101] (1) The execution method is a method for causing a robot (for example, robot 1) to execute a predetermined operation. The execution method includes the following steps. In this specification, the following (a) to (d) do not mean that each step is executed in this order. (a) A step of causing an artificial intelligence model to learn an execution command sentence example (for example, execution command sentence example EI) for instructing the execution of a predetermined operation and a combination example of execution units (for example, execution unit combination example EP) for executing the predetermined operation corresponding to the execution command sentence example. (b) A step of inputting an execution command sentence for instructing the robot to execute a predetermined operation into the artificial intelligence model. (c) A step of obtaining a program (for example, application program AP) constituted by a combination of execution units generated by the artificial intelligence model in response to the input of the execution command sentence. (d) A step of causing the robot to execute a predetermined operation by causing the robot to execute the program.

[0102] In the above execution method for causing a robot to execute a predetermined operation, an artificial intelligence model is trained with an example of an execution command sentence (execution command sentence example) for executing a predetermined operation and an example of a combination of execution units (combination example) for executing the predetermined operation corresponding to this execution command sentence example. By inputting an execution command sentence for a desired operation to be executed by the robot into the trained artificial intelligence model, a program composed of a combination of execution units necessary for executing this operation is generated.

[0103] In this way, in the above execution method, just by inputting an execution command sentence for an operation desired by the user into the artificial intelligence model, a program for executing the operation is generated. Therefore, even a user who does not have advanced knowledge about robots and programming can generate a program for causing the robot to execute a desired operation and cause the robot to execute this operation. As a result, the working burden and cost burden on the user for creating a program for causing the robot to execute a desired operation can be reduced.

[0104] Also, by training an artificial intelligence model with an example of an execution command sentence (execution command sentence example) for instructing the execution of a predetermined operation and an example of a combination of execution units corresponding thereto, even if there is a linguistic difference between the input execution command sentence and the execution command sentence example, if these contents are similar to each other, an appropriate combination of execution units corresponding to the input execution command sentence can be obtained. As a result, the burden of devising an appropriate execution command sentence to input into the artificial intelligence model can be reduced.

[0105] (2) The execution method in (1) above may further include a step of training an artificial intelligence model with a keyword (for example, keyword KW) representing a work unit executed by an execution unit and an execution unit (for example, execution unit EU) corresponding to the keyword. According to this configuration, even if there is a linguistic difference between the input execution command sentence and the execution command sentence example, the execution command sentence can be interpreted more flexibly and an appropriate combination of execution units can be output.

[0106] (3) In the execution method described in (2) above, the keyword may be composed of a noun and a verb-noun representing a work unit. According to this configuration, since the correspondence between the execution unit and the keyword representing the work unit can be clarified, a more appropriate combination of execution units can be output for the execution command sentence.

[0107] (4) Any of the execution methods described in (1) to (3) above may instruct the artificial intelligence model to generate a program for the input of the execution command sentence. According to this configuration, it is possible to notify the artificial intelligence model that the input of the execution command sentence means that a command to generate a program composed of a corresponding combination of execution units has been given. As a result, it is possible to prevent an unintended answer from being returned for the input of the execution command sentence.

[0108] (5) In any of the execution methods described in (1) to (4) above, the execution command sentence may be input by voice. According to this configuration, it is possible to facilitate the command of the work to be executed by the robot. As a result, the working burden and cost burden of the user can be reduced.

[0109] (6) The program (for example, the generation program MP) is a program for causing a computer (for example, the control server 2, the user terminal 4) to execute any of the execution methods described in (1) to (5) above.

[0110] (7) The robot control system (for example, the robot control system 100) includes a robot and a server (for example, the control server 2) for causing the robot to execute a predetermined work. The server causes the artificial intelligence model to learn an example of an execution command sentence for instructing the execution of a predetermined work and an example of a combination of execution units for executing the predetermined work corresponding to the example of the execution command sentence, inputs an execution command sentence for instructing the robot to execute the predetermined work to the artificial intelligence model, obtains a program composed of the combination of execution units generated by the artificial intelligence model for the input of the execution command sentence, and causes the robot to execute the program, thereby causing the robot to execute the predetermined work.

[0111] In the above robot control system, an artificial intelligence model is trained with an example of an execution command sentence (execution command sentence example) for executing a predetermined operation, and an example of a combination of execution units (combination example) for executing the predetermined operation corresponding to this execution command sentence example. By inputting an execution command sentence for a desired operation to be executed by the robot into the trained artificial intelligence model, a program composed of a combination of execution units required to execute this operation is generated.

[0112] In this way, in the above robot control system, just by inputting an execution command sentence for an operation desired by the user into the artificial intelligence model, a program for executing the operation is generated. Therefore, even a user who does not have advanced knowledge about robots and programming can generate a program for causing the robot to execute a desired operation and cause the robot to execute this operation. As a result, the working burden and cost burden on the user for creating a program for causing the robot to execute a desired operation can be reduced.

[0113] Also, by training an artificial intelligence model with an example of an execution command sentence (execution command sentence example) for instructing the execution of a predetermined operation and an example of a combination of corresponding execution units, even if there is a linguistic difference between the input execution command sentence and the execution command sentence example, if their contents are similar to each other, an appropriate combination of execution units corresponding to the input execution command sentence can be obtained. As a result, the burden of devising an appropriate execution command sentence to input into the artificial intelligence model can be reduced.

Description of Reference Numerals

[0114] 100: Robot control system 1: Robot 11: First control device 111: First control unit 113: First storage unit P1: Autonomous driving program P2: Voice output program P3: Voice input program P4: Object Discrimination Program P5: Face Authentication Program 12: Traveling Unit 13: First Communication Unit 14: Sensor 15: Camera 16: Voice Output Unit 17: Voice Input Unit 2: Control Server 21: Second Control Unit 22: Second Memory Unit AP: Application Program MP: Generation Program 23: Second Communication Unit 3: Artificial Intelligence Server 4: User Terminal 9: Network

Claims

1. A method for causing a robot to perform a predetermined operation, comprising: a step of causing an artificial intelligence model to learn an execution command sentence example for instructing execution of the predetermined operation and a combination example of execution units for executing the predetermined operation corresponding to the execution command sentence example; a step of inputting an execution command sentence for instructing the robot to perform the predetermined operation into the artificial intelligence model; a step of obtaining a program constituted by the combination of execution units generated by the artificial intelligence model in response to the input of the execution command sentence; a step of causing the robot to execute the predetermined operation by causing the robot to execute the program; An execution method comprising the above steps.

2. The method further comprises a step of causing the artificial intelligence model to learn a keyword representing a work unit executed by the execution unit and the execution unit corresponding to the keyword. The execution method according to Claim 1.

3. The execution method according to Claim 2, wherein the keyword is composed of a noun and a verb representing the work unit.

4. The execution method according to Claim 1, further comprising a step of instructing the artificial intelligence model to generate the program in response to the input of the execution command sentence.

5. The execution method according to Claim 1, wherein the execution command sentence is input by voice.

6. A program for causing a computer to execute the execution method according to any one of Claims 1 to 5.

7. A robot and a server for causing the robot to perform a predetermined operation, A robot control system comprising: wherein the server causes an artificial intelligence model to learn an execution command sentence example for instructing execution of the predetermined operation and a combination example of execution units for executing the predetermined operation corresponding to the execution command sentence example; inputs an execution command sentence for instructing the robot to perform the predetermined operation into the artificial intelligence model; obtains a program constituted by the combination of execution units generated by the artificial intelligence model in response to the input of the execution command sentence; and causes the robot to execute the predetermined operation by causing the robot to execute the program. A robot control system.

Images