Robot operation task control system and robot

Abstract

The invention provides a robot operation task control system and a robot, the robot operation task control system comprises a processing unit, a display unit and a receiving unit, the display unit displays a robot task setting interface, and the task setting interface comprises an AI creation service function control; the receiving unit receives a trigger instruction of a user for the AI creation service function control, pops up an intelligent chat program Chatbox window on the upper layer of the current task setting interface, receives text information input by the user and displays the text information in a text dialog box; and the processing unit translates the text information into codes and tests the codes, and generates a robot operation task control flow diagram for controlling the robot to execute corresponding tasks. Therefore, the method can be suitable for personalized robot operation control scenes, code compiling does not need to be carried out by professional technicians, codes can be automatically generated by inputting natural language texts from a user side, an interface is friendly and easy to operate, the code development period is greatly shortened, and compiling efficiency is high.

Description

Background Art

[0002] With the development of artificial intelligence technology, natural language processing technology has gradually matured. In natural language processing technology, it is possible to understand the user's intention based on the user's description information and give corresponding results.

[0003] For existing robot operation control, generally, a program pre-compiled by a programmer is loaded, and the motion trajectory, operation tasks, etc. of the robot are simulated and tested according to the function controls preset on the human-machine interface. After the test is completed, it is saved for future use by the user.

[0004] However, the above method is not applicable to personalized operation control settings. Especially for users who cannot compile programs, if they want to add some special operation modes, they need to re-develop the corresponding programs by programmers, and the whole cycle is long and the efficiency is low. Summary of the Invention

[0005] Aiming at the deficiencies in the prior art, the purpose of the present invention is to provide a robot operation task control system and a robot.

[0006] In a first aspect, an embodiment of the present application provides a robot operation task control system, including: a processing unit, a display unit, and a receiving unit. Among them, the display unit is used to display a robot task setting interface, and an AI creation service function control is included in the task setting interface;

[0007] The receiving unit is used to receive a trigger instruction from the user for the AI creation service function control, so that the display unit pops up an intelligent chat program Chatbox window on the current task setting interface. Among them, the Chatbox window includes: a text dialog box and a program compilation box;

[0008] The receiving unit is also used to receive the text information input by the user and display it in the text dialog box;

[0009] The processing unit is used to translate the text information into code and perform tests to generate a robot operation task control flow diagram for controlling the robot to execute corresponding tasks.

[0010] Optionally, the processing unit includes: a keyword extraction unit,

[0011] The keyword extraction unit is used to extract keywords matching task control from the text information input by the user according to the built-in chatbot program ChatGPT;

[0012] The code generation unit is used to generate python format code according to the keywords;

[0013] A running effect simulation unit, which is used to call relevant application programming interfaces (APIs) through the generated code to simulate the effect after the code runs.

[0014] Optionally, the processing unit further includes: a task flow diagram generation unit, and the task flow diagram generation unit is used for:

[0015] Receiving first touch information input by a user;

[0016] Running the generated code according to the first touch information, and generating a robot operation task control flow diagram during the running process.

[0017] Optionally, it further includes: a manual verification unit, and the manual verification unit is used for:

[0018] Manually verifying the generated robot operation task control flow diagram, and if an error is found, performing an editing process on the robot operation control flow diagram;

[0019] After all errors are excluded, re-running the robot operation control flow diagram and generating a corresponding visual programming (VP) module.

[0020] Optionally, it further includes: an error feedback unit, and the error feedback unit is used for:

[0021] During the code testing process, automatically returning the position where an error exists and prompting the reason for the error.

[0022] Optionally, the Chatbox window is a semi-transparent floating window and is always displayed on top of the application.

[0023] Optionally, the receiving unit is further used for: receiving second touch information of the user for the Chatbox window;

[0024] The processing unit is further used for performing any one of the following operations according to the second touch information:

[0025] Dragging the Chatbox window;

[0026] Hiding or closing the Chatbox window;

[0027] Zooming in or out the Chatbox window;

[0028] Displaying the help information of the Chatbox.

[0029] Optionally, it further includes: an information management unit, and the information management unit is used for:

[0030] Managing the text information input in the text dialog box, and the management includes: performing a save or clear operation on the chat record.

[0031] Optionally, the processing unit is further configured to:

[0032] Determine whether there is an advanced instruction in the text information input by the user. The advanced instructions include: an automatic execution instruction, a direct code editing instruction, a refresh instruction, and an interval instruction; where

[0033] The automatic execution instruction is used to indicate that the generated code is automatically executed;

[0034] The direct code editing instruction is used to indicate editing the code and returning the error during operation;

[0035] The refresh instruction is used to indicate refreshing the running process, and the interval instruction is used to indicate running the code in two parts separately;

[0036] When it is detected that there is the advanced instruction, the processing unit directly executes the corresponding advanced instruction during the code running process.

[0037] In a second aspect, an embodiment of the present application provides a robot, which is characterized by including: a base, a robotic arm, a drive system, a vision sensor, a processor, and a robot operation task control system according to any one of the first aspect, where:

[0038] After the robot operation task control system generates a robot operation task control flow diagram, it sends it to the processor;

[0039] The vision sensor acquires an environmental image of the operation space and sends it to the processor;

[0040] The processor generates a drive instruction according to the robot operation task control flow diagram and the environmental image of the operation space;

[0041] The drive system drives the robotic arm to execute the corresponding operation task according to the drive instruction.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] In this application, by receiving an AI creation service start instruction input by the user on the task setting page; according to the AI creation service start instruction, a smart chat program Chatbox window is popped up, where the Chatbox window includes: a text dialog box and a program compilation box; receiving the text information input by the user and displaying it in the text dialog box; translating the text information into code and testing it to generate a robot operation task control flow diagram for controlling the robot to execute the corresponding task. Thus, it can be applied to personalized robot operation control scenarios. Without the need for professional technicians to compile code, by inputting natural language text on the user side, code can be automatically generated. The interface is user-friendly and easy to operate, greatly shortening the code development cycle and having high compilation efficiency. Brief Description of the Drawings

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objectives, and advantages of the present invention will become more obvious:

[0045] Figure 1 Schematic diagram of the interface of the Chatbox window provided by the embodiment of the present application;

[0046] Figure 2 Schematic diagram of the structure of a robot operation task control system provided by the embodiment of the present application;

[0047] Figure 3 Schematic diagram of the human-machine interaction interface when the Chatbox window pops up provided by the embodiment of the present application;

[0048] Figure 4 Schematic diagram of the code result generated by the Chatbox window provided by the embodiment of the present application;

[0049] Figure 5 Schematic diagram of the effect when code compilation is performed in the Chatbox window provided by the embodiment of the present application;

[0050] Figure 6 Schematic diagram of the structure of another robot operation task control system provided by the embodiment of the present application;

[0051] Figure 7 Schematic diagram of the structure of yet another robot operation task control system provided by the embodiment of the present application;

[0052] Figure 8 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0053] Figure 9 Schematic diagram of the structure of the computer-readable storage medium in the embodiment of the present invention. Detailed Embodiments

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts shall fall within the scope of protection of this application.

[0055] It should be noted that when a component is referred to as being "fixed to" another component, it can be directly on the other component or there may also be an intermediate component. When a component is considered to be "connected to" another component, it can be directly connected to the other component or there may be an intermediate component at the same time.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used in the specification of this application herein are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0057] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this invention and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0058] The following will specifically describe in detail the technical solutions of the present invention and how the technical solutions of this application solve the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0059] The following will describe in detail some embodiments of this application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0060] Embodiments of the present application provide a method for controlling robot operation tasks based on an intelligent chat program, which can be applied to various types of robots. For example, mobile logistics robots, robots on large-scale operation assembly lines, simple handling robots, etc. Embodiments of the present application are not limited thereto.

[0061] Figure 1 It is a schematic diagram of the interface of the Chatbox window provided by the embodiments of the present application. As Figure 1 shown, it includes a program compilation box located on the upper side of the interface area, a text dialog box located on the lower side of the interface area. Three function controls are sequentially distributed on the right side of the program compilation box, representing the return, delete, and help functions respectively. There is also a progress bar between the program compilation box and the text dialog box, used to indicate the progress of code generation. The button "APPLY!" represents running, and the button "QUIT" represents exiting (closing the Chatbox).

[0062] Exemplarily, input natural language information in the text dialog box, and then click Enter to input the text information and wait for the code to be generated in the program compilation box.

[0063] Figure 2 It is a schematic diagram of the structure of a robot operation task control system provided by the embodiments of the present application. As Figure 2 shown, the robot operation task control system 200 in this embodiment may include: a processing unit 210, a display unit 220, and a receiving unit 230. Among them, the display unit 220 is used to display the robot task setting interface, and an AI creation service function control is included in the task setting interface; the receiving unit 230 is used to receive a trigger instruction from the user for the AI creation service function control, so that the display unit pops up an intelligent chat program Chatbox window on the upper layer of the current task setting interface, where the Chatbox window includes: a text dialog box and a program compilation box; the receiving unit 230 is also used to receive the text information input by the user and display it in the text dialog box; the processing unit 210 is used to translate the text information into code and perform tests to generate a robot operation task control flowchart for controlling the robot to execute corresponding tasks.

[0064] In this embodiment, on the human-machine interaction interface for robot control, open the robot operation control software, click "Task" on the home page, and find the AI creation service in the drop-down menu of the tool, and click the AI creation service with the mouse to generate a start instruction.

[0065] In this embodiment, when an AI creation service start instruction is received from the user, a smart chat program Chatbox window pops up. The Chatbox window includes: a text dialog box and a program compilation box. The ChatBox in this embodiment is a cross-platform desktop client for the OpenAI API and also an OpenAI API prompt debugging and management tool.

[0066] Exemplarily, Figure 3 FIG. is a schematic diagram of the human-computer interaction interface when the Chatbox window provided by the embodiment of the present application pops up. As Figure 3 shown, the Chatbox window floats on top of other applications, and the Chatbox window is in a semi-transparent state.

[0067] In this embodiment, referring to Figure 1 or Figure 3 as shown, enter text information in the text dialog box. For example, you can enter: "I now have a bunch of cartons of the same size. These cartons are sent one by one from the conveyor belt. I need to control the robotic arm to pick up the cartons and stack them in a large box." Click the Enter key to complete the input of the text information.

[0068] Exemplarily, the processing unit 210 includes: a keyword extraction unit 211, a code generation unit 212, and a running effect simulation unit 213. Among them: the keyword extraction unit 211 is used to extract keywords matching the task control from the text information input by the user according to the built-in chatbot program ChatGPT; the code generation unit 212 is used to generate python-format code according to the keywords; the running effect simulation unit 213 is used to call relevant application program interfaces API through the generated code to simulate the effect after the code runs.

[0069] Exemplarily, Figure 4 FIG. is a schematic diagram of the code result generated by the Chatbox window provided by the embodiment of the present application. Figure 5 FIG. is a schematic diagram of the effect when code compilation is performed in the Chatbox window provided by the embodiment of the present application.

[0070] Exemplarily, receive the first touch information input by the user; run the generated code according to the first touch information, and generate a robot operation task control flow diagram during the running process. Specifically, referring to Figure 1 as shown, generate the first touch information by clicking "APPLY!" to instruct to run the code generated by Chatbox.

[0071] Exemplarily, the receiving unit 220 is further configured to: receive the second touch information of the user for the Chatbox window; the processing unit 210 is further configured to perform any of the following operations according to the second touch information:

[0072] Drag the Chatbox window;

[0073] Hide or close the Chatbox window;

[0074] Enlarge or reduce the Chatbox window;

[0075] Display the help information of Chatbox.

[0076] In this embodiment, the processing unit 210 is further configured to: determine whether there is an advanced instruction in the text information input by the user, and the advanced instructions include: an automatic execution instruction, a direct code editing instruction, a refresh instruction, and an interval instruction; wherein, the automatic execution instruction is used to indicate automatic execution of the generated code; the direct code editing instruction is used to indicate editing of the code and returning the error during operation; the refresh instruction is used to indicate refreshing of the running process, and the interval instruction is used to indicate running the code in two parts separately; when it is detected that there is an advanced instruction, the processing unit directly executes the corresponding advanced instruction during the code running process.

[0077] Exemplarily, this embodiment further provides a management function for chat information, and the user can manage the text information input in the text box, such as saving or clearing the text. In addition, keyword search and other processing can also be performed on the text information.

[0078] In a possible implementation manner, when the processing unit 210 recognizes the text information in the dialog box, it may first determine whether there is an advanced instruction in the text information input by the user, and the advanced instructions include: an automatic execution instruction (apply), a direct code editing instruction (exec), a refresh instruction (reboot), and an interval instruction (split); wherein, the automatic execution instruction is used to indicate automatic execution of the generated code, the direct code editing instruction is used to indicate editing of the code and returning the error during operation, the refresh instruction is used to indicate refreshing of the running process, and the interval instruction is used to indicate running the code in two parts separately; if there is an advanced instruction, the corresponding advanced instruction is directly executed during the code running process.

[0079] In this embodiment, the "apply" instruction has the exact same effect as clicking the APPLY button. For example, one can first ask for a piece of text and then add ": apply" after it, which is equivalent to auto-applying this piece of code. The "exec" instruction is used to provide the function of directly editing and running code. Below this line of instruction, the user can write custom code. Chatbox will run the code written below ": exec" and return any error messages during the running process. If one doesn't want to spend time tuning the Chatbot, they can also directly generate the initial code generated by it with some minor modifications. The "reboot" instruction is equivalent to clicking the refresh button. When the user feels that there are too many questions being asked at this time, they can insert a ": reboot" and then write other questions after it. The "split" instruction will treat the parts before and after this instruction as two separate parts. For example, if a piece of custom code is written after ": exec" and then the user wants to ask Chatbot some questions, they need to write ": split" after the code and then write the questions they want to ask.

[0080] In this embodiment, operations such as dragging, pulling, hiding, closing, enlarging, and shrinking can be performed on Chatbox. The Chatbox window is a semi-transparent floating window and always appears on top of the application, thus ensuring that it does not affect the development of other functions during use. In addition, all functions in the middle of Chatbox are asynchronous, so they do not block other processes and ensure that they do not affect the development of other functions during use.

[0081] In this embodiment, on the task setting page, it receives the AI creation service start instruction input by the user; according to the AI creation service start instruction, it pops up the intelligent chat program Chatbox window, where the Chatbox window includes: a text dialog box and a program compilation box; it receives the text information input by the user and displays it in the text dialog box; it translates the text information into code and conducts tests to generate a robot job task control flow diagram for controlling the robot to execute corresponding tasks. Thus, it can be applied to personalized robot job control scenarios. Without the need for professional technicians to compile code, by the user inputting natural language text on the user side, code can be automatically generated. The interface is friendly and easy to operate, greatly shortening the code development cycle and having high compilation efficiency.

[0082] Figure 6 It is a schematic structural diagram of another robot job task control system provided by an embodiment of this application, as Figure 6As shown in the figure, the system in this embodiment may include: a processing unit 210, a display unit 220, a receiving unit 230, a task flow diagram generation unit 240, a manual verification unit 250, and a task flow diagram generation unit 240, which is configured to: receive first touch information input by a user; run the generated code according to the first touch information, and generate a robot operation task control flow diagram during the running process. The manual verification unit 250 is configured to: manually verify the generated robot operation task control flow diagram, and if an error is found, perform an editing process on the robot operation control flow diagram; after all errors are eliminated, re-run the robot operation control flow diagram, and generate a corresponding visual programming VP module.

[0083] In this embodiment, since the code generated by the Chatbot may have some errors, in order to make the code run successfully, it is necessary to manually verify the code.

[0084] Exemplarily, Figure 4 taking the generated code as an example, its goal is to complete a task of repeatedly grasping workpieces from container A and placing them in container B. The specific process is mainly divided into five parts: initialization, calling vision, registering grasping, motion planning, and motion execution. However, it can be found that the code generated by the Chatbot has problems, that is, the asynchronous execution should only include #4, and the initialization should be in a separate function, and vision, registering grasping, and planning should be in a function, and the id of the function should be the same as the id of the module it contains. Therefore, these errors can be modified through manual verification.

[0085] In a possible implementation manner, the Chatbot can also be repeatedly tuned to continuously improve the accuracy rate of the code generated by the Chatbot, so that the code can be run without manual verification.

[0086] In this embodiment, by manually verifying the code generated by the Chatbot, correcting or adjusting the code generated by the Chatbot, the accuracy of the Chatbot code is improved, ensuring that it can run normally.

[0087] Figure 7 FIG. is a schematic structural diagram of another robot operation task control system provided by an embodiment of the present application. As Figure 7 shown, the system in this embodiment may include: a processing unit 210, a display unit 220, a receiving unit 230, a task flow diagram generation unit 240, a manual verification unit 250, an error feedback unit 260, and an information management unit 270. The error feedback unit 260 is configured to: automatically return the position where an error exists and prompt the error reason during the code test process; the information management unit 270 is configured to: manage the text information input in the text dialog box, and the management includes: saving or clearing the chat record.

[0088] In this embodiment, during code testing, Chatbot can automatically return the location where an error exists and prompt the reason for the error, thus facilitating the user to correct the code.

[0089] In this embodiment, some APIs specifically for Chatbot are built into the task setting interface. Chatbot can directly generate VP (Visual Programming) modules by calling these APIs. It can also ask Chatbot to obtain the latest API (Application Program Interface) and usage methods.

[0090] The embodiment of the present application also provides a method for controlling a robot operation task based on an intelligent chat program, which may include the following steps:

[0091] Step S1: On the task setting page, receive the AI creation service start instruction input by the user.

[0092] In this embodiment, on the human-computer interaction interface for robot control, start the robot operation control software, click "Task" on the home page, and find the AI creation service in the drop-down menu of the tool, then click the AI creation service with the mouse to generate the start instruction.

[0093] Step S2: According to the AI creation service start instruction, pop up the intelligent chat program Chatbox window.

[0094] In this embodiment, when receiving the AI creation service start instruction input by the user, pop up the intelligent chat program Chatbox window. Among them, the Chatbox window includes: a text dialog box and a program compilation box. ChatBox in this embodiment is a cross-platform desktop client of the OpenAI API and also a debugging and management tool for the OpenAI API prompt.

[0095] Exemplarily, Figure 3 is a schematic diagram of the human-computer interaction interface when the Chatbox window provided by the embodiment of the present application pops up. As Figure 3 shown, the Chatbox window floats on top of other applications, and the Chatbox window is in a semi-transparent state.

[0096] Step S3: Receive the text information input by the user and display it in the text dialog box.

[0097] In this embodiment, refer to Figure 1 , or Figure 3As shown, enter text information in the text dialog box. For example, you can enter: "I now have a bunch of cartons of the same size, and these cartons are sent one by one from the conveyor belt. I need to control the robotic arm to pick up the cartons and stack them in a large box." Click the Enter key to complete the input of the text information.

[0098] Step S4: Translate the text information into code and conduct tests to generate a control flow diagram for the robot operation task, so as to control the robot to execute corresponding tasks.

[0099] In this embodiment, according to the built-in chatbot program ChatGPT, extract keywords matching the task control from the text information input by the user; generate Python-formatted code according to the keywords; call relevant application programming interfaces (APIs) through the generated code to simulate the effect after the code runs.

[0100] Exemplarily, Figure 4 is a schematic diagram of the code result generated by the Chatbox window provided by the embodiment of the present application. Figure 5 is a schematic diagram of the effect during code compilation in the Chatbox window provided by the embodiment of the present application.

[0101] Exemplarily, receive the first touch information input by the user; run the generated code according to the first touch information, and generate a control flow diagram for the robot operation task during the running process. Specifically, refer to Figure 1 , generate the first touch information by clicking "APPLY!" to indicate running the code generated by Chatbox.

[0102] In this embodiment, by receiving the AI creation service start instruction input by the user on the task setting page; according to the AI creation service start instruction, pop up the intelligent chat program Chatbox window, where the Chatbox window includes: a text dialog box and a program compilation box; receive the text information input by the user and display it in the text dialog box; translate the text information into code and conduct tests to generate a control flow diagram for the robot operation task, so as to control the robot to execute corresponding tasks. Thus, it can be applied to personalized robot operation control scenarios. Without the need for professional technical personnel to compile code, by inputting natural language text on the user side, code can be automatically generated, with a friendly and easy-to-operate interface, greatly shortening the code development cycle and having high compilation efficiency.

[0103] Step S5: Manually verify the generated control flow diagram for the robot operation task. If an error is found, edit the control flow diagram for the robot operation.

[0104] In this embodiment, since there may be some errors in the code generated by Chatbot, in order to make the code run successfully, the code needs to be manually verified.

[0105] Exemplarily, taking Figure 4 the generated code as an example, its goal is to complete a task of repeatedly grasping workpieces from container A and placing them in container B. The specific process is mainly divided into five parts: initialization, calling vision, registering grasping, motion planning, and motion execution. However, it can be found that the code generated by Chatbot has problems, that is, asynchronous execution should only include #4, and initialization should be in a separate function, vision, registering grasping, and planning should be in a function, and the id of the function should be the same as the id of the module it contains. Therefore, these errors can be modified by manual verification.

[0106] In one possible implementation, Chatbot can also be repeatedly trained to continuously improve the accuracy of the code generated by Chatbot, so that the code can be run without manual verification.

[0107] In another possible implementation, during code testing, Chatbot can automatically return the location of the error and prompt the reason for the error. Thus, it is convenient for users to correct the code.

[0108] Step S6: After eliminating all errors, re-run the robot job control flow diagram and generate the corresponding visual programming VP module.

[0109] In this embodiment, some APIs specifically for Chatbot are built into the task setting interface. Chatbot can directly generate VP (Visual Programming) modules by calling these APIs. It can also ask Chatbot to obtain the latest API (Application Program Interface) and usage methods.

[0110] In this embodiment, by manually verifying the code generated by Chatbot, correcting or adjusting the code generated by Chatbot, the accuracy of Chatbot code is improved to ensure its normal operation.

[0111] Step S7: Receive the second touch information of the user for the Chatbox window and perform corresponding operations according to the second touch information.

[0112] In this embodiment, the corresponding operations may include any one of the following:

[0113] Drag the Chatbox window;

[0114] Hide or close the Chatbox window;

[0115] Enlarge or shrink the Chatbox window;

[0116] Display the help information of Chatbox.

[0117] Exemplarily, this embodiment also provides a management function for chat information. Users can manage the text information input in the text box, such as saving or clearing the text. In addition, keyword search and other processing can also be performed on the text information.

[0118] In a possible implementation manner, when identifying the text information in the dialog box, it is also possible to first determine whether there is a high-level instruction in the text information input by the user. The high-level instructions include: automatic execution instruction (apply), direct code editing instruction (exec), refresh instruction (reboot), interval instruction (split); among them, the automatic execution instruction is used to indicate that the generated code is automatically executed, the direct code editing instruction is used to indicate editing the code and returning the error reported during operation, the refresh instruction is used to indicate refreshing the running process, and the interval instruction is used to indicate splitting the code into two parts and running them separately; if there is a high-level instruction, the corresponding high-level instruction is directly executed during the code running process.

[0119] In this embodiment, the "apply" instruction has exactly the same effect as clicking the APPLY button. For example, it is possible to first ask a piece of text and add ": apply" after it, which is equivalent to auto applying this piece of code (automatically running / applying). The "exec" instruction is used to provide the function of directly editing and running the code. Below this line of instruction, the user-defined code can be written. Chatbox will run the code written below ": exec" and return the error reported during operation. If you don't want to spend time tuning the Chatbot, you can also directly generate it after slightly modifying the preliminary code it generates. The "reboot" instruction is equivalent to clicking the refresh button. When the user feels that there are too many questions being asked at this time, a ": reboot" can be inserted and other questions can be written after it. The "split" instruction will treat the parts before and after this instruction as two parts. For example, if a piece of user-defined code is written after ": exec" and then the user wants to ask Chatbot some questions, a ": split" needs to be written after the code and then the questions to be asked.

[0120] In this embodiment, operations such as dragging, pulling, hiding, closing, enlarging, and shrinking can be performed on Chatbox. The Chatbox window is a semi-transparent floating window and is always displayed on top of the application, so as to ensure that it will not affect the development of other functions when in use. In addition, all functions in the middle of Chatbox are asynchronous, so they will not block other processes and ensure that they will not affect the development of other functions when in use.

[0121] Figure 8 A schematic structural diagram of an electronic device provided in an embodiment of the present application. The electronic device 800 in this embodiment may include: a processor 801 and a memory 802.

[0122] The memory 802 is used to store programs; the memory 802 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviation: RAM), such as a static random access memory (English: static random-access memory, abbreviation: SRAM), a double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviation: DDR SDRAM), etc.; the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory). The memory 802 is used to store computer programs (such as application programs and functional modules for implementing the above methods), computer instructions, etc. The above computer programs, computer instructions, etc. may be partitioned and stored in one or more memories 802. And the above computer programs, computer instructions, data, etc. may be called by the processor 801.

[0123] The above computer programs, computer instructions, etc. may be partitioned and stored in one or more memories 802. And the above computer programs, computer instructions, data, etc. may be called by the processor 801.

[0124] The processor 801 is used to execute the computer programs stored in the memory 802 to implement each step in the method involved in the above embodiment.

[0125] Specifically, reference may be made to the relevant descriptions in the foregoing method embodiments.

[0126] The processor 801 and the memory 802 may be independent structures or integrated structures integrated together. When the processor 801 and the memory 802 are independent structures, the memory 802 and the processor 801 may be coupled and connected through a bus 803.

[0127] The electronic device 800 in this embodiment may execute the technical solutions in the above method. The specific implementation process and technical principle are referred to the relevant descriptions in the above method, and will not be elaborated here.

[0128] Those skilled in the art can understand that various aspects of the present invention can be implemented as a system, a method, or a program product. Therefore, various aspects of the present invention can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "platform" here.

[0129] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When at least one processor of the user equipment executes the computer-executable instructions, the user equipment executes the above various possible methods.

[0130] Among them, the computer-readable medium includes a computer storage medium and a communication medium. The communication medium includes any medium facilitating the transmission of a computer program from one place to another. The storage medium can be any available medium accessible by a general-purpose or special-purpose computer. An exemplary storage medium is coupled to the processor, enabling the processor to read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. Additionally, the ASIC can be located in the user equipment. Of course, the processor and the storage medium can also exist as discrete components in the communication device.

[0131] The present application also provides a program product, which includes a computer program stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium, and the execution of the computer program by at least one processor enables the server to implement the method according to any one of the above embodiments of the present invention.

[0132] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: various media such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disc that can store program codes.

[0133] Figure 9 is a schematic structural diagram of the computer-readable storage medium in the embodiment of the present invention. Refer to Figure 9As shown, a program product 900 for implementing the above method according to an embodiment of the present invention is described. It may be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, a readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0134] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but not be limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0135] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0136] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages - such as Java, C++, etc., and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).

[0137] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather will conform to the broadest scope consistent with the principles and novel features disclosed herein.

[0138] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A robot operation task control system, characterized in that, it includes: a processing unit, a display unit, and a receiving unit. Among them, the display unit is used to display a robot task setting interface, and an AI creation service function control is included in the task setting interface; the receiving unit is used to receive a trigger instruction from the user for the AI creation service function control, so that the display unit pops up an intelligent chat program Chatbox window on the upper layer of the current task setting interface. Among them, the Chatbox window includes: a text dialog box and a program compilation box; the receiving unit is also used to receive text information input by the user and display it in the text dialog box; the processing unit is used to translate the text information into code and perform tests, generate a robot operation task control flow diagram, and control the robot to execute corresponding tasks.

2. The robot operation task control system according to claim 1, characterized in that, the processing unit includes: a keyword extraction unit, a code generation unit, and a running effect simulation unit. Among them: the keyword extraction unit is used to extract keywords matching task control from the text information input by the user according to the built-in chatbot program ChatGPT; the code generation unit is used to generate python format code according to the keywords; the running effect simulation unit is used to call relevant application programming interfaces API through the generated code to simulate the effect after the code runs.

3. The robot operation task control system according to claim 1, characterized in that, the processing unit further includes: a task flow diagram generation unit, and the task flow diagram generation unit is used for: receiving the first touch information input by the user; running the generated code according to the first touch information and generating a robot operation task control flow diagram during the running process.

4. The robot operation task control system according to any one of claims 1-3, characterized in that, it further includes: an artificial verification unit, and the artificial verification unit is used for: artificially verifying the generated robot operation task control flow diagram. If an error is found, editing processing is performed on the robot operation control flow diagram; after all errors are excluded, the robot operation control flow diagram is run again, and a corresponding visual programming VP module is generated.

5. The robot operation task control system according to any one of claims 1-3, characterized in that, it further includes: an error feedback unit, and the error feedback unit is used for: automatically returning the position where an error exists and prompting the error reason during the code test.

6. The robot operation task control method according to any one of claims 1-3, characterized in that, the Chatbox window is a semi-transparent floating window and is always displayed on the upper layer of the application.

7. The robot operation task control system according to claim 6, characterized in that, the receiving unit is also used for: receiving the second touch information from the user for the Chatbox window; the processing unit is also used to perform any of the following operations according to the second touch information: Drag the Chatbox window; Hide or close the Chatbox window; Enlarge or reduce the Chatbox window; Display the help information of Chatbox.

8. The robot operation task control system according to any one of claims 1-3, characterized in that, further comprising: An information management unit, which is used for: Managing the text information input in the text dialog box, and the management includes: saving or clearing the chat history.

9. The robot operation task control system according to any one of claims 1-3, characterized in that, The processing unit is further used for: Judging whether there is a high-level instruction in the text information input by the user, and the high-level instructions include: automatic operation instruction, direct code editing instruction, refresh instruction, interval instruction; wherein, The automatic operation instruction is used to indicate that the generated code is automatically run; The direct code editing instruction is used to indicate editing the code and returning the error during operation; The refresh instruction is used to indicate refreshing the running process, and the interval instruction is used to indicate running the code in two parts respectively; When it is detected that there is the high-level instruction, the processing unit directly executes the corresponding high-level instruction during the code running process.

10. A robot, characterized in that, comprising: A base, a robotic arm, a drive system, a vision sensor, a processor, and a robot operation task control system according to any one of claims 1-9, wherein: After the robot operation task control system generates a robot operation task control flow diagram, it sends it to the processor; The vision sensor acquires the environmental image of the operation space and sends it to the processor; The processor generates a drive instruction according to the robot operation task control flow diagram and the environmental image of the operation space; The drive system drives the robotic arm to perform corresponding operation tasks according to the drive instruction.

Images