Visual programming interaction processing method, terminal equipment and storage medium

Abstract

The invention provides a visual programming interaction processing method, terminal equipment and a storage medium, and relates to the technical field of robots. The method comprises the steps that in response to a first operation of a user for a first point location node in an offline simulation interface, a first instruction block corresponding to the first point location node is generated and displayed in a script programming interface, the first point location node is a node in offline simulation software, and the first instruction block is an instruction block in script programming software; and in response to a second operation of the user for the first instruction block in the script programming interface, modifying the point location information of the first point location node in the offline simulation software according to the modification data of the first instruction block in the script programming software. According to the method, the offline simulation interface and the script programming interface are integrated in the same graphical user interface, and the bidirectional mapping relationship between the point location nodes and the instruction blocks is established, so that the problem that simulation and programming are separated and difficult to cooperate in the prior art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a visual programming interaction processing method, a terminal device and a storage medium. BACKGROUND

[0002] In the application of robot programming and simulation technology, offline simulation software is widely used in trajectory planning and program generation. Usually, the user plans path points and motion trajectories in advance in the offline simulation software, and generates an executable robot script program through a post-processor, and then adjusts and verifies the running of the point, coordinate system and other parameters in the script program in the field debugging stage by relying on a teach pendant or a mobile application (App).

[0003] However, the offline simulation software in the prior art usually updates the robot script program in two ways, one is to completely cover the original script, and the other is to generate a subprogram, but due to the lack of effective data synchronization mechanism between the simulation environment and the script program, the trajectory and point in the simulation environment and the instruction in the actual script program are independent of each other, so it is difficult to realize incremental synchronization regardless of which way is used. Conversely, if the script program is modified in the mobile APP, such as adding a transition point or modifying some path, it is also difficult to synchronize the modified data back to the offline simulation software, especially when the script program contains complex logic, variable reference or dynamic coordinate calculation, it is difficult to parse and restore the trajectory information to the offline simulation interface. Therefore, the prior art has significant deficiencies in the bidirectional collaboration of simulation and programming. SUMMARY

[0004] The present application aims to solve the problem of significant deficiencies in the bidirectional collaboration of simulation and programming in the prior art by providing a visual programming interaction processing method, a terminal device and a storage medium.

[0005] To achieve the above-mentioned purpose, the technical solutions adopted by the embodiments of the present application are as follows: In a first aspect, the embodiments of the present application provide a visual programming interaction processing method. A terminal provides a graphical user interface, which includes an offline simulation interface of offline simulation software and a script programming interface of script programming software. The offline simulation interface displays a point node for describing a point in a robot motion trajectory. The method includes: in response to a first operation of a user on a first point node in the offline simulation interface, generating and displaying a first instruction block corresponding to the first point node in the script programming interface. The first point node is a node in the offline simulation software, and the first instruction block is an instruction block in the script programming software. In response to a second operation of the user on the first instruction block in the script programming interface, modifying point information of the first point node in the offline simulation software according to modification data of the first instruction block in the script programming software.

[0006] As a possible implementation manner, the generating and displaying the first instruction block corresponding to the first point node in the script programming interface in response to the first operation of the user on the first point node in the offline simulation interface includes: in response to the user selecting and starting dragging the first point node in the offline simulation interface, obtaining first index information of the first point node. The first index information is used to indicate a storage path of point information of the first point node in a first database corresponding to the offline simulation software. In response to the user dragging the first point node from the offline simulation interface to the script programming interface, obtaining the point information of the first point node from the first database according to the first index information of the first point node, and generating instruction block information of the first instruction block according to a data conversion relationship between the offline simulation software and the script programming software, storing the instruction block information of the first instruction block into a second database corresponding to the script programming software, and generating and displaying the first instruction block in the script programming interface according to the instruction block information of the first instruction block.

[0007] As a possible implementation manner, the generating and displaying the first instruction block in the script programming interface according to the instruction block information of the first instruction block includes: in response to a mouse release operation of the user in the script programming interface, generating and displaying the first instruction block at a mouse release position according to the instruction block information of the first instruction block.

[0008] As a possible implementation manner, the response of the user to the second operation on the first instruction block in the script programming interface, the modification of the point position information of the first point position node in the offline simulation software according to the modification data of the first instruction block in the script programming software, comprises: in response to the click operation of the user on the first instruction block, a first modification interface is displayed, so that the user modifies the instruction block information of the first instruction block in the first modification interface, and updates the modification data to the corresponding second database of the script programming software; the modification data is converted into the modification data in the offline simulation software according to the data conversion relationship between the offline simulation software and the script programming software, and the point position information of the first point position node in the first database corresponding to the offline simulation software is modified based on the modification data in the offline simulation software.

[0009] As a possible implementation manner, the method further comprises: in response to the creation operation of the user in the script programming interface, generating a second instruction block in the script programming interface; in response to the third operation of the user on the second instruction block in the script programming interface, generating and displaying a second point position node corresponding to the second instruction block in the offline simulation interface.

[0010] As a possible implementation manner, the response of the user to the third operation on the second instruction block in the script programming interface, the generation and display of the second point position node corresponding to the second instruction block in the offline simulation interface, comprises: in response to the selection and start of the drag of the second instruction block in the script programming interface, the second index information of the second instruction block is acquired, the second index information is used to indicate the storage path of the instruction block information of the second instruction block in the second database corresponding to the script programming software; in response to the drag of the second instruction block from the script programming interface to the offline simulation interface, the instruction block information of the second instruction block is acquired from the second database according to the second index information of the second instruction block, and the point position information of the second point position node is generated according to the data conversion relationship between the offline simulation software and the script programming software, the point position information of the second point position node is stored in the first database corresponding to the offline simulation software, and the second point position node is generated and displayed in the offline simulation interface according to the point position information of the second point position node.

[0011] As a possible implementation manner, the generation and display of the second point position node in the offline simulation interface according to the point position information of the second point position node, comprises: in response to the first drag operation of the user in the offline simulation interface, a first position corresponding to the first drag operation is determined, and the second point position node is generated and displayed in the first position according to the point position information of the second point position node.

[0012] As a possible implementation manner, the generating and displaying the second point node in the offline simulation interface according to the point information of the second point node comprises: in response to a second drag operation of a user in the offline simulation interface, determining a second position corresponding to the second drag operation, and generating and displaying the second point node in the second position according to the point information of the second point node.

[0013] As a possible implementation manner, the method further comprises: in response to a fourth operation of a user on a third point node in the offline simulation interface, modifying instruction block information of a third instruction block corresponding to the third point node in the script programming interface according to modification data of the third point node in the offline simulation software.

[0014] As a possible implementation manner, the modifying instruction block information of the third instruction block in the script programming interface according to the modification data of the third point node in the offline simulation software in response to the fourth operation of the user on the third point node in the offline simulation interface comprises: in response to a click operation of the user on the third point node, displaying a second modification interface, so that the user modifies point information of the third point node in the second modification interface, and updates the modified point information to a first database corresponding to the offline simulation software; converting the modified point information into modification data in the script programming software according to a data conversion relationship between the offline simulation software and the script programming software, and modifying instruction block information of the third instruction block in a second database corresponding to the script programming software based on the modification data in the script programming software.

[0015] As a possible implementation manner, the method further comprises: generating a motion trajectory script program according to a connection order of the plurality of instruction blocks displayed in the script programming interface, and sending the motion trajectory script program to the robot, so that the robot runs the motion trajectory script program.

[0016] In a second aspect, the embodiments of the present application provide a visual programming interaction processing apparatus. A terminal provides a graphical user interface, which includes an offline simulation interface of offline simulation software and a script programming interface of script programming software. The offline simulation interface displays a point node for describing a point in a motion trajectory of a robot. The apparatus includes a generation module configured to, in response to a first operation of a user on a first point node in the offline simulation interface, generate and display a first instruction block corresponding to the first point node in the script programming interface. The first point node is a node in the offline simulation software, and the first instruction block is an instruction block in the script programming software. The apparatus also includes a modification module configured to, in response to a second operation of the user on the first instruction block in the script programming interface, modify point information of the first point node in the offline simulation software according to modification data of the first instruction block in the script programming software.

[0017] As a possible implementation manner, the generation module is specifically configured to: in response to that the user selects and starts dragging the first point node in the offline simulation interface, acquire first index information of the first point node, the first index information being used to indicate a storage path of point information of the first point node in a first database corresponding to the offline simulation software; and in response to that the user drags the first point node from the offline simulation interface to the script programming interface, acquire the point information of the first point node from the first database according to the first index information of the first point node, generate instruction block information of the first instruction block according to a data conversion relationship between the offline simulation software and the script programming software, store the instruction block information of the first instruction block into a second database corresponding to the script programming software, and generate and display the first instruction block in the script programming interface according to the instruction block information of the first instruction block.

[0018] As a possible implementation manner, the generation module is specifically configured to: in response to a mouse release operation of the user in the script programming interface, generate and display the first instruction block at a mouse release position according to the instruction block information of the first instruction block.

[0019] As a possible implementation manner, the modification module is specifically configured to: in response to a click operation of a user on the first instruction block, display a first modification interface, so that the user modifies instruction block information of the first instruction block in the first modification interface, and updates modification data to a second database corresponding to the script programming software; convert the modification data into modification data in the offline simulation software according to a data conversion relationship between the offline simulation software and the script programming software, and modify the point information of the first point node in a first database corresponding to the offline simulation software based on the modification data in the offline simulation software.

[0020] As a possible implementation manner, the generation module is further configured to: in response to a creation operation of a user in the script programming interface, generate a second instruction block in the script programming interface; and in response to a third operation of the user on the second instruction block in the script programming interface, generate and display a second point node corresponding to the second instruction block in the offline simulation interface.

[0021] As a possible implementation manner, the generation module is further configured to: in response to the user selecting and starting to drag the second instruction block in the script programming interface, acquire second index information of the second instruction block, the second index information being used to indicate a storage path of instruction block information of the second instruction block in a second database corresponding to the script programming software; and in response to the user dragging the second instruction block from the script programming interface to the offline simulation interface, acquire the instruction block information of the second instruction block from the second database according to the second index information of the second instruction block, generate point information of the second point node according to a data conversion relationship between the offline simulation software and the script programming software, store the point information of the second point node into a first database corresponding to the offline simulation software, and generate and display the second point node in the offline simulation interface according to the point information of the second point node.

[0022] As a possible implementation manner, the generation module is further configured to: in response to a first dragging operation of a user in the offline simulation interface, determine a first position corresponding to the first dragging operation, and generate and display the second point node at the first position according to the point information of the second point node.

[0023] As a possible implementation manner, the generation module is further configured to: in response to a second dragging operation of a user in the offline simulation interface, determine a second position corresponding to the second dragging operation, and generate and display the second point node at the second position according to the point information of the second point node.

[0024] As a possible implementation manner, the modification module is further configured to: in response to a fourth operation of a user on a third point node in the offline simulation interface, modify instruction block information of a third instruction block corresponding to the third point node in the script programming interface according to modification data of the third point node in the offline simulation software.

[0025] As a possible implementation manner, the modification module is further configured to: in response to a click operation of a user on the third point node, display a second modification interface, so that the user modifies point information of the third point node in the second modification interface, and updates the modified point information to a first database corresponding to the offline simulation software; convert the modified point information into modification data in the script programming software according to a data conversion relationship between the offline simulation software and the script programming software, and modify instruction block information of the third instruction block in a second database corresponding to the script programming software based on the modification data in the script programming software.

[0026] As a possible implementation manner, the generation module is further configured to: generate a motion trajectory script program according to a connection order of the plurality of instruction blocks displayed in the script programming interface, and send the motion trajectory script program to the robot, so that the robot runs the motion trajectory script program.

[0027] In a third aspect, an embodiment of the present application provides a terminal device, including: a processor, a storage medium, and a bus, the storage medium stores machine readable instructions executable by the processor, when the terminal device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine readable instructions to perform steps of the visual programming interaction processing method in any one of the first aspect.

[0028] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program, when the computer program is run by a processor, steps of the visual programming interaction processing method in any one of the first aspect are executed.

[0029] According to the visual programming interaction processing method, the terminal device and the storage medium provided in the embodiments of the present application, in response to a first operation of a user on a first point node in an offline simulation interface, a first instruction block corresponding to the first point node is generated and displayed in a script programming interface. The first point node is a node in offline simulation software, and the first instruction block is an instruction block in script programming software. In response to a second operation of the user on the first instruction block in the script programming interface, point information of the first point node in the offline simulation software is modified according to modification data of the first instruction block in the script programming software. According to the embodiments of the present application, the offline simulation interface and the script programming interface are integrated in the same graphical user interface, and a bidirectional mapping relationship between the point node and the instruction block is established, thereby solving the problem that simulation and programming are disconnected and difficult to cooperate in the prior art. When the user operates the point node in the simulation interface, the corresponding visual instruction block is generated. Conversely, when the instruction block is modified in the programming interface, the coordinates of the corresponding point in the simulation interface are updated in real time according to the logic of the instruction block, thereby realizing bidirectional synchronization and incremental linkage of the simulation trajectory and the script program at the semantic level, breaking the one-way mode of generating a script from simulation and the script being separated from simulation in the traditional process, and significantly improving the efficiency and accuracy of robot programming. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0031] Figure 1 A schematic diagram of a graphical user interface provided by an embodiment of the present application is shown; Figure 2 A flowchart of a visual programming interaction processing method provided by an embodiment of the present application is shown; Figure 3 A flowchart of an instruction block generation method provided by an embodiment of the present application is shown; Figure 4 An interface operation schematic diagram provided by an embodiment of the present application is shown; Figure 5 A flowchart of a point information modification method provided by an embodiment of the present application is shown; Figure 6 A flowchart of a point node generation method provided by an embodiment of the present application is shown; Figure 7 Another interface operation schematic diagram provided by an embodiment of the present application is shown; Figure 8A flow diagram of a method for modifying instruction block information is shown. Figure 9 A structural diagram of a visual programming interactive processing device is shown. Figure 10 A structural diagram of a terminal device is shown. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of description and illustration, and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn according to the actual proportions. The flowcharts show the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can not be implemented in sequence, and the steps without logical context relationship can be reversed in sequence or implemented simultaneously. In addition, one or more other operations can be added to the flowcharts or one or more operations can be removed from the flowcharts under the guidance of the content of the present application by those skilled in the art.

[0033] In addition, the described embodiments are only some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0034] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0035] In many robot application scenarios, a user needs to first generate points and trajectories through offline simulation software, and then generate a script program that can be run by a robot through post-processing. The generated script program is imported into the robot on site, and a teach pendant or related application (APP) is used to fine-tune points, coordinate systems, and run and verify programs. In the above process, if the points in the simulation environment change or trajectories are added, it is relatively complex to update the script program. In the operation process of the simulation software, points, trajectories, and script programs are completely different two sets of things. Although the script program also uses points and trajectories, it can be understood that the points and trajectories in the script program and the points and trajectories in the simulation software are two different copies and cannot be synchronized by themselves. Similarly, if the script program is modified in the teach pendant, for example, a transition point or some path is added, it is difficult to synchronize back to the simulation environment because it is difficult to extract relevant information from the robot script program, especially when it is not running. When the robot runs the script, although the robot parser can parse the script program, this is a dynamic process, and it is difficult to specify to extract a certain specific information to the interface.

[0036] In addition, on the one hand, the current simulation software cannot modify specific points and trajectories in the simulation environment and then synchronize the modified content alone. The processing method is to generate a complete program or a subprogram. If a complete program is generated, the user's previous modifications on the script program through the teach pendant or the APP will be lost. If a subprogram is generated, the user also needs to load this subprogram on the APP and modify the previous program to reference this subprogram. On the other hand, the current offline programming software does not support graphical programming. For example, the robot simulation and offline programming software RobotStudio only supports script programming in text form. The offline simulation environment and script programming environment can only be operated through synchronization in large particles, such as operating a trajectory group as a whole.

[0037] In summary, the prior art has the following problems: on the one hand, in offline simulation, only trajectories are involved, and no logic is involved in simulating trajectories, but in script programs, usually a lot of logic instructions are involved, and the values of points, coordinate systems and the like used by the trajectories may come from some variables, and the values may be updated during running, which will cause difficulties in importing the script program into the offline simulation software. On the other hand, the information storage in the program and the information storage in the offline simulation software are inconsistent, and it is difficult to correspond. Therefore, when the offline simulation is synchronized to the script, the main program is overwritten or a new subprogram is created, and then the main program is manually modified to reference the new main program, and when the script is synchronized to the offline simulation software, the order of the trajectories may not be correct, and the coordinate system or point value of the trajectory can only use the default value or a certain initial value, so no matter which synchronization is used, it is not very intuitive and not very smooth, and the user needs to make additional adjustments.

[0038] To solve the above problems, the present application provides a visual programming interactive processing method. Considering that the instruction blocks in the script programming interface are placed in the canvas by dragging and releasing, and the trajectories in the offline simulation interface are generated after being dragged and released into the trajectory group, the operations of the two are both dragging and releasing, therefore, the script editing interface of the script programming software and the offline simulation interface of the offline simulation software are fused together, and the dragging and releasing operation can be operated between the script editing interface and the offline simulation interface. Moreover, since the user performs the dragging operation and confirms the specific position that needs to be synchronized, for example, which instruction in the script editing interface and the offline simulation interface needs to be synchronized, and the direction of synchronization, the influence of the logic instructions in the script can be reduced as much as possible, the possibility of error can be reduced, and when the trajectory parameters in the script come from variables, a prompt can be given in time, which significantly improves the user operation experience.

[0039] Figure 1 A schematic diagram of a graphical user interface provided by an embodiment of the present application is shown. Referring to Figure 1 The present application provides a graphical user interface through a terminal, which includes an offline simulation interface of offline simulation software and a script programming interface of script programming software. The offline simulation interface displays a TreeView view, and the TreeView view displays point nodes for describing points in a robot motion trajectory, such as tar_1, tar_2, tar_3 and tar_4 shown in Figure 1

[0040] The visual programming interactive processing method provided by the present application will be described in detail below in combination with the above graphical user interface.

[0041] Figure 2 ​A flowchart of a visual programming interaction processing method provided by an embodiment of the present application is shown, and the execution subject of the method is a terminal device. Referring to Figure 2 The method specifically includes the following steps: S201, in response to a first operation of a user on a first point node in an offline simulation interface, generating and displaying a first instruction block corresponding to the first point node in a script programming interface.

[0042] Optionally, the first point node is a node in offline simulation software, and the first instruction block is an instruction block in script programming software.

[0043] Optionally, when the user selects and drags the first point node in the offline simulation interface, the user operation is captured, and first index information of real point information pointing to the first point node is obtained, and the first index information is encapsulated in a QMimeData class and transmitted through a Multipurpose Internet Mail Extensions (MIME) mechanism. When the user drags the first point node into the script programming interface and releases the mouse, an instruction block generation process is triggered at the mouse release position, the complete point information of the first point node is read from a first database according to the first index information, and the point information is converted into structured instruction block information suitable for the programming interface according to a preset semantic mapping rule, and finally a corresponding visual instruction block is generated and displayed at the accurate coordinates of the mouse release. In this way, by integrating the drag interaction and cross-system data mapping, the visual instruction generation from the offline simulation interface to the script programming interface is realized, and the seamless connection of simulation data and programming logic is realized, ensuring the intuitiveness of user operation, data consistency and efficient cooperation between interfaces.

[0044] S202, in response to a second operation of a user on a first instruction block in a script programming interface, modifying the point information of a first point node in offline simulation software according to the modification data of the first instruction block in the script programming software.

[0045] Optionally, when the user clicks and modifies the first instruction block in the script programming interface, a modification interface is popped up to receive input, and the changed parameters are updated to the second database, then the modified data is inversely analyzed into a data format recognizable by the offline simulation software according to the semantic mapping rules between the script programming software and the offline simulation software, and the corresponding point node in the first database is accurately located through the first index information associated with the instruction block, and the point information such as the coordinates is synchronously updated. The process ensures that any adjustment of the script program can be real-time and accurately synchronized to the simulation model, realizes dynamic consistency of the programming logic and the simulation trajectory, and effectively solves the problem of disconnection between simulation and programming in the traditional method. In this way, the application realizes closed-loop updating from the programming side to the simulation side by establishing a bidirectional data mapping mechanism between script programming and offline simulation.

[0046] Based on this, the visual programming interaction processing method according to the embodiments of the application solves the problem of disconnection between simulation and programming in the prior art by integrating the offline simulation interface and the script programming interface in the same graphical user interface and establishing a bidirectional mapping relationship between the point node and the instruction block. When the user operates the point node in the simulation interface, the corresponding visual instruction block is generated, and conversely, when the instruction block is modified in the programming interface, the logic is analyzed and the coordinate information of the corresponding point in the simulation interface is real-time reversely updated, realizing bidirectional synchronization and incremental linkage of the simulation trajectory and the script program at the semantic level, thereby breaking the one-way mode of simulation generating script and script being separated from simulation in the traditional process, and significantly improving the efficiency and accuracy of robot programming.

[0047] Figure 3 A flowchart of an instruction block generation method provided by an embodiment of the application is shown. Referring to Figure 3 The above step S201 generates and displays the first instruction block corresponding to the first point node in the script programming interface in response to the first operation of the user on the first point node in the offline simulation interface, and specifically includes the following steps: S301, in response to the user selecting and starting to drag the first point node in the offline simulation interface, acquiring first index information of the first point node.

[0048] Optionally, the first index information is used to indicate a storage path of the point information of the first point node in the first database corresponding to the offline simulation software.

[0049] Exemplarily, when the user selects the first point node and presses the mouse in the offline simulation interface, the operation of selecting the first point node and pressing the mouse is captured, and the metadata of the first point node is queried. It is worth noting that in the process of querying, the first index information of the first point node is acquired in response to the user operation, which is not the actual coordinate value of the first point node, but a logical reference or path identifier pointing to the real data of the first point node in the corresponding first database of the offline simulation software, for example, a variable name or a unique identifier in the first database. The first index information is used for subsequent fast positioning and reading of the complete point information of the first point node, such as name, value, coordinate system, etc.

[0050] Further, to support the drag operation, the first index information of the first point node can be encapsulated by using a MIME mechanism. The MIME mechanism is a general type standard used to express the type of information. The application utilizes the MIME mechanism to pass the data type and content between different interface components to ensure that the information can be safely carried in the drag process. Specifically, the first index information of the first point node can be encapsulated in a QMimeData class. The QMimeData class is used to describe the data stored on the clipboard and to pass the data on the clipboard through the drag-and-drop event, so as to pass the data between different programs, or pass the data within the same program.

[0051] S302, in response to the user dragging the first point node from the offline simulation interface to the script programming interface, acquiring the point information of the first point node from the first database according to the first index information of the first point node, and generating the instruction block information of the first instruction block according to the data conversion relationship between the offline simulation software and the script programming software, storing the instruction block information of the first instruction block into the second database corresponding to the script programming software, and generating and displaying the first instruction block in the script programming interface according to the instruction block information of the first instruction block.

[0052] Optionally, when the user drags the mouse into the script programming interface, it is detected that the user drag action has entered any position of the script programming interface, and a visual instruction block preview moving with the mouse is rendered in real time. When the user releases the mouse, the first index information encapsulated in the QMimeData class is extracted, and the complete point information of the first point node is read from the first database according to the first index information.

[0053] Optionally, the data conversion relationship between the offline simulation software and the script programming software refers to a semantic mapping rule between the offline simulation software and the script programming software. After the complete point information of the first point node is read from the first database, the original point information of the first point node is converted into a structured instruction format suitable for a script programming interface according to the data conversion relationship, and instruction block information of the first instruction block is obtained. The instruction block information includes icon, text, color, parameter binding and other attribute information corresponding to the first instruction block. The converted instruction block information is stored in the second database of the script programming software, and the corresponding first instruction block is generated and displayed at a specified position of the script programming interface.

[0054] Optionally, the first instruction block is generated and displayed in the script programming interface according to the instruction block information of the first instruction block, including: in response to a mouse release operation of a user in the script programming interface, the first instruction block is generated and displayed at a position of the mouse release according to the instruction block information of the first instruction block.

[0055] Exemplarily, when the user drags the first instruction block, which represents the first point node, from the offline simulation interface to the script programming interface and releases the mouse button at a certain position of the script programming interface, a mouse release event is captured, which indicates the completion of the dragging operation and is an explicit instruction to generate the first instruction block. Moreover, the cursor coordinates when the mouse is released are recorded in real time, that is, the final position of the mouse pointer on the script programming interface, which is the accurate position where the first instruction block should be placed in the script programming interface. On this basis, a visual first instruction block is created and rendered in real time at the coordinate position of the mouse by using the instruction block information of the first instruction block converted during the dragging process, and the user can see in real time whether the first instruction block is placed at the desired position.

[0056] Exemplarily, referring to FIG. 6, the user selects the first point node “tar_4” in the offline simulation interface as an example. The first point node is selected and dragged until it is dragged into the script programming interface, that is, the first instruction block shown in FIG. 7 is displayed in the script programming interface. The first instruction block displays the node attribute “joint motion” corresponding to the first point node “tar_4” and the icon indicating that the first point node is “new point”. In addition, the icon “new programming” is displayed above the first instruction block, and subsequent programming can be performed in the script programming interface where the first instruction block has been created through the icon. Figure 4 Figure 4

[0057] ​​Based on this, the MIME mechanism is used to carry the index information of the user-dragged point position node, and the target position is accurately obtained when the mouse is released, and the corresponding visual instruction block is dynamically generated and positioned. This process not only automatically converts the abstract database point position information into interactive programming instructions, but also ensures the synchronization of simulation data and programming logic in semantics and space, significantly improving the intuitiveness and efficiency of cross-interface operations, and laying a data interaction foundation for realizing bidirectional collaborative visual programming.

[0058] Figure 5 A flowchart of a point position information modification method provided by an embodiment of the application is shown. Referring to Figure 5 The above step S202 modifies the point position information of the first point position node in the offline simulation software according to the modification data of the first instruction block in the script programming software in response to a second operation of the user on the first instruction block in the script programming interface, and specifically includes the following steps: S501, in response to a click operation of the user on the first instruction block, a first modification interface is displayed to enable the user to modify the instruction block information of the first instruction block in the first modification interface, and the modification data is updated to the corresponding second database of the script programming software.

[0059] Optionally, when the user clicks the first instruction block in the script programming interface, the click event is captured, and a first modification interface is popped up, for example, a property panel or an editing dialog box, and the first modification interface displays and allows the user to modify each parameter of the first instruction block. After the user completes the modification, the changes to the first instruction block are recorded as modification data, and the updated parameter values are written into the corresponding second database of the script programming software, ensuring that the data state on the programming side is kept up to date.

[0060] S502, according to the data conversion relationship between the offline simulation software and the script programming software, the modification data is converted into modification data in the offline simulation software, and the point position information of the first point position node in the corresponding first database of the offline simulation software is modified based on the modification data in the offline simulation software.

[0061] Optionally, after the second database is updated, the modification data is reversely analyzed based on the data conversion relationship between the offline simulation software and the script programming software, that is, the semantic mapping rule, the modification data is converted from the script programming format back to the data structure recognizable by the offline simulation software, then the first point position node in the corresponding first database of the offline simulation software is located by using the first index information referenced by the first instruction block, and the point position information thereof is updated to the converted value. In this way, reverse data synchronization from the programming side to the simulation side is realized, ensuring that the trajectory points in the simulation environment and the script instructions always remain consistent, forming a closed-loop collaboration.

[0062] Based on this, after the user modifies the instruction block parameters in the script programming interface, not only can the changes be stored persistently, but also the modified data can be converted into a format recognizable by the offline simulation software according to the preset semantic mapping rule, and the coordinate information of the corresponding point in the simulation database can be accurately updated through the original index. This processing process breaks the limitations of traditional one-way generation, so that script modification can be reflected in the simulation model in real time and accurately, ensuring the consistency of programming logic and simulation trajectory. In this way, not only is the reverse data synchronization and closed-loop update from the script programming interface to the offline simulation interface realized, but also the efficiency and reliability of robot program debugging are significantly improved.

[0063] As a possible implementation manner, the method further includes: in response to a creation operation of the user in the script programming interface, generating a second instruction block in the script programming interface, and in response to a third operation of the user on the second instruction block in the script programming interface, generating and displaying a second point node corresponding to the second instruction block in the offline simulation interface.

[0064] Optionally, the user first creates the second instruction block in the script programming interface and drags it into the offline simulation interface, extracts the second index information of the second instruction block to obtain the storage path of the second instruction block in the second database, reads the complete instruction block information of the second instruction block, and then reversely converts the instruction block information of the second instruction block into point data recognizable by the offline simulation software according to the semantic mapping rule between the script programming software and the offline simulation software, and stores the point data into the first database.

[0065] Further, the intention is intelligently judged according to the position where the user releases the dragging, specifically, if the release is on the point list, a new node is generated in the logical structure, and if the release is on the trajectory segment, the point is inserted in the space path and the trajectory connection is updated. In this way, the user can intuitively perform graphical operation to synchronize the newly added path point on the programming side to the simulation environment, realizing flexible point layout and closed-loop collaboration across interfaces and multiple modes, and realizing reverse point creation from script programming to offline simulation through the bidirectional dragging and data mapping mechanism.

[0066] Figure 6 A flowchart of a point node generation method provided by an embodiment of the application is shown. Referring to Figure 6 As shown in the above step, in response to the third operation of the user on the second instruction block in the script programming interface, a second point node corresponding to the second instruction block is generated and displayed in the offline simulation interface, specifically including the following steps: S601, in response to the user selecting and starting to drag the second instruction block in the script programming interface, obtaining second index information of the second instruction block.

[0067] Optionally, the second index information is used to indicate the storage path of the instruction block information of the second instruction block in the second database corresponding to the script programming software.

[0068] Optionally, when a user selects an existing second instruction block in the script programming interface and starts dragging with the mouse, the operation of the instruction block object is identified, and the metadata corresponding to the second instruction block is queried to obtain the second index information of the second instruction block. Similarly, the second index information is not a visual attribute of the second instruction block, but a logical reference or storage path pointing to the actual data of the second instruction block in the second database inside the script programming software. The second index information is used to read the complete instruction block information of the second instruction block in the future.

[0069] S602. In response to the user dragging the second instruction block from the script programming interface to the offline simulation interface, the instruction block information of the second instruction block is obtained from the second database according to the second index information of the second instruction block, and the point information of the second point node is generated according to the data conversion relationship between the offline simulation software and the script programming software. The point information of the second point node is stored in the first database corresponding to the offline simulation software, and the second point node is generated and displayed in the offline simulation interface according to the point information of the second point node.

[0070] Optionally, refer to Figure 7 As shown, when the user drags the second instruction block into the offline simulation interface and releases the mouse, the second index information is extracted, and the complete instruction block information of the second instruction block is read from the second database based on the second index information. Then, according to the semantic mapping rules between the offline simulation software and the script programming software, the instruction information in the script programming format is reverse-converted into a point data format that the offline simulation software can recognize. The converted point information is stored in the first database of the offline simulation software, and the corresponding second point node is generated in the TreeView view of the offline simulation interface, such as... Figure 7 The “tar_new” shown enables reverse data injection from the scripting environment to the simulation environment, allowing newly created or modified pathpoints to be visualized in the simulation model.

[0071] Optionally, generating and displaying the second point node in the offline simulation interface based on the point information of the second point node includes: responding to the user's first drag operation in the offline simulation interface, determining the first position corresponding to the first drag operation, and generating and displaying the second point node at the first position based on the point information of the second point node.

[0072] Exemplarily, the point list and the trajectory segment are included in the TreeView view of the offline simulation interface. When it is detected that the user's drag operation is finally released in the point list of the offline simulation interface, the first position corresponding to the operation is determined as the point area. At this time, according to the point information converted by the second instruction block, a new second point node is generated at a specified position of the point list, such as name sorting or user mouse hovering position, and the logical index thereof is registered to the first database.

[0073] Optionally, the second point node is generated and displayed in the offline simulation interface according to the point information of the second point node, including: in response to a second drag operation of the user in the offline simulation interface, determining a second position corresponding to the second drag operation, and generating and displaying the second point node at the second position according to the point information of the second point node.

[0074] Exemplarily, when it is detected that the user's drag operation is finally released on the trajectory segment of the offline simulation interface, the second position corresponding to the operation is determined as the trajectory area. At this time, not only a new second point node is generated according to the point information of the second instruction block, but also the new second point node is directly inserted into a specified position of the trajectory, that is, the 3D coordinates where the mouse is released. On this basis, the new point can be automatically connected with adjacent path points, the motion trajectory is updated, and the spatial relationship thereof is maintained in the database. This mode is suitable for intuitively expanding or correcting the motion path of the robot in space, and provides a more direct geometric editing experience.

[0075] Optionally, based on the above two modes, flexible point creation and layout are realized according to different user intentions.

[0076] Based on this, the present application allows the user to intuitively drag the newly created or modified instruction block in the script programming interface to the offline simulation interface, and intelligently generates the corresponding point node in the logical structure or spatial trajectory according to the user's intention, so as to quickly adjust the motion path of the robot. At the same time, by using the data mapping and semantic conversion mechanism, the data consistency and interoperability between the software platforms are ensured, and the collaborative workflow of programming and simulation is simplified. In this way, the present application realizes seamless data conversion and visual presentation from script programming to offline simulation, improves the operation efficiency and flexibility of the user in complex robot path planning, and the method not only supports accurate spatial editing experience, but also improves the accuracy and efficiency of path planning, so that the creation and correction of the robot motion path are more efficient and intuitive.

[0077] As a possible implementation manner, the method further includes: in response to a fourth operation of the user on a third point node in the offline simulation interface, modifying instruction block information of a third instruction block corresponding to the third point node in the script programming interface according to modification data of the third point node in the offline simulation software.

[0078] Optionally, when the user performs a fourth operation, such as a click operation, on a certain existing third point node in the offline simulation interface, the event is captured and a second modification interface, such as a property panel or a coordinate input box, is popped up to display detailed parameter information of the point node. The user can directly edit the point data in the second modification interface, and after the user completes the modification, the updated point information is written into the corresponding first database of the offline simulation software, ensuring that the data state in the simulation environment is real-time and accurately reflects the latest configuration.

[0079] Optionally, after the simulation data is updated, according to the semantic mapping rule between the offline simulation software and the script programming software, the modified point information in the first database is converted into a structured format that can be recognized by the script programming software, and then through the index information of the third point node, the corresponding third instruction block in the script programming interface is located, and the internal parameters of the instruction block are updated using the converted data. Finally, the change result is written into the second database of the script programming software, ensuring that the instruction content in the script program and the point data in the simulation model are consistent.

[0080] Figure 8 A flowchart of an instruction block information modification method provided by an embodiment of the application is shown. Referring to FIG. 8, the above steps modify the instruction block information of the third instruction block corresponding to the third point node in the script programming interface according to the modification data of the third point node in the offline simulation software in response to a fourth operation of the user on the third point node in the offline simulation interface, and specifically include the following steps: Figure 8 S801, in response to a click operation of the user on the third point node, a second modification interface is displayed to enable the user to modify the point information of the third point node in the second modification interface and update the modified point information to the corresponding first database of the offline simulation software.

[0081] Optionally, when the user clicks on a certain existing third point node in the offline simulation interface, the click event is captured, and a second modification interface, such as a property editing panel or a coordinate input dialog box, is popped up to display detailed information of the third point node. The user can directly modify the parameters of the third point node in the second modification interface, and after completing the editing, the changed content is recorded as modified point information which is written into the corresponding first database of the offline simulation software through a data binding mechanism, ensuring that the data state in the simulation environment is updated in real time.

[0082] ​S802, convert the modified point information into modified data in the script programming software according to a data conversion relationship between the offline simulation software and the script programming software, and modify instruction block information of the third instruction block in a second database corresponding to the script programming software based on the modified data in the script programming software.

[0083] Optionally, after the first database is updated, the updated point information is converted from an offline simulation format into a structured instruction format recognizable by the script programming software according to a semantic mapping rule between the offline simulation software and the script programming software. Then, a third instruction block corresponding to the third point node in a script programming interface is found through index information of the third point node, and internal parameters of the instruction block are updated by using the converted data. Finally, the change is written into the second database of the script programming software, so that the instruction block content on the programming side is consistent with the point data on the simulation side, and reverse loop synchronization from simulation to programming is realized.

[0084] Based on this, after the user modifies the point parameter on the simulation side, not only can the update be persisted to the simulation database, but also the change can be automatically converted and synchronized to the corresponding instruction block on the programming side according to the semantic mapping rule, so as to ensure that the path instruction in the script program and the point information in the simulation model are always consistent. In this way, the present application realizes reverse data synchronization loop from the offline simulation interface to the script programming interface, and the method breaks the limitation of traditional one-way generation, supports bidirectional real-time linkage, and significantly improves the debugging efficiency, data consistency and operation reliability of robot programming.

[0085] As a possible implementation manner, the method further includes: generating a motion trajectory script program according to a connection order of the plurality of instruction blocks displayed in the script programming interface, and sending the motion trajectory script program to the robot, so that the robot runs the motion trajectory script program.

[0086] Optionally, the present application generates an executable motion trajectory script program by analyzing the connection topology structure of the plurality of visual instruction blocks in the script programming interface. Each instruction block represents an atomized robot action, and the output port of each instruction block is connected to the input port of the next instruction block through a connection line to form an ordered execution chain.

[0087] Exemplarily, starting from a starting instruction block, the entire instruction flow is traversed according to the connection order, the type, parameters and bound data index of each instruction block are extracted, and according to a preset code generation template and a syntax specification of a target robot controller, these structured instructions are translated into textual robot program codes one by one. Further, after the generated script program is checked for syntax and logic, the script program is sent to a field robot controller through a network communication protocol, triggering the robot controller to load and run the motion trajectory script.

[0088] Based on this, the present application realizes seamless connection from visual programming to real robot execution, and users can intuitively construct a complete robot motion process by dragging and connecting instruction blocks without manually writing complex codes. Moreover, the present application automatically generates a script program in a standard format and completes deployment, thereby significantly reducing the programming threshold, improving the development efficiency and program reliability, and ensuring the consistency of offline simulation logic and on-site execution.

[0089] Based on the same inventive concept, the present application also provides a visual programming interaction processing device corresponding to the visual programming interaction processing method. Since the visual programming interaction processing device solves problems in the same principle as the visual programming interaction processing method, the implementation of the visual programming interaction processing device can be referred to the implementation of the visual programming interaction processing method, and repeated parts will not be described.

[0090] Referring to Figure 9 Fig. 9 is a structural schematic diagram of a visual programming interaction processing device provided by an embodiment of the present application. The visual programming interaction processing device 900 includes a generation module 901 and a modification module 902, wherein: The generation module 901 is configured to, in response to a first operation of a user on a first point node in an offline simulation interface, generate and display a first instruction block corresponding to the first point node in a script programming interface. The first point node is a node in offline simulation software, and the first instruction block is an instruction block in script programming software. The modification module 902 is configured to, in response to a second operation of the user on the first instruction block in the script programming interface, modify point information of the first point node in the offline simulation software according to modification data of the first instruction block in the script programming software.

[0091] Based on this, the visual programming interaction processing device according to the present application integrates the offline simulation interface and the script programming interface in the same graphical user interface and establishes a bidirectional mapping relationship between the point node and the instruction block, thereby solving the problem of the disconnection between simulation and programming in the prior art and the difficulty in cooperation. When the user operates the point node in the simulation interface, the corresponding visual instruction block is generated, and vice versa. When the instruction block is modified in the programming interface, the coordinates of the corresponding point in the simulation interface are updated in real time, thereby realizing bidirectional synchronization and incremental linkage of the simulation trajectory and the script program at the semantic level, breaking the one-way mode of script generation from simulation in the traditional process, and significantly improving the efficiency and accuracy of robot programming.

[0092] In a possible implementation, the generation module 901 is specifically configured to: in response to the user selecting and starting to drag the first point node in the offline simulation interface, acquire first index information of the first point node, the first index information being used to indicate a storage path of point information of the first point node in a first database corresponding to the offline simulation software; in response to the user dragging the first point node from the offline simulation interface to the script programming interface, acquiring the point information of the first point node from the first database according to the first index information of the first point node, and generating instruction block information of a first instruction block according to a data conversion relationship between the offline simulation software and the script programming software, storing the instruction block information of the first instruction block into a second database corresponding to the script programming software, and generating and displaying the first instruction block in the script programming interface according to the instruction block information of the first instruction block.

[0093] In a possible implementation, the generation module 901 is specifically configured to: in response to a mouse release operation of the user in the script programming interface, generating and displaying the first instruction block at a mouse release position according to the instruction block information of the first instruction block.

[0094] In a possible implementation, the modification module 902 is specifically configured to: in response to a click operation of the user on the first instruction block, displaying a first modification interface, so that the user modifies the instruction block information of the first instruction block in the first modification interface, and updating modification data to the second database corresponding to the script programming software; converting the modification data into modification data in the offline simulation software according to the data conversion relationship between the offline simulation software and the script programming software, and modifying the point information of the first point node in the first database corresponding to the offline simulation software based on the modification data in the offline simulation software.

[0095] In a possible implementation, the generation module 901 is further configured to: in response to a creation operation of the user in the script programming interface, generating a second instruction block in the script programming interface; and in response to a third operation of the user on the second instruction block in the script programming interface, generating and displaying a second point node corresponding to the second instruction block in the offline simulation interface.

[0096] In a possible implementation, the generation module 901 is further configured to: in response to the user selecting and starting to drag the second instruction block in the script programming interface, acquire second index information of the second instruction block, the second index information being used to indicate a storage path of instruction block information of the second instruction block in a second database corresponding to the script programming software; in response to the user dragging the second instruction block from the script programming interface to the offline simulation interface, acquiring the instruction block information of the second instruction block from the second database according to the second index information of the second instruction block, and generating point information of a second point node according to a data conversion relationship between the offline simulation software and the script programming software, storing the point information of the second point node into a first database corresponding to the offline simulation software, and generating and displaying the second point node in the offline simulation interface according to the point information of the second point node.

[0097] In a possible implementation, the generation module 901 is further configured to: in response to a first dragging operation of the user in the offline simulation interface, determining a first position corresponding to the first dragging operation, and generating and displaying the second point node at the first position according to the point information of the second point node.

[0098] In a possible implementation, the generation module 901 is further configured to: in response to a second dragging operation of the user in the offline simulation interface, determining a second position corresponding to the second dragging operation, and generating and displaying the second point node at the second position according to the point information of the second point node.

[0099] In a possible implementation, the modification module 902 is further configured to: in response to a fourth operation of the user on the third point node in the offline simulation interface, modifying instruction block information of a third instruction block corresponding to the third point node in the script programming interface according to modification data of the third point node in the offline simulation software.

[0100] In a possible implementation, the modification module 902 is further configured to: in response to a clicking operation of the user on the third point node, displaying a second modification interface, so that the user modifies point information of the third point node in the second modification interface, and updating the modified point information into the first database corresponding to the offline simulation software; converting the modified point information into modification data in the script programming software according to the data conversion relationship between the offline simulation software and the script programming software, and modifying instruction block information of the third instruction block in the second database corresponding to the script programming software based on the modification data in the script programming software.

[0101] In a possible implementation, the generation module 901 is further configured to: generating a motion trajectory script program according to a connection order of the plurality of instruction blocks displayed in the script programming interface, and sending the motion trajectory script program to the robot, so that the robot runs the motion trajectory script program.

[0102] The description of the processing flow of each module in the device and the interaction flow between the modules can refer to the related description in the above method embodiments, and will not be described in detail here.

[0103] The embodiment of the present application also provides a terminal device 1000, as shown in the structure schematic diagram of the terminal device 1000 provided by the embodiment of the present application, which comprises a processor 1001, a memory 1002, and optionally a bus 1003. The memory 1002 stores machine readable instructions executable by the processor 1001, when the terminal device 1000 runs, the processor 1001 and the memory 1002 communicate through the bus 1003, and the machine readable instructions are executed by the processor 1001 to perform the steps of the visual programming interaction processing method according to any one of the above. Figure 10 The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the visual programming interaction processing method according to any one of the above.

[0104] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to perform the steps of the visual programming interaction processing method according to any one of the above.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working process of the above-described system and device can refer to the corresponding process in the method embodiments, and will not be described in detail in the present application. In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. The device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some communication interface, indirect coupling or communication connection between the devices or modules, which can be electrical, mechanical or other forms.

[0106] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. When the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that are essential or contribute to the prior art can be embodied in the form of software products, which are stored in a storage medium and include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0107] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A method of visualizing programming interaction processing, the method comprising: A graphical user interface is provided via a terminal, the graphical user interface including: an offline simulation interface of offline simulation software and a script programming interface of script programming software, wherein the offline simulation interface displays point nodes for describing points in the robot's motion trajectory; the method includes: In response to a user's first operation on a first point node in the offline simulation interface, a first instruction block corresponding to the first point node is generated and displayed in the script programming interface. The first point node is a node in the offline simulation software, and the first instruction block is an instruction block in the script programming software. In response to the user's second operation on the first instruction block in the script programming interface, the point information of the first point node in the offline simulation software is modified according to the modification data of the first instruction block in the script programming software.

2. The method of claim 1, wherein, The response to the user's first operation on the first point node in the offline simulation interface generates and displays the first instruction block corresponding to the first point node in the script programming interface, including: In response to the user selecting and starting to drag the first point node in the offline simulation interface, the first index information of the first point node is obtained. The first index information is used to indicate the storage path of the point information of the first point node in the first database corresponding to the offline simulation software. In response to the user dragging the first point node from the offline simulation interface to the script programming interface, the system retrieves the point information of the first point node from the first database based on the first index information of the first point node, generates the instruction block information of the first instruction block according to the data conversion relationship between the offline simulation software and the script programming software, stores the instruction block information of the first instruction block in the second database corresponding to the script programming software, and generates and displays the first instruction block in the script programming interface according to the instruction block information of the first instruction block.

3. The method of claim 2, wherein, The step of generating and displaying the first instruction block in the script programming interface based on the instruction block information of the first instruction block includes: In response to the user's mouse release operation in the script programming interface, the first instruction block is generated and displayed at the mouse release position according to the instruction block information of the first instruction block.

4. The method according to claim 1, characterized in that, The response to the user's second operation on the first instruction block in the script programming interface, modifying the point information of the first point node in the offline simulation software according to the modification data of the first instruction block in the script programming software, includes: In response to a user's click on the first instruction block, a first modification interface is displayed, allowing the user to modify the instruction block information of the first instruction block on the first modification interface, and updating the modified data to the second database corresponding to the script programming software; Based on the data conversion relationship between the offline simulation software and the script programming software, the modified data is converted into modified data in the offline simulation software, and the point information of the first point node in the first database corresponding to the offline simulation software is modified based on the modified data in the offline simulation software.

5. The method according to claim 1, characterized in that, The method further includes: In response to the user's creation operation in the script programming interface, a second instruction block is generated in the script programming interface; In response to the user's third operation on the second instruction block in the script programming interface, the second point node corresponding to the second instruction block is generated and displayed in the offline simulation interface.

6. The method according to claim 5, characterized in that, The response to the user's third operation on the second instruction block in the script programming interface, generating and displaying the second point node corresponding to the second instruction block in the offline simulation interface, includes: In response to the user selecting and starting to drag the second instruction block in the script programming interface, the second index information of the second instruction block is obtained. The second index information is used to indicate the storage path of the instruction block information of the second instruction block in the second database corresponding to the script programming software. In response to the user dragging the second instruction block from the script programming interface to the offline simulation interface, the system retrieves the instruction block information of the second instruction block from the second database according to the second index information of the second instruction block, generates the point information of the second point node according to the data conversion relationship between the offline simulation software and the script programming software, stores the point information of the second point node in the first database corresponding to the offline simulation software, and generates and displays the second point node in the offline simulation interface according to the point information of the second point node.

7. The method according to claim 6, characterized in that, The step of generating and displaying the second point node in the offline simulation interface based on the point information of the second point node includes: In response to the user's first drag operation in the offline simulation interface, the system determines the first position corresponding to the first drag operation, and generates and displays the second point node at the first position based on the point information of the second point node.

8. The method according to claim 6, characterized in that, The step of generating and displaying the second point node in the offline simulation interface based on the point information of the second point node includes: In response to the user's second drag operation in the offline simulation interface, the system determines the second position corresponding to the second drag operation, and generates and displays the second point node at the second position based on the point information of the second point node.

9. The method according to claim 1, characterized in that, The method further includes: In response to the user's fourth operation on the third point node in the offline simulation interface, the instruction block information of the third instruction block corresponding to the third point node in the script programming interface is modified according to the modification data of the third point node in the offline simulation software.

10. The method according to claim 9, characterized in that, The response to the user's fourth operation on the third node in the offline simulation interface involves modifying the instruction block information of the third instruction block corresponding to the third node in the script programming interface according to the modification data of the third node in the offline simulation software, including: In response to the user's click operation on the third point node, a second modification interface is displayed so that the user can modify the point information of the third point node in the second modification interface, and the modified point information is updated to the first database corresponding to the offline simulation software. Based on the data conversion relationship between the offline simulation software and the script programming software, the modified point information is converted into modified data in the script programming software, and the instruction block information of the third instruction block in the second database corresponding to the script programming software is modified based on the modified data in the script programming software.

11. The method according to claim 1, characterized in that, The method further includes: Based on the connection order of multiple instruction blocks displayed in the script programming interface, a motion trajectory script program is generated and sent to the robot so that the robot can run the motion trajectory script program.

12. A terminal device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, wherein when the terminal device is running, the processor executes the machine-readable instructions to perform the steps of the visual programming interactive processing method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the visual programming interactive processing method as described in any one of claims 1 to 11.

Images