Graphical programming method, processor, terminal and computer program product

Abstract

The present invention discloses a task management method and a processor, a terminal and a computer program product. The method comprises: in a first interface, displaying a programming flowchart matching a target task to be executed, wherein the programming flowchart includes at least two function blocks for realizing the target task, and the first interface is used to configure the function blocks in the programming flowchart and the execution logic between the function blocks; determining a first function block and a second function block from the programming flowchart, wherein the first function block is used to obtain at least two first information to be processed, different first information corresponds to different information processing results, and the second function block is used to obtain the information processing result corresponding to the first information; configuring a first control between the first function block and the second function block, wherein the first control is used to determine the current information processing result according to the first information currently detected from the first function block, and transmit the current processing result to the second function block. The present invention solves the problem that the programming processing operation in the related art is relatively complicated.

Description

Technical Field

[0001] The present invention relates to the field of electronic control, and in particular to a graphical programming method, a processor, a terminal and a computer program product. Background Art

[0002] In many tools for automating production line projects through robots, multiple tasks are often set for different objects on the production line. The graphical programming flowcharts used to manage different objects and their corresponding tasks usually include many confusing cross lines, making it difficult for many engineers to understand the integration or flow of various tasks on the production line.

[0003] At present, the common way of backend system is for programmers to manually write program files to call different functional modules or data ports to achieve the purpose of task management. However, the above functional modules or data ports are independent of each other and have no association with the user's actual application scenarios from the user's perspective. Summary of the invention

[0004] The main purpose of the present invention is to provide a graphical programming method and a processor, a terminal and a computer program product to solve the problem of complex programming processing operations in the prior art.

[0005] In order to achieve the above-mentioned purpose, according to one embodiment of the present invention, in a first interface, a programming flowchart matching the target task to be executed is displayed, wherein the programming flowchart includes at least two function blocks for realizing the target task, and the first interface is used to configure the function blocks in the programming flowchart and the execution logic between the function blocks; a first function block and a second function block are determined from the programming flowchart, wherein the first function block is used to obtain at least two first information to be processed, different first information corresponds to different information processing results, and the second function block is used to obtain the information processing result corresponding to the first information; a first control is configured between the first function block and the second function block, wherein the first control is used to determine the current information processing result according to the first information currently detected from the first function block, and transfer the current processing result to the second function block.

[0006] In this way, the first function block and the second function block are reused through the first control, so that at least two first information that perform the same processing action can complete the application programming through the first control, without the need to configure different programming flowcharts for different first information, thereby achieving the purpose of simplifying the application programming operation. Furthermore, directly programming and configuring the graphically displayed function blocks and their execution logic in the first interface will also be beneficial from the user's perspective, combining the application scenario to complete the programming configuration process, assisting novice engineers to more easily complete the application programming process, and achieving the effect of reducing the complexity and time cost of programming operations.

[0007] Further, according to an embodiment of the present invention, the above-mentioned determination of the first function block and the second function block from the above-mentioned programming flowchart includes: obtaining attribute description information corresponding to each function block in the above-mentioned programming flowchart, wherein the above-mentioned attribute description information is used to describe the attributes of the information to be processed in the above-mentioned function block; determining the above-mentioned function block indicated by the above-mentioned attribute description information including at least two information as the above-mentioned first function block; obtaining a first result type of the information processing result corresponding to the first information to be processed in the above-mentioned first function block; searching for a function block corresponding to the above-mentioned first result type in the above-mentioned each function block as the above-mentioned second function block matching the above-mentioned first function block.

[0008] In this way, the first function block and the second function block in the programming flowchart are determined according to the attribute description information of each function block in the programming flowchart, so as to quickly determine the first function block and the second function block to be reused from multiple function blocks, thereby shortening the time for creating the first control between the first function block and the second function block, thereby achieving the purpose of improving programming efficiency.

[0009] Further, according to an embodiment of the present invention, configuring the first control between the first functional block and the second functional block includes: creating the first control in the first interface; connecting the input end of the first control to the first functional block, and connecting the output end of the first control to the second functional block.

[0010] In this way, after determining the first function block and the second function block from the programming flowchart, a first control is created between the two to facilitate the reuse of the function blocks based on the first control without establishing multiple complex process information chains, thereby simplifying the programming process and programming operations, thereby achieving the effect of reducing the complexity of programming operations.

[0011] Further, according to an embodiment of the present invention, the creation of the first control in the first interface includes: obtaining selection branches matching the first functional block and output results corresponding to each selection branch through guidance information, wherein the number of the first information to be processed in the first functional block is consistent with the number of selection branches set in the first control, and each of the output results includes the information processing results configured for each of the first information; creating the first control according to the selection branches and the output results.

[0012] In this way, the selection branches matching the first functional block and the output results corresponding to each selection branch are obtained through the guidance information, so as to assist novice engineers to gradually complete the graphical programming process in the first interface through the guidance information, and configure a corresponding programming flowchart for the target task, so as to achieve the purpose of simplifying the complexity of the programming operation. Further, under the premise of simplifying the programming process, it will be helpful to expand the scope of application of the graphical programming tool and make it applicable to more scenarios.

[0013] Further, according to an embodiment of the present invention, obtaining a selection branch matching the above-mentioned first functional block through guidance information includes: obtaining an image including multiple objects to be processed; identifying object information of each object from the above-mentioned image; grouping the above-mentioned identified object information to obtain at least two of the above-mentioned first information, wherein each of the above-mentioned selection branches corresponds to one of the above-mentioned first information.

[0014] In this way, after identifying the object information of each object from the acquired image, it is grouped to obtain multiple first information, and then each first information is used as a selection branch in the first control to achieve the purpose of creating the first control based on the guide information, thereby guiding the user to create the first control based on the guide information, simplifying the creation process, and making it easier for novice engineers to understand and use the programming tool.

[0015] Further, according to an embodiment of the present invention, obtaining the output result corresponding to each selection branch through guidance information includes: in a second interface associated with the first interface, displaying input prompt information corresponding to each of the above-mentioned selection branches, wherein the second interface is used to obtain the above-mentioned information processing result configured for the above-mentioned first information; according to the above-mentioned input prompt information, obtaining the above-mentioned information processing result corresponding to the above-mentioned selection branch as the above-mentioned output result of the above-mentioned selection branch.

[0016] In this way, the information processing result corresponding to each selection branch is obtained according to the input prompt information corresponding to each selection branch, and the first control is created as the corresponding output result, so that the first function block can be reused directly using the first control in subsequent application programming, so as to achieve the purpose of efficiently processing the different first information therein.

[0017] Further, according to an embodiment of the present invention, the creation of the first control in the first interface includes: automatically generating the corresponding first control in the first interface according to the number of the first information to be processed in the first function block, wherein the number of selection branches set in the first control is consistent with the number of the first information; in each selection branch of the first control, respectively configuring the information processing result corresponding to each of the first information as the output result of the selection branch.

[0018] In this way, the first control is created based on the object information of the registered and identified object and the configured information processing result. This allows the graphical programming process to be quickly completed in the first interface, and a corresponding programming flowchart is configured for the target task, so as to achieve the purpose of simplifying the complexity of the programming operation. Further, under the premise of simplifying the programming process, it will be beneficial to expand the scope of application of the graphical programming tool and make it applicable to more scenarios.

[0019] Further, according to an embodiment of the present invention, before automatically generating the corresponding first control in the first interface according to the number of the first information to be processed in the first functional block, it also includes: acquiring an image including multiple objects to be processed; identifying object information of each object from the image; and determining at least two of the first information based on the identified object information.

[0020] In this way, multiple first information are directly obtained through the results obtained after object recognition, and then each first information is used as a selection branch in the first control to achieve the purpose of automatically creating the first control, thereby simplifying the creation process and making it easier for novice engineers to understand and use the programming tool.

[0021] Further, according to an embodiment of the present invention, in each selection branch of the above-mentioned first control, the above-mentioned information processing results corresponding to each of the above-mentioned first information are respectively configured, and the output results of the above-mentioned selection branch include: displaying a candidate result list matching the above-mentioned first control, wherein the above-mentioned candidate result list includes the above-mentioned information processing results corresponding to each of the above-mentioned first information respectively; determining the target information processing result corresponding to the above-mentioned first information to be configured currently from the above-mentioned candidate result list; obtaining a drag instruction generated by performing a drag operation on the above-mentioned target information processing result; and in response to the above-mentioned drag instruction, dragging the above-mentioned target information processing result to the output result corresponding to the above-mentioned first information.

[0022] In this way, the information processing result corresponding to each selection branch is obtained by dragging and dropping, and used as the corresponding output result to create a first control, so that the first function block can be reused directly using the first control in subsequent application programming, thereby achieving the purpose of efficiently processing different first information therein.

[0023] Further, according to an embodiment of the present invention, after configuring the first control between the first functional block and the second functional block, it also includes: obtaining a first operation performed on the first control; in response to the first operation, adjusting the first control to a disabled state, wherein the disabled state is used to indicate that the first control is terminated; or obtaining a second operation performed on the first control; in response to the second operation, adjusting the first control to an available state, wherein the available state is used to indicate that the first control is resumed.

[0024] In this way, through the interface of the application programming, the operating state of the first control and its selection branch can also be directly adjusted, thereby achieving the purpose of improving control flexibility.

[0025] According to another aspect of the present invention, a processor is provided, wherein the processor is used to run a program, wherein the method described herein is performed when the program is run.

[0026] According to another aspect of the present invention, a terminal is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include means for executing the described method.

[0027] According to yet another aspect of the present invention, there is also provided a computer program product, which is tangibly stored on a computer-readable medium and includes computer-executable instructions, which when executed cause at least one processor to perform the method described.

[0028] The technical solution of the present invention is applied, by providing a graphical programming, after displaying a programming flowchart matching the target task to be executed in the first interface, a first function block and a second function block are determined from the programming flowchart, wherein the programming flowchart includes at least two function blocks, and the first function block determined above is used to obtain at least two first information to be processed, and different first information corresponds to different information processing results. The second function block is used to obtain the information processing result corresponding to the first information. Then, a first control is configured between the first function block and the second function block, and the first control is used to determine the current information processing result according to the first information currently detected from the first function block, and pass the information processing result to the second function block, so that the second function block further processes the information processing result, and then passes the further processed result to the subsequent function block in the programming flowchart, thereby completing the above target task. That is to say, in this embodiment, the first function block and the second function block are reused by the first control, so that at least two first information that perform the same processing action can complete the application programming through the first control, without configuring different programming flowcharts for different first information, thereby achieving the purpose of simplifying the operation of application programming. Furthermore, directly programming and configuring the graphically displayed function blocks and their execution logic in the first interface will also be beneficial for completing the programming configuration process from the user's perspective and combining it with the application scenario, assisting novice engineers to complete the application programming process more easily and achieving the effect of reducing the complexity and time cost of programming operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0030] Figure 1 A flowchart showing an optional graphical programming method according to the present invention; and

[0031] Figure 2 A schematic diagram showing a programming flow chart in an optional graphical programming method according to the present invention;

[0032] Figure 3 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0033] Figure 4 A schematic diagram showing a programming flow chart in an optional graphical programming method according to the present invention;

[0034] Figure 5A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0035] Figure 6 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0036] Figure 7 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0037] Figure 8 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0038] Fig. 9 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0039] Fig.10 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0040] Fig.11 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0041] Fig.12 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0042] Fig.13 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0043] Fig.14 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0044] Fig.15 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0045] Fig.16 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0046] Fig.17 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention;

[0047] Fig.18 A schematic diagram showing a programming flow chart in another optional graphical programming method according to the present invention; and

[0048] Fig.19 A schematic diagram of a programming flow chart in yet another optional graphical programming method according to the present invention is shown.

[0049] The above drawings include the following reference numerals:

[0050] S102: Displaying a programming flowchart matching the target task to be executed in the first interface, wherein the programming first interface is used to configure the function blocks in the programming flowchart and the execution logic between the function blocks;

[0051] S104: determining a first function block and a second function block from the programming flow chart, wherein the first function block is used to obtain at least two first information to be processed, different first information corresponds to different information processing results, and the second function block is used to obtain the information processing result corresponding to the first information;

[0052] S106: configuring a first control between the first functional block and the second functional block, wherein the first control is used to determine a current information processing result according to first information currently detected from the first functional block, and transmit the current processing result to the second functional block.

[0053] S202: Triggering the start of the task in the programming flow chart;

[0054] S204: Execute the processing action in FB-1;

[0055] S206: Execute the processing action in FB-2;

[0056] S208: Execute the processing action in FB-3;

[0057] S210: Execute the processing action in FB-4;

[0058] S212: Execute the processing action in FB-5;

[0059] S214: Execute the processing action in FB-6;

[0060] S216: Complete the task.

[0061] 30: first control;

[0062] 31: Object;

[0063] 32: Object.

[0064] 40: Dimension;

[0065] 41: Dimension;

[0066] 42: Dimension;

[0067] 43: Dimension;

[0068] 44: Dimension;

[0069] 45: Dimension;

[0070] 46: Image display area;

[0071] 46-1: Scissors;

[0072] 46-2: Battery;

[0073] 46-3: Circle object;

[0074] 46-4: rectangular object;

[0075] S402: Add the image obtained after taking the photo.

[0076] S502: Perform object recognition on the image.

[0077] 60: Region;

[0078] 61: circular object;

[0079] 62: Rectangle object.

[0080] 70: Group adding area;

[0081] 71: Object adding area;

[0082] 72: Information configuration area.

[0083] 80: object;

[0084] 81: Object;

[0085] 82: Object.

[0086] 100: Collaboration system interface;

[0087] 101: input box;

[0088] 102: Input box.

[0089] 110: first control;

[0090] 111: Information;

[0091] 112: Information.

[0092] 120: Select area;

[0093] 121: Confirm the area.

[0094] 130: Object;

[0095] 131: Object.

[0096] 140: first control;

[0097] S1402: Configure first information of a first control connected to FB-2.

[0098] 150: trigger button;

[0099] 151: first control;

[0100] 152: Select branch;

[0101] 153: output box;

[0102] 154: Candidate result list.

[0103] 160: Fill the area with diagonal lines.

[0104] 180: First control.

[0105] 190: First control. DETAILED DESCRIPTION

[0106] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0107] It should be noted that, unless otherwise specified, all technical and scientific terms used in this application have the same meanings as commonly understood by ordinary technicians in the technical field to which this application belongs.

[0108] According to one aspect of an embodiment of the present invention, a graphical programming method is provided. Optionally, as an optional implementation, Figure 1 As shown, the above graphical programming method can be but is not limited to:

[0109] S102, in a first interface, displaying a programming flowchart matching a target task to be executed, wherein the programming flowchart includes at least two function blocks for implementing the target task, and the first interface is used to configure the function blocks in the programming flowchart and the execution logic between the function blocks;

[0110] S104, determining a first function block and a second function block from the programming flow chart, wherein the first function block is used to obtain at least two first information to be processed, different first information corresponds to different information processing results, and the second function block is used to obtain the information processing result corresponding to the first information;

[0111] S106, configuring a first control between the first functional block and the second functional block, wherein the first control is used to determine a current information processing result according to first information currently detected from the first functional block, and transmit the current processing result to the second functional block.

[0112] Optionally, in this embodiment, the above-mentioned graphical programming method can be applied to, but not limited to, a computer programming tool, and the function blocks and execution logic between the function blocks required in the target task to be executed can be configured through the human-computer interaction interface provided in the computer programming tool. The above-mentioned interaction interface can be, but is not limited to, a channel for information exchange between people and computers. The user inputs information and performs operations to the computer through the interaction interface, and the computer provides information to the user through the interaction interface for reading, analysis and judgment. In addition, each function block (Function Block, FB for short) in this embodiment will be presented in a graphical form in the above-mentioned human-computer interaction interface, so as to complete the application programming process of the target task directly through the configuration operation on the human-computer interaction interface. Among them, each of the above-mentioned function blocks respectively integrates a group of program instructions in application programming, and the group of program instructions is used to perform a processing action. For example, in the case of Figure 2 In the first interface shown, multiple functional blocks required to achieve the target task can be configured, such as Figure 2 The programming flow chart shown includes at least six function blocks, which are: FB-1 for moving the object to the shooting position, FB-2 for obtaining the sorting position of the object, FB-3 for moving to the shooting position, FB-4 for grabbing the object, FB-5 for moving the object to the box position to be released, and FB-6 for releasing the object. Figure 2 As shown, the corresponding actions in the programming flowchart are executed in sequence through the above-mentioned functional blocks, such as S202, triggering the start of the task in the programming flowchart; such as S204-S214, sequentially executing the processing actions in FB-1 to FB-6, and finally, such as S216, completing the task. The above is an example, and this embodiment does not make any limitation to this.

[0113] It should be noted that, in this embodiment, after displaying the programming flowchart matching the target task to be executed in the first interface, the first function block and the second function block are determined from the programming flowchart, wherein the programming flowchart includes at least two function blocks, and the first function block determined above is used to obtain at least two first information to be processed, and different first information corresponds to different information processing results. The second function block is used to obtain the information processing result corresponding to the first information. Then, a first control is configured between the first function block and the second function block, and the first control is used to determine the current information processing result according to the first information currently detected from the first function block, and pass the information processing result to the second function block, so that the second function block further processes the information processing result, and then passes the further processed result to the subsequent function block in the programming flowchart, thereby completing the above target task. That is to say, in this embodiment, the first function block and the second function block are reused by the first control, so that at least two first information that perform the same processing action can complete the application programming through the first control, without configuring different programming flowcharts for different first information, thereby achieving the purpose of simplifying the operation of application programming. Furthermore, directly programming and configuring the graphically displayed function blocks and their execution logic in the first interface will also be beneficial for completing the programming configuration process from the user's perspective and combining it with the application scenario, assisting novice engineers to complete the application programming process more easily and achieving the effect of reducing the complexity and time cost of programming operations.

[0114] Optionally, in this embodiment, the method of determining the first function block and the second function block from the programming flowchart may include but is not limited to: obtaining attribute description information of each function block; determining a function block that will process at least two information based on the attribute description information, and using it as the first function block; obtaining a first result type of an information processing result obtained after processing the first information in the first function block, and then determining a second function block that matches the above-mentioned first function block based on the first result type.

[0115] It should be noted that, in this embodiment, the attribute description information may be, but is not limited to, used to indicate the type and quantity of information to be processed by the function block. When the number of information is at least two, it indicates that the function block will be reused, and the function block may be determined as the first function block. Further, the result type of the information processing result after the function block processes the information is obtained, and from the function blocks after the first function block, another function block using the result type is determined as the second function block matching the first function block.

[0116] In addition, in this embodiment, the first control may include but is not limited to a conditional selection control (such as Switch) for selecting different information to be processed. Through the first control, the corresponding information processing result can be directly selected and determined according to the matching relationship based on the detected current input information.

[0117] For example, Figure 3 As shown, assuming that Figure 2 Take the programming flowchart shown as an example. In FB-2, it is determined that the two first information to be moved include: object 31 (abbreviated as Obj_1) and object 32 (abbreviated as Obj_2), then FB-2 can be determined as the first functional block. Then it is determined that the information processing result after the first functional block is processed is the release position, such as object 31 (abbreviated as Obj_1) corresponds to position 1 (abbreviated as P1), object 32 (abbreviated as Obj_2) corresponds to position 2 (abbreviated as P2), then FB-5 used to obtain the release position can be determined as the second functional block. Then configure the first control between the above FB-2 and FB-5, such as Figure 3 The input end of the first control 30 is connected to FB-2, and the output end is connected to FB-5, wherein the first control 30 records two mapping relationships between the first information and the information processing results (i.e., the mapping relationship between object 31 and position P1, and the mapping relationship between object 32 and position P2).

[0118] Optionally, in this embodiment, the method of creating the first control may include but is not limited to one of the following:

[0119] 1) Guiding the user to create the first control through guidance information. For example, guiding the user to determine each selection branch and the output result corresponding to each selection branch. The number of selection branches is consistent with the number of first information to be processed in the first functional block, and the output result includes the information processing result; then selecting the branch and the output result to create the first control;

[0120] 2) Automatically create a first control according to the pre-acquired first information to be processed. For example, automatically generate the first control according to the number of pre-acquired first information, where the number of selection branches in the first control is consistent with the number of the first information; then configure the output result for the first control, and the output result includes the information processing result.

[0121] Optionally, in this embodiment, the output result of the first control (that is, the result type of the information processing result) may include but is not limited to: the position coordinates of the moving position, the moving path, the storage path of the read and write data, the modified data content or value, etc. This is an example and is not limited in this embodiment.

[0122] That is to say, in this embodiment, different first controls can be flexibly created according to the actual needs of users in different application scenarios, so as to achieve more efficient reuse of function blocks, thereby achieving the effect of improving the efficiency of programming using function blocks.

[0123] Through the embodiments provided by the present application, the first function block and the second function block are reused through the first control, so that at least two first information that perform the same processing action can complete the application programming through the first control, without the need to configure different programming flow charts for different first information, thereby achieving the purpose of simplifying the application programming operation. Furthermore, directly programming and configuring the graphically displayed function blocks and their execution logic in the first interface will also be beneficial from the user's perspective, combining the application scenario to complete the programming configuration process, assisting novice engineers to more easily complete the application programming process, and achieving the effect of reducing the complexity and time cost of programming operations.

[0124] As an optional solution, determining the first functional block and the second functional block from the programming flow chart includes:

[0125] S1, obtaining attribute description information corresponding to each function block in the programming flow chart, wherein the attribute description information is used to describe the attributes of the information to be processed in the function block;

[0126] S2, determining the function block including at least two pieces of information indicated by the attribute description information as the first function block;

[0127] S3, obtaining a first result type of the information processing result corresponding to the first information to be processed in the first functional block;

[0128] S4, searching for a function block corresponding to the first result type in each function block as a second function block matching the first function block.

[0129] Optionally, in this embodiment, each function block in the above programming flow chart may be configured with corresponding attribute description information, but not limited to, respectively. The attributes of the information described in the attribute description information may include but are not limited to the following: information type, information quantity, size of the object indicated by the information, weight of the object indicated by the information, etc. For example, assuming that the current target task is an item delivery task, and the object indicated by the information here is the item to be delivered, then the multiple function blocks included in the programming flow chart for realizing the item delivery task here will control the machine equipment (such as hardware equipment such as a robot or a mechanical arm) through the programming program to cooperate to complete the above task.

[0130] For example, in this scenario, the above-mentioned multiple function blocks may include: function block FB-1 for taking images of items to be delivered, function block FB-2 for obtaining the position of the sorting table where the items are located, function block FB-3 for moving the items to the sorting table, function block FB-4 for grabbing the items, function block FB-5 for moving the items to the position of the packaging box, and function block FB-6 for releasing the items. Further, in the case where the items to be delivered include at least two items identified by the image, the information corresponding to the function block FB-2 for obtaining the position of the sorting table where the items are located includes at least two, corresponding to the two items respectively. Then, FB-2 is determined as the first function block, and then it is determined that each item will eventually be placed in different packaging boxes, and the function block FB-5 for moving the items to the position of the packaging box is determined as the second function block.

[0131] Through the embodiments provided in the present application, the first function block and the second function block in the programming flowchart are determined according to the attribute description information of each function block in the programming flowchart, so as to quickly determine the first function block and the second function block to be reused from multiple function blocks, thereby shortening the time for creating the first control between the first function block and the second function block, thereby achieving the purpose of improving programming efficiency.

[0132] As an optional solution, configuring the first control between the first functional block and the second functional block includes:

[0133] S1, creating a first control in a first interface;

[0134] S2, connecting the input end of the first control to the first functional block, and connecting the output end of the first control to the second functional block.

[0135] Optionally, in this embodiment, the first control may be, but is not limited to, a conditional selection control with a selection branch, such as Switch. It should be noted that the first control may set a selection branch according to the type of information or the amount of information. The following description will take the selection branch in the first control constructed by the amount of information as an example.

[0136] For example, Figure 3As shown, assuming that the number of first information is 2, a first control can be created, which includes two selection branches, object 31 (abbreviated as Obj_1) and object 32 (abbreviated as Obj_2). If (if) the currently detected information is object 31, the first branch will be selected; if (if) the currently detected information is object 32, the second branch will be selected. Further configure the information processing result for each selection branch, such as configuring the information processing result P1 for object 31 and configuring the information processing result P2 for object 32. It should be noted that the content of the output results of the two branches here (such as the value Value) is different, but they are the same result type, and are both used for the second functional block.

[0137] After creating the first control, just Figure 3 As shown, the input end of the first control is connected to a first functional block (such as FB-2) that obtains multiple information, and the output end of the first control is connected to a second functional block (such as FB-5) that obtains information processing results of the multiple information.

[0138] Through the embodiments provided in the present application, after determining the first function block and the second function block from the programming flowchart, a first control is created between the two to facilitate the reuse of the function blocks based on the first control without establishing multiple complex process information chains, thereby simplifying the programming process and programming operations, thereby achieving the effect of reducing the complexity of programming operations.

[0139] As an optional solution, creating a first control in the first interface includes:

[0140] S1, obtaining a selection branch matching the first function block and an output result corresponding to each selection branch through the guidance information, wherein the number of first information to be processed in the first function block is consistent with the number of selection branches set in the first control, and each output result includes an information processing result configured for each first information;

[0141] S2, creating a first control according to the selection branch and the output result.

[0142] Optionally, in this embodiment, the above-mentioned guidance information can be used, but is not limited to, to guide the user to input selection branches and output results corresponding to each selection branch. Here, the number of selection branches is consistent with the number of first information to be processed, so that each first information corresponds to a selection branch, so that when the first information is detected, the output result in the corresponding selection branch can be obtained as the information processing result of the first information.

[0143] Optionally, in this embodiment, the method of obtaining the selection branch matching the first functional block may be, but is not limited to: grouping the objects to be processed in the target task according to the guidance information, and using the object information in a group as a first information. That is, in this embodiment, the selection branch required in the first control is determined by configuring the above objects in groups. It should be noted that, in this embodiment, the above objects to be processed may be, but are not limited to, determined by performing object recognition in the image of the image acquisition device.

[0144] In addition, in the present embodiment, the above-mentioned method of obtaining the output result corresponding to the selection branch may be, but is not limited to: configuring the information processing result in each selection branch according to the guidance information. For example, in the case where the processing action of the first functional block is to move multiple objects, the first information is used to indicate the objects after different groupings, and the output result of each selection branch is used to indicate the moving path or the position after movement of the objects in this group. For another example, in the case where the processing action of the first functional block is to identify multiple objects, the first information is used to indicate the objects after different groupings, and the output result of each selection branch is used to indicate the storage location of the identification result of the objects in this group. This is an example, and no limitation is made to this in the present embodiment.

[0145] Through the embodiments provided by the present application, the selection branches matching the first functional block and the output results corresponding to each selection branch are obtained through the guidance information, so as to achieve the purpose of assisting novice engineers to gradually complete the graphical programming process in the above-mentioned first interface through the guidance information, and configure a corresponding programming flowchart for the target task, so as to achieve the purpose of simplifying the complexity of the programming operation. Further, under the premise of simplifying the programming process, it will be helpful to expand the scope of application of the graphical programming tool and make it applicable to more scenarios.

[0146] As an optional solution, obtaining a selection branch matching the first functional block through the guidance information includes:

[0147] S1, acquiring an image including a plurality of objects to be processed;

[0148] S2, identifying the object information of each object from the image;

[0149] S3, grouping the identified object information to obtain at least two first information, wherein each selection branch corresponds to one first information.

[0150] Optionally, in this embodiment, images of multiple objects to be processed on the production line may be collected by an image acquisition device, but not limited to. The image here may be, but not limited to, a global image including multiple objects. In addition, the image acquisition device in this embodiment may be, but not limited to, a camera installed at different positions on the production line. The camera here may be different cameras, such as an ordinary camera, an infrared night vision camera, etc., which is not limited in this embodiment.

[0151] The following example is used to illustrate: a programming configuration interface is displayed in the application programming tool provided in the embodiment of the present application, and the programming configuration interface includes at least the following tabs: a project setting tab (such as Project Settings) for configuring the task information of the target task, a system configuration tab (such as System Configuration) for configuring the relevant information of the operating device for executing the target task, an application programming tab (such as Application Programming) for configuring the execution logic of the function block, and a simulation tab for simulating the completion of programming. Further, the following programming configuration process can be performed in the application programming tab (i.e., the first interface) but is not limited to:

[0152] Assume that the guidance information here may include but is not limited to the following Figure 4 The six dimensions shown in the sidebar on the left, such as dimensions 40 to 45. Here, in the perspective detection dimension (i.e., dimension 40), the object to be processed by the target task can be photographed by a camera. As in step S402, the image obtained after taking the photo is added, and the image can be Figure 4 The image in the middle shows the contents shown in area 46, which includes seven objects: scissors 46-1, battery 46-2, circle object 46-3, and rectangle object 46-4. Figure 5 In step S502, object recognition is performed on the image to obtain object information of each object, such as identifying the appearance shape of each object: five rectangles of different sizes and two circles.

[0153] Further, if Figure 6 As shown in the figure, each appearance shape is selected in turn and the object is registered for the appearance shape. The registration result is shown in Figure 6 As shown in the right area 60: a circular object is registered as "61", and the registration result includes the appearance shape and registration name of the circular object as shown in the figure. A rectangular object is registered as "62", and the registration result includes the appearance shape and registration name of the circular object as shown in the figure. It should be noted that the registration name here can be customized, and the registered objects here can also be manually deleted.

[0154] Then, in the dimension of determining the path to the location, the identified registered objects can be grouped, and the grouping interface can be as follows Figure 7-Figure 9 As shown:

[0155] like Figure 7 As shown, at least two groups that have been established will be displayed by default, such as Group 1 (Group 1, as shown in the figure G1) and Group 2 (Group 2, as shown in the figure G2). You can also add a new group (Add Group) by clicking on the group adding area 70. In addition, you can directly add objects (Select Object) to each group, such as clicking on the object adding area 71 to add new objects to G1 or G2. The source of the new object here is the object that has been identified and registered as mentioned above. Figure 7 Other information associated with the group may also be configured in the information configuration area 72 shown.

[0156] Assuming that the objects that have been identified and registered include "80", "81" and "82", then Figure 8 As shown in the figure, add the object "80" in group 1 (G1). Fig. 9 As shown, add objects "81" and "82" in group 2 (G2). Correspondingly, group 1 (G1) group 1 can be used as a first information in the first functional block, and group 2 (G2) group 2 can be used as another first information in the first functional block. Here, each first information will correspond to a different selection branch.

[0157] Through the embodiments provided by the present application, after identifying the object information of each object from the acquired image, it is grouped to obtain multiple first information, and then each first information is used as a selection branch in the first control to achieve the purpose of creating the first control based on the guide information, thereby guiding the user to create the first control based on the guide information, simplifying the creation process, and making it easier for novice engineers to understand and use the programming tool.

[0158] As an optional solution, obtaining the output result corresponding to each selection branch through the guide information includes:

[0159] S1, in a second interface associated with the first interface, displaying input prompt information corresponding to each selection branch, wherein the second interface is used to obtain an information processing result configured for the first information;

[0160] S2, according to the input prompt information, obtain the information processing result corresponding to the selection branch as the output result of the selection branch.

[0161] Optionally, in this embodiment, the second interface may be a sub-interface included in the first interface, or may be an interface jumped to after an operation is performed on a function key in the first interface, and this is not limited in this embodiment.

[0162] The following description is still based on the scenario assumed in the above embodiment:

[0163] Here we take the output result of group 1 (G1) configuration as an example. Fig.10 As shown, input prompt information is displayed in the drop-down interface (collaborative system interface) 100 of group 1 (G1) to prompt the user to input the information processing result corresponding to group 1 (G1). After obtaining the information processing result obtained by the input, it is used as the output result of the selection branch corresponding to group 1 (G1). For example, the input box 101 prompts the user to input object information, and the input box 102 prompts the user to input the placement position P1. In order to further configure the output result for group 2 (G2), the user can switch to the drop-down interface (collaborative system interface) of group 2 (G2). The configuration process refers to the above embodiment, which will not be repeated in this embodiment.

[0164] After configuring the output results of each selection branch, the created first control 110 can be obtained. Fig.11 As shown, the first control includes two selection branches, wherein the result corresponding to information 111 is P1, and the result corresponding to information 112 is P2, and the input and output ends of the first control have established information transmission connections with FB-2 and FB-5 in the programming flowchart, respectively.

[0165] In addition, assuming that the objects in group 1 (G1) in this example are used to perform grasping actions, Fig.12 As shown, the width of the gripper can be configured, such as selecting a width value in the selection area 120 to adapt it to the width of the objects in group 1 (G1), and then obtaining a confirmation operation performed on the confirmation area 121.

[0166] Through the embodiments provided in the present application, the information processing result corresponding to each selection branch is obtained according to the input prompt information corresponding to each selection branch, and the first control is created as the corresponding output result, so that the first function block can be reused directly using the first control in subsequent application programming, so as to achieve the purpose of efficiently processing different first information therein.

[0167] As an optional solution, creating a first control in the first interface includes:

[0168] S1, automatically generating a corresponding first control in a first interface according to the amount of first information to be processed in a first functional block, wherein the number of selection branches set in the first control is consistent with the number of first information;

[0169] S2. In each selection branch of the first control, respectively configure the information processing result corresponding to each first information as the output result of the selection branch.

[0170] It should be noted that, in this embodiment, the first control can also be automatically generated based on the object information of the object that has been identified and registered. For example, the first control can be automatically generated based on the number of registered objects (or grouping results), so that the number of registered objects is consistent with the number of selection branches in the first control. Then, the information processing results of each object are obtained as the output results corresponding to each selection branch.

[0171] Through the embodiment provided by the present application, a first control is created based on the object information of the registered and identified object and the configured information processing result. Thus, the graphical programming process can be quickly completed in the above-mentioned first interface, and a corresponding programming flowchart can be configured for the target task to achieve the purpose of simplifying the complexity of the programming operation. Further, under the premise of simplifying the programming process, it will be beneficial to expand the scope of application of the graphical programming tool and make it applicable to more scenarios.

[0172] As an optional solution, before automatically generating a corresponding first control in the first interface according to the amount of first information to be processed in the first functional block, the method further includes:

[0173] S1, acquiring an image including a plurality of objects to be processed;

[0174] S2, identifying the object information of each object from the image;

[0175] S3: Determine at least two first information according to the identified object information.

[0176] The following example is used to illustrate: Figure 4-Figure 5 The example shown in the figure is used to obtain images of multiple objects, and the object information of each object is recognized from the image. The image acquisition and image recognition process will not be described in detail here.

[0177] Further select objects to register, and the registration results can be as follows Fig.13 As shown, the first object includes two circular shapes named "130", and the second object includes two batteries named "131". It should be noted that the registered names here can be customized, and the registered objects here can also be manually deleted.

[0178] Then, each object is determined as different information in the first functional block, such as determining "130" as one first information in the first functional block, and determining "131" as another first information in the first functional block. Here, each first information will correspond to a different selection branch.

[0179] Through the embodiments provided in the present application, multiple first information are directly obtained through the results obtained after object recognition, and then each first information is used as a selection branch in the first control to achieve the purpose of automatically creating the first control, thereby simplifying the creation process and making it easier for novice engineers to understand and use the programming tool.

[0180] As an optional solution, in the output result corresponding to each selection branch in the first control, configuring a corresponding information processing result for each first information includes:

[0181] S1, displaying a candidate result list matching the first control, wherein the candidate result list includes information processing results corresponding to each piece of first information;

[0182] S2, determining a target information processing result corresponding to the first information currently to be configured from the candidate result list;

[0183] S3, obtaining a drag instruction generated by performing a drag operation on the target information processing result;

[0184] S4, responding to the drag instruction, dragging the target information processing result to the output result corresponding to the first information.

[0185] Assume that Fig.14 As shown in the figure, the first function block currently determined is FB-2, then as in step S1402, the first information of the first control 140 connected to FB-2 (such as the oblique line area in 140 in the figure) is configured first, and then the respective information processing results are further configured. Fig.15 As shown, it is assumed that the output result is currently being configured for the first control "151" in the selection branch 152. After the click operation on the trigger button 150 in FB-2 is obtained, the candidate result list 154 is displayed, and after the output result P1 corresponding to 152 is determined, P1 is dragged to the output box 153 corresponding to 152.

[0186] The candidate result list 154 may be as follows: Fig.16 The candidate result list shown in the left sidebar, where the candidate result list 154 may include but is not limited to: path, coordinates, data and value. Then, the target information processing result (such as Fig.16After the slashed filled area 160 is shown in the figure, a drag instruction is obtained to drag the target information processing result (such as the "coordinate position") to the output box 154 of the corresponding output result in the first control 151, and the drag instruction is responded to to complete the drag action, and at the same time, the target information processing result (such as the "coordinate position") is configured to the output result of the selection branch where 152 is located.

[0187] After configuring the output results of each selection branch, the first control created can be obtained, as shown in the following figure: Fig.17 As shown, the first control includes two selection branches, and the input end and the output end of the first control have established information transmission connections with the programming flow chart respectively.

[0188] Through the embodiments provided in the present application, the information processing result corresponding to each selection branch is obtained by dragging and dropping, and is used as the corresponding output result to create a first control, so that the first function block can be reused directly using the first control in subsequent application programming, thereby achieving the purpose of efficiently processing different first information therein.

[0189] As an optional solution, after configuring the first control between the first functional block and the second functional block, the method further includes:

[0190] 1) obtaining a first operation performed on a first control; in response to the first operation, adjusting the first control to a disabled state, wherein the disabled state is used to indicate that the first control is stopped; or

[0191] 2) Acquire a second operation performed on the first control; in response to the second operation, adjust the first control to an available state, wherein the available state is used to indicate that the first control is resumed.

[0192] Optionally, in this embodiment, the above-mentioned first control can also accept an operation for indicating adjustment of the running status, such as the first operation is used to indicate that the running first control is adjusted to a disabled state, and the first control in the disabled state is marked and displayed; such as the second operation indicates that the stopped first control is adjusted to an available state, and the first control that has restored the available state is restored to normal display.

[0193] It should be noted that the first operation here can be, but is not limited to, a click operation performed on the control button corresponding to the first control, such as a short press operation, a long press operation, and a combination thereof. For example, a short press switches once; for another example, a short press resumes operation, and a long press stops operation. This is only an example, and no limitation is made in this embodiment.

[0194] For example, assuming that an adjustment instruction for instructing to adjust a running first control to a disabled state is obtained, the first control in the disabled state may be marked and displayed, such as Fig.18 As shown, the display color of the first control 180 is adjusted (the grid is filled as shown in the figure), and the connection between the first control 180 and the second functional block FB-5 is disconnected, while the connection between the first control 180 and the first functional block FB-2 is retained.

[0195] In addition, in this embodiment, it is also possible but not limited to accepting an operation for instructing to adjust the running state of any one branch in the first control, so as to realize flexible control of the running state of each selection branch.

[0196] For example, assuming that an adjustment instruction for instructing to adjust the second selection branch in the running first control to a disabled state is obtained, then Fig.19 As shown, the second selection branch in the first control 190 in the disabled state can be marked and displayed, and the display color of the selection branch can be adjusted (filling the grid as shown in the figure).

[0197] Through the embodiments provided in the present application, the operating state of the first control and its selection branch can also be directly adjusted through the application programming interface, thereby achieving the purpose of improving control flexibility.

[0198] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0199] According to another aspect of the present invention, a processor for implementing the above graphical programming method is provided, wherein the processor is used to run a program, wherein the above method is executed when the program is run.

[0200] In addition, a memory may be included for storing software programs and modules, such as program instructions / modules corresponding to the graphical programming method in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the above-mentioned graphical programming method is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0201] According to another aspect of an embodiment of the present invention, a terminal for implementing the above-mentioned graphical programming method is also provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include means for executing the above-mentioned method.

[0202] Optionally, those skilled in the art will appreciate that the terminal here may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices.

[0203] In other embodiments, the terminal may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer (P2P) network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0204] According to another aspect of an embodiment of the present invention, a computer program product for implementing the above-mentioned graphical programming method is also provided. The computer program product is tangibly stored on a computer-readable medium and includes computer-executable instructions, which, when executed, enable at least one processor to perform the above-mentioned method.

[0205] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0206] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0207] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0208] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0211] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A graphical programming method, characterized in that: include: In a first interface, a programming flowchart matching a target task to be executed is displayed, wherein the programming flowchart includes at least two function blocks for realizing the target task, and the first interface is used to configure the function blocks in the programming flowchart and the execution logic between the function blocks; Determine a first function block and a second function block from the programming flowchart, wherein the first function block is used to obtain at least two first information to be processed, different first information corresponds to different information processing results, and the second function block is used to obtain the information processing result corresponding to the first information; A first control is configured between the first functional block and the second functional block, wherein the first control is used to determine a current information processing result according to the first information currently detected from the first functional block, and transmit the current processing result to the second functional block; Wherein, determining the first functional block and the second functional block from the programming flowchart comprises: Acquire attribute description information corresponding to each function block in the programming flow chart, wherein the attribute description information is used to describe the attributes of the information to be processed in the function block; Determine the function block indicated by the attribute description information as including at least two pieces of information as the first function block; Acquire a first result type of an information processing result corresponding to the first information to be processed in the first function block; Searching for a function block corresponding to the first result type in each of the function blocks as the second function block matching the first function block; Wherein, configuring a first control between the first functional block and the second functional block includes: Creating the first control in the first interface; Connecting the input end of the first control to the first functional block, and connecting the output end of the first control to the second functional block; The step of creating the first control in the first interface includes: Acquire, through guidance information, selection branches matching the first functional block and output results corresponding to each selection branch, wherein the number of the first information to be processed in the first functional block is consistent with the number of selection branches set in the first control, and each of the output results includes the information processing result configured for each of the first information; The first control is created according to the selection branch and the output result.

2. The method according to claim 1, characterized in that: Acquiring a selection branch matching the first functional block through the guidance information includes: acquiring an image including a plurality of objects to be processed; identifying object information of each object from the image; The identified object information is grouped to obtain at least two first information, wherein each selection branch corresponds to one first information.

3. The method according to claim 1, characterized in that The output results corresponding to each selection branch obtained through the guidance information include: In a second interface associated with the first interface, input prompt information corresponding to each selection branch is displayed, wherein the second interface is used to obtain the information processing result configured for the first information; According to the input prompt information, the information processing result corresponding to the selection branch is obtained as the output result of the selection branch.

4. The method according to claim 1, characterized in that: The creating the first control in the first interface includes: According to the amount of the first information to be processed in the first functional block, automatically generating the corresponding first control in the first interface, wherein the amount of selection branches set in the first control is consistent with the amount of the first information; In each selection branch of the first control, the information processing result corresponding to each first information is respectively configured as the output result of the selection branch.

5. The method according to claim 4, characterized in that Before automatically generating the corresponding first control in the first interface according to the amount of the first information to be processed in the first functional block, the method further includes: acquiring an image including a plurality of objects to be processed; identifying object information of each object from the image; At least two pieces of the first information are determined according to the identified object information.

6. The method according to claim 4, characterized in that In each selection branch of the first control, the information processing result corresponding to each first information is respectively configured, and the output result of the selection branch includes: Displaying a candidate result list matching the first control, wherein the candidate result list includes the information processing results corresponding to each of the first information; Determining a target information processing result corresponding to the first information currently to be configured from the candidate result list; Acquire a drag instruction generated by performing a drag operation on the target information processing result; In response to the drag instruction, the target information processing result is dragged to the output result corresponding to the first information.

7. The method according to claim 1, characterized in that After configuring a first control between the first functional block and the second functional block, the method further includes: Acquire a first operation performed on the first control; in response to the first operation, adjust the first control to a disabled state, wherein the disabled state is used to indicate that the first control is stopped; or Acquire a second operation performed on the first control; and in response to the second operation, adjust the first control to an available state, wherein the available state is used to indicate that the first control is resumed.

8. A processor, characterized in that The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 7 when running.

9. A terminal, characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product is tangibly stored on a computer readable medium and comprises computer executable instructions which, when executed, cause at least one processor to perform the method according to any one of claims 1 to 7.

Images