A control method, device, and apparatus for a robot, and a storage medium

Abstract

The application discloses a kind of robot control method, device, equipment and storage medium, it is related to robot technical field, method includes: display robot control interface, robot control interface includes target control robot operating environment information and at least one process operation mode;Display robot control interface, robot control interface includes target control robot operating environment information and at least one process operation mode;Based on the operation interface of target process operation mode, in response to the adjustment operation to workpiece image, display adjusted workpiece image;Based on the operation interface of target process operation mode, in response to the control operation to adjusted workpiece image, generate and send control instruction to target control robot, to make target control robot according to control instruction to the workpiece to be operated under target process operation mode executes operation, to realize high-precision control to robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots, and in particular, to a robot control method, device, equipment and storage medium. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, robots can replace humans to complete many repetitive, dangerous or tedious work, so that people can be freed from tedious or dangerous work. Industrial robots are a kind of robots, which are automatic machine devices with many movement axes, and can complete different work tasks by converting different actions. For example, industrial robots can be applied to workpiece assembly, welding and stacking, etc., to improve work efficiency.

[0003] With the wide application of robots, robot debugging has become a common work in robot operation, and debugging control terminal has also become an indispensable tool for controlling robots. However, the existing debugging control terminal for robot debugging control still has the problems of low accuracy and efficiency. SUMMARY

[0004] The main purpose of the present application is to provide a robot control method, device, equipment and storage medium, which aims to improve the control accuracy of the robot.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a robot control method, which comprises:

[0007] displaying a robot control interface, the robot control interface comprising operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated;

[0008] in response to a selection operation for the process operation mode, displaying an operation interface of a target process operation mode, wherein the operation interface comprises a display state of the operation environment information under the target process operation mode, the display state of the operation environment information comprises a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0009] based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, displaying an adjusted workpiece image;

[0010] generate and send a control instruction to the target control robot according to the adjustment operation on the workpiece image, so that the target control robot performs the operation in the target process operation mode according to the control instruction.

[0011] Correspondingly, the application further provides a robot control device, comprising:

[0012] a first display unit configured to display a robot control interface, the robot control interface comprising operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated;

[0013] a second display unit configured to display an operation interface of a target process operation mode in response to a selection operation on the process operation mode, wherein the operation interface comprises a display state of the operation environment information in the target process operation mode, the display state of the operation environment information comprises a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0014] a third display unit configured to display an adjusted workpiece image in response to an adjustment operation on the workpiece image based on the operation interface of the target process operation mode;

[0015] a sending unit configured to generate and send a control instruction to the target control robot according to the adjustment operation on the workpiece image based on the operation interface of the target process operation mode, so that the target control robot performs the operation in the target process operation mode according to the control instruction.

[0016] In a third aspect, the application provides a robot control device, comprising a processor and a memory, wherein the memory stores a robot control program, and the robot control program is executed by the processor to implement the robot control method as described above.

[0017] In a fourth aspect, the application provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is executed by one or more processors to implement the robot control method as described above.

[0018] The above one or more technical solutions provided by the application can have the following advantages or at least achieve the following technical effects:

[0019] The application provides a robot control method, device, equipment and storage medium. The robot control method comprises the following steps: displaying a robot control interface, the robot control interface comprising target control robot operation environment information and at least one process operation mode, wherein the target control robot operation environment information is generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated; in response to a selection operation on the process operation mode, displaying an operation interface of a target process operation mode, wherein the operation interface comprises a display state of the operation environment information in the target process operation mode, the display state of the operation environment information comprises a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation on the workpiece image, displaying an adjusted workpiece image; based on the operation interface of the target process operation mode, in response to a control operation on the adjusted workpiece image, generating and sending a control instruction to the target control robot, so that the target control robot performs an operation in the target process operation mode on the workpiece to be operated according to the control instruction, and high-precision control of the robot can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0021] Figure 1 A flowchart of a first embodiment of the robot control method of the present application;

[0022] Figure 2 A hardware structure diagram of the robot control device related to the present application;

[0023] Figure 3 A function module diagram of a first embodiment of the robot control device of the present application;

[0024] Figure 4 A scene diagram of a first embodiment of the robot control method of the present application;

[0025] Figure 5 Another scene diagram of a first embodiment of the robot control method of the present application;

[0026] Figure 6 Another scene diagram of a first embodiment of the robot control method of the present application;

[0027] Figure 7Another scene schematic view for the first embodiment of the robot control method of the present application;

[0028] Figure 8 Another scene schematic view for the first embodiment of the robot control method of the present application;

[0029] Figure 9 Another scene schematic view for the first embodiment of the robot control method of the present application;

[0030] Figure 10 Another scene schematic view for the first embodiment of the robot control method of the present application;

[0031] Figure 11 Flow schematic view for the second embodiment of the robot control method of the present application.

[0032] The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0034] It should be noted that in the present application, the terms "comprising", "containing" or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or system that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or system. Without more limitations, the elements defined by the phrase "comprising" do not exclude the presence of additional identical elements in the process, method, article or system that includes the element. In addition, the meaning of "and / or" appearing throughout the text includes three parallel solutions, for example, "A and / or B" includes A solution, or B solution, or A and B solutions. In the present application, if there is a description of "first", "second", etc., the description of "first", "second", etc. is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the present application, the suffix such as "module", "component" or "unit" used to represent an element is only for the convenience of the description of the present application, and has no specific meaning. Therefore, "module", "component" or "unit" can be mixed. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In addition, the technical solutions of various embodiments can be combined with each other, but it is based on the fact that the technical solutions can be realized by those skilled in the art. When the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist, and is not within the scope of protection claimed by the present application.

[0035] With the continuous development of artificial intelligence technology, robots can complete many repetitive, dangerous or tedious work, so that people can more efficiently complete these work and reduce labor costs. With the wide application of robots, robot point position debugging has become a common work. In the prior art, the point position debugging of the robot generally depends on the control of the operator on the teach pendant. The operator needs to go to the working site of the robot to watch the to-be-operated control and realize the point position debugging of the robot by controlling the teach pendant. This process is complex and inefficient, and if the to-be-operated workpiece is a high-precision workpiece, the debugging accuracy may exceed the limit of the human eye, thereby causing low efficiency of robot debugging.

[0036] In view of the technical problem of low efficiency of robot debugging in the related art, the present application provides a control method of a robot, and the general idea is as follows:

[0037] The robot control interface is displayed, the robot control interface includes target control robot operating environment information and at least one process operation mode, wherein the target control robot operating environment information is generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated; in response to a selection operation for the process operation mode, an operation interface of the target process operation mode is displayed, wherein the operation interface includes a display state of the operating environment information under the target process operation mode, and the display state of the operating environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, an adjusted workpiece image is displayed; based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instruction.

[0038] Through the above technical solution, the operating environment information of the target control robot can be displayed on the control device, so that the operator can realize the point position debugging of the target control robot through the interface of the control device, and the accuracy and precision of the robot point position debugging are improved.

[0039] The robot control method, device, equipment and storage medium provided by the application will be described in detail below with reference to the drawings, specific embodiments and implementation manners.

[0040] Embodiment one

[0041] Referring to Figure 1 , the first embodiment of the robot control method of the application is proposed, and the robot control method is applied to the control device of the robot.

[0042] The control device of the robot refers to a terminal device or a network device that can realize network connection, which can be a terminal device such as a mobile phone, a computer, a tablet computer, a portable computer, an embedded industrial computer, or a network device such as a server, a cloud platform, etc.

[0043] As Figure 2 shown, it is a schematic diagram of the hardware structure of the control device of the robot. The control device of the robot can include a processor 1001 such as a CPU (Central Processing Unit, central processor), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005.

[0044] Specifically, the communication bus 1002 is configured to realize the connection communication between the components; the user interface 1003 is configured to connect the client and communicate data with the client, and the user interface 1003 can include an output unit and an input unit; the network interface 1004 is configured to connect the background server and communicate data with the background server, and the network interface 1004 can include an input / output interface; the memory 1005 is configured to store various types of data, which can include, for example, instructions of any application program or method in the control device of the robot, and application-related data, and the memory 1005 can be an internal memory; optionally, the memory 1005 can also be a storage device independent of the processor 1001, and the memory 1005 can include an operating system, a network communication module, a user interface module, and a control program of the robot, and the processor 1001 is configured to call the control program of the robot stored in the memory 1005 and perform the following operations: Figure 2

[0045] display a robot control interface, the robot control interface including target control robot operating environment information and at least one process operation mode, wherein the target control robot operating environment information is generated by an intelligent camera carried by the target control robot, and the operating environment information represents an operating environment between the target control robot and a corresponding workpiece to be operated;

[0046] in response to a selection operation for the process operation mode, display an operation interface of a target process operation mode, wherein the operation interface includes a display state of the operating environment information under the target process operation mode, and the display state of the operating environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0047] based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, display an adjusted workpiece image;

[0048] based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, generate and send a control instruction to the target control robot, so that the target control robot performs an operation under the target process operation mode on the workpiece to be operated according to the control instruction.

[0049] based on the above-mentioned control device of the robot, the control method of the robot of the present embodiment will be described in detail in combination with the flowchart shown in Figure 1 the method can include the following steps:

[0050] ​101. displaying a robot control interface, the robot control interface comprising operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated.

[0051] The target control robot can refer to a robot controlled by a control device. The robot can be a robot with certain automation, which can realize various processing and manufacturing functions by relying on its own power source and control capability. For example, the robot mentioned in the present application can be an industrial robot. For example, the robot mentioned in the present application can be a collaborative robot, a six-axis robot, a four-axis robot (Selective Compliance Assembly Robot Arm, SCARA), or a robot with a mechanical hand, etc.

[0052] The process operation mode can include operations that can be performed by the target control operation robot. For example, the at least one process operation mode can include spot welding, grabbing, adsorbing, screwing, carrying, spraying, etc.

[0053] The workpiece to be operated can include a workpiece that needs to be operated by the target control robot. For example, when the target control robot needs to perform a spot welding operation, the workpiece to be operated can include a workpiece that needs to be spot welded by the target control robot.

[0054] The operation environment information can represent the operation environment between the target control robot and the corresponding workpiece to be operated. For example, through the operation environment information, an image corresponding to the workpiece to be operated, the layout of objects in the operation environment of the target control robot, whether there are obstacles, etc. can be obtained.

[0055] In an embodiment, the operation environment information can be a two-dimensional planar image or a three-dimensional image. For example, the operation environment information can be a two-dimensional planar map or a three-dimensional map, etc.

[0056] In an embodiment, the robot control interface can be displayed, and the robot control interface comprises operation environment information of a target control robot and at least one process operation mode. For example, as shown in Figure 4 The process operation mode displayed in the robot control interface can include carrying, screwing, spot welding, adsorbing, gluing, and spraying, etc. Figure 4

[0057] In an embodiment, the operation environment information can be a two-dimensional planar image or a three-dimensional image. For example, as shown in Figure 5 The operation environment information is a two-dimensional planar image.​Figure 5 In an embodiment, the operation environment information of the target control robot 003, the workpiece to be operated 002, the feeding table 004, the processing table 005, etc. are displayed in a two-dimensional form. For example, as shown in FIG. 2, the operation environment information of the target control robot 003, the workpiece to be operated 002, the feeding table 004, the processing table 005, etc. are displayed in a two-dimensional form. Figure 5 In an embodiment, the operation environment information of the target control robot 003, the workpiece to be operated 002, the feeding table 004, the processing table 005, etc. are displayed in a two-dimensional form. For example, as shown in FIG. 2, the operation environment information of the target control robot 003, the workpiece to be operated 002, the feeding table 004, the processing table 005, etc. are displayed in a two-dimensional form. Figure 5 In an embodiment, the interface display selection control 001 is provided in the target control robot 003. Through the interface display selection control 001, the operator can decide whether the operation environment information is displayed in a two-dimensional form or a three-dimensional form. For example, if the operator wants to view the workpiece to be operated in more detail, the operation environment information can be displayed in a two-dimensional form. If the operator wants to view the workpiece to be operated from different perspectives, the operation environment information can be displayed in a three-dimensional form.

[0058] In an embodiment, the target control robot 003 can be equipped with an intelligent camera, so that the operation environment image of the target control robot 003 can be captured by the intelligent camera. In addition, in addition to the target control robot 003 being equipped with an intelligent camera, the environment in which the target control robot 003 is located can also be equipped with an intelligent camera, so that the operation environment image of the target control robot 003 can be more comprehensively captured.

[0059] In an embodiment, the operation environment information can have multiple forms of representation. For example, the operation environment information can be a two-dimensional plan view, and can also be a three-dimensional plan view. For example, as shown in FIG. 2, the operation environment information can be a two-dimensional plan view. As can be seen from the figure, the robot 003 and the workpiece to be operated 002 are displayed in a two-dimensional planar form. Figure 5 Figure 6 In an embodiment, the operation environment information can have multiple forms of representation. For example, the operation environment information can be a two-dimensional plan view, and can also be a three-dimensional plan view. For example, as shown in FIG. 2, the operation environment information can be a two-dimensional plan view. As can be seen from the figure, the robot 003 and the workpiece to be operated 002 are displayed in a two-dimensional planar form.

[0060] 102、In response to a selection operation for a process operation mode, an operation interface of the target process operation mode is displayed, wherein the operation interface includes a display state of operation environment information in the target process operation mode, and the display state of operation environment information includes a workpiece image of a workpiece to be operated, and the target process operation mode is the selected process operation mode.

[0061] In an embodiment, the robot control interface includes at least one process operation mode. For example, spot welding, grabbing, adsorbing, screwing, etc. The user can select the work that the robot needs to operate according to the needs. For example, the user can select the target control robot to perform a grabbing operation. For example, the user can select the target control robot to perform an adsorbing operation. ​

[0062] In an embodiment, the interface has different display modes for different process operation modes. For example, the interface displays different operation tools, operation setting areas, etc. for different process operation modes. In addition, the display state of the operation environment information can also be different for different process operation modes.

[0063] Therefore, in response to the selection operation for the process operation mode, the operation interface of the target process operation mode is displayed, wherein the operation interface includes the display state of the operation environment information in the target process operation mode, and the display state of the operation environment information includes the workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode.

[0064] For example, the at least one process operation mode includes spot welding, grabbing, adsorbing, and screwing. When the user selects the spot welding process operation mode, the operation interface corresponding to the spot welding operation can be displayed, which can include the operation tools corresponding to the spot welding operation, and the workpiece image of the workpiece to be operated displayed in the spot welding process operation mode. For example, the position of spot welding can be marked in the workpiece image, etc.

[0065] For another example, when the user selects the screwing process operation mode, the operation interface corresponding to the screwing operation can be displayed, which can include the operation tools corresponding to the screwing operation, and the workpiece image of the workpiece to be operated displayed in the screwing process operation mode. For example, the position of screwing can be marked in the workpiece image, etc.

[0066] In an embodiment, the workpiece image of the workpiece to be operated can be the display form of the workpiece to be operated in the robot control interface. For example, as shown in FIG. 2, the workpiece image 002 in FIG. 2 can be a schematic diagram of the workpiece image of the workpiece to be operated, and it can be seen that the workpiece image of the workpiece to be operated can be a certain display area in the robot control interface. Therefore, the user can adjust the display state of the workpiece image by operating the interface of the control device. Figure 5 Figure 5

[0067] 103、Based on the operation interface of the target process operation mode, in response to the adjustment operation for the workpiece image, the adjusted workpiece image is displayed.

[0068] ​​In an embodiment, in the prior art, robot point position debugging becomes a common work, and the teach pendant also constitutes an indispensable debugging control terminal for robot point position adjustment. However, when the teach pendant is used for point position debugging, the operator needs to constantly debug the robot while looking at the work. However, in the debugging process, factors such as the debugging environment or the debugging precision exceeding the limit of the human eye cause low efficiency of robot debugging. Through the embodiment of the present application, the operator can directly adjust the workpiece image through the control device, thereby achieving accurate positioning.

[0069] For example, the operator can perform zoom-in operation, zoom-out operation, rotation operation, positioning operation, etc. on the workpiece image.

[0070] In an embodiment, the workpiece image is an image corresponding to at least one workpiece to be operated. For example, the workpiece image can have only one workpiece to be operated. For another example, the workpiece image can have multiple workpieces to be operated.

[0071] In an embodiment, when there is only one workpiece to be operated in the workpiece image, the operator can adjust the workpiece image, so as to adjust the workpiece to be operated to a size convenient for positioning. For example, the workpiece image can be zoomed in to a suitable size. For another example, the workpiece image can be rotated and zoomed in to a suitable size.

[0072] For example, if the operation environment information is a two-dimensional plan view, the operation interface is also a two-dimensional plan view, and the operation interface includes a robot, a workpiece to be operated, a feeding table, a machining table, etc. If the operator wants to position the workpiece to be operated, so as to realize positioning and debugging of the robot, the operator can select the workpiece to be operated, and then the workpiece to be operated can be displayed in a zoomed-in manner. Then, the operator can further zoom in the workpiece to be operated, so as to realize high-precision positioning.

[0073] In an embodiment, when there are multiple workpieces to be operated in the workpiece image, the multiple workpieces to be operated can be multiple workpieces to be operated of the same type, or can be workpieces to be operated of different types. For example, there are 30 workpieces to be operated in the workpiece image, of which 10 are workpieces to be operated of type A, 10 are workpieces to be operated of type B, and 10 are workpieces to be operated of type C.

[0074] When there are multiple workpieces to be operated, the operator can select a target workpiece to be adjusted, and then perform adjustment. Specifically, the step of “displaying an adjusted workpiece image based on the operation interface of the target process operation mode, in response to an adjustment operation on the workpiece image” can include:

[0075] The operation interface based on the target process operation mode displays a selected workpiece image in response to a selection operation on at least one workpiece to be operated in the workpiece image, wherein the selected workpiece image includes a target workpiece selected from the workpiece to be operated.

[0076] The operation interface based on the target process operation mode displays an updated workpiece image in response to an update operation on the selected workpiece image.

[0077] The operation interface based on the target process operation mode displays an adjusted workpiece image in response to a pointing operation on the updated workpiece image.

[0078] In an embodiment, the operation interface based on the target process operation mode displays a selected workpiece image in response to a selection operation on at least one workpiece to be operated in the workpiece image, wherein the selected workpiece image includes a target workpiece selected from the workpiece to be operated.

[0079] For example, when the target process operation mode is welding, if there are multiple workpieces in the workpiece image, the operator can select the workpiece to be welded. For another example, when the target process operation mode is adsorption, the operator can select the workpiece to be adsorbed. For another example, if the operator needs to operate all the workpieces, the operator can also arbitrarily select one workpiece for more detailed operation.

[0080] For another example, assuming that there are multiple workpieces to be operated of different types, the operator can select the type of workpiece to be operated. For another example, if the types of multiple workpieces to be operated are the same, the operator can arbitrarily select one workpiece to be operated as a reference for debugging positioning.

[0081] For example, as shown in FIG. 9, there are multiple workpieces to be operated, and the operator selects one workpiece to be operated 009. The display interface of the control device can further magnify the workpiece to be operated 009, thereby facilitating the operator to further operate the workpiece to be operated. Figure 7

[0082] In an embodiment, after the operator selects the workpiece to be operated, the operator can also fine-tune the selected workpiece image. Therefore, the operation interface based on the target process operation mode displays an updated workpiece image in response to an update operation on the selected workpiece image.

[0083] In an embodiment, the update operation includes at least one of a magnification operation, a reduction operation, and a rotation operation.

[0084] ​For example, when the operating environment information is a two-dimensional planar view, the operator can further zoom in on the selected workpiece image. Alternatively, the operator can repeatedly zoom in or out on the selected workpiece image to adjust it to a suitable size. For instance, the operator can zoom in, zoom out, and zoom in again on the selected workpiece image to adjust it to a suitable size.

[0085] For example, when the operating environment information is a three-dimensional view, the selected workpiece image can also be a three-dimensional view, and the operator can also rotate the selected workpiece image to a suitable viewing angle and then zoom in.

[0086] In one embodiment, the operating environment information may be a two-dimensional plan view or a three-dimensional view.

[0087] When the operating environment information is a two-dimensional planar view, the content of the workpiece image can be the front of the workpiece to be operated. However, when the operating environment information is a three-dimensional view, the content of the workpiece image can be a three-dimensional view of the workpiece to be operated, allowing viewing of all faces of the workpiece. For example, the workpiece image can show the front, side, and back of the workpiece, and so on.

[0088] Specifically, when the operating environment information is a three-dimensional operating image, in response to the update operation for the selected workpiece image, the updated workpiece image is displayed, including:

[0089] In response to a rotation operation on the selected workpiece image, display the side view image corresponding to the target workpiece;

[0090] In response to a zoom-in operation on the side image, an updated workpiece image is displayed.

[0091] For example, such as Figure 8 As shown, if an operator wants to process the side of a workpiece, they can rotate the selected workpiece image to display the corresponding side image of the target workpiece. Therefore, in response to a selection operation on the selected workpiece image, the corresponding side image of the target workpiece can be displayed. The operator can then zoom in on the workpiece to adjust the side image to a suitable size for further processing.

[0092] In one embodiment, different calibration tools may be used for different process operation modes. The calibration tool is used to pinpoint the workpiece to be operated on, and then the target control robot can perform operations on the workpiece based on the pinpointed position.

[0093] Specifically, the operation interface of the target process operation mode includes at least one calibration tool. Therefore, the step of "displaying the adjusted workpiece image in response to the pointing operation on the updated workpiece image based on the operation interface of the target process operation mode" can include:

[0094] displaying a pointing mark corresponding to the calibration tool in the updated workpiece image in response to the triggering operation on the at least one calibration tool;

[0095] displaying the adjusted workpiece image in response to the confirmation operation on the pointing mark.

[0096] The pointing mark can be an instruction of the operator operating the calibration tool.

[0097] In an embodiment, the calibration tool can have various forms. For example, the calibration tool can be a rectangle, a circle, a dot, or a custom shape, etc. For example, as shown in FIG. 006, the tools displayed in the region of FIG. 006 can be schematic diagrams of the calibration tools. Figure 4

[0098] For example, if the operator selects a rectangular calibration tool, the interface can display an instruction corresponding to the rectangle. Then, the operator can position the image by operating the instruction.

[0099] For example, when the target process operation mode is spot welding, the calibration tools in the operation interface are dot calibration tools and custom shape calibration tools. For another example, when the target process operation mode is screw locking, the calibration tools in the operation interface can be calibration tools corresponding to screws of different sizes or different types. For another example, when the target process operation mode is glue coating, the calibration tools in the operation interface can be circular calibration tools and custom shape calibration tools.

[0100] For example, when the target process operation mode is spot welding, the target control robot can know the positions of the workpiece to be spot welded through the pointing operation. For another example, when the target process operation mode is glue coating, the target control robot can know the positions of the workpiece to be coated with glue through the pointing. For another example, when the target process operation mode is suction, the target control robot can know the positions of the suction through the pointing operation.

[0101] In an embodiment, the pointing mark corresponding to the calibration tool can be displayed in the updated workpiece image in response to the triggering operation on the at least one calibration tool, and the adjusted workpiece image can be displayed in response to the confirmation operation on the pointing mark.

[0102] ​For example, when the target process operation mode is conveyance, the calibration tools of the operation interface can include a rectangular calibration tool and a custom shape calibration tool. If the user selects the rectangular calibration tool, a rectangular positioning mark can be displayed in the interface. Then, the user can point in the updated workpiece image by dragging the rectangular positioning mark to obtain an adjusted workpiece image. For example, if the user points two points in the updated workpiece image by using the rectangular positioning mark, two rectangular frames can be displayed in the adjusted workpiece image.

[0103] For another example, when the target process operation mode is spot welding, the calibration tools of the operation interface can include a circular dot calibration tool and a custom shape calibration tool. If the user selects the circular dot calibration tool, a circular dot positioning mark can be displayed in the interface. Then, the user can point in the updated workpiece image by dragging the circular dot positioning mark to obtain an adjusted workpiece image. For example, if the user points two points in the updated workpiece image by using the circular dot positioning mark, two circular dots can be displayed in the adjusted workpiece image.

[0104] In an embodiment, in response to a triggering operation on at least one calibration tool, a positioning mark corresponding to the calibration tool can be automatically displayed. For example, after the operator selects the corresponding calibration tool, the control device can automatically identify the updated workpiece image according to the calibration tool, so as to identify the position in the updated workpiece image that needs to be pointed, and display the position in the adjusted workpiece image in the form of a positioning mark.

[0105] In an embodiment, the positioning of the operator on the updated workpiece image can include two steps, i.e., selecting a target point to be positioned, and determining the execution order of the target point to be positioned. Therefore, the adjusted workpiece image can include an execution order indication of the target point to be positioned.

[0106] For example, when the target process operation mode is spot welding, the adjusted workpiece image can display the points to be spot welded on the workpiece to be operated. If the workpiece to be operated has multiple points to be spot welded, the adjusted workpiece image can also display the order of the points to be spot welded. Generally, the order of the points to be positioned selected by the operator is the execution order of the positioning mark.

[0107] Specifically, the step of “displaying the adjusted workpiece image in response to the confirmation operation on the positioning mark” can include:

[0108] in response to the selection operation on the positioning mark, intensively displaying the target point to be positioned;

[0109] in response to the order confirmation operation on the target point to be positioned, displaying an execution order indication of the target point to be positioned.

[0110] For example, in response to the triggering operation for the at least one calibration tool, the control device displays a pointing mark corresponding to the calibration tool in the updated workpiece image. Then, the operator can operate the pointing mark to point, and thus, in response to a selection operation for the pointing mark, the control device can highlight the target pointing mark. For example, in response to the selection operation for the pointing mark, the control device can highlight the target pointing mark.

[0111] For another example, if the operator selects multiple target pointing marks, the control device can also display an execution order indication of the multiple target pointing marks.

[0112] In an embodiment, the workpiece to be operated can be a high-precision workpiece. For example, the workpiece to be operated can be a chip, etc. Thus, the updated workpiece image can have at least one precision structure of the workpiece to be operated. The precision structure is a structure with a size that is too small to be distinguished by the human eye.

[0113] Thus, when positioning the updated workpiece image, the precision structure in the updated workpiece image can be positioned. Specifically, the step of "in response to the triggering operation for the at least one calibration tool, displaying a pointing mark corresponding to the calibration tool in the updated workpiece image" can include:

[0114] in response to a selection operation for the at least one calibration tool, displaying a control mark corresponding to the calibration tool;

[0115] in response to a control operation for the control mark, displaying a pointing mark corresponding to the precision structure in the updated workpiece image.

[0116] For example, when the workpiece to be operated is a chip with a very small size, the operator can zoom in the workpiece image to an appropriate size, so that the structure of the chip can be clearly viewed. Then, the operator can control the calibration tool to achieve high-precision positioning of the chip structure.

[0117] 104. The operation interface based on the target process operation mode, in response to a control operation for the adjusted workpiece image, generates and sends a control instruction to the target control robot, so that the target control robot performs the operation of the target process operation mode on the workpiece to be operated according to the control instruction.

[0118] In one embodiment, the user interface includes a control setting area corresponding to the target process operation mode. The control setting area can be used to configure the operation process of the target process operation mode. For example, when the target process operation mode is spot welding, the control setting area can set the spot welding temperature range, vibration amplitude, etc. As another example, when the target process operation mode is gripping, the control setting area can set the gripping force, etc. As yet another example, when the target process operation mode is adsorption, the control setting area can set the adsorption force, duration, and release time, etc.

[0119] Specifically, the step "based on the operation interface of the target process operation mode, responding to the control operation for the adjusted workpiece image, generating and sending control commands to the target control robot" may include:

[0120] Retrieve control setting information in response to editing operations on the control settings area;

[0121] Based on the control settings information, generate at least one trajectory route corresponding to the target controlled robot;

[0122] In response to a selection operation for at least one trajectory route, control commands are generated and sent to the target control robot based on the selected target trajectory route.

[0123] In one embodiment, the operator can edit the control settings area. For example, the operator can input or select corresponding parameters in the control settings area to set the operation process for the target process operation mode.

[0124] In one embodiment, after obtaining the control setting information, this application embodiment can generate at least one trajectory route corresponding to the target controlled robot based on the control setting information, so that the operator can view the movement trajectory of the target controlled robot on the control device. Then, the operator can select the trajectory route that the target controlled robot wants to run. Therefore, in response to the selection operation for at least one trajectory route, control commands can be generated and sent to the target controlled robot based on the selected target trajectory route.

[0125] For example, such as Figure 9 As shown, operators can configure the process executed by the target controlled robot through the user interface. For example, when the target process operation mode is material handling, the operator can set the material handling sequence, and so on. Then, the operator can trigger the "Automatically Generate Trajectory" control, and the control device will automatically generate the corresponding trajectory route. For example, as... Figure 10As shown, the operation route corresponding to the target control robot can be displayed in the operation interface. Then, the operator can select the corresponding trajectory route for the target control robot, so that the target control robot can perform the corresponding operation according to the target trajectory route.

[0126] In an embodiment, the method provided by the embodiments of the present application can further include:

[0127] When it is detected that the operation environment of the target control robot is updated, the updated operation environment information is acquired;

[0128] The at least one trajectory route is updated based on the updated operation environment information, and the updated trajectory route is obtained and displayed;

[0129] For example, when the control device detects that the operation environment information of the target control robot is updated through the intelligent camera, the control device can acquire the updated operation environment information. Then, the control device can update the at least one trajectory route based on the updated operation environment information, and obtain and display the updated trajectory route.

[0130] Then, the step of "in response to the selection operation for the at least one trajectory route, generating and sending the control instruction to the target control robot based on the selected target trajectory route" can include:

[0131] In response to the selection operation for the at least one updated trajectory route, generating and sending the control instruction to the target control robot based on the selected target updated trajectory route.

[0132] After updating the trajectory route, the control device can display the updated trajectory route on the control device in real time, so that the operator can view the updated trajectory route in time and select the execution route of the target control robot based on the updated trajectory route.

[0133] The robot control method provided by the embodiment displays a robot control interface, the robot control interface includes target control robot operating environment information and at least one process operation mode, wherein the target control robot operating environment information is generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated; in response to a selection operation for the process operation mode, an operation interface of the target process operation mode is displayed, wherein the operation interface includes a display state of the operating environment information in the target process operation mode, the display state of the operating environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, an adjusted workpiece image is displayed; based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs an operation in the target process operation mode on the workpiece to be operated according to the control instruction. The operating environment information of the target control robot is collected, and the operating environment information is displayed on the device, and the operating environment information includes the workpiece image of the workpiece to be operated, so that the operator can directly look at the workpiece image on the control device for point debugging, thereby improving the efficiency and accuracy of point debugging. Moreover, the embodiment can also support operation on the workpiece image in different process operation modes, thereby improving the applicability of the embodiment.

[0134] Embodiment two

[0135] Based on the same inventive concept, a second embodiment of a robot control method of the application is proposed, which is applied to a robot control system of the robot, and the control system includes a target control robot, an intelligent camera, a control device and a cloud server.

[0136] The robot can be a robot with certain automation, which can realize various processing and manufacturing functions by relying on its own power source and control capability. For example, the robot mentioned in the application can be an industrial robot. For example, the robot proposed in the application can be a six-axis robot, a four-axis robot (Selective Compliance Assembly Robot Arm, SCARA), or a robot with a mechanical hand, etc.

[0137] The target control robot can refer to a robot controlled by a control device. A corresponding robot control interface can be displayed on the control device, and a user can control the robot to perform a corresponding operation by operating the robot control interface. For example, the user can control the robot to perform a carrying operation, a spot welding operation, a suction operation, or a glue coating operation, etc. by operating the control device. For example, the control device can be a teach pendant, a tablet computer, or a laptop computer, etc.

[0138] The smart camera can be a highly integrated miniature machine vision system. It integrates image acquisition, processing, and communication functions in a single camera, thereby providing a machine vision solution with multiple functions, modularity, high reliability, and ease of implementation.

[0139] In an embodiment, the target control robot can be loaded with a smart camera, so that the operating environment image of the target control robot can be acquired by the smart camera. In addition, in addition to the target control robot being loaded with a smart camera, the environment in which the target control robot is located can also be loaded with a smart camera, so that the operating environment image of the target control robot can be more comprehensively acquired.

[0140] In an embodiment, the operating environment information of the target control robot can be acquired by the smart camera, so as to determine the boundary and object position under the reachable coverage of the target control robot. For example, the target control robot can be loaded with a smart camera, and then the target control robot can acquire the operating environment information by self-rotation. The smart camera can feed back the acquired operating environment data to a cloud server, and the cloud server can model the operating environment information. The operating environment information can be a two-dimensional planar view or a three-dimensional view. After the cloud server completes the modeling of the operating environment information, the cloud server can send the operating environment information to a control device, so that the control device can display the operating environment information. For example, when the operating environment information is a two-dimensional planar view, the control device can display a two-dimensional planar view. When the operating environment information is a three-dimensional view, the control device can display a three-dimensional view. Then, a user can perform a point debugging on a workpiece to be operated by operating the control device, and then control the target control robot to perform a carrying operation, a spot welding operation, a suction operation, or a glue coating operation, etc. according to the point debugging result.

[0141] In addition, the control device can also display different process operation modes according to the scene function executed by the target control robot. For example, when selecting spot welding, the adjustable amplitude of the jitter, the temperature range of the welding, the single dwell time, and other variable functions are displayed. When selecting grabbing, the adjustable variables of the grabbing position and the grabbing force are displayed. When selecting using the adsorption operation, the adjustable variables of the adsorption force, the adsorption time, and the release time are displayed. Different operations can be performed on the plan view for different operation controls, so as to control the target control robot to perform different operations. For example, when the target control robot is a mechanical hand, when the mechanical hand selects the grabbed object, the digital plan view is enlarged and the object to be grabbed is framed, without understanding the three-dimensional tool coordinates of the mechanical hand and the object to be operated in the same space. Through the system, the control mechanical hand can be conveniently and quickly adjusted to grab the corresponding workpiece, so as to meet the actual use requirements of the user and efficiently and conveniently use the operation.

[0142] The control system of the robot of the present embodiment will be described in detail below. As shown in Figure 11 The system can include the following steps:

[0143] 201. The intelligent camera collects spatial environment information.

[0144] For example, the target control robot can be a mechanical hand. The intelligent camera can be loaded in the gripper of the mechanical hand. Then, the mechanical hand can rotate by itself and collect spatial environment information through the intelligent camera, and determine the boundary and object position in the reachable coverage range of the mechanical hand.

[0145] 202. The intelligent camera transmits the spatial environment information to the cloud server.

[0146] 203. The cloud server identifies and processes the spatial environment information to generate environment operation information.

[0147] For example, the cloud server can use the Simultaneous Localization and Mapping (SLAM) algorithm and the triangular surveying method to model the environment information and draw a two-dimensional plan view or a three-dimensional view.

[0148] 204. The cloud server sends the environment operation information to the control device.

[0149] 205. The control device displays a robot control interface, and the robot control interface includes operation environment information of the target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is generated by the intelligent camera carried by the target control robot, and the operation environment information represents the operation environment between the target control robot and the corresponding workpiece to be operated.

[0150] 206、the control device displays an operation interface of the target process operation mode in response to a selection operation for the process operation mode, wherein the operation interface includes a display state of operation environment information in the target process operation mode, and the display state of the operation environment information includes a workpiece image of a workpiece to be operated, and the target process operation mode is the selected process operation mode.

[0151] For example, the process operation mode can include spot welding, grabbing, adsorbing, screwing, etc.

[0152] 207、the control device displays an adjusted workpiece image in response to an adjustment operation for the workpiece image based on the operation interface of the target process operation mode.

[0153] For example, the control device can freely zoom in or out the workpiece image (including the digital map) according to needs (control precision), and determine the range position points of different shapes and sizes on the corresponding map.

[0154] 208、the control device generates and sends a control instruction to the target control robot in response to a control operation for the adjusted workpiece image based on the operation interface of the target process operation mode.

[0155] For example, detailed parameters can be adjusted based on the target process operation mode. For example, when spot welding, set the temperature value, specific range, and intensity of shaking of spot welding; when grabbing, set the force, direction, and time for completing grabbing, etc.

[0156] 209、the target control robot performs operation in the target process operation mode on the workpiece to be operated according to the control instruction.

[0157] For example, the control device can feed back the operation settings of the digital map by the operator to the system, calculate the feedback to the actual physical space, and command the robot to execute the operation command to the specified position with the optimal trajectory.

[0158] For another example, during the movement of the manipulator, the intelligent camera continuously updates the current position and the surrounding environment information, to ensure the layout and change state within the coverage range of the robot.

[0159] More implementation details of the above method steps can be found in the description of the specific implementation of Example 1, which will not be repeated here for the sake of brevity of the description.

[0160] The robot control system provided by the embodiment comprises an intelligent camera collecting space environment information; the intelligent camera transmits the space environment information to a cloud server; the cloud server identifies and processes the space environment information to generate environment operation information; the cloud server sends the environment operation information to a control device; the control device displays a robot control interface, the robot control interface comprising operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated; the control device displays an operation interface of a target process operation mode in response to a selection operation on the process operation mode, wherein the operation interface comprises a display state of the operation environment information under the target process operation mode, the display state of the operation environment information comprising a workpiece image of the workpiece to be operated, and the target process operation mode being the selected process operation mode; the control device displays an adjusted workpiece image in response to an adjustment operation on the workpiece image based on the operation interface of the target process operation mode; the control device generates and sends a control instruction to the target control robot in response to a control operation on the adjusted workpiece image based on the operation interface of the target process operation mode; and the target control robot performs an operation under the target process operation mode on the workpiece to be operated according to the control instruction. The control system provided by the embodiment can ensure that an operator can perform spot debugging on the operation environment information, thereby ensuring the efficiency and accuracy of spot debugging.

[0161] Embodiment three

[0162] Based on the same inventive concept, referring to Figure 3 , the first embodiment of the robot control device of the present application is proposed, which can be a virtual device and is applied to a robot control device.

[0163] The robot control device provided by the embodiment will be described in detail below in combination with the functional module schematic diagram shown in Figure 3 , which can comprise:

[0164] a first display unit configured to display a robot control interface, the robot control interface comprising operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated;

[0165] a second display unit, configured to display, in response to a selection operation for the process operation mode, an operation interface of a target process operation mode, wherein the operation interface comprises a display state of the operation environment information in the target process operation mode, and the display state of the operation environment information comprises a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0166] a third display unit, configured to display, in response to an adjustment operation for the workpiece image, an adjusted workpiece image based on the operation interface of the target process operation mode;

[0167] a sending unit, configured to generate and send, in response to a control operation for the adjusted workpiece image, a control instruction to the target control robot based on the operation interface of the target process operation mode, so that the target control robot performs operation in the target process operation mode on the workpiece to be operated according to the control instruction.

[0168] In an embodiment, the third display unit can comprise:

[0169] a first display sub-unit, configured to display, in response to a selection operation for at least one workpiece to be operated in the workpiece image, a selected workpiece image based on the operation interface of the target process operation mode, wherein the selected workpiece image comprises a target workpiece selected from the workpiece to be operated;

[0170] a second display sub-unit, configured to display, in response to an update operation for the selected workpiece image, an updated workpiece image;

[0171] a third display sub-unit, configured to display, in response to a pointing operation for the updated workpiece image, the adjusted workpiece image based on the operation interface of the target process operation mode.

[0172] In an embodiment, the third display sub-unit can comprise:

[0173] a first display module, configured to display, in response to a triggering operation for the at least one calibration tool, a pointing identifier corresponding to the calibration tool in the updated workpiece image;

[0174] a second display module, configured to display, in response to a confirmation operation for the pointing identifier, the adjusted workpiece image.

[0175] In an embodiment, the first display module can comprise:

[0176] a first display sub-module, configured to display, in response to a selection operation for the at least one calibration tool, a control identifier corresponding to the calibration tool;

[0177] The second display sub-module is configured to display a pointing mark corresponding to the fine structure on the updated workpiece image in response to a control operation on the control mark.

[0178] In an embodiment, the second display module can include:

[0179] The third display sub-module is configured to intensively display the target pointing mark in response to a selection operation on the pointing mark.

[0180] The fourth display sub-module is configured to display an execution sequence indication of the target pointing mark in response to a sequence confirmation operation on the target pointing mark.

[0181] In an embodiment, the second display sub-module can include:

[0182] The third display module is configured to display a side surface image corresponding to the target workpiece in response to a rotation operation on the selected workpiece image.

[0183] The fourth display module is configured to display the updated workpiece image in response to a zoom-in operation on the side surface image.

[0184] In an embodiment, the sending unit can include:

[0185] The first acquisition sub-unit is configured to acquire control setting information in response to an editing operation on the control setting area.

[0186] The first generation sub-unit is configured to generate at least one trajectory route corresponding to the target control robot based on the control setting information.

[0187] The second generation sub-unit is configured to generate and send a control instruction to the target control robot based on the selected target trajectory route in response to a selection operation on the at least one trajectory route.

[0188] In an embodiment, the sending unit can further include:

[0189] The second acquisition sub-unit is configured to acquire updated operation environment information when detecting that the operation environment of the target control robot is updated.

[0190] The updating sub-unit is configured to update the at least one trajectory route based on the updated operation environment information to obtain and display an updated trajectory route.

[0191] In an embodiment, the second generation sub-unit can include:

[0192] The generating module is configured to generate and send a control instruction to the target control robot based on the selected target updated trajectory route in response to a selection operation for the at least one updated trajectory route.

[0193] It should be noted that the functions and technical effects of the modules in the robot control device provided in the embodiments can refer to the descriptions of the specific embodiments of the robot control method of the application. For the sake of brevity of the description, the functions and technical effects of the modules will not be described here.

[0194] Embodiment Four

[0195] Based on the same inventive concept, referring to the hardware structure diagram of Figure 2 The robot control device provided in the embodiments can include a processor and a memory. The memory stores a robot control program. When the robot control program is executed by the processor, all or part of the steps of the embodiments of the robot control method of the application are implemented.

[0196] Specifically, the robot control device refers to a terminal device or a network device that can realize network connection. The terminal device can be a mobile phone, a computer, a tablet computer, a portable computer, an embedded industrial computer, etc. The network device can be a server, a cloud platform, etc.

[0197] It is understandable that the robot's control equipment may also include a communication bus, a user interface, and a network interface. The communication bus is used to connect and communicate between these components; the user interface is used to connect to the client and communicate data with the client. The user interface may include output units such as displays and speakers, and input units such as keyboards and microphones; the network interface is used to connect to the backend server and communicate data with the backend server. The network interface may include input / output interfaces, such as standard wired interfaces and wireless interfaces such as Wi-Fi interfaces; the memory is used to store various types of data. This data may include, for example, instructions for any application or method in the robot's control device, as well as application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), and Electrically Erasable Programmable Read-Only Memory (EPROM). The memory can be a storage device independent of the processor, such as an EEPROM, magnetic storage, flash memory, disk, or optical disk. The processor is used to call the robot control program stored in the memory and execute the robot control method described above. The processor can be an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components to execute all or part of the steps of the various embodiments of the robot control method described above.

[0198] It needs to be explained that, Figure 2 The hardware structure shown does not constitute a limitation on the control device of the robot of the present invention, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0199] Example 5

[0200] Based on the same inventive concept, the embodiment provides a computer readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, a server, etc., which stores a computer program. The computer program can be executed by one or more processors. When the computer program is executed by the processor, all or part of the steps of the control method of the robot of the embodiment can be implemented.

[0201] It should be noted that the above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above embodiments are only optional embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent flow conversion, or direct or indirect application in other related technical fields, which is made under the inventive concept of the present application, using the contents of the present application specification and drawings, is included in the patent protection scope of the present application.

Claims

1. A control method of a robot characterized by, The method comprises: displaying a robot control interface, the robot control interface comprising target control robot operating environment information and at least one process operation mode, wherein the target control robot operating environment information is generated by an intelligent camera carried by the target control robot, and the operating environment information represents an operating environment between the target control robot and a corresponding workpiece to be operated; in response to a selection operation for the process operation mode, displaying an operation interface of a target process operation mode, wherein the operation interface comprises a display state of the operating environment information under the target process operation mode, and the display state of the operating environment information comprises a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, displaying an adjusted workpiece image; the workpiece image is an image corresponding to at least one workpiece to be operated; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, displaying an adjusted workpiece image, comprising: based on the operation interface of the target process operation mode, in response to a selection operation for at least one workpiece to be operated in the workpiece image, displaying a selected workpiece image, wherein the selected workpiece image comprises a target workpiece selected from the workpiece to be operated; in response to an update operation for the selected workpiece image, displaying an updated workpiece image; based on the operation interface of the target process operation mode, in response to a pointing operation for the updated workpiece image, displaying the adjusted workpiece image; the operation interface of the target process operation mode comprises at least one calibration tool; based on the operation interface of the target process operation mode, in response to a pointing operation for the updated workpiece image, displaying the adjusted workpiece image, comprising: in response to a trigger operation for the at least one calibration tool, displaying a pointing identifier corresponding to the calibration tool in the updated workpiece image; in response to a confirmation operation for the pointing identifier, displaying the adjusted workpiece image; the updated workpiece image comprises at least one precision structure; the response to the trigger operation for the at least one calibration tool, displaying the pointing identifier corresponding to the calibration tool in the updated workpiece image, comprising: in response to a selection operation for the at least one calibration tool, displaying a control identifier corresponding to the calibration tool; in response to a control operation for the control identifier, displaying a pointing identifier corresponding to the precision structure on the updated workpiece image; based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, generating and sending a control instruction to the target control robot, so that the target control robot performs an operation under the target process operation mode on the workpiece to be operated according to the control instruction.

2. The method of claim 1, wherein, the adjusted workpiece image comprises a target pointing and an execution order indication of the target pointing; The response to the confirmation operation for the pointing identification includes: In response to a selection operation for the pointing identification, the target pointing is highlighted for display; In response to a sequence confirmation operation for the target pointing, an execution sequence indication of the target pointing is displayed.

3. The method of claim 1, wherein, When the operation environment information is a three-dimensional operation image, the response to the update operation for the selected workpiece image includes: In response to a rotation operation for the selected workpiece image, a side image corresponding to the target workpiece is displayed; In response to a zoom-in operation for the side image, the updated workpiece image is displayed.

4. The method of claim 1, wherein, The update operation includes at least one of a zoom-in operation, a zoom-out operation, and a rotation operation.

5. The method according to any one of claims 1 to 3, characterized in that, The operation interface includes a control setting area corresponding to the target process operation mode; The operation interface based on the target process operation mode generates and sends a control instruction to the target control robot in response to a control operation for the adjusted workpiece image, including: In response to an editing operation for the control setting area, control setting information is obtained; Based on the control setting information, at least one trajectory route corresponding to the target control robot is generated; In response to a selection operation for the at least one trajectory route, a control instruction is generated and sent to the target control robot based on the selected target trajectory route.

6. The method of claim 5, wherein, The method further includes: When the operation environment of the target control robot is detected to be updated, updated operation environment information is obtained; Based on the updated operation environment information, the at least one trajectory route is updated to obtain and display an updated trajectory route; The response to the selection operation for the at least one trajectory route includes: In response to a selection operation for at least one updated trajectory route, a control instruction is generated and sent to the target control robot based on the selected target updated trajectory route.

7. A control device of a robot characterized by comprising: The control device includes: A first display unit for displaying a robot control interface, the robot control interface including operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is generated by an intelligent camera carried by the target control robot, and the operation environment information represents an operation environment between the target control robot and a corresponding workpiece to be operated; A second display unit for displaying an operation interface of a target process operation mode in response to a selection operation for the process operation mode, wherein the operation interface includes a display state of the operation environment information under the target process operation mode, the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; The third display unit is configured to display an adjusted workpiece image in response to an adjustment operation on the workpiece image based on the operation interface of the target process operation mode, the workpiece image being an image corresponding to at least one workpiece to be operated; the operation interface of the target process operation mode is configured to display an adjusted workpiece image in response to an adjustment operation on the workpiece image, including: the operation interface of the target process operation mode is configured to display a selected workpiece image in response to a selection operation on at least one workpiece to be operated in the workpiece image, the selected workpiece image including a target workpiece selected from the workpieces to be operated; the operation interface of the target process operation mode is configured to display an updated workpiece image in response to an update operation on the selected workpiece image; the operation interface of the target process operation mode is configured to display the adjusted workpiece image in response to a pointing operation on the updated workpiece image; the operation interface of the target process operation mode includes at least one calibration tool; the operation interface of the target process operation mode is configured to display the adjusted workpiece image in response to a pointing operation on the updated workpiece image, including: the operation interface of the target process operation mode is configured to display a pointing identifier corresponding to the calibration tool in the updated workpiece image in response to a triggering operation on the at least one calibration tool; the operation interface of the target process operation mode is configured to display the adjusted workpiece image in response to a confirmation operation on the pointing identifier; the updated workpiece image includes at least one precision structure; the operation interface of the target process operation mode is configured to display a pointing identifier corresponding to the calibration tool in the updated workpiece image in response to a triggering operation on the at least one calibration tool, including: the operation interface of the target process operation mode is configured to display a control identifier corresponding to the calibration tool in response to a selection operation on the at least one calibration tool; the operation interface of the target process operation mode is configured to display a pointing identifier corresponding to the precision structure on the updated workpiece image in response to a control operation on the control identifier. The sending unit is configured to generate and send a control instruction to the target control robot in response to a control operation on the adjusted workpiece image based on the operation interface of the target process operation mode, so that the target control robot performs the operation in the target process operation mode on the workpiece to be operated according to the control instruction.

8. A control device of a robot characterized by comprising: The device includes a processor and a memory, and the memory stores a control program of a robot, and the control program of the robot is executed by the processor to implement the control method of the robot in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by one or more processors to implement the control method of the robot in any one of claims 1-6.

Images