Robot control method, apparatus and device, and storage medium

Abstract

A robot control method, apparatus and device, and a storage medium, which relate to the technical field of robots. The method comprises: displaying a robot control interface, which comprises operational environment information of a target control robot and at least one process operation mode; in response to a selection operation concerning the process operation mode, displaying an operation interface of a target process operation mode; on the basis of the operation interface of the target process operation mode, in response to an adjustment operation concerning a workpiece image, displaying an adjusted workpiece image; and on the basis of the operation interface of the target process operation mode, in response to a control operation concerning the adjusted workpiece image, generating a control instruction and sending same to the target control robot, such that the target control robot executes, on the basis of the control instruction, an operation in the target process operation mode on a workpiece to be operated. Therefore, high-precision control over a robot is realized.

Description

A robot control method, device, equipment and storage medium

[0001] Related applications

[0002] This application claims priority to Chinese patent application No. 202311551640.7 filed on November 20, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of robotics, and in particular to a robot control method, device, equipment, and storage medium. Background Art

[0004] With the continuous development of artificial intelligence technology, robots can replace humans in many repetitive, dangerous, or tedious tasks, freeing people from tedious or dangerous work. Industrial robots are a type of robot. They are automatic machines with multiple motion axes that can perform different tasks by switching between different movements. For example, industrial robots can be used in tasks such as workpiece assembly, welding, and stacking, improving work efficiency.

[0005] With the widespread use of robots, robot debugging has become a common task in robot operation, and the debugging control terminal has become an indispensable tool for controlling robots. However, the existing debugging and control of robots using the debugging control terminal still has problems with low accuracy and efficiency.

[0006] Application Contents

[0007] The main purpose of this application is to provide a robot control method, device, equipment and storage medium, aiming to improve the control accuracy of the robot.

[0008] To achieve the above objectives, this application adopts the following technical solutions:

[0009] In a first aspect, the present application provides a robot control method, the method comprising:

[0010] Displaying a robot control interface, the robot control interface including operating environment information of the target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated;

[0011] In response to a selection operation for the process operation mode, an operation interface of a target process operation mode is displayed, wherein the operation interface includes a display state of the operation environment information in the target process operation mode, the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0012] Based on the operation interface of the target process operation mode, in response to the adjustment operation on the workpiece image, displaying the adjusted workpiece image;

[0013] Based on the operation interface of the target process operation mode, in response to the control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instruction.

[0014] Accordingly, the present application also proposes a robot control device, comprising:

[0015] a first display unit, configured to display a robot control interface, the robot control interface including operating environment information of the target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated;

[0016] a second display unit, configured to display an operation interface of a target process operation mode in response to a selection operation for the process operation mode, wherein the operation interface includes a display state of the operation environment information in the target process operation mode, the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0017] a third display unit, configured to display an adjusted workpiece image in response to an adjustment operation on the workpiece image based on an operation interface of the target process operation mode;

[0018] A sending unit is used to generate and send control instructions to the target control robot in response to the control operation on the adjusted workpiece image based on the operation interface of the target process operation mode, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instructions.

[0019] On the third aspect, the present application provides a robot control device, which includes a processor and a memory, wherein the memory stores a robot control program, and when the robot control program is executed by the processor, the robot control method as described above is implemented.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by one or more processors, the robot control method as described above is implemented.

[0021] The above one or more technical solutions provided by this application may have the following advantages or at least achieve the following technical effects:

[0022] The present application proposes a robot control method, device, equipment and storage medium, which displays a robot control interface, wherein the robot control interface includes operating environment information of a target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated; in response to a selection operation for the process operation mode, an operation interface of the target process operation mode is displayed, wherein the operation interface includes a display status of the operating environment information under the target process operation mode, and the display status of the operating environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, an adjusted workpiece image is displayed; based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instruction, thereby achieving high-precision control of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0024] FIG1 is a flow chart of a first embodiment of a control method for a robot according to the present application;

[0025] FIG2 is a schematic diagram of the hardware structure of the control device of the robot involved in this application;

[0026] FIG3 is a schematic diagram of functional modules of a first embodiment of a control device for a robot according to the present application;

[0027] FIG4 is a schematic diagram of a scenario of a first embodiment of a robot control method of the present application;

[0028] FIG5 is another schematic diagram of a scenario of the first embodiment of the robot control method of the present application;

[0029] FIG6 is another schematic diagram of a scenario of the first embodiment of the robot control method of the present application;

[0030] FIG7 is another schematic diagram of a scenario of the first embodiment of the robot control method of the present application;

[0031] FIG8 is another schematic diagram of a scenario of the first embodiment of the robot control method of the present application;

[0032] FIG9 is another schematic diagram of a scenario of the first embodiment of the robot control method of the present application;

[0033] FIG10 is another schematic diagram of a scenario of the first embodiment of the robot control method of the present application;

[0034] FIG11 is a flow chart of a second embodiment of a robot control method of the present application.

[0035] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0036] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of this application.

[0037] It should be noted that, in this application, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprise..." does not preclude the presence of other identical elements in the process, method, article, or system comprising that element. Furthermore, the meaning of "and / or" appearing throughout this document includes three parallel solutions. For example, "A and / or B" includes Solution A, Solution B, or solutions where both A and B are satisfied. In this application, if there are descriptions involving "first," "second," etc., such descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In this application, suffixes such as "module", "component" or "unit" used to represent elements are used only to facilitate the description of this application and have no specific meaning in themselves. Therefore, "module", "component" or "unit" can be used interchangeably. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances. In addition, the technical solutions of the various embodiments can be combined with each other, but this is based on the ability of those of ordinary skill in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0038] With the continuous development of artificial intelligence technology, robots can complete many repetitive, dangerous or tedious tasks, allowing people to complete these tasks more efficiently and reduce labor costs. With the widespread application of robots, robot point debugging has become a common task. In the existing technology, robot point debugging generally relies on the operator's control of the teach pendant. The operator needs to go to the robot's work site to look at the controls to be operated and debug the robot's points by controlling the teach pendant. This process is complicated and inefficient. Moreover, if the workpiece to be operated is a high-precision workpiece, the debugging accuracy may exceed the limit of the human eye, resulting in low robot debugging efficiency.

[0039] In view of the technical problem of low robot debugging efficiency in related technologies, this application provides a robot control method. The overall idea is as follows:

[0040] A robot control interface is displayed, wherein the robot control interface includes operating environment information of the target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated; in response to a selection operation for the process operation mode, an operation interface of the target process operation mode is displayed, wherein the operation interface includes a display status of the operating environment information under the target process operation mode, the display status of the operating environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, an adjusted workpiece image is displayed; based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs operations under the target process operation mode on the workpiece to be operated according to the control instruction.

[0041] Through the above technical solution, the operating environment information of the target control robot can be displayed on the control device, so that the operator can realize the point debugging of the target control robot through the interface of the control device, thereby improving the accuracy and precision of the robot point debugging.

[0042] The robot control method, device, equipment and storage medium provided by the present application are described in detail below with reference to specific embodiments and implementation methods in conjunction with the accompanying drawings.

[0043] Example 1

[0044] 1 , a first embodiment of a robot control method of the present application is provided. The robot control method is applied to a control device of the robot.

[0045] The control device of a robot refers to a terminal device or network device that can achieve network connection. It can be a terminal device such as a mobile phone, computer, tablet computer, portable computer, embedded industrial computer, or a network device such as a server or cloud platform.

[0046] As shown in Figure 2, which is a schematic diagram of the hardware structure of the robot's control device, the robot's control device may include: a processor 1001, such as a CPU (Central Processing Unit), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005.

[0047] Specifically, the communication bus 1002 is used to realize the connection and communication between these components; the user interface 1003 is used to connect to the client and communicate data with the client, and the user interface 1003 may include an output unit and an input unit; the network interface 1004 is used to connect to the backend server and communicate data with the backend server, and the network interface 1004 may include an input / output interface; the memory 1005 is used to store various types of data, which may include, for example, instructions of any application or method in the control device of the robot, as well as application-related data, and the memory 1005 may be a built-in memory; in some embodiments, the memory 1005 may also be a storage device independent of the processor 1001. Continuing with FIG2 , the memory 1005 may include an operating system, a network communication module, a user interface module, and a robot control program; the processor 1001 is used to call the robot control program stored in the memory 1005 and perform the following operations:

[0048] Displaying a robot control interface, the robot control interface including operating environment information of the target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated;

[0049] In response to a selection operation for the process operation mode, an operation interface of a target process operation mode is displayed, wherein the operation interface includes a display state of the operation environment information in the target process operation mode, the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0050] Based on the operation interface of the target process operation mode, in response to the adjustment operation on the workpiece image, displaying the adjusted workpiece image;

[0051] Based on the operation interface of the target process operation mode, in response to the control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instruction.

[0052] Based on the above-mentioned robot control device, the robot control method of this embodiment is described in detail below in conjunction with the flow chart shown in Figure 1. The method may include the following steps:

[0053] 101. Display a robot control interface, which includes operating environment information of the target control robot and at least one process operation mode. The operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated.

[0054] The target control robot may refer to a robot controlled by a control device. A robot may be a robot that has a certain degree of automation and can rely on its own power and control capabilities to perform various processing and manufacturing functions. For example, the robot mentioned in this application may be an industrial robot. For example, the robot proposed in this application may be a collaborative robot, a six-axis robot, a four-axis robot (Selective Compliance Assembly Robot Arm, SCARA), or a robot with a manipulator, etc.

[0055] The process operation mode may include operations that the target control operation robot can perform. For example, the at least one process operation mode may include spot welding, grasping, adsorption, screwing, handling, spraying, etc.

[0056] The workpiece to be operated may include a workpiece that the target control robot needs to operate. For example, when the target control robot needs to perform a spot welding operation, the workpiece to be operated may include a workpiece that the target control robot needs to perform spot welding.

[0057] The operating environment information can represent the operating environment between the target control robot and the corresponding operating workpiece. For example, the operating environment information can be used to obtain the image corresponding to the operating workpiece, the layout of objects in the target control robot's operating environment, whether there are obstacles, etc.

[0058] In one embodiment, the operating environment information may be a two-dimensional plane image or a three-dimensional image. For example, the operating environment information may be a two-dimensional plane map or a three-dimensional map, etc.

[0059] In one embodiment, a robot control interface may be displayed, including operating environment information for the target robot and at least one process operation mode. For example, FIG4 illustrates a schematic diagram of process operation modes within the robot control interface. The process operation modes shown in FIG4 include handling, screwing, spot welding, adsorption, gluing, and spraying, among others.

[0060] In one embodiment, the operating environment information can be a two-dimensional plane image or a three-dimensional image. For example, as shown in Figure 5, it is a schematic diagram of the operating environment information being a two-dimensional plane graphic. In Figure 5, the environmental operating information such as the target control robot 003, the workpiece to be operated 002, the unloading table 004, and the processing table 005 are all displayed in a two-dimensional form. For another example, as shown in Figure 5, there is an interface display selection control 001 in Figure 5. Through the interface display selection control 001, the operator can decide whether the operating environment information is displayed in a two-dimensional or three-dimensional form. For example, if the operator wants to view the workpiece to be operated in more detail, the operating environment information can be displayed in a two-dimensional form. If the operator wants to view the workpiece to be operated from different perspectives, the operating environment information can be displayed in a three-dimensional form.

[0061] In one embodiment, the target control robot may be equipped with a smart camera, thereby enabling the smart camera to capture images of the target control robot's operating environment. Furthermore, in addition to the smart camera being equipped on the target control robot, the environment surrounding the target control robot may also be equipped with a smart camera, thereby enabling more comprehensive capture of images of the target control robot's operating environment.

[0062] In one embodiment, the operating environment information can be presented in a variety of forms. For example, the operating environment information can be a two-dimensional plan view or a three-dimensional plan view. For example, as shown in FIG5 , it can be a two-dimensional plan view of the operating environment information, from which it can be seen that the robot 003 and the workpiece to be operated 002 are both displayed in the form of a two-dimensional plane. For another example, as shown in FIG6 , it can be a three-dimensional plan view of the operating environment information, from which it can be seen that the robot 008 and the workpiece to be operated 007, as well as other unloading tables and processing tables, are all displayed in a three-dimensional form. When the operating environment information is displayed in a three-dimensional plan view, the operating environment information can be regarded as a three-dimensional map, so the operator can view the operating environment information from different perspectives by operating the control device.

[0063] 102. In response to a selection operation for a process operation mode, an operation interface of a target process operation mode is displayed, wherein the operation interface includes a display status of operation environment information under the target process operation mode, the display status of the operation environment information includes a workpiece image of a workpiece to be operated, and the target process operation mode is the selected process operation mode.

[0064] In one embodiment, the robot control interface includes at least one process operation mode, such as spot welding, gripping, suction, screwdriving, etc. The user can select the task the robot needs to perform based on their needs. For example, the user can select a target to control the robot to perform a gripping operation. For another example, the user can select a target to control the robot to perform a suction operation.

[0065] In one embodiment, the interface may have different display modes for different process operation modes. For example, the interface may display different operation tools, operation setting areas, etc. for different process operation modes. In addition, the display state of the operating environment information may also vary for different process operation modes.

[0066] Therefore, in response to the selection operation for the process operation mode, the operation interface of the target process operation mode is displayed, wherein the operation interface includes the display status of the operation environment information under the target process operation mode, the display status of the operation environment information includes the workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode.

[0067] For example, at least one process operation mode includes spot welding, gripping, adsorption, and screw driving. When a user selects the spot welding process operation mode, an operation interface corresponding to the spot welding operation may be displayed. The operation interface may include operating tools corresponding to the spot welding operation and a workpiece image of the workpiece to be operated in the spot welding process operation mode. For example, the workpiece image may identify the location of the spot weld, etc.

[0068] For example, when a user selects the screwdriving operation mode, an operation interface corresponding to the screwdriving operation may be displayed. This operation interface may include the corresponding operation tools for the screwdriving operation and an image of the workpiece to be operated in the screwdriving operation mode. For example, the workpiece image may identify the location where the screws are to be driven, etc.

[0069] In one embodiment, the workpiece image of the workpiece to be operated can be the display format of the workpiece to be operated on the robot control interface. For example, as shown in Figure 5, 002 in Figure 5 can be a schematic diagram of the workpiece image of the workpiece to be operated. Therefore, the workpiece image of the workpiece to be operated can be a display area in the robot control interface. Therefore, the user can adjust the display status of the workpiece image by operating the control device interface.

[0070] 103. Based on the operation interface of the target process operation mode, in response to the adjustment operation on the workpiece image, display the adjusted workpiece image.

[0071] In one embodiment, in the prior art, robot point debugging has become a common task, and the teach pendant also constitutes an indispensable debugging control terminal for robot point adjustment. However, when using the teach pendant for point debugging, the operator needs to watch the work and continuously debug the robot. However, during the debugging process, the robot debugging efficiency is low due to factors such as the debugging environment or the debugging accuracy exceeding the limit of the human eye. With the embodiment of the present application, the operator can directly adjust the workpiece image through the control device, thereby achieving precise positioning.

[0072] For example, the operator can zoom in, zoom out, rotate, position, etc. the workpiece image.

[0073] In one embodiment, the workpiece image is an image corresponding to at least one workpiece to be operated. For example, the workpiece image may only have one workpiece to be operated. For another example, the workpiece image may have multiple workpieces to be operated.

[0074] In one embodiment, when the workpiece image contains only one workpiece to be operated, the operator can adjust the workpiece image to a size that facilitates positioning of the workpiece to be operated. For example, the workpiece image can be enlarged to a suitable size. For another example, the workpiece image can be rotated and enlarged to a suitable size.

[0075] For example, if the operating environment is a 2D planar view, the operation interface is also a 2D planar view, and includes the robot, a workpiece, a loading platform, and a processing table. To position the workpiece and debug the robot, the operator can select the workpiece, which will be magnified. The operator can then further zoom in on the workpiece, enabling high-precision positioning.

[0076] In one embodiment, when a workpiece image contains multiple workpieces to be operated, the multiple workpieces to be operated may be of the same type or of different types. For example, a workpiece image contains 30 workpieces to be operated, 10 of which are of type A, 10 of which are of type B, and 10 of which are of type C.

[0077] When there are multiple workpieces to be operated, the operator can select the target workpiece that needs to be adjusted and then make the adjustment. Specifically, the step of "displaying the adjusted workpiece image in response to the adjustment operation on the workpiece image based on the operation interface of the target process operation mode" may include:

[0078] Based on the operation interface of the target process operation mode, in response to a selection operation for at least one workpiece to be operated in the workpiece images, a selected workpiece image is displayed, wherein the selected workpiece image includes the target workpiece selected in the workpieces to be operated;

[0079] In response to an update operation on the selected artifact image, displaying the updated artifact image;

[0080] The operation interface based on the target process operation mode displays the adjusted workpiece image in response to a pointing operation on the updated workpiece image.

[0081] In one embodiment, based on the operation interface of the target process operation mode, in response to a selection operation on at least one workpiece to be operated in the workpiece images, a selected workpiece image is displayed, wherein the selected workpiece image includes the selected target workpiece in the workpieces to be operated.

[0082] For example, if the target process operation mode is welding, and there are multiple workpiece images, the operator can select the workpiece to be welded. Another example is if the target process operation mode is suction, the operator can select the workpiece to be suctioned. Another example is if the operator needs to operate on all workpieces, the operator can also select any workpiece for more detailed operation.

[0083] For another example, if there are multiple different types of workpieces to be operated, the operator can select the type of workpiece to be operated. For another example, if the types of multiple workpieces to be operated are the same, the operator can arbitrarily select a workpiece to be operated as a reference for debugging and positioning.

[0084] For example, as shown in FIG7 , there are multiple workpieces to be operated, and the operator selects one of the workpieces to be operated 009 . The display interface of the control device can further magnify the workpiece to be operated 009 , thereby facilitating the operator to perform further operations on the workpiece to be operated.

[0085] In one embodiment, after the operator selects a workpiece to be operated, the operator may further fine-tune the selected workpiece image. Therefore, in response to an update operation on the selected workpiece image, an updated workpiece image is displayed.

[0086] In one embodiment, the update operation includes at least one of a zoom-in operation, a zoom-out operation, and a rotation operation.

[0087] For example, when the operating environment information is a two-dimensional plan view, the operator can further zoom in on the selected workpiece image. For another example, the operator can continuously zoom in or out on the selected workpiece image to adjust the selected workpiece image to a suitable size. For example, the operator can zoom in on the selected workpiece image, then zoom out, and then zoom in again to adjust the selected workpiece image to a suitable size.

[0088] For another example, when the operating environment information is a three-dimensional view, the selected workpiece image may also be a three-dimensional view, and the operator may further rotate the selected workpiece image to a suitable viewing angle and then zoom in on it.

[0089] In one embodiment, the operating environment information may be a two-dimensional plan view or a three-dimensional view.

[0090] When the operating environment information is a two-dimensional planar view, the workpiece image may contain the front view of the workpiece to be operated. When the operating environment information is a three-dimensional view, the workpiece image may contain a stereoscopic view of the workpiece to be operated. The workpiece image allows viewing of all sides of the workpiece to be operated. For example, the workpiece image allows viewing of the front, side, and back of the workpiece to be operated, etc.

[0091] Specifically, when the operating environment information is a three-dimensional operating image, in response to an update operation on the selected workpiece image, displaying the updated workpiece image includes:

[0092] In response to a rotation operation on the selected workpiece image, displaying a side image corresponding to the target workpiece;

[0093] In response to a zoom-in operation on the side image, an updated workpiece image is displayed.

[0094] For example, as shown in Figure 8, if the operator wishes to process the side of a workpiece, the operator can rotate the selected workpiece image to display the corresponding side image of the target workpiece. Thus, in response to a selection operation on the selected workpiece image, the corresponding side image of the target workpiece can be displayed. The operator can then zoom in on the workpiece to adjust the side image to a suitable size for further processing.

[0095] In one embodiment, there may be different calibration tools for different process operation modes.

[0096] Among them, the calibration tool is used to locate the workpiece to be operated, and then the target control robot can operate the workpiece to be operated according to the determined point.

[0097] Specifically, the operation interface of the target process operation mode includes at least one calibration tool. Therefore, the step of "displaying the adjusted workpiece image in response to a fixed-point operation on the updated workpiece image based on the operation interface of the target process operation mode" may include:

[0098] In response to a triggering operation on at least one calibration tool, displaying a fixed point mark corresponding to the calibration tool in the updated workpiece image;

[0099] In response to the confirmation operation for the fixed-point mark, the adjusted workpiece image is displayed.

[0100] The fixed point mark may be an instruction for an operator when operating a calibration tool.

[0101] In one embodiment, the calibration tool can have various forms. For example, the calibration tool can be a rectangle, a circle, a dot, or a custom graphic, etc. For example, as shown in FIG4 , the tools shown in the area 006 in the figure can all be schematic diagrams of the calibration tool.

[0102] For example, if the operator selects a rectangular calibration tool, an indication corresponding to the rectangle may be displayed in the interface, and the operator can then position the image by operating the indication.

[0103] For example, when the target process operation mode is spot welding, the calibration tools in the operation interface are the dot calibration tool and the custom shape calibration tool. For another example, when the target process operation mode is screwing, the calibration tools in the operation interface can be calibration tools corresponding to screws of different sizes or types. For another example, when the target process operation mode is gluing, the calibration tools in the operation interface can be the circle calibration tool and the custom shape calibration tool.

[0104] For example, when the target process operation mode is spot welding, the target control robot can determine the location of the workpiece where spot welding is required through fixed-point operation. For another example, when the target process operation mode is gluing, the target control robot can determine the location of the workpiece where gluing is required through fixed-point operation. For another example, when the target process operation mode is suction, the target control robot can determine the suction location through fixed-point operation.

[0105] In one embodiment, in response to a trigger operation on at least one calibration tool, a fixed point marker corresponding to the calibration tool is displayed in the updated workpiece image; in response to a confirmation operation on the fixed point marker, the adjusted workpiece image may be displayed.

[0106] For example, when the target process operation mode is transport, the calibration tools on the operation interface may include a rectangle calibration tool and a custom shape calibration tool. If the user selects the rectangle calibration tool, a rectangular positioning marker may be displayed in the interface. The user can then drag the rectangular positioning marker to locate points in the updated workpiece image to obtain the adjusted workpiece image. For example, if the user uses the rectangular positioning marker to locate two points in the updated workpiece image, two rectangular boxes may be displayed in the adjusted workpiece image.

[0107] For another example, when the target process operation mode is spot welding, the calibration tools in the operation interface may include a dot calibration tool and a custom shape calibration tool. If the user selects the dot calibration tool, a dot positioning marker will appear in the interface. The user can then drag the dot positioning marker to locate a point in the updated workpiece image, resulting in an adjusted workpiece image. For example, if the user uses the circular positioning marker to locate two points in the updated workpiece image, two dots will appear in the adjusted workpiece image.

[0108] In one embodiment, in response to a triggering operation for at least one calibration tool, a fixed point marker corresponding to the calibration tool may be automatically displayed. For example, after an operator selects the corresponding calibration tool, the control device may automatically identify the updated workpiece image based on the calibration tool, thereby identifying the location in the updated workpiece image where fixed points are required, and displaying the location as a fixed point marker in the adjusted workpiece image.

[0109] In one embodiment, the operator's positioning of the updated workpiece image may include two steps: selecting a target point for positioning and determining an execution order corresponding to the target point. Therefore, the adjusted workpiece image may include an execution order indication of the target point.

[0110] For example, if the target process operation mode is spot welding, the adjusted workpiece image can display the points on the workpiece that need to be spot welded. If the workpiece has multiple points requiring spot welding, the adjusted workpiece image can also display the order in which these points need to be welded. Generally, the order in which the operator selects the points is the order in which the spot marking is executed.

[0111] Specifically, the step of “displaying the adjusted workpiece image in response to the confirmation operation on the fixed-point mark” may include:

[0112] In response to a selection operation on the fixed point identifier, emphasizing and displaying the target fixed point;

[0113] In response to the order confirmation operation for the target pointing, an execution order indication of the target pointing is displayed.

[0114] For example, in response to a trigger operation on at least one calibration tool, a fixed point marker corresponding to the calibration tool can be displayed in the updated workpiece image. An operator can then operate the fixed point marker to locate the target. Therefore, in response to a selection operation on the fixed point marker, the control device can emphasize the display of the target fixed point. For example, in response to a selection operation on the fixed point marker, the control device can highlight the target fixed point.

[0115] For another example, if the operator selects multiple target points, the execution order instructions of the multiple target points may also be displayed.

[0116] In one embodiment, the workpiece to be operated may be a high-precision workpiece. For example, the workpiece to be operated may be a chip, etc. Therefore, the updated workpiece image will contain at least one precise structure of the workpiece to be operated. Precision structures are structures that are extremely small and beyond the range of human visual perception.

[0117] Therefore, when positioning the updated workpiece image, it is possible to position the precise structure in the updated workpiece image. Specifically, the step of "displaying a fixed point marker corresponding to the calibration tool in the updated workpiece image in response to a triggering operation on at least one calibration tool" may include:

[0118] In response to a selection operation on at least one calibration tool, displaying a control mark corresponding to the calibration tool;

[0119] In response to a control operation on the control mark, a fixed-point mark corresponding to the precise structure is displayed on the updated workpiece image.

[0120] For example, when the workpiece to be operated is a very small chip, the operator can magnify the workpiece image to the appropriate size so that the chip structure can be clearly viewed. The operator can then use the control calibration tool to achieve high-precision positioning of the chip structure.

[0121] 104. Based on the operation interface of the target process operation mode, in response to the control operation on the adjusted workpiece image, generate and send a control instruction to the target control robot, so that the target control robot performs the operation in the target process operation mode on the workpiece to be operated according to the control instruction.

[0122] In one embodiment, the operation interface includes a control setting area corresponding to the target process operation mode. The control setting area can be used to set the operation process of the target process operation mode. For example, when the target process operation mode is spot welding, the control setting area can set the temperature range, dithering amplitude, etc. of the spot welding. For another example, when the target process operation mode is grasping, the control setting area can set the grasping force, etc. For another example, when the target process operation mode is adsorption, the control setting area can set the adsorption force, duration, and release time, etc.

[0123] Specifically, the step of “generating and sending a control instruction to the target control robot in response to a control operation on the adjusted workpiece image based on an operation interface of the target process operation mode” may include:

[0124] In response to an edit operation on the control setting area, acquiring control setting information;

[0125] Based on the control setting information, generating at least one trajectory route corresponding to the target control robot;

[0126] In response to a selection operation for at least one trajectory route, a control instruction is generated and sent to the target control robot based on the selected target trajectory route.

[0127] In one embodiment, the operator can edit the control setting area. For example, the operator can input corresponding parameters in the control setting area or select corresponding parameters to implement the setting of the operation process of the target process operation mode.

[0128] In one embodiment, after acquiring the control setting information, the present invention can also generate at least one trajectory route corresponding to the target control robot based on the control setting information, so that the operator can view the motion trajectory of the target control robot on the control device. The operator can then select the trajectory route that the target control robot wants to run. Therefore, in response to the selection operation for the at least one trajectory route, a control instruction can be generated and sent to the target control robot based on the selected target trajectory route.

[0129] For example, as shown in Figure 9, the operator can use the operation interface to set the process for the target control robot to execute the process. For example, when the target process operation mode is transport, the operator can set the transport order, etc. The operator can then trigger the "Automatically Generate Trajectory" control, and the control device will automatically generate the corresponding trajectory route. For example, as shown in Figure 10, the operation route corresponding to the target control robot can be displayed in the operation interface. The operator can then select the corresponding trajectory route for the target control robot, so that the target control robot can perform the corresponding operation according to the target trajectory route.

[0130] In one embodiment, the method proposed in the embodiment of the present application may further include:

[0131] When it is detected that the operating environment of the target control robot is updated, the updated operating environment information is obtained;

[0132] updating at least one trajectory route based on the updated operating environment information, obtaining and displaying the updated trajectory route;

[0133] For example, when the control device detects an update in the operating environment information of the target control robot through a smart camera, the control device can obtain the updated operating environment information. Then, the control device can update at least one trajectory route based on the updated operating environment information, obtain and display the updated trajectory route.

[0134] Then, the step of “generating and sending a control instruction to the target control robot based on the selected target trajectory route in response to the selection operation for the at least one trajectory route” may include:

[0135] In response to a selection operation for at least one updated trajectory route, a control instruction is generated and sent to the target control robot based on the selected target updated trajectory route.

[0136] After the trajectory route is updated, the control device can display the updated trajectory route on the control device in real time, so that the operator can view the updated trajectory route in time and select the execution route of the target control robot based on the updated trajectory route.

[0137] The robot control method provided in this embodiment displays a robot control interface, wherein the robot control interface includes operating environment information of the target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated; in response to a selection operation for the process operation mode, an operation interface of the target process operation mode is displayed, wherein the operation interface includes a display status of the operating environment information under the target process operation mode, the display status of the operating environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, in response to an adjustment operation for the workpiece image, an adjusted workpiece image is displayed; based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, a control instruction is generated and sent to the target control robot, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instruction. The embodiments of the present application collect operating environment information of the target control robot and display the operating environment information on the device. The operating environment information includes an image of the workpiece to be operated. This allows the operator to perform fixed-point debugging directly by looking at the workpiece image on the control device, thereby improving the efficiency and accuracy of fixed-point debugging. Furthermore, the embodiments of the present application can also support the operation of workpiece images in different process operation modes, thereby improving the applicability of the embodiments of the present application.

[0138] Example 2

[0139] Based on the same application concept, a second embodiment of the control method of the robot of this application is proposed. The control method of the robot is applied to the control system of the robot, which includes a target control robot, an intelligent camera, a control device and a cloud server.

[0140] The robot may be a robot with a certain degree of automation that can perform various processing and manufacturing functions by relying on its own power source and control capabilities. For example, the robot mentioned in this application may be an industrial robot. For example, the robot proposed in this application may be a six-axis robot, a four-axis robot (Selective Compliance Assembly Robot Arm, SCARA), or a robot with a manipulator, etc.

[0141] The target controlled robot may refer to a robot controlled by a control device. The control device may display a corresponding robot control interface, and the user may operate the robot control interface to control the robot to perform corresponding operations. For example, the user may operate the control device to control the robot to perform a handling operation, spot welding operation, suction operation, or gluing operation, etc. The control device may be, for example, a teach pendant, a tablet computer, or a laptop computer.

[0142] Smart cameras are highly integrated miniature machine vision systems that integrate image acquisition, processing, and communication functions into a single camera, providing a versatile, modular, highly reliable, and easy-to-implement machine vision solution.

[0143] In one embodiment, the target control robot may be equipped with a smart camera, thereby enabling the smart camera to capture images of the target control robot's operating environment. Furthermore, in addition to the smart camera being equipped on the target control robot, the environment surrounding the target control robot may also be equipped with a smart camera, thereby enabling more comprehensive capture of images of the target control robot's operating environment.

[0144] In one embodiment, a smart camera can be used to collect operating environment information of a target control robot, thereby determining the boundaries and object locations within the target control robot's reachable coverage range. For example, the target control robot can be equipped with a smart camera, which can then collect operating environment information through self-rotation. The smart camera can feed the collected operating environment data back to a cloud server, which can then model the operating environment information. The operating environment information can be a two-dimensional plan view or a three-dimensional view. After modeling the operating environment information, the cloud server can send the operating environment information to a control device, allowing the control device to display the operating environment information. For example, if the operating environment information is a two-dimensional plan view, the two-dimensional plan view can be displayed on the control device. If the operating environment information is a three-dimensional view, the three-dimensional view can be displayed on the control device. The user can then operate the control device to perform fixed-point debugging of the workpiece to be operated. Based on the fixed-point debugging results, the target control robot can be controlled to perform handling operations, spot welding operations, suction operations, gluing operations, etc.

[0145] Furthermore, the control device can display different process operation modes based on the target robot's execution scenario. For example, when selecting spot welding, adjustable variables such as the vibration amplitude, welding temperature range, and single dwell time are displayed. When selecting grasping, adjustable variables such as grasping position and grasping force are displayed. When selecting the suction operation, adjustable variables such as suction force, suction duration, and release time are displayed. Different operations can be controlled by performing different operations on the plan view, thereby controlling the target robot. For example, when selecting the target robot to grasp an object, the digital plan view can be used to zoom in and select the object to be grasped, eliminating the need to understand the 3D tool coordinates of the robot and the workpiece in the same space. This system allows for convenient and mobile debugging of the robot to quickly grasp the corresponding workpiece, meeting the user's actual needs and providing efficient and convenient operation.

[0146] The control system of the robot of this embodiment is described in detail below. As shown in FIG11 , the system may include the following steps:

[0147] 201. Smart cameras collect spatial environment information.

[0148] For example, the target control robot can be a manipulator. The smart camera can be mounted in the manipulator's gripper. The manipulator can then rotate autonomously and collect spatial environment information through the smart camera to determine the boundaries and object locations within the manipulator's reachable coverage area.

[0149] 202. The smart camera transmits the spatial environment information to the cloud server.

[0150] 203. The cloud server identifies and processes the spatial environment information and generates environmental operation information.

[0151] For example, the cloud server can use the Simultaneous Localization and Mapping (SLAM) algorithm and triangulation to model environmental information and draw a two-dimensional plane view or a three-dimensional view.

[0152] 204. The cloud server sends the environmental operation information to the control device.

[0153] 205. The control device displays a robot control interface, where the robot control interface includes operating environment information of the target control robot and at least one process operation mode. The operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated.

[0154] 206. The control device displays an operation interface of the target process operation mode in response to a selection operation for the process operation mode, wherein the operation interface includes a display status of the operation environment information under the target process operation mode, the display status of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode.

[0155] For example, process operation modes may include spot welding, gripping, adsorption, screwing, etc.

[0156] 207 . The control device displays the adjusted workpiece image in response to the adjustment operation on the workpiece image based on the operation interface of the target process operation mode.

[0157] For example, the control device can freely zoom in and out (control accuracy) the workpiece image (including the digitized map) as needed, and determine and draw range position points of different shapes and sizes on the corresponding map.

[0158] 208. The control device generates and sends a control instruction to the target control robot in response to the control operation on the adjusted workpiece image based on the operation interface of the target process operation mode.

[0159] For example, you can adjust detailed parameters based on the target process operation mode. For example, when spot welding, you can set the temperature value, specific range, and vibration intensity; when grasping, the force, direction, and time to complete the grasp, etc.

[0160] 209. The target control robot performs operations in the target process operation mode on the workpiece to be operated according to the control instructions.

[0161] For example, the control device can feed back the operator's operation settings on the digital map to the system, calculate the feedback to the actual physical space and command the robot to execute the operation command at the specified position along the optimal trajectory.

[0162] For example, during the movement of the robot, the smart camera will continuously update the current position and surrounding environment information to ensure the layout and change status within the range covered by the robot.

[0163] For more implementation details of the specific implementation of the above method steps, please refer to the description of the specific implementation in Example 1. For the sake of brevity of the description, they will not be repeated here.

[0164] The control system of the robot provided in this embodiment includes an intelligent camera collecting spatial environment information; the intelligent camera transmits the spatial environment information to a cloud server; the cloud server identifies and processes the spatial environment information to generate environment operation information; the cloud server sends the environment operation information to a control device; the control device displays a robot control interface, the robot control interface including the operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is collected and generated by the intelligent camera carried by the target control robot, and the operation environment information represents the operation environment between the target control robot and the corresponding workpiece to be operated; the control device displays an operation interface of the target process operation mode in response to a selection operation for the process operation mode, wherein the operation interface includes a display state of the operation environment information under the target process operation mode, the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, the control device displays an adjusted workpiece image in response to an adjustment operation on the workpiece image; based on the operation interface of the target process operation mode, the control device generates and sends a control instruction to the target control robot in response to a control operation on the adjusted workpiece image; the target control robot performs an operation under the target process operation mode on the workpiece to be operated according to the control instruction. The control system proposed in the embodiment of the present application can ensure that the operator can perform fixed-point debugging of the operating environment information, thereby ensuring the efficiency and accuracy of the fixed-point debugging.

[0165] Example 3

[0166] Based on the same application concept, with reference to FIG3 , a first embodiment of the control device of the robot of the present application is proposed. The control device of the robot may be a virtual device applied to the control equipment of the robot.

[0167] The following describes in detail the control device of the robot provided in this embodiment with reference to the functional module schematic diagram shown in FIG3 . The device may include:

[0168] a first display unit, configured to display a robot control interface, the robot control interface including operating environment information of the target control robot and at least one process operation mode, wherein the operating environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operating environment information represents the operating environment between the target control robot and the corresponding workpiece to be operated;

[0169] a second display unit, configured to display an operation interface of a target process operation mode in response to a selection operation for the process operation mode, wherein the operation interface includes a display state of the operation environment information in the target process operation mode, the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode;

[0170] a third display unit, configured to display an adjusted workpiece image in response to an adjustment operation on the workpiece image based on an operation interface of the target process operation mode;

[0171] A sending unit is used to generate and send control instructions to the target control robot in response to the control operation on the adjusted workpiece image based on the operation interface of the target process operation mode, so that the target control robot performs the operation under the target process operation mode on the workpiece to be operated according to the control instructions.

[0172] In one embodiment, the third display unit may include:

[0173] a first display subunit, configured to display a selected workpiece image in response to a selection operation on at least one workpiece to be operated in the workpiece images based on an operation interface of the target process operation mode, wherein the selected workpiece image includes the selected target workpiece in the workpieces to be operated;

[0174] a second display subunit, configured to display an updated workpiece image in response to an update operation on the selected workpiece image;

[0175] The third display subunit is configured to display the adjusted workpiece image in response to a fixed-point operation on the updated workpiece image based on an operation interface of the target process operation mode.

[0176] In one embodiment, the third display sub-unit may include:

[0177] a first display module, configured to display a fixed point marker corresponding to the at least one calibration tool in the updated workpiece image in response to a triggering operation on the at least one calibration tool;

[0178] The second display module is configured to display the adjusted workpiece image in response to a confirmation operation on the fixed-point mark.

[0179] In one embodiment, the first display module may include:

[0180] A first display submodule, configured to display a control mark corresponding to the at least one calibration tool in response to a selection operation on the at least one calibration tool;

[0181] The second display submodule is configured to display the fixed-point mark corresponding to the precise structure on the updated workpiece image in response to a control operation on the control mark.

[0182] In one embodiment, the second display module may include:

[0183] A third display submodule is configured to, in response to a selection operation on the fixed point identifier, intensively display the target fixed point;

[0184] The fourth display submodule is configured to display an execution order indication of the target pointing in response to a sequence confirmation operation for the target pointing.

[0185] In one embodiment, the second display sub-unit may include:

[0186] a third display module, configured to display a side image corresponding to the target workpiece in response to a rotation operation on the selected workpiece image;

[0187] The fourth display module is configured to display the updated workpiece image in response to a zoom-in operation on the side image.

[0188] In one embodiment, the sending unit may include:

[0189] a first acquiring subunit, configured to acquire control setting information in response to an editing operation on the control setting area;

[0190] A first generating subunit is configured to generate at least one trajectory route corresponding to the target control robot based on the control setting information;

[0191] The second generating subunit is configured to generate and send a control instruction to the target control robot based on the selected target trajectory route in response to a selection operation on the at least one trajectory route.

[0192] In one embodiment, the sending unit may further include:

[0193] A second acquisition subunit is configured to acquire updated operating environment information when detecting that the operating environment of the target control robot is updated;

[0194] The updating subunit is configured to update the at least one trajectory route based on the updated operating environment information, and obtain and display the updated trajectory route.

[0195] In one embodiment, the second generating subunit may include:

[0196] A generating module is configured to generate and send a control instruction to the target control robot based on the selected target updated trajectory route in response to a selection operation on the at least one updated trajectory route.

[0197] It should be noted that the functions that can be realized by each module in the control device of the robot provided in this embodiment and the corresponding technical effects achieved can refer to the description of the specific implementation methods in each embodiment of the control method of the robot of this application. For the sake of brevity of the specification, they will not be repeated here.

[0198] Example 4

[0199] Based on the same application concept, referring to the hardware structure diagram of Figure 2, this embodiment provides a robot control device, which may include a processor and a memory, in which a robot control program is stored. When the robot control program is executed by the processor, all or part of the steps of each embodiment of the robot control method of the present application are implemented.

[0200] Specifically, the robot's control device refers to a terminal device or network device that can achieve network connection. It can be a terminal device such as a mobile phone, computer, tablet computer, portable computer, embedded industrial computer, or a network device such as a server or cloud platform.

[0201] It can be understood that the control device of the robot may also include a communication bus, a user interface and a network interface. Among them, the communication bus is used to realize the connection and communication between these components; the user interface is used to connect to the client and communicate data with the client. The user interface may include output units such as display screens, speakers, etc., and input units such as keyboards, microphones, etc.; the network interface is used to connect to the background server and communicate data with the background server. The network interface may include input / output interfaces, such as standard wired interfaces and wireless interfaces such as Wi-Fi interfaces; the memory is used to store various types of data, which may include instructions of any application or method in the control device of the robot, as well as application-related data. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (ELC). Memory, EEPROM), magnetic memory, flash memory, disk or optical disk, etc.; in some embodiments, the memory can also be a storage device independent of the processor; the processor is used to call the robot control program stored in the memory and execute the robot control method as described above. The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, which is used to execute all or part of the steps of each embodiment of the robot control method as described above.

[0202] It should be noted that the hardware structure shown in FIG2 does not constitute a limitation on the control device of the robot of the present application, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0203] Example 5

[0204] Based on the same application concept, this embodiment provides a computer-readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a disk, an optical disk, a server, etc. A computer program is stored on the storage medium, and the computer program can be executed by one or more processors. When the computer program is executed by the processor, it can implement all or part of the steps of each embodiment of the control method of the robot of the present application.

[0205] It should be noted that the serial numbers of the above embodiments of this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above embodiments are only optional embodiments of this application and do not limit the patent scope of this application. All equivalent structures or equivalent process changes made by using the contents of this application specification and drawings under the application concept of this application, or directly or indirectly applied in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method for a robot, wherein, the method includes: displaying a robot control interface, the robot control interface including operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operation environment information characterizes the operation environment between the target control robot and a corresponding workpiece to be operated; responding to a selection operation for the process operation mode, displaying an operation interface of the target process operation mode, wherein the operation interface includes a display state of the operation environment information in the target process operation mode, and the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; based on the operation interface of the target process operation mode, responding to an adjustment operation for the workpiece image, displaying an adjusted workpiece image; based on the operation interface of the target process operation mode, responding to a control operation for the adjusted workpiece image, generating and sending a control instruction to the target control robot, so that the target control robot performs an operation in the target process operation mode on the workpiece to be operated according to the control instruction.

2. The method according to claim 1, wherein, the workpiece image is an image corresponding to at least one workpiece to be operated; the step of, based on the operation interface of the target process operation mode, responding to an adjustment operation for the workpiece image and displaying an adjusted workpiece image includes: based on the operation interface of the target process operation mode, responding to a selection operation for at least one workpiece to be operated in the workpiece image, displaying a selected workpiece image, wherein the selected workpiece image includes a target workpiece selected from the workpieces to be operated; responding to an update operation for the selected workpiece image, displaying an updated workpiece image; based on the operation interface of the target process operation mode, responding to a fixed-point operation for the updated workpiece image, displaying the adjusted workpiece image.

3. The method according to claim 2, wherein, the operation interface of the target process operation mode includes at least one calibration tool; the step of, based on the operation interface of the target process operation mode, responding to a fixed-point operation for the updated workpiece image and displaying the adjusted workpiece image includes: responding to a trigger operation for the at least one calibration tool, displaying a fixed-point identifier corresponding to the calibration tool in the updated workpiece image; responding to a confirmation operation for the fixed-point identifier, displaying the adjusted workpiece image.

4. The method according to claim 3, wherein, the updated workpiece image includes at least one precision structure; the step of, responding to a trigger operation for the at least one calibration tool and displaying a fixed-point identifier corresponding to the calibration tool in the updated workpiece image includes: responding to a selection operation for the at least one calibration tool, displaying a control identifier corresponding to the calibration tool; In response to a control operation for the control identifier, a fixed-point identifier corresponding to the precision structure is displayed on the updated workpiece image.

5. The method according to claim 3, wherein, the adjusted workpiece image includes a target fixed point and an execution sequence indication of the target fixed point; The displaying the adjusted workpiece image in response to an acknowledgement operation for the fixed-point identifier includes: intensifying and displaying the target fixed point in response to a selection operation for the fixed-point identifier; displaying the execution sequence indication of the target fixed point in response to a sequence acknowledgement operation for the target fixed point.

6. The method according to claim 2, wherein, when the operation environment information is a three-dimensional operation image, the displaying the updated workpiece image in response to an update operation for the selected workpiece image includes: displaying a side image corresponding to the target workpiece in response to a rotation operation for the selected workpiece image; displaying the updated workpiece image in response to a magnification operation for the side image.

7. The method according to claim 2, wherein, the update operation includes at least one of a magnification operation, a reduction operation, and a rotation operation.

8. The method according to any one of claims 1 to 6, wherein, the operation interface includes a control setting area corresponding to the target process operation mode; Based on the operation interface of the target process operation mode, in response to a control operation for the adjusted workpiece image, generating and sending a control instruction to the target control robot includes: acquiring control setting information in response to an editing operation for the control setting area; generating at least one trajectory route corresponding to the target control robot based on the control setting information; in response to a selection operation for the at least one trajectory route, generating and sending a control instruction to the target control robot based on the selected target trajectory route.

9. The method according to claim 8, wherein, the method further includes: when it is detected that the operation environment of the target control robot is updated, acquiring updated operation environment information; performing an update process on the at least one trajectory route based on the updated operation environment information, obtaining and displaying an updated trajectory route; The in response to a selection operation for the at least one trajectory route, generating and sending a control instruction to the target control robot based on the selected target trajectory route includes: in response to a selection operation for the at least one updated trajectory route, generating and sending a control instruction to the target control robot based on the selected target updated trajectory route.

10. A control device for a robot, wherein, the method includes: a first display unit configured to display a robot control interface, the robot control interface including operation environment information of a target control robot and at least one process operation mode, wherein the operation environment information of the target control robot is collected and generated by an intelligent camera carried by the target control robot, and the operation environment information characterizes the operation environment between the target control robot and a corresponding workpiece to be operated; A second display unit, configured to display an operation interface of a target process operation mode in response to a selection operation for the process operation mode, where the operation interface includes a display state of the operation environment information in the target process operation mode, and the display state of the operation environment information includes a workpiece image of the workpiece to be operated, and the target process operation mode is the selected process operation mode; A third display unit, configured to display an adjusted workpiece image in response to an adjustment operation for the workpiece image based on the operation interface of the target process operation mode; A sending unit, configured to generate and send a control instruction to the target control robot in response to a control operation for the adjusted workpiece image based on the operation interface of the target process operation mode, so that the target control robot performs an operation in the target process operation mode on the workpiece to be operated according to the control instruction.

11. A control device for a robot, wherein, the device includes a processor and a memory, and a control program for the robot is stored on the memory. When the control program for the robot is executed by the processor, the control method for the robot according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium, wherein, a computer program is stored on the storage medium. When the computer program is executed by one or more processors, the control method for the robot according to any one of claims 1 to 9 is implemented.

Images