Robot control method, system, device and storage medium

Abstract

Embodiments of the present application disclose a control method, system and device applied to a robot and a storage medium, the method comprising: determining at least one operation stage in an operation process in which a target robot places a target object in a target area; sending, by the target robot, an image acquisition request of a current operation stage to a vision device for the at least one operation stage; receiving, by the vision device, the image acquisition request of the current operation stage, and acquiring an operation image corresponding to the current operation stage based on the image acquisition request; processing, by the vision device, the current operation image to obtain operation feedback information corresponding to the current operation stage, and sending the operation feedback information to the target robot, so that the target robot can receive the operation feedback information corresponding to the operation stage, and then place the target object based on the operation feedback information, thereby improving the accuracy of placing the target object.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and particularly relate to a control method and system applied to a Fanuc robot, a device, and a storage medium. BACKGROUND

[0002] When the Fanuc robot is working, it grasps a target object and places the target object at a target position on a corresponding target device. In order to ensure the accuracy of the target object placed at the target position, the initial position of the target object is usually accurately positioned, the target object is grasped by the Fanuc robot, and the target object is placed at the target position.

[0003] However, if the initial position of the target object is not accurate enough, or if the target object grasped by the Fanuc robot is not at the preset position due to problems such as force, or if the position of the target device deviates, causing the target position in the target device to be not at the preset position, these reasons may cause the target object to be unable to be accurately placed at the target position. In the existing solution, the Fanuc robot cannot know whether the target object is accurately placed at the target position, and if not, the same operation is still performed to place the target object, which may cause more target objects to be placed inaccurately, and thus cause waste of materials. SUMMARY

[0004] Embodiments of the present application provide a control method and system applied to a Fanuc robot, a device, and a storage medium, to process position deviation information of a target object grasped by the Fanuc robot and position deviation information of a target region of a target device, and to place the target object at a target position according to total position deviation information, thereby improving the accuracy of the target object placed at the target position.

[0005] In a first aspect, embodiments of the present application provide a control method applied to a Fanuc robot, the method comprising:

[0006] determining at least one operation stage in an operation process of placing a target object by a target robot in a target region;

[0007] for at least one operation stage, sending an image acquisition request of a current operation stage to a vision device by the target robot;

[0008] receiving the image acquisition request of the current operation stage by the vision device, and acquiring an operation image corresponding to the current operation stage based on the image acquisition request;

[0009] processing the current operation image by the vision device to obtain operation feedback information corresponding to the current operation stage, and sending the operation feedback information to the target robot.

[0010] In a second aspect, the embodiments of the present application further provide a control system applied to the Genovis robot, and the system comprises:

[0011] an operation stage determination module configured to determine at least one operation stage in an operation process in which the target robot places the target object in the target area;

[0012] a request sending module configured to send, for the at least one operation stage, an image acquisition request of a current operation stage to the vision device through the target robot;

[0013] a request receiving module configured to receive the image acquisition request of the current operation stage through the vision device, and acquire an operation image corresponding to the current operation stage based on the image acquisition request;

[0014] an image processing module configured to process the current operation image through the vision device, to obtain operation feedback information corresponding to the current operation stage, and to send the operation feedback information to the target robot.

[0015] In a third aspect, the embodiments of the present application further provide an electronic device, and the electronic device comprises:

[0016] one or more processors;

[0017] a storage device configured to store one or more programs,

[0018] when the one or more programs are executed by the one or more processors, the one or more processors implement the control method applied to the Genovis robot as any of the embodiments of the present application.

[0019] In a fourth aspect, the embodiments of the present application further provide a storage medium containing computer executable instructions, when the computer executable instructions are executed by a computer processor, the computer executable instructions are used to execute the control method applied to the Genovis robot as any of the embodiments of the present application.

[0020] The technical solution of this invention determines at least one operation stage in the process of a target robot placing a target object in a target area. The target robot sends an image acquisition request for each operation stage to a vision device. The vision device receives the image acquisition request for the current operation stage, acquires the operation image corresponding to the current operation stage based on the image acquisition request, processes the current operation image to obtain operation feedback information corresponding to the current operation stage, and sends the operation feedback information to the target robot. This technical solution of the present invention achieves the determination of operation feedback information based on the operation images of the target robot during the process of placing the target object in the target area, and feeds the operation feedback information back to the target robot. This allows the target robot to understand whether there were any operational errors during the entire operation and whether the target object was accurately placed in the target area, realizing two-way communication between the vision device and the target robot. The target robot can make corresponding adjustments to its robotic arm based on the operation feedback information. The technical solution of this invention improves the accuracy of target object placement. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0022] Figure 1 This is a flowchart illustrating a control method for a FANUC robot provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart illustrating a control method for a FANUC robot provided in Embodiment 2 of the present invention;

[0024] Figure 3 This is a schematic diagram of a control system for a FANUC robot provided in Embodiment 3 of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures. Example 1

[0027] Figure 1 A flowchart of a control method for a FANUC robot is provided in the embodiments of the present application. The embodiments of the present application can be applied to the control of a FANUC robot in operation. The method can be executed by a control system for a FANUC robot, which can be implemented in the form of software and / or hardware.

[0028] As Figure 1 The control method for a FANUC robot according to the embodiments of the present application specifically includes the following steps:

[0029] S110, determining at least one operation stage in an operation process of a target robot placing a target object in a target region.

[0030] The target robot refers to a robot performing the placement of a target object. In the embodiments of the present application, the target robot refers to a FANUC robot. The target object refers to an object requiring placement, such as an icon, a workpiece, etc. The target region refers to a region where the target object needs to be placed, for example, when the target object refers to a computer icon, the target region refers to a region where the computer icon needs to be placed on the front of a computer display (for example, the lower left corner of the front of the computer display). The operation process refers to the process of the target robot grasping the target object and placing the target object in the target region. The operation process includes at least one operation stage, for example, when the target object refers to a computer icon and the target region refers to the lower left corner of the front of the computer display, the operation stage refers to the grasping operation of the target robot on the computer icon, the movement of the target robot to the position of the computer display after grasping the computer icon, and the placement of the target robot on the computer icon in the preset lower left corner of the front of the computer display. It should be understood that the movement of each operation stage in the operation process of the target robot is pre-set.

[0031] Specifically, the target robot places the target object in the target region according to the pre-set operation process. First, at least one operation stage of the operation process is determined. Optionally, the manner of obtaining at least one operation stage of the operation process includes but is not limited to: determining the operation process according to the correspondence between the target object and the pre-set operation process corresponding to the object, and then determining at least one operation stage included in the operation process. This is preparatory work for subsequent sending of image acquisition requests for at least one operation stage.

[0032] S120, for at least one operation stage, sending an image acquisition request of the current operation stage to a vision device through the target robot.

[0033] The image acquisition request is a request for acquisition of an operation image of an operation stage. The vision device is a device with a shooting function and a computing function, which can receive the image acquisition request and shoot an image according to the image acquisition request and subsequent image processing. It should be noted that the communication protocol between the Fanuc robot and the external device is the TCP / IP protocol, so the communication protocol in the embodiment of the present application is set to the TCP / IP protocol. Moreover, the kernel language of the Fanuc robot is KAREL, so the image acquisition request sent by the target robot to the vision device and the subsequent feedback information received by the target robot are written in the form of the KAREL language. It should be noted that the port number of the target robot in the embodiment of the present application can be changed according to the port number of the vision device to realize the connection between the target robot and the vision device.

[0034] Specifically, for at least one operation stage in the operation process, the target robot sends an image acquisition request of the current operation stage to the vision device, so that the vision device shoots an operation image corresponding to the current operation stage based on the image acquisition request. For example, the current operation stage is that the target robot grasps a target object, and the vision device needs to shoot the target object grasped by the target robot. It should be noted that the vision device in the embodiment of the present application can have multiple cameras, and a camera is arranged at a corresponding position of each operation stage to avoid the movement of the camera causing errors in the shooting angle and affecting the subsequent processing of the shot image.

[0035] In the embodiment of the present application, the target robot sends an image acquisition request of the current operation stage to the vision device, including: when the target robot detects that the mechanical arm moves to a preset shooting position corresponding to each operation stage, the target robot sends an image acquisition request corresponding to each operation stage to the vision device.

[0036] The target robot includes a mechanical arm. It should be noted that the degrees of freedom of the mechanical arm can be set as needed, such as 4 degrees of freedom, 5 degrees of freedom, etc. The shooting position of each operation stage is set in advance, and when the mechanical arm moves to the preset shooting position, the image acquisition request corresponding to the operation stage is sent to the vision device.

[0037] Specifically, the target robot detects whether the mechanical arm moves to a preset shooting position corresponding to the current operation stage. If yes, the target robot sends an image acquisition request corresponding to the current operation stage to the vision device. Optionally, the image acquisition request corresponding to each operation stage is set in advance. For example, the image acquisition requests for each operation stage are distinguished, and different image acquisition requests are assigned corresponding stage identifiers.

[0038] In S130, the vision device receives the image acquisition request of the current operation stage and acquires an operation image corresponding to the current operation stage based on the image acquisition request.

[0039] Specifically, the vision device acquires an operation image corresponding to the current operation stage according to the image acquisition request. Optionally, at least one camera corresponding to the vision device is set in each operation stage, and the position and angle of the at least one camera are determined. When the image acquisition request of the current operation stage is received, the at least one camera corresponding to the current operation stage is turned on to shoot the operation image. This optional scheme can realize camera fixation, avoiding the problem that the movement of the camera causes the operation image to be not accurate enough in subsequent processing. Of course, the camera of the vision device can also be movable. When the image acquisition request of the current operation stage is received, the camera is moved to a preset position corresponding to the current operation stage according to the preset correspondence between the position information of the camera and the operation stage, and the operation image is shot at a preset angle. This optional scheme can use only one camera, reducing resource waste.

[0040] In S140, the vision device processes the current operation image to obtain operation feedback information corresponding to the current operation stage, and sends the operation feedback information to the target robot.

[0041] The operation feedback information refers to feedback information for the current operation stage obtained after processing the current operation image. The target robot can determine whether the target object is accurately placed in the target area according to the operation feedback information.

[0042] Specifically, the current operation image is processed by the visual device, and the processing manner includes but is not limited to image recognition on the current operation image, and operation feedback information is determined according to the image recognition result; the current operation image is compared with a preset operation image corresponding to the current operation stage, and operation feedback information is determined according to the comparison result. The operation feedback information is sent to the target robot, so that the target robot knows the feedback information of the target object in the operation process of being placed in the target area. The target robot can adjust the mechanical arm and the like according to the feedback information, so that the target object is accurately placed in the target area.

[0043] The technical scheme of the embodiment of the application determines at least one operation stage in the operation process of the target robot placing the target object in the target area, sends an image acquisition request of each operation stage to the visual device by the target robot, receives the image acquisition request of the current operation stage by the visual device, and acquires the operation image corresponding to the current operation stage based on the image acquisition request. The current operation image is processed by the visual device to obtain operation feedback information corresponding to the current operation stage, and the operation feedback information is sent to the target robot. Through the technical scheme of the embodiment of the application, operation feedback information is determined according to the operation image of the target robot in the operation process of placing the target object in the target area, and the operation feedback information is fed back to the target robot, so that the target robot can know whether there is an operation error in the entire operation process and whether the target object is accurately placed in the target area, and bidirectional communication between the visual device and the target robot is realized. The target robot can adjust the corresponding mechanical arm according to the operation feedback information, and the technical scheme of the embodiment of the application improves the accuracy of placing the target object. Embodiment two

[0044] Figure 2 is a flowchart of a control method applied to a Genasys robot provided by the embodiment of the application. The embodiment of the application refines step 140 on the basis of the optional scheme of the above-mentioned embodiment, and the specific refinement process is described in detail in the embodiment of the application. Technical terms same as or similar to those in the above-mentioned embodiment will not be described again.

[0045] As Figure 2 described, the control method applied to the Genasys robot provided by the embodiment of the application specifically includes the following steps:

[0046] S210, at least one operation stage in the operation process of the target robot placing the target object in the target area is determined.

[0047] S220, for at least one operation stage, an image acquisition request of the current operation stage is sent to the visual device by the target robot.

[0048] S230, receiving, by the vision device, an image acquisition request of a current operation stage, and acquiring an operation image corresponding to the current operation stage based on the image acquisition request.

[0049] S240, processing, by the vision device, the current operation image to determine whether the current operation image includes a target operation object.

[0050] It should be noted that the target operation object corresponding to each operation stage is set in advance, for example, in the operation stage of placing a target object, the target operation object refers to a target region.

[0051] Specifically, the vision device processes the current operation image, compares the target operation object corresponding to the operation stage with the object to be processed identified from the current operation image, and determines whether the current operation image includes the target operation object. If yes, step 250 is performed, and if no, step 260 is performed.

[0052] In the embodiment of the application, the processing, by the vision device, of the current operation image to determine whether the current operation image includes the target operation object includes: comparing, by the vision device, the current operation image with a preset operation image corresponding to an operation stage to which the current operation image belongs and pre-stored in the vision device, to determine whether the current operation image includes the target operation object.

[0053] It should be noted that the target operation object corresponding to each operation stage is set in advance, for example, in the operation stage of placing a target object, the target operation object refers to a target region.

[0054] Specifically, the vision device processes the current operation image, compares the target operation object corresponding to the operation stage with the object to be processed identified from the current operation image, and determines whether the current operation image includes the target operation object. If yes, step 250 is performed, and if no, step 260 is performed.

[0055] S250, determining, according to current position information and preset position information of the target operation object, position offset information of the current operation object, and generating operation feedback information corresponding to the current operation stage based on state information of the target operation object and the position offset information.

[0056] The position information refers to position information of the target operation object in an image. In the embodiment of the present application, the position information can refer to position information of the target operation object in a spatial coordinate system. A spatial coordinate system is set in advance, for example, an origin is set on one end of a mechanical arm of the target robot that grasps the target object, and then a spatial coordinate system is established. Through the establishment of the spatial coordinate system, the coordinates of the target object grasped by the mechanical arm can be known. It should be understood that the coordinates of the target region in the spatial coordinate system can also be calculated. The preset position information refers to position information set in advance, including position information of the target object on the mechanical arm when the mechanical arm grasps the target object and / or position information of the target region. Optionally, in the embodiment of the present application, the position information can only include position information on a plane to which the target object belongs, and the plane to which the target region belongs and the plane to which the target object belongs are regarded as parallel planes, facilitating subsequent calculation. The state information of the target operation object refers to whether the target operation object is included in the current operation image.

[0057] Specifically, the position offset information of the current operation object is determined according to the current position information of the target operation object and the preset position information. The operation feedback information is generated according to the position offset information and the state information of the target operation object, for example, the state information is OK (indicating that the target operation object is included in the current operation image), the x-axis direction offset is positive 3 unit lengths, the y-axis direction offset is negative 1 unit length, and the offset angle is 4 degrees from the positive direction of the x-axis to the positive direction of the y-axis.

[0058] In another optional scheme of the embodiment of the present application, the visual device determines the current operation stage according to the image acquisition request. In the operation stage before the step of placing the target object in the target region, the visual device feeds back the state information of the target operation object as the operation feedback information, and the position offset information is stored in the visual device. The position offset information of all operation stages is integrated and processed to obtain the final position offset information. When the target object is placed in the target region, the final position offset information and the state information are fed back as the operation feedback information, so that the target robot moves the mechanical arm according to the final position offset information, and the target object is accurately placed in the target region.

[0059] S260, generating operation feedback information corresponding to the current operation stage based on the state information of the target operation object.

[0060] Specifically, the state information of the target operation object is determined as the operation feedback information corresponding to the current operation stage, for example, the state information is NG (indicating that the target operation object is not included in the current operation image).

[0061] In the embodiment of the present application, the operation feedback information is received by the target robot, and the operation of placing the target object in the target region is performed based on the operation feedback information.

[0062] Specifically, the target robot receives the operation feedback information sent by the vision device, and places the target object in the target area according to the operation feedback information. The target robot moves the mechanical arm according to the position offset information in the operation feedback information, so that the target object can be accurately placed in the target area.

[0063] In an optional solution of the embodiment of the present application, when the operation feedback information only has state information as NG, the target robot determines the operation stage corresponding to the operation feedback information, and determines the adjustment scheme corresponding to the operation stage. When the target operation object corresponding to the operation stage is the target object, the movement track of the mechanical arm or the force of the mechanical arm grabbing the target object is adjusted, so that the target object can be avoided from falling when the target object is re-grabbed. When the target operation object is the target area, the warning information is sent, so that the system re-sends the target device to the preset position, and then determines the target area in the target device.

[0064] In the embodiment of the present application, the target robot receives the operation feedback information, including:

[0065] The target robot receives the operation feedback information corresponding to each operation stage respectively, and stores the operation feedback information in the target data storage area.

[0066] The target data storage area is used for storing operation feedback information, and the target data storage area includes a numerical register.

[0067] Specifically, the operation feedback information is analyzed, and the state information and the position offset information included in the operation feedback information are stored in the target data storage area in the form of numerical values, so as to facilitate the target robot to retrieve the operation feedback information from the target data storage area.

[0068] In the embodiment of the present application, the operation of placing the target object in the target area based on the operation feedback information includes: when the current operation stage is the operation of placing the target object in the target area, and the state information of the target operation object in the received operation feedback information determines that the current operation stage contains the target operation object, the target robot reads the position offset information in the target data storage area, and performs the operation of placing the target object in the target area based on the position offset information.

[0069] Specifically, in the operation stage of placing the target object in the target region, when the operation feedback information is received, the operation feedback information is stored in the target data storage area. The target robot reads the state information in the target data storage area. According to the state information, it is determined whether the target operation object is included in the current operation stage. If yes, the target robot reads the position offset information in the target data storage area, and places the target object in the target region according to the position offset information; if no, an adjustment scheme corresponding to the operation stage is determined, and the placing operation of the target object is re-performed according to the adjustment scheme.

[0070] S270, the operation feedback information is sent to the target robot.

[0071] The technical scheme of the embodiment of the application determines at least one operation stage in the operation process of placing the target object in the target region by the target robot, sends an image acquisition request of each operation stage to the vision device by the target robot, receives the image acquisition request of the current operation stage by the vision device, and acquires the operation image corresponding to the current operation stage based on the image acquisition request. The current operation image is processed by the vision device to determine whether the target operation object is included in the current operation stage. If yes, the position offset information is determined according to the current position information of the target operation object and the preset position information, the operation feedback information is generated based on the state information and the position offset information, and if no, the operation feedback information is generated based on the state information. And send the operation feedback information to the target robot. Through the technical scheme of the embodiment of the application, it is determined whether the target operation object exists in the corresponding operation stage according to the operation image of the target robot in the operation process of placing the target object in the target region. Two kinds of operation feedback information are determined according to the two results, and the operation feedback information is fed back to the target robot, so that the target robot makes different responses according to different operation feedback information, thereby improving the accuracy of placing the target object. Embodiment three

[0072] Figure 3 A structural schematic diagram of a control system applied to a Genasys robot is provided for the embodiment of the application. The control system applied to the Genasys robot provided by the embodiment of the application can execute the control method applied to the Genasys robot provided by any embodiment of the application, and has the corresponding function modules and beneficial effects of the execution method. The system comprises an operation stage determination module 310, a request sending module 320, a request receiving module 330 and an image processing module 340; wherein:

[0073] The operation stage determination module 310 is used for determining at least one operation stage in the operation process of placing the target object in the target region by the target robot.

[0074] The request sending module 320 is configured to send, by the target robot, an image acquisition request of a current operation stage to the vision device for the at least one operation stage.

[0075] The request receiving module 330 is configured to receive, by the vision device, the image acquisition request of the current operation stage, and acquire an operation image corresponding to the current operation stage based on the image acquisition request.

[0076] The image processing module 340 is configured to process, by the vision device, the current operation image to obtain operation feedback information corresponding to the current operation stage, and send the operation feedback information to the target robot.

[0077] Further, the image processing module 340 comprises:

[0078] The target operation object determination submodule is configured to process, by the vision device, the current operation image to determine whether the target operation object is included in the current operation image, wherein the target operation object comprises the target object and / or the target region; if yes, determine position offset information of the target operation object according to current position information of the target operation object and preset position information, and generate the operation feedback information corresponding to the current operation stage based on state information of the target operation object and the position offset information; if no, generate the operation feedback information corresponding to the current operation stage based on the state information of the target operation object.

[0079] Further, the target operation object determination submodule comprises:

[0080] The image comparison unit is configured to compare, by the vision device, the current operation image with a preset operation image corresponding to an operation stage to which the current operation image belongs and pre-stored in the vision device, to determine whether the target operation object is included in the current operation image, wherein the preset operation image comprises the target operation object.

[0081] Further, the system further comprises:

[0082] The operation feedback information receiving module is configured to receive, by the target robot, the operation feedback information, and perform the operation of placing the target object in the target region based on the operation feedback information.

[0083] Further, the operation feedback information receiving module comprises:

[0084] The operation feedback information storage submodule is configured to receive, by the target robot, the operation feedback information corresponding to each operation stage respectively, and store the operation feedback information in a target data storage area.

[0085] Further, the operation feedback information receiving module comprises:

[0086] The target object placing sub-module is configured to read the position offset information in the target data storage area by the target robot when the current operation stage is the operation of placing the target object in the target region and the state information of the target operation object in the received operation feedback information indicates that the current operation stage contains the target operation object, and execute the operation of placing the target object in the target region based on the position offset information.

[0087] Further, the request sending module 320 comprises:

[0088] The image acquisition request sending sub-module is configured to send an image acquisition request corresponding to each operation stage to the vision device by the target robot when the robot arm is detected to move to the preset shooting position corresponding to each operation stage by the target robot.

[0089] The technical scheme of the embodiment of the present application determines at least one operation stage in the operation process of placing the target object in the target region by the target robot, sends an image acquisition request of each operation stage to the vision device by the target robot, receives the image acquisition request of the current operation stage by the vision device, acquires the operation image corresponding to the current operation stage based on the image acquisition request, processes the current operation image by the vision device to obtain the operation feedback information corresponding to the current operation stage, and sends the operation feedback information to the target robot. Through the technical scheme of the embodiment of the present application, the operation feedback information is determined according to the operation image of the target robot in the operation process of placing the target object in the target region, and the operation feedback information is fed back to the target robot, so that the target robot can know whether there is an operation error in the entire operation process or whether the target object is accurately placed in the target region, and the bidirectional communication between the vision device and the target robot is realized. The target robot can adjust the robot arm according to the operation feedback information, and the technical scheme of the embodiment of the present application improves the accuracy of placing the target object.

[0090] It should be noted that the units and modules included in the above system are only divided according to the function logic, and are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for convenient mutual distinction, and are not used to limit the protection scope of the embodiment of the present application. Embodiment four

[0091] Figure 4 A structural schematic diagram of an electronic device provided by the embodiment of the present application. Figure 4A block diagram is shown of an exemplary electronic device 50 suitable for implementing embodiments of the present invention. Figure 4 The electronic device 50 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0092] like Figure 4 As shown, the electronic device 50 is represented in the form of a general-purpose computing device. The components of the electronic device 50 may include, but are not limited to: one or more processors or processing units 501, system memory 502, and bus 503 connecting different system components (including system memory 502 and processing unit 501).

[0093] Bus 503 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0094] Electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0095] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 503 via one or more data media interfaces. Memory 502 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0096] Program / utility 508 having a set of program modules 507 can be stored in system memory 502 by way of example, and not limitation, including an operating system, one or more application programs, other program modules, and program data, each or any combination thereof, which may

[0097] Electronic device 50 can also communicate with one or more external devices 509 such as a keyboard or a pointing device, displays 510, etc.; one or more devices that enable a user to interact with electronic device 50; and / or one or more devices that enable electronic device 50 to communicate with one or more other computing devices. Such communication can occur via input / output (I / O) interfaces 511. Still yet, electronic device 50 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 512. As depicted, network adapter 512 can communicate with the other components of electronic device 50 via bus 503. It should be appreciated that although not shown, other hardware and / or software components could be used in conjunction with electronic device 50. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc. Figure 4

[0098] Processing unit(s) 501 can execute instructions, for example, stored in system memory 502 to implement embodiments of the application, for example, to implement a control method applied to a Fanuc robot. Embodiment five

[0099] Embodiments of the application also provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to perform a control method applied to a Fanuc robot, the method comprising:

[0100] determining at least one operation stage in an operation process in which a target robot places a target object in a target area; sending, by the target robot, an image acquisition request of a current operation stage to a vision device for the at least one operation stage; receiving, by the vision device, the image acquisition request of the current operation stage, and acquiring an operation image corresponding to the current operation stage based on the image acquisition request; processing, by the vision device, the current operation image to obtain operation feedback information corresponding to the current operation stage, and sending the operation feedback information to the target robot.

[0101] ​The computer storage medium of the embodiments of the present application can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0102] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave, in which computer readable program code is embodied. Such propagated data signals can take a wide variety of forms, including but not limited to electro-magnetic signals, optical signals, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a storage medium, that is capable of storing the program for use by or in connection with the instruction execution system, apparatus or device.

[0103] The program code embodied on the computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0104] The computer program code for carrying out operations of the embodiments of the present application can be written in one or more programming languages or combinations of languages including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0105] Note that the above merely describes preferred embodiments of the present application and the principles of the technology applied. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, modifications and substitutions can be made to the present application without departing from the scope of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments, and can include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A control method applied to a robot, characterized by, The method comprises the following steps: determining at least one operation stage in the operation process of placing the target object in the target area by the target robot; wherein the action of each operation stage in the operation process of the target robot is pre-set; for at least one operation stage, when the target robot detects that the mechanical arm moves to the pre-set shooting position corresponding to each operation stage, the target robot sends an image acquisition request corresponding to each operation stage to the vision device; the vision device receives the image acquisition request of the current operation stage, and acquires the operation image corresponding to the current operation stage based on the image acquisition request; the vision device processes the current operation image to obtain operation feedback information corresponding to the current operation stage, and sends the operation feedback information to the target robot; wherein the processing of the current operation image by the vision device to obtain the operation feedback information corresponding to the current operation stage comprises: processing the current operation image by the vision device to determine whether the target operation object is included in the current operation image; wherein the target operation object includes the target object and / or the target area; if yes, determining the position offset information of the target operation object according to the current position information and the pre-set position information of the target operation object, and generating the operation feedback information corresponding to the current operation stage based on the state information of the target operation object and the position offset information; wherein the operation feedback information includes the final position offset information and the state information; the final position offset information refers to the integrated processing of the position offset information of all operation stages; if no, generating the operation feedback information corresponding to the current operation stage based on the state information of the target operation object; the target robot determines the operation stage corresponding to the operation feedback information, and determines the adjustment scheme corresponding to the operation stage, and when the target operation object corresponding to the operation stage is the target object, adjusts the movement track of the mechanical arm or the force of the mechanical arm grabbing the target object, so as to avoid the target object from falling when the target object is re-grabbed; when the target operation object is the target area, sends a warning information to make the system resend the target device to the pre-set position, and then determines the target area in the target device.

2. The method of claim 1, wherein, the processing of the current operation image by the vision device to determine whether the target operation object is included in the current operation image comprises: comparing the current operation image with the pre-set operation image corresponding to the operation stage to which the current operation image belongs and pre-stored in the vision device by the vision device to determine whether the target operation object is included in the current operation image, wherein the pre-set operation image includes the target operation object.

3. The method of claim 1, wherein, Further comprising: receiving the operation feedback information by the target robot, and executing the operation of placing the target object in the target area based on the operation feedback information.

4. The method of claim 3, wherein, the receiving of the operation feedback information by the target robot comprises: The target robot receives the operation feedback information corresponding to each operation stage respectively, and stores the operation feedback information in a target data storage area.

5. The method of claim 4, wherein, The operation of placing the target object in the target area based on the operation feedback information comprises: When the current operation stage is the operation of placing the target object in the target area, and it is determined based on the state information of the target operation object in the received operation feedback information that the current operation stage contains the target operation object, the target robot reads the position offset information in the target data storage area, and performs the operation of placing the target object in the target area based on the position offset information.

6. A control system applied to a robot, applied to the control method of the robot as claimed in any one of claims 1 to 5, characterized in that, Comprise: An operation stage determination module is configured to determine at least one operation stage in the process of placing a target object in a target area by a target robot; A request sending module is configured to send an image acquisition request of a current operation stage to a vision device through the target robot for at least one operation stage; A request receiving module is configured to receive the image acquisition request of the current operation stage through the vision device, and acquire an operation image corresponding to the current operation stage based on the image acquisition request; An image processing module is configured to process the current operation image through the vision device to obtain operation feedback information corresponding to the current operation stage, and send the operation feedback information to the target robot.

7. An electronic device, comprising: The electronic device comprises: One or more processors; A storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the control method for robots as claimed in any one of claims 1-5.

8. A storage medium containing computer-executable instructions, characterized in that, The computer executable instructions, when executed by a computer processor, are used to perform the control method for robots as claimed in any one of claims 1-5.

Images