Intelligent robot control method, device, equipment and storage medium

Abstract

This invention relates to the field of intelligent control technology, and discloses an intelligent robot control method, device, equipment, and storage medium. The method disclosed in this invention includes acquiring work information; controlling the intelligent robot to move to the work location; acquiring a field of view by using the intelligent robot's 2D camera; capturing a calibration object in the field of view that matches the calibration object's feature information; acquiring the corner coordinate parameters of the calibration object; and determining the target pose matrix of the intelligent robot's robotic arm based on the corner coordinate parameters. The method provided by this invention can acquire the positional relationship between the intelligent robot and the 2D camera using a 2D camera, and obtain the target pose matrix based on this positional relationship. The robotic arm is then adjusted to the target operating pose using the target pose matrix, ensuring the accuracy of the intelligent robot's operation and reducing the cost of the intelligent robot.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to an intelligent robot control method, device, equipment and storage medium. Background Art

[0002] At present, the intelligent robots used in the power industry go to the work site to perform work under the guidance of cameras. However, due to positioning errors or bumps and collisions during the journey, the position conditions of the intelligent robots when they arrive at the work site may deviate, causing the intelligent robots to not start the work in a suitable position, resulting in the inability to accurately complete the work objectives. Summary of the invention

[0003] Embodiments of the present invention provide an intelligent robot control method, device, equipment and storage medium, which are used to solve the problem of lack of a method for obtaining the positional relationship between the intelligent robot and the 2D camera based on the 2D camera and guiding the mechanical arm of the intelligent robot to achieve the optimal operating posture for the power equipment.

[0004] In a first aspect, an embodiment of the present invention provides a method for controlling an intelligent robot, including: obtaining operation information, the operation information including the operation location of the intelligent robot and characteristic information of a calibration object; controlling the intelligent robot to move to the operation location; acquiring a field of view picture by performing image acquisition through a 2D camera of the intelligent robot; capturing a calibration object that meets the characteristic information of the calibration object in the field of view picture according to the characteristic information of the calibration object; obtaining corner point coordinate parameters of the calibration object; and determining a target pose matrix of a robotic arm of the intelligent robot according to the corner point coordinate parameters, the target pose matrix being used to adjust the robotic arm to a target pose.

[0005] In combination with the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where capturing a calibration object that meets the calibration object feature information in a field of view according to the calibration object feature information includes:

[0006] Control the intelligent robot to enter the search mode; in the search mode, if there is a calibration object in the field of view that completely meets the characteristic information of the calibration object, the search mode is terminated; if the calibration object that completely meets the characteristic information of the calibration object is still not captured in the field of view after the search mode is repeated for a preset number of rounds, the intelligent robot is controlled to return and the intelligent robot is controlled to move to the work location again according to the work information until a calibration object that completely meets the characteristic information of the calibration object is captured in the field of view.

[0007] In combination with the first aspect, the embodiment of the present invention provides a second possible implementation of the first aspect, where the preset rounds are three rounds.

[0008] In combination with the first aspect, an embodiment of the present invention provides a third possible implementation of the first aspect, where if a calibration object that completely meets the characteristic information of the calibration object is not captured in the field of view after the repeated search mode reaches a preset number of rounds, a search failure signal is fed back.

[0009] In combination with the first aspect, an embodiment of the present invention provides a fourth possible implementation of the first aspect, wherein a 2D camera is disposed on a robotic arm, and a calibration object that meets the characteristic information of the calibration object is captured in a field of view according to the characteristic information of the calibration object, including:

[0010] In each round of search mode, the calibration object is searched from the current field of view. If the calibration object that meets the characteristic information of the calibration object is not captured in the field of view, a first field of view adjustment signal is generated. According to the first field of view adjustment signal, the robotic arm is controlled to move a preset distance in at least any direction of up, down, left, and right with the center point of the current field of view as a reference and then continue to search for the calibration object.

[0011] In combination with the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where capturing a calibration object that meets the calibration object feature information in a field of view according to the calibration object feature information includes:

[0012] In each round of search mode, the calibration object is searched from the current field of view. If a calibration object that partially meets the characteristic information of the calibration object is captured in the field of view, the positional relationship between the position of the calibration object in the current field of view and the center point of the field of view is obtained; a second field of view adjustment signal is obtained according to the positional relationship; and the robot arm is controlled to adjust its posture according to the second field of view adjustment signal so that a calibration object that completely meets the characteristic information of the calibration object exists in the field of view.

[0013] In combination with the first aspect, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein the calibration object is an identifier including four corner points, and obtaining the corner point coordinate parameters of the calibration object includes: obtaining the two-dimensional position coordinates of the four corner points of the calibration object through the current field of view.

[0014] In combination with the first aspect, an embodiment of the present invention provides a seventh possible implementation of the first aspect, wherein a target pose matrix of a manipulator arm of an intelligent robot is determined according to corner point coordinate parameters, and the target pose matrix is ​​used to adjust the manipulator arm to a target pose, including:

[0015] According to the corner point coordinate parameters, the current position relationship matrix between the 2D camera of the intelligent robot and the calibration object in the current posture of the manipulator is obtained; and the target posture matrix is ​​determined according to the current position relationship matrix.

[0016] In combination with the first aspect, an embodiment of the present invention provides an eighth possible implementation of the first aspect, where determining a target pose matrix according to a current position relationship matrix includes:

[0017] The target pose matrix is ​​calculated based on the current position relationship matrix, the hand-eye matrix, the current pose matrix, and the preset position relationship matrix, wherein the hand-eye matrix is ​​a matrix of the relationship between the end coordinate system of the robot arm and the coordinate system of the 2D camera, the current pose matrix is ​​a matrix of the relationship between the end coordinate system of the robot arm and the base coordinate system of the robot arm in the current pose, and the preset position relationship matrix is ​​a matrix of the relationship between the 2D camera of the trained preset intelligent robot and the calibration object.

[0018] In combination with the first aspect, an embodiment of the present invention provides a ninth possible implementation of the first aspect, wherein the target posture matrix is ​​calculated according to the current position relationship matrix, the hand-eye matrix, the current posture matrix, and the preset position relationship matrix, including: the target posture matrix is ​​calculated according to a preset equation, and the preset equation is:

[0019] { * * } = { * * },in, is the spatial position relationship matrix between the 2D camera and the calibration object; is the hand-eye matrix, which is the spatial relationship between the end coordinate system of the robot and the coordinate system of the 2D camera. The hand-eye matrix is ​​a constant; is the current pose matrix of the robot arm, which is obtained by obtaining the pose information of the end coordinate system of the robot arm in the base coordinate system of the robot arm; is a preset position relationship matrix; is the target pose matrix.

[0020] In combination with the first aspect, an embodiment of the present invention provides a tenth possible implementation of the first aspect, and the intelligent robot control method also includes: judging whether the robot arm has the operating conditions according to the current position relationship matrix.

[0021] In combination with the first aspect, an embodiment of the present invention provides an eleventh possible implementation of the first aspect, and the intelligent robot control method also includes: generating motion guidance parameters according to the target posture matrix; controlling the intelligent robot to adjust the robotic arm to the target posture according to the motion guidance parameters.

[0022] In a second aspect, an embodiment of the present invention provides an intelligent robot control device, including: a first acquisition module, a control module, a collection module, an identification module, a second acquisition module and a determination module.

[0023] The first acquisition module is used to acquire operation information, which includes the operation location of the intelligent robot and the characteristic information of the calibration object; the control module is used to control the intelligent robot to move to the operation location; the acquisition module is used to acquire images through the 2D camera of the intelligent robot to obtain a field of view; the recognition module is used to capture the calibration object that meets the characteristic information of the calibration object in the field of view according to the characteristic information of the calibration object; the second acquisition module is used to acquire the corner point coordinate parameters of the calibration object; the determination module is used to determine the target posture matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target posture matrix is ​​used to adjust the robotic arm to the target posture.

[0024] In a third aspect, an embodiment of the present invention provides an intelligent robot control device, comprising: a memory and a processor, wherein instructions are stored in the memory; the processor calls the instructions in the memory so that the processor executes the intelligent robot control method of any of the aforementioned embodiments of the first aspect of the present invention.

[0025] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed by a processor, the intelligent robot control method of any of the aforementioned embodiments of the first aspect of the present invention is implemented.

[0026] The method provided by an embodiment of the present invention includes obtaining operation information, the operation information includes the operation location of the intelligent robot and the characteristic information of the calibration object; controlling the intelligent robot to move to the operation location; acquiring a field of view picture by performing image acquisition through the 2D camera of the intelligent robot; capturing a calibration object that meets the characteristic information of the calibration object in the field of view picture according to the characteristic information of the calibration object; obtaining the corner point coordinate parameters of the calibration object; determining the target posture matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target posture matrix is ​​used to adjust the robotic arm to the target posture.

[0027] The method provided in the embodiment of the present invention can obtain the positional relationship between the intelligent robot and the 2D camera through a 2D camera, and obtain the target posture matrix based on the positional relationship. The robotic arm is adjusted to the target operating posture through the target posture matrix, thereby ensuring the accuracy of the operation of the intelligent robot. At the same time, since the method provided in the embodiment of the present invention uses a 2D camera to realize the calculation of the target posture matrix and the adjustment of the posture of the intelligent robot, the cost of the intelligent robot is reduced.

[0028] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0029] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flow chart of an embodiment of the intelligent robot control method of the present invention;

[0031] Figure 2 A flowchart of the steps of capturing a calibration object that meets the characteristic information of the calibration object in a field of view according to the characteristic information of the calibration object in an embodiment of the intelligent robot control method of the present invention;

[0032] Figure 3 A flowchart of the steps of determining the target pose matrix of the mechanical arm of the intelligent robot according to the corner point coordinate parameters in one embodiment of the intelligent robot control method of the present invention;

[0033] Figure 4 This is a flow chart after the step of determining the target pose matrix of the mechanical arm of the intelligent robot according to the corner point coordinate parameters in one embodiment of the intelligent robot control method of the present invention;

[0034] Figure 5 A structural block diagram of an embodiment of the intelligent robot control device of the present invention;

[0035] Figure 6 The figure is a structural block diagram of an embodiment of the intelligent robot control device of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0037] Embodiments of the present invention provide an intelligent robot control method, device, equipment and storage medium, which are used to solve the problem of lack of a method for obtaining the positional relationship between the intelligent robot and the 2D camera based on the 2D camera and guiding the mechanical arm of the intelligent robot to achieve the optimal operating posture for the power equipment.

[0038] For ease of understanding, the intelligent robot control method of the embodiment of the present invention is described below. Figure 1 As shown, the intelligent robot control method in the embodiment of the present invention includes steps S110 to S160.

[0039] In step S110, operation information is obtained, where the operation information includes the operation location of the intelligent robot and characteristic information of the calibration object.

[0040] In step S120, the intelligent robot is controlled to move to the working location.

[0041] In step S130, the 2D camera of the intelligent robot is used to collect images to obtain a field of view.

[0042] In step S140 , a calibration object that meets the calibration object feature information is captured in the field of view according to the calibration object feature information.

[0043] In some optional embodiments, such as Figure 2 As shown, the step of capturing the calibration object that meets the calibration object characteristic information in the field of view according to the calibration object characteristic information includes steps S141 to S143.

[0044] In step S141, the intelligent robot is controlled to enter a search mode.

[0045] In step S142 , in the search mode, if there is a calibration object in the field of view that completely matches the characteristic information of the calibration object, the search mode is terminated.

[0046] In step S143, if the calibration object that completely meets the characteristic information of the calibration object is not captured in the field of view after the repeated search mode reaches the preset rounds, the intelligent robot is controlled to return, and the intelligent robot is controlled to move to the work location again according to the work task until the calibration object that completely meets the characteristic information of the calibration object is captured in the field of view.

[0047] In some optional embodiments, the preset number of rounds is three rounds.

[0048] In some optional embodiments, if the calibration object that completely meets the characteristic information of the calibration object is still not captured in the field of view after the search mode is repeated for a preset number of times, a search failure signal is fed back.

[0049] In some optional embodiments, the 2D camera is disposed on a robotic arm, and the step of capturing a calibration object that meets the characteristic information of the calibration object in a field of view according to the characteristic information of the calibration object includes: searching for the calibration object from the current field of view in each round of search mode, and generating a first field of view adjustment signal if the calibration object that meets the characteristic information of the calibration object is not captured in the field of view; and controlling the robotic arm according to the first field of view adjustment signal to move a preset distance in at least any direction of up, down, left, and right with the center point of the current field of view as a reference and continue to search for the calibration object.

[0050] In some optional embodiments, the step of capturing a calibration object that meets the characteristic information of the calibration object in the field of view according to the characteristic information of the calibration object includes: searching for the calibration object from the current field of view in each round of search mode, and if a calibration object that partially meets the characteristic information of the calibration object is captured in the field of view, obtaining the positional relationship between the position of the calibration object in the current field of view and the center point of the field of view, obtaining a second field of view adjustment signal according to the positional relationship, and controlling the robot arm to adjust the posture according to the second field of view adjustment signal, so that a calibration object that completely meets the characteristic information of the calibration object exists in the field of view.

[0051] In step S150, the coordinate parameters of the corner points of the calibration object are obtained.

[0052] In some optional embodiments, the calibration object is a marker including four corner points, and the step of obtaining the corner point coordinate parameters of the calibration object includes: obtaining the two-dimensional position coordinates of the four corner points of the calibration object through the current field of view.

[0053] In step S160, a target pose matrix of the robotic arm of the intelligent robot is determined according to the corner point coordinate parameters, and the target pose matrix is ​​used to adjust the robotic arm to the target pose.

[0054] In some optional embodiments, such as Figure 3 As shown, the target posture matrix of the mechanical arm of the intelligent robot is determined according to the corner point coordinate parameters, and the step of using the target posture matrix to adjust the mechanical arm to the target posture includes steps S161 to S162.

[0055] In step S161, the current position relationship matrix between the 2D camera of the intelligent robot and the calibration object in the current posture of the robotic arm is obtained according to the corner point coordinate parameters.

[0056] In step S162, the target posture matrix is ​​determined according to the current position relationship matrix.

[0057] In some optional embodiments, the step of determining the target pose matrix based on the current position relationship matrix includes: calculating the target pose matrix based on the current position relationship matrix, the hand-eye matrix, the current pose matrix, and the preset position relationship matrix, wherein the hand-eye matrix is ​​a matrix of the relationship between the end coordinate system of the robot arm and the coordinate system of the 2D camera, the current pose matrix is ​​a matrix of the relationship between the end coordinate system of the robot arm and the base coordinate system of the robot arm in the current pose, and the preset position relationship matrix is ​​a matrix of the relationship between the 2D camera of a trained preset intelligent robot and the calibration object.

[0058] In some optional embodiments, the step of calculating the target posture matrix according to the current position relationship matrix, the hand-eye matrix, the current posture matrix, and the preset position relationship matrix includes: calculating the target posture matrix according to a preset equation, and the preset equation is:

[0059] { * * } = { * * },in, is the spatial position relationship matrix between the 2D camera and the calibration object; is the hand-eye matrix, which is the spatial relationship between the end coordinate system of the robot and the coordinate system of the 2D camera. The hand-eye matrix is ​​a constant; is the current pose matrix of the robot arm, which is obtained by obtaining the pose information of the end coordinate system of the robot arm in the base coordinate system of the robot arm; is a preset position relationship matrix; is the target pose matrix, and the preset position relationship matrix is ​​the spatial position relationship matrix between the 2D camera and the calibration object that has been set in the previous training.

[0060] In some optional embodiments, the above-mentioned intelligent robot control method also includes judging whether the robot arm has the operating conditions according to the current position relationship matrix.

[0061] In some optional embodiments, such as Figure 4 As shown, after step S160, the above intelligent robot control method also includes step S170 and step S180.

[0062] In step S170, motion guidance parameters are generated according to the target pose matrix.

[0063] In step S180, the intelligent robot is controlled to adjust the robotic arm to the target posture according to the motion guidance parameters.

[0064] The intelligent robot control method provided by an embodiment of the present invention includes: obtaining operation information, the operation information includes the operation location of the intelligent robot and the characteristic information of the calibration object; controlling the intelligent robot to move to the operation location; acquiring a field of view picture by performing image acquisition through the 2D camera of the intelligent robot; capturing a calibration object that meets the characteristic information of the calibration object in the field of view picture according to the characteristic information of the calibration object; obtaining the corner point coordinate parameters of the calibration object; determining the target posture matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target posture matrix is ​​used to adjust the robotic arm to the target posture.

[0065] The method provided in the embodiment of the present invention can obtain the positional relationship between the intelligent robot and the 2D camera through a 2D camera, and obtain the target posture matrix based on the positional relationship. The robotic arm is adjusted to the target operating posture through the target posture matrix, thereby ensuring the accuracy of the operation of the intelligent robot. At the same time, since the method provided in the embodiment of the present invention uses a 2D camera to realize the calculation of the target posture matrix and the adjustment of the posture of the intelligent robot, the cost of the intelligent robot is reduced.

[0066] For the above method embodiment, the embodiment of the present invention also provides Figure 5 The intelligent robot control device shown includes: a first acquisition module 301, a control module 302, a collection module 303, a recognition module 304, a second acquisition module 305 and a determination module 306.

[0067] The first acquisition module 301 is used to acquire operation information, which includes the operation location of the intelligent robot and the characteristic information of the calibration object; the control module 302 is used to control the intelligent robot to move to the operation location; the acquisition module 303 is used to acquire images through the 2D camera of the intelligent robot to obtain a field of view; the recognition module 304 is used to capture the calibration object that meets the characteristic information of the calibration object in the field of view according to the characteristic information of the calibration object; the second acquisition module 305 is used to acquire the corner point coordinate parameters of the calibration object; the determination module 306 is used to determine the target pose matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target pose matrix is ​​used to adjust the robotic arm to the target pose.

[0068] The intelligent robot control device provided by the embodiment of the present invention includes: a first acquisition module 301, used to acquire operation information, the operation information includes the operation location of the intelligent robot and the characteristic information of the calibration object; a control module 302, used to control the intelligent robot to move to the operation location; an acquisition module 303, used to acquire images through the 2D camera of the intelligent robot to obtain a field of view picture; an identification module 304, used to capture a calibration object that meets the characteristic information of the calibration object in the field of view picture according to the characteristic information of the calibration object; a second acquisition module 305, used to acquire the corner point coordinate parameters of the calibration object; a determination module 306, used to determine the target pose matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target pose matrix is ​​used to adjust the robotic arm to the target pose.

[0069] The intelligent robot control device provided by the embodiment of the present invention can obtain the positional relationship between the intelligent robot and the 2D camera based on the 2D camera by implementing the above method, and obtain the target posture matrix according to the positional relationship, and adjust the robotic arm to the target operation posture through the target posture matrix, thereby ensuring the accuracy of the intelligent robot operation. At the same time, since the intelligent robot control device provided by the embodiment of the present invention uses a 2D camera to realize the calculation of the target posture matrix and the adjustment of the posture of the intelligent robot, the cost of the intelligent robot is reduced.

[0070] For the above method embodiment, the embodiment of the present invention also provides Figure 6 An intelligent robot control device shown includes: a processor 401 and a memory 402, in which instructions are stored; the processor 401 calls the instructions in the memory 402 to implement the above-mentioned intelligent robot control method.

[0071] The intelligent robot control device provided by the embodiment of the present invention includes a memory 402 and a processor 401, the memory 402 stores instructions, and the processor 401 calls the instructions in the memory 402 to implement the above-mentioned intelligent robot control method, and the above-mentioned intelligent robot control method includes: obtaining operation information, the operation information includes the operation location of the intelligent robot and the characteristic information of the calibration object; controlling the intelligent robot to move to the operation location; acquiring a field of view picture by performing image acquisition through the 2D camera of the intelligent robot; capturing a calibration object that meets the characteristic information of the calibration object in the field of view picture according to the characteristic information of the calibration object; obtaining the corner point coordinate parameters of the calibration object; determining the target posture matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target posture matrix is ​​used to adjust the robotic arm to the target posture.

[0072] The intelligent robot control device provided by the embodiment of the present invention can obtain the positional relationship between the intelligent robot and the 2D camera based on the 2D camera by implementing the above method, and obtain the target posture matrix according to the positional relationship. The robotic arm is adjusted to the target operating posture through the target posture matrix, thereby ensuring the accuracy of the intelligent robot's operation and reducing the cost of the intelligent robot.

[0073] Furthermore, the intelligent robot control device provided in the embodiment of the present invention may also include a communication interface 403 and a bus 404 , and the processor 401 , the memory 402 and the communication interface 403 are electrically connected via the bus 404 .

[0074] The memory 402 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 403 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 404 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0075] The processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 401. The above processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 402, and the processor 401 reads the information in the memory 402 and completes the steps of the method of the above embodiment in combination with its hardware.

[0076] An embodiment of the present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned intelligent robot control method.

[0077] The computer-readable storage medium provided by an embodiment of the present invention stores data and computer executable instructions of the above-mentioned intelligent robot control method, and the above-mentioned intelligent robot control method includes obtaining operation information, the operation information includes the operation location of the intelligent robot and calibration object feature information; controlling the intelligent robot to move to the operation location; acquiring a field of view picture by performing image acquisition through the 2D camera of the intelligent robot; capturing a calibration object that meets the calibration object feature information in the field of view picture according to the calibration object feature information; obtaining the corner point coordinate parameters of the calibration object; and determining the target pose matrix of the robotic arm of the intelligent robot according to the corner point coordinate parameters, and the target pose matrix is ​​used to adjust the robotic arm to the target pose.

[0078] The computer-readable storage medium provided by an embodiment of the present invention can obtain the positional relationship between the intelligent robot and the 2D camera based on the 2D camera by implementing the above method, and obtain the target posture matrix according to the positional relationship. The robotic arm is adjusted to the target operating posture through the target posture matrix, thereby ensuring the accuracy of the intelligent robot's operation and reducing the cost of the intelligent robot.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0080] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0081] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0082] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling an intelligent robot, characterized in that: The method comprises: Acquire operation information, wherein the operation information includes the operation location of the intelligent robot and characteristic information of the calibration object; Controlling the intelligent robot to move to the operation location; Acquire a visual field by using a 2D camera of the intelligent robot; According to the characteristic information of the calibration object, a calibration object that meets the characteristic information of the calibration object is captured in the field of view; Obtaining the coordinate parameters of the corner points of the calibration object; According to the corner point coordinate parameters, a current position relationship matrix between the 2D camera of the intelligent robot and the calibration object is obtained at the current posture of the mechanical arm of the intelligent robot, and a target posture matrix of the mechanical arm of the intelligent robot is determined according to the current position relationship matrix, and the target posture matrix is ​​used to adjust the mechanical arm to the target posture, Wherein, capturing the calibration object that meets the calibration object characteristic information in the field of view according to the calibration object characteristic information comprises: Controlling the intelligent robot to enter a search mode; In the search mode, if the calibration object that completely matches the characteristic information of the calibration object exists in the field of view, the search mode is terminated; If the calibration object that completely matches the characteristic information of the calibration object is still not captured in the field of view after repeating the search mode for a preset number of rounds, the intelligent robot is controlled to return and the intelligent robot is controlled to move to the work location again according to the work information until the calibration object that completely matches the characteristic information of the calibration object is captured in the field of view; In each round of the search mode, the calibration object is searched from the current field of view, and if a portion of the calibration object that meets the characteristic information of the calibration object is captured in the field of view, the positional relationship between the position of the calibration object in the current field of view and the center point of the field of view is obtained; Obtaining a second field of view adjustment signal according to the positional relationship; The robot arm is controlled to adjust its posture according to the second field of view adjustment signal, so that the calibration object that completely conforms to the characteristic information of the calibration object exists in the field of view picture.

2. The method according to claim 1, characterized in that The preset rounds are three rounds.

3. The method according to claim 1, characterized in that If the calibration object that completely matches the characteristic information of the calibration object is still not captured in the field of view after the search mode is repeated for a preset number of times, a search failure signal is fed back.

4. The method according to claim 1, characterized in that: The 2D camera is disposed on the robotic arm, and capturing the calibration object that meets the calibration object feature information in the field of view according to the calibration object feature information includes: In each round of the search mode, the calibration object is searched from the current field of view picture, and if the calibration object that meets the characteristic information of the calibration object is not captured in the field of view picture, a first field of view adjustment signal is generated; According to the first field of view adjustment signal, the robot arm is controlled to move a preset distance in at least any one direction of up, down, left, and right based on the center point of the current field of view, and then continue to search for the calibration object.

5. The method according to claim 1, characterized in that: The calibration object is a mark including four corner points, and obtaining the corner point coordinate parameters of the calibration object includes: The two-dimensional position coordinates of the four corner points of the calibration object are obtained through the current field of view.

6. The method according to claim 1, characterized in that Determining the target posture matrix according to the current position relationship matrix includes: The target posture matrix is ​​calculated based on the current position relationship matrix, the hand-eye matrix, the current posture matrix, and the preset position relationship matrix. Among them, the hand-eye matrix is ​​a matrix of the relationship between the end coordinate system of the robotic arm and the coordinate system of the 2D camera, the current pose matrix is ​​a matrix of the relationship between the end coordinate system of the robotic arm and the base coordinate system of the robotic arm in the current pose, and the preset position relationship matrix is ​​a matrix of the relationship between the trained preset 2D camera of the intelligent robot and the calibration object.

7. The method according to claim 6, characterized in that The target posture matrix is ​​calculated based on the current position relationship matrix, the hand-eye matrix, the current posture matrix, and the preset position relationship matrix, including: the target posture matrix is ​​calculated based on a preset equation, and the preset equation is: { * * } = { * * }， Among them, the is the spatial position relationship matrix between the 2D camera and the calibration object; Said is the hand-eye matrix, the hand-eye matrix is ​​the spatial relationship between the end coordinate system of the robot arm and the coordinate system of the 2D camera, and the hand-eye matrix is ​​a constant; Said is the current pose matrix of the robot arm, and the current pose matrix is ​​obtained by acquiring the pose information of the end coordinate system of the robot arm in the base coordinate system of the robot arm; Said is the preset position relationship matrix; Said is the target pose matrix.

8. The method according to claim 1, characterized in that: The intelligent robot control method also includes: It is determined whether the robot arm has the operating conditions according to the current position relationship matrix.

9. The method according to claim 1, characterized in that: The intelligent robot control method also includes: generating motion guidance parameters according to the target posture matrix; The intelligent robot is controlled to adjust the mechanical arm to the target posture according to the motion guidance parameters.

10. An intelligent robot control device, characterized in that: The device comprises: A first acquisition module is used to acquire operation information, wherein the operation information includes the operation location of the intelligent robot and characteristic information of the calibration object; A control module, used for controlling the intelligent robot to move to the operation site; An acquisition module, used for acquiring images through a 2D camera of the intelligent robot to obtain a visual field picture; An identification module, configured to capture a calibration object that meets the calibration object characteristic information in the field of view according to the calibration object characteristic information; A second acquisition module is used to acquire the coordinate parameters of the corner points of the calibration object; and A determination module is used to determine a target pose matrix of the mechanical arm of the intelligent robot according to the corner point coordinate parameters, and the target pose matrix is ​​used to adjust the mechanical arm to the target pose. The determination module is configured to: obtain a current position relationship matrix between the 2D camera of the intelligent robot and the calibration object in the current pose of the mechanical arm of the intelligent robot according to the corner point coordinate parameters, and determine a target pose matrix of the mechanical arm of the intelligent robot according to the current position relationship matrix. The identification module is configured to: control the intelligent robot to enter a search mode; in the search mode, if the calibration object that completely meets the characteristic information of the calibration object exists in the field of view, then end the search mode; if the calibration object that completely meets the characteristic information of the calibration object is still not captured in the field of view after repeating the search mode for a preset number of rounds, then control the intelligent robot to return and control the intelligent robot to move to the work location again according to the operation information until the calibration object that completely meets the characteristic information of the calibration object is captured in the field of view; in each round of the search mode, search for the calibration object from the current field of view, if the calibration object that partially meets the characteristic information of the calibration object is captured in the field of view, then obtain the positional relationship between the position of the calibration object in the current field of view and the center point of the field of view; obtain a second field of view adjustment signal according to the positional relationship; and control the robot arm to adjust its posture according to the second field of view adjustment signal so that the calibration object that completely meets the characteristic information of the calibration object exists in the field of view.

11. An intelligent robot control device, characterized in that: The device comprises: a memory and a processor, wherein instructions are stored in the memory; and the processor calls the instructions in the memory so that the processor executes the intelligent robot control method according to any one of claims 1 to 9.

12. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the intelligent robot control method according to any one of claims 1 to 9 is implemented.

Images