Mechanical arm pose control method and device, electronic equipment and storage medium

Abstract

The invention provides a mechanical arm pose control method and device, electronic equipment and a storage medium, the mechanical arm pose control method is applied to the electronic equipment, the electronic equipment is in communication connection with a mechanical arm, the mechanical arm is provided with a camera device, and the camera device is in communication connection with the electronic equipment. The method is used for shooting a target object arranged in a target area, and comprises the following steps: controlling a camera device to shoot the target object to obtain target point cloud data; a first difference between the target point cloud data and pre-stored template point cloud data is determined, and the pre-stored template point cloud data is the point cloud data obtained when the camera device shoots the target object when the mechanical arm is in the target pose; and the mechanical arm is controlled to be adjusted to the target pose according to the first difference. According to the method, the accuracy of mechanical arm pose control can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent control technology, and in particular to a method, device, electronic device and storage medium for controlling the posture of a robotic arm. Background Art

[0002] With the continuous progress of the manufacturing industry, more and more manufacturers tend to use automated production lines and robotic arms to cooperate with each other to improve production efficiency. At present, automated production lines usually use robotic arms to replace manual work for single, complex, and high-risk operations, and the robotic arms require workers to manually configure points in advance before they can autonomously operate to a preset position. This way of controlling the robotic arm is inefficient and the accuracy of controlling the robotic arm to operate to a preset position is low. Summary of the invention

[0003] In view of the above, it is necessary to propose a method, device, electronic device and storage medium for controlling the posture of a robotic arm to solve the technical problem of low accuracy when controlling the posture of a robotic arm.

[0004] The present application provides a robot arm posture control method, which is applied to an electronic device, wherein the electronic device is communicatively connected to the robot arm, and a camera device is installed on the robot arm, and the camera device is communicatively connected to the electronic device for photographing a target object set in a target area. The method includes: controlling the camera device to photograph the target object to obtain target point cloud data; determining a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robot arm is in a target posture; and controlling the robot arm to adjust to the target posture according to the first difference.

[0005] In some embodiments, the method for obtaining the pre-stored template point cloud data includes: receiving a first point position input by a user; controlling the robotic arm to move to a target position according to the first point position; controlling the camera device to photograph the target object in the target area to obtain the pre-stored template point cloud data.

[0006] In some embodiments, determining the first difference between the target point cloud data and the pre-stored template point cloud data includes: determining the first coordinate of each first point in the target point cloud data in the camera coordinate system and the second coordinate of each second point in the template point cloud data in the camera coordinate system; matching all the second points according to the first coordinate of each first point to obtain the second coordinate of the second point closest to the first coordinate of each first point, and determining multiple point pairs according to the first coordinate of each first point and the second coordinate of the matched second point; calculating the difference between the first coordinate of the first point and the second coordinate of the second point in each point pair to obtain the first difference.

[0007] In some embodiments, controlling the robotic arm to adjust to the target posture according to the first difference includes: mapping the first difference from the camera coordinate system to the world coordinate system to obtain a second difference; determining a second point position according to the first point position and the second difference; and controlling the robotic arm to adjust to the target posture according to the second point position.

[0008] In some embodiments, the method further includes: when the first difference is greater than a preset threshold, continuing to adjust the posture of the robotic arm.

[0009] In some embodiments, continuing to adjust the posture of the robotic arm includes: continuing to control the camera device to photograph the target object to obtain the target point cloud data; determining the first difference between the target point cloud data and the pre-stored template point cloud data, and continuing to adjust the posture of the robotic arm according to the first difference until the first difference is less than or equal to the preset threshold.

[0010] In some embodiments, when the electronic device is communicatively connected to multiple robotic arms, the method further includes: sending the first point position to the multiple robotic arms, controlling the camera device installed on each of the robotic arms to shoot the target object, and obtaining target point cloud data corresponding to each of the robotic arms; determining a first difference between the target point cloud data and the pre-stored template point cloud data; and controlling each of the robotic arms to adjust to the target posture according to the first difference.

[0011] An embodiment of the present application also provides a robot arm posture control device, which includes: a control module, used to control the camera device to photograph the target object and obtain target point cloud data; a determination module, used to determine a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robot arm is in the target posture; the control module is also used to control the robot arm to adjust to the target posture according to the first difference.

[0012] An embodiment of the present application also provides an electronic device, comprising: a memory storing at least one instruction; and a processor executing the instruction stored in the memory to implement the robot arm posture control method.

[0013] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the robotic arm posture control method.

[0014] It can be seen from the above technical solutions that the embodiment of the present application can use the camera device on the robot arm to shoot the target object when the robot arm is in any posture, obtain the target point cloud data, and judge the first difference between the current posture of the robot arm and the target posture according to the target point cloud data and the pre-stored template point cloud data, and adjust the robot arm to the target posture according to the first difference. In this way, adjusting the posture of the robot arm according to the principle of computer vision can improve the accuracy of controlling the posture of the robot arm, and there is no need to manually adjust the posture of the robot arm, thereby improving the efficiency of adjusting the posture of the robot arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is an application scenario diagram of a robot arm posture control method provided in one embodiment of the present application.

[0016] Figure 2 It is a flow chart of a robot arm posture control method provided in one embodiment of the present application.

[0017] Figure 3 This is a flowchart of a method for obtaining template point cloud data provided by an embodiment of the present application.

[0018] Figure 4 It is a flowchart of a method for determining a first difference provided by an embodiment of the present application.

[0019] Figure 5 This is a flow chart of a method for controlling a robotic arm to adjust to a target posture provided by an embodiment of the present application.

[0020] Figure 6 This is a flowchart of a method for continuously controlling a robotic arm to a target position provided by an embodiment of the present application.

[0021] Figure 7 This is a flowchart of a method for controlling multiple robotic arms to move to a target position provided by an embodiment of the present application.

[0022] Figure 8 It is a functional module diagram of a robotic arm posture control device provided in one embodiment of the present application.

[0023] Fig. 9It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.

[0025] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0027] An embodiment of the present application provides a method for controlling the posture of a robotic arm, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0028] An electronic device can be any electronic product that can interact with a customer, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0029] The electronic device may also include a network device, which includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0030] The network where the electronic device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0031] like Figure 1 As shown, the robot arm posture control method provided in the present application can be applied to an electronic device 100, and the electronic device 100 is communicatively connected with a robot arm 200. The electronic device 100 is used to receive the point information input by the user, and control the robot arm 200 to run to the posture corresponding to the point information according to the point information, so that the robot arm can perform a preset operation at the posture (for example, grabbing a fixture located in a preset area). Among them, a camera device 300 is installed on the robot arm 200, and the camera device 300 is used to shoot a target object 400 located in a preset area (for example, products and fixtures in a production line, etc.), obtain point cloud data of the target object 400, and adjust the posture of the robot arm 200 according to the point cloud data and the pre-stored template point cloud data, so that the posture of the robot arm is more accurate.

[0032] In one embodiment of the present application, the electronic device 100 may be an external electronic device that is communicatively connected to the robotic arm 200, or may be a central control device of the robotic arm itself, which is not limited in the present application.

[0033] like Figure 2 , is a flow chart of a method for controlling the position and posture of a robotic arm provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for controlling the position and posture of a robotic arm provided by an embodiment of the present application includes the following steps.

[0034] S20, controlling the camera device to photograph the target object to obtain target point cloud data.

[0035] In one embodiment of the present application, when the robotic arm is in any posture, the electronic device sends a shooting instruction to the camera device to control the camera device to shoot the target object located in the target area, and obtains the target point cloud data corresponding to the target object.

[0036] In one embodiment of the present application, the target point cloud data includes a plurality of first points, each of which corresponds to a first coordinate. The first coordinate may be a three-dimensional coordinate, which is used to represent the position of the first point in the camera coordinate system corresponding to the camera device. For example, the coordinates of a first point may be represented as (x1, y1, z1).

[0037] S21, determining a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robotic arm is in the target posture.

[0038] In one embodiment of the present application, since the target point cloud data is a set of first points in the camera coordinate system, the target point cloud data can represent the positional relationship between the camera device and the target object. In order to adjust the position and posture of the robot arm, the first difference between the target point cloud data and the pre-stored template point cloud data can be determined, and the position and posture of the robot arm can be adjusted according to the first difference.

[0039] In one embodiment of the present application, the pre-stored template point cloud data is the point cloud data obtained by the camera device photographing the target object when the robot arm is in the target posture. The point cloud data can characterize the positional relationship between the camera device and the target object when the robot arm is in the target posture. When the first difference between the target point cloud data and the pre-stored template point cloud data is smaller, it indicates that the posture of the robot arm is closer to the target posture.

[0040] For more information on how to obtain pre-stored template point cloud data, see Figure 3 Corresponding instructions; for determining the first difference, see Figure 4 Corresponding description.

[0041] S22: Control the robotic arm to adjust to the target posture according to the first difference.

[0042] In one embodiment of the present application, the first difference is used to characterize the difference between the posture of the robot arm when obtaining the target point cloud and the target posture. Therefore, the posture of the robot arm can be compensated and calculated based on the first difference, so that the electronic device controls the robot arm to adjust to the target posture, ensuring that the robot arm can complete the preset operation at the target posture. For example, the preset operation can be to grab the target object in the target area. For specific methods of controlling the robot arm to adjust to the target posture, please refer to Figure 5 Corresponding description.

[0043] It can be seen from the above technical solutions that the embodiment of the present application can use the camera device on the robot arm to shoot the target object when the robot arm is in any posture, obtain the target point cloud data, and judge the first difference between the current posture of the robot arm and the target posture according to the target point cloud data and the pre-stored template point cloud data, and adjust the robot arm to the target posture according to the first difference. In this way, adjusting the posture of the robot arm according to the principle of computer vision can improve the accuracy of controlling the posture of the robot arm, and there is no need to manually adjust the posture of the robot arm, thereby improving the efficiency of adjusting the posture of the robot arm.

[0044] like Figure 3, is a flowchart of a method for obtaining the pre-stored template point cloud data provided by an embodiment of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The method for obtaining the pre-stored template point cloud data provided by an embodiment of the present application includes the following steps.

[0045] S30, receiving a first point input by a user.

[0046] In one embodiment of the present application, the first point position may be a multidimensional vector input by a user into an electronic device, which is used to represent the position of the robot arm in three-dimensional space desired by the user. For example, the form of the multidimensional vector corresponding to the first point position may be Wherein, x, y, and z are respectively used to represent the coordinates of the position of the robot arm corresponding to the first point in the three-dimensional space; It is used to represent the angle between the line connecting the end of the robot arm and the center of the robot arm base and the x-axis. It is used to represent the angle between the line connecting the end of the robot arm and the center of the robot arm base and the y-axis. It is used to represent the angle between the line connecting the end of the robot arm and the center of the robot arm base and the z-axis.

[0047] S31, controlling the robot arm to move to a target posture according to the first point.

[0048] In one embodiment of the present application, the electronic device controls the robot arm to move to a target posture according to the first point, and the target posture is used to represent the posture that the user expects the robot arm to adjust to according to the first point. The posture of the robot arm includes multiple different categories, and the robot arm is used to achieve different functions when it is in each category of posture. For example, when the posture corresponding to the first point is a grasping position, the robot arm moves to the target posture to grasp the target in the target area.

[0049] S32, controlling the camera device to photograph the target object in the target area to obtain the pre-stored template point cloud data.

[0050] In one embodiment of the present application, after the robot arm moves to the target posture, the camera device can be controlled to shoot the target object in the target area to obtain template point cloud data. The template point cloud data includes multiple second points, each second point corresponds to a second coordinate, and the second coordinate is used to characterize the position of the second point in the camera coordinate system corresponding to the camera device. Exemplarily, the second coordinate can be in the form of: (x2, y2, z2).

[0051] In one embodiment of the present application, in the subsequent process of controlling the posture of the robotic arm, the first difference between the target point cloud data and the template point cloud data can be calculated based on the first coordinate and the second coordinate, so that the electronic device controls the robotic arm to move to the target posture, thereby improving the accuracy of the robotic arm posture control.

[0052] like Figure 4 , is a flow chart of a method for determining a first difference provided in an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for determining a first difference provided in an embodiment of the present application includes the following steps.

[0053] S40, determining a first coordinate of each first point in the target point cloud data in a camera coordinate system and a second coordinate of each second point in the template point cloud data in the camera coordinate system.

[0054] In one embodiment of the present application, in order to improve the accuracy of adjusting the posture of the robotic arm, the first coordinates of each first point in the target point cloud data and the second coordinates of each second point in the template point cloud data can be first determined, and then the first difference can be determined based on all the first coordinates and all the second coordinates.

[0055] S41, matching all second points according to the first coordinates of each first point to obtain the second coordinates of the second point closest to the first coordinates of each first point, and determining a plurality of point pairs according to the first coordinates of each first point and the second coordinates of the matched second points.

[0056] In one embodiment of the present application, in order to comprehensively compare the global features of the target point cloud data and the template point cloud data, the first point and the second point can be matched according to the first coordinate and the second coordinate to obtain multiple point pairs. Each point pair includes a first point and a second point, and the distance between the first coordinate of the first point and the second coordinate of the second point is the shortest.

[0057] Exemplarily, when the first coordinate of a first point is (1, 2, 3), the second coordinate of a second point is (1, 1, 2), and the distance between the first coordinate and the second coordinate is the shortest, then the first point and the second point match each other to form a point pair.

[0058] S42, calculating the difference between the first coordinate of the first point and the second coordinate of the second point in each point pair to obtain the first difference.

[0059] In one embodiment of the present application, the first coordinate and the second coordinate both include multiple dimensions, and determining the first coordinate difference by determining the difference between the first coordinate of the first point and the second coordinate of the second point in each point pair includes: obtaining the difference corresponding to each point pair in each dimension by calculating the difference between the first coordinate and the second coordinate in the same dimension; respectively calculating the mean of the differences corresponding to all point pairs in each dimension to obtain the mean of the differences in each dimension; and determining the mean of the differences corresponding to all dimensions as the first coordinate difference.

[0060] For example, after the first point matches the second point, there are two point pairs, and the first coordinate and the second coordinate in one of the point pairs are (1, 2, 3) and (1, 1, 2), respectively; the first coordinate and the second coordinate in the other point pair are (4, 5, 6) and (4, 5, 7), respectively. Then the mean difference in the x-coordinate dimension is 0, the mean difference in the y-coordinate dimension is -0.5, and the mean difference in the z-coordinate dimension is 0, and the first difference is (0, -0.5, 0).

[0061] like Figure 5 , is a flow chart of a method for controlling a mechanical arm to adjust to a target posture provided in an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for controlling a mechanical arm to adjust to a target posture provided in an embodiment of the present application includes the following steps.

[0062] S50: Map the first difference from the camera coordinate system to the world coordinate system to obtain a second difference.

[0063] In one embodiment of the present application, since the first difference is obtained based on the first coordinate and the second coordinate in the camera coordinate system, the coordinate difference represented by the first difference is the difference between the target point cloud and the template point cloud in the camera coordinate system. In order to adjust the robot arm to the target posture according to the first difference, the first difference can first be mapped to the world coordinate system to obtain the second difference. The world coordinate system can be a three-dimensional coordinate system constructed with the base center of the robot arm as the origin.

[0064] S51: Determine a second point according to the first point and the second difference.

[0065] In an embodiment of the present application, the first three dimensions of the vector corresponding to the first point position and the coordinates corresponding to the second difference may be summed to obtain the second point position.

[0066] Exemplarily, when the first point position is (1, 2, 3, 45°, 20°, 60°) and the second difference is (0, -0.5, 0), the second point position is (1, 1.5, 3, 45°, 20°, 60°).

[0067] S52: Control the robotic arm to adjust to the target posture according to the second point.

[0068] In one embodiment of the present application, the second point position can represent the position corresponding to the posture obtained after the current posture is corrected in the world coordinate system.

[0069] In one embodiment of the present application, after determining the first difference, the electronic device further determines whether the robotic arm is in the target posture. When the first difference is less than or equal to a preset threshold, the robotic arm is determined to be in the target posture; when the first difference is greater than the preset threshold, it is determined that the current posture of the robotic arm is significantly different from the target posture, and the posture of the robotic arm continues to be adjusted. For specific methods of continuing to adjust the posture of the robotic arm, please refer to Figure 6 Corresponding description.

[0070] like Figure 6 The flowchart of the method for continuously adjusting the posture of the robot arm provided by an embodiment of the present application is shown. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The method for continuously adjusting the posture of the robot arm provided by an embodiment of the present application includes the following steps.

[0071] S60, continue to control the camera device to photograph the target object to obtain the target point cloud data.

[0072] In one embodiment of the present application, in order to determine whether the robotic arm is in the target posture, after the electronic device controls the robotic arm to adjust the posture according to the second point, it can continue to control the camera device to shoot the target object to obtain target point cloud data.

[0073] S61, determining the first difference between the target point cloud data and the pre-stored template point cloud data.

[0074] In one embodiment of the present application, in order to determine whether there is still a large difference between the target point cloud data and the template point cloud data, the first difference between the target point cloud and the template point cloud may be determined again. The method of determining the first difference again is the same as that described in steps S40 to S42, and will not be repeated here.

[0075] S62, comparing the first difference with the preset threshold, when the first difference is greater than the preset threshold, executing step S63; when the first difference is less than or equal to the preset threshold, executing step S64.

[0076] In one embodiment of the present application, if the first difference is greater than a preset threshold, it indicates that the difference between the target point cloud data and the template point cloud data is still large, so the difference between the current position of the robot arm and the target position is still large, and the position of the robot arm needs to be further adjusted.

[0077] S63, continue adjusting the posture of the robot arm according to the first difference, and return to step S60.

[0078] In one embodiment of the present application, the method for continuing to adjust the posture of the robot arm according to the first difference is the same as the method described in steps S50 to S52, and will not be repeated here.

[0079] S64, determining that the current posture of the robotic arm is the target posture.

[0080] In one embodiment of the present application, when the first difference is less than or equal to a preset threshold, it indicates that the difference between the target point cloud data and the template point cloud data is small, and therefore the similarity between the current position of the robot arm and the target position is high, and it can be confirmed that the current position of the robot arm is the target position.

[0081] In one embodiment of the present application, when the electronic device is in communication connection with multiple robotic arms, the electronic device also copies the first point position to the multiple robotic arms, thereby improving the efficiency of controlling the multiple robotic arms to adjust their positions. Figure 7 As shown, it is a flow chart of a method for controlling multiple robotic arms to move to a target posture provided by an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The method for controlling multiple robotic arms to move to a target posture provided by an embodiment of the present application includes the following steps.

[0082] S70, sending the first point position to the plurality of robotic arms, controlling the camera device installed on each of the robotic arms to photograph the target object, and obtaining target point cloud data corresponding to each of the robotic arms.

[0083] In one embodiment of the present application, when the electronic device is connected to multiple robotic arms for communication, it is also used to control the multiple robotic arms to adjust to a target posture. In order to improve the efficiency of controlling the multiple robotic arms to adjust the posture, the electronic device can send a first point to the multiple robotic arms to directly copy the first point to each robotic arm, without the need for the staff to adjust the point of each robotic arm one by one, thereby improving the efficiency of controlling the multiple robotic arms to adjust the posture.

[0084] In one embodiment of the present application, after adjusting the posture of each mechanical arm according to the first point, it is also necessary to determine whether the posture of each mechanical arm is adjusted in place, thereby improving the accuracy of the posture of each mechanical arm. Therefore, the camera device installed on each mechanical arm can be controlled to shoot the target object to obtain the target point cloud data corresponding to each mechanical arm.

[0085] S71, determining a first difference between the target point cloud data and the pre-stored template point cloud data.

[0086] In one embodiment of the present application, in order to determine whether the posture of each robotic arm is adjusted in place, the first difference between the target point cloud data corresponding to each robotic arm and the pre-stored template point cloud data can be determined respectively, and the posture of each robotic arm can be adjusted respectively according to the first difference. The smaller the first difference is, the smaller the difference between the current posture of the robotic arm and the target posture is after the robotic arm is controlled to adjust its posture according to the first point position; the larger the first difference is, the larger the difference between the current posture of the robotic arm and the target posture is.

[0087] S72: Control each of the robotic arms to adjust to the target posture according to the first difference.

[0088] In one embodiment of the present application, the current posture of the robot arm can be corrected based on the first difference, reducing the difference between the current posture of the robot arm and the target posture, thereby improving the accuracy of controlling the posture adjustment of multiple robot arms. Specifically, the method of controlling each robot arm to adjust to the target posture according to the first difference is the same as the detailed description corresponding to steps S50 to S52, and will not be repeated here.

[0089] In this way, by copying the first point position to each robot arm in the production environment, the posture control of multiple robot arms can be realized, which can avoid the inefficiency caused by manual adjustment of the robot arm posture. And the first difference corresponding to each robot arm is determined according to the target point cloud and the pre-stored template point cloud, and the posture of each robot arm is adjusted according to the first difference, so as to improve the accuracy of the posture of each robot arm.

[0090] See also Figure 8 , Figure 8 81 is a functional module diagram of a robot arm posture control device provided in an embodiment of the present application. The robot arm posture control device 81 includes a control module 810 and a determination module 811. The module / unit referred to in the present application refers to a type of device that can be controlled by the processor 13 (see Fig. 9 ) and can perform a series of computer readable instruction segments that are executed and can complete fixed functions, which are stored in the memory 12 (see Fig. 9 In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0091] The control module 810 is used to control the camera to photograph the target object and obtain target point cloud data.

[0092] The determination module 811 is used to determine a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robotic arm is in the target posture.

[0093] The control module 810 is further configured to control the robotic arm to adjust to the target posture according to the first difference.

[0094] In one embodiment of the present application, the control module 810 is specifically used to: receive a first point position input by a user; control the robotic arm to move to a target position according to the first point position; control the camera device to photograph the target object in the target area to obtain the pre-stored template point cloud data.

[0095] In one embodiment of the present application, the determination module 811 is specifically used to: determine the first coordinate of each first point in the target point cloud data in the camera coordinate system and the second coordinate of each second point in the template point cloud data in the camera coordinate system; match all second points according to the first coordinate of each first point to obtain the second coordinate of the second point closest to the first coordinate of each first point, and determine multiple point pairs according to the first coordinate of each first point and the second coordinate of the matched second point; calculate the difference between the first coordinate of the first point and the second coordinate of the second point in each point pair to obtain the first difference.

[0096] In one embodiment of the present application, the control module 810 is also used to: map the first difference from the camera coordinate system to the world coordinate system to obtain a second difference; determine the second point position based on the first point position and the second difference; and control the robotic arm to adjust to the target posture based on the second point position.

[0097] In an embodiment of the present application, the control module 810 is further used to: when the first difference is greater than a preset threshold, continue to adjust the posture of the robotic arm.

[0098] In one embodiment of the present application, the control module 810 is also used to: continue to control the camera device to photograph the target object and obtain the target point cloud data; determine the first difference between the target point cloud data and the pre-stored template point cloud data, and continue to adjust the posture of the robotic arm according to the first difference until the first difference is less than or equal to the preset threshold.

[0099] In one embodiment of the present application, the control module 810 is also used to: input the first point position to the multiple robotic arms, control the camera device installed on each of the robotic arms to shoot the target object, and obtain target point cloud data corresponding to each of the robotic arms; determine the first difference between the target point cloud data and the pre-stored template point cloud data; and control each of the robotic arms to adjust to the target posture according to the first difference.

[0100] It can be seen from the above technical solutions that the embodiment of the present application can use the camera device on the robot arm to shoot the target object when the robot arm is in any posture, obtain the target point cloud data, and judge the first difference between the current posture of the robot arm and the target posture according to the target point cloud data and the pre-stored template point cloud data, and adjust the robot arm to the target posture according to the first difference. In this way, adjusting the posture of the robot arm according to the principle of computer vision can improve the accuracy of controlling the posture of the robot arm, and there is no need to manually adjust the posture of the robot arm, thereby improving the efficiency of adjusting the posture of the robot arm.

[0101] See also Fig. 9 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 is used to execute the computer-readable instructions stored in the memory to implement the robot arm posture control method described in any of the above embodiments.

[0102] In an embodiment of the present application, the electronic device 1 further includes a bus, and a computer program stored in the memory 12 and executable on the processor 13, such as a robot arm posture control program.

[0103] Fig. 9 Only the electronic device 1 having the memory 12 and the processor 13 is shown, and those skilled in the art can understand that Fig. 9 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0104] Combination Figure 2 The memory 12 in the electronic device 1 stores a plurality of computer-readable instructions to implement a method for controlling a robot arm posture, and the processor 13 can execute the plurality of instructions to implement: controlling the camera device to photograph the target object to obtain target point cloud data; determining a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robot arm is in the target posture; and controlling the robot arm to adjust to the target posture according to the first difference.

[0105] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0106] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may be a bus structure or a star structure. The electronic device 1 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the electronic device 1 may also include input and output devices, network access devices, etc.

[0107] It should be noted that the electronic device 1 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.

[0108] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the robot arm posture control program, etc., but also can be used to temporarily store data that has been output or is to be output.

[0109] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect the various components of the entire electronic device 1, and executes or executes programs or modules stored in the memory 12 (for example, executing a robot arm posture control program, etc.), and calls the data stored in the memory 12 to execute various functions of the electronic device 1 and process data.

[0110] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the robot arm posture control method, for example Figure 2 Steps shown.

[0111] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a control module 810 and a determination module 811.

[0112] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute a part of the robot arm posture control method described in each embodiment of the present application.

[0113] If the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware devices to complete through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0114] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory, etc.

[0115] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0116] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. Fig. 9 Only one arrow is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0117] An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable storage medium is stored computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the robot arm posture control method described in any of the above embodiments.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0119] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0121] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A method for controlling the position and posture of a robot arm, applied to an electronic device, wherein the electronic device is communicatively connected to the robot arm, characterized in that: The robotic arm is provided with a camera device, which is communicatively connected with the electronic device and is used to photograph a target object disposed in a target area. The method includes: Controlling the camera to photograph the target object to obtain target point cloud data; Determining a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robotic arm is in the target posture; The robotic arm is controlled to adjust to the target posture according to the first difference.

2. The method for controlling the position and posture of a robotic arm according to claim 1, wherein: The method for obtaining the pre-stored template point cloud data includes: Receive the first point input by the user; Control the robot arm to move to a target position according to the first point; The camera device is controlled to photograph the target object in the target area to obtain the pre-stored template point cloud data.

3. The robot arm posture control method according to claim 1, characterized in that: The determining a first difference between the target point cloud data and the pre-stored template point cloud data comprises: Determine a first coordinate of each first point in the target point cloud data in a camera coordinate system and a second coordinate of each second point in the template point cloud data in the camera coordinate system; Match all the second points according to the first coordinate of each of the first points to obtain the second coordinate of the second point closest to the first coordinate of each of the first points, and determine a plurality of point pairs according to the first coordinate of each of the first points and the second coordinate of the matched second points; The difference between the first coordinate of the first point and the second coordinate of the second point in each of the point pairs is calculated to obtain the first difference.

4. The robot arm posture control method according to claim 2, characterized in that: The controlling the robotic arm to adjust to the target posture according to the first difference comprises: Mapping the first difference from the camera coordinate system to the world coordinate system to obtain a second difference; Determine a second point according to the first point and the second difference; The robotic arm is controlled to adjust to the target posture according to the second point position.

5. The robot arm posture control method according to claim 1 or 4, characterized in that: The method further includes: when the first difference is greater than a preset threshold, continuing to adjust the posture of the robotic arm.

6. The method for controlling the position and posture of a robotic arm according to claim 5, wherein: The step of continuing to adjust the posture of the robotic arm includes: Continue to control the camera device to photograph the target object to obtain the target point cloud data; Determine the first difference between the target point cloud data and the pre-stored template point cloud data, and continue to adjust the posture of the robotic arm according to the first difference until the first difference is less than or equal to the preset threshold.

7. The method for controlling the position and posture of a robotic arm according to claim 2, wherein: When the electronic device is communicatively connected with a plurality of the mechanical arms, the method further comprises: Sending the first point position to the plurality of robotic arms, controlling the camera device installed on each of the robotic arms to shoot the target object, and obtaining the target point cloud data corresponding to each of the robotic arms; Determining the first difference between the target point cloud data and the pre-stored template point cloud data; Each of the robotic arms is controlled to adjust to the target posture according to the first difference.

8. A robot arm posture control device, characterized in that: The device comprises a module for implementing the robot arm posture control method according to any one of claims 1 to 7, and the device comprises: A control module, used for controlling the camera to photograph the target object and obtain target point cloud data; A determination module, configured to determine a first difference between the target point cloud data and pre-stored template point cloud data, wherein the pre-stored template point cloud data is point cloud data obtained by the camera device photographing the target object when the robotic arm is in the target posture; The control module is further used to control the robotic arm to adjust to the target posture according to the first difference.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing computer-readable instructions; and A processor executes computer-readable instructions stored in the memory to implement the robot arm posture control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the robot arm posture control method according to any one of claims 1 to 7 is implemented.

Images