Operation control method and device, equipment and medium

By using robotic arms and control modules in the operation control system, the planned operation points are obtained and moved to achieve precise control of the operation points, the problem of inconsistency in the traditional operation control methods is solved, and the accuracy and safety of the operation are improved.

CN118906052AInactive Publication Date: 2024-11-08BEIJING NATONG MEDICAL ROBOT TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410991232.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional homework control methods rely on user self-control, and it is difficult to ensure that the pre-planned homework points are consistent with the actual homework points, which affects the beauty effect.

Method used

By using robotic arms, control modules and end tools in the operation control system, the target position of the planned work point on the target object surface under the coordinate system of the robotic arm base, and based on the target position of the adjacent two planned work points. Control the movement of the robotic arm to ensure that the end tool performs preset work depth in a direction perpendicular to the target surface.

Benefits of technology

The consistency between the pre-planned work points and the actual work points is achieved, the accuracy and safety of work control are improved, and the dependence of user self-control is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118906052A_ABST
    Figure CN118906052A_ABST
Patent Text Reader

Abstract

The invention relates to an operation control method and device, equipment and a medium. Acquiring a target pose of a planned operation point on the surface of the target object under a base coordinate system of the mechanical arm; for a current planned operation point in the planned operation points, based on a first target pose of the current planned operation point and a second target pose of a previous planned operation point, controlling the mechanical arm to move from the previous planned operation point to the current planned operation point; and the control module controls the end tool to move the preset operation depth from the current planned operation point to perform operation in the inward direction perpendicular to the surface of the target object. Therefore, based on the target poses corresponding to the two adjacent planning operation points, the mechanical arm is controlled to move on the surface of the target object, and at each planning operation point, the tail end tool is controlled to move and operate. Therefore, the method does not need to depend on the self-control of a user, achieves the consistency of a pre-planned operation point and an actual operation point, and improves the precision and safety of operation control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of robot arm control, and in particular to an operation control method, device, equipment and medium. Background Art

[0002] Traditional operation control methods rely on users to manually control the operation tools. However, manual operation control methods rely on the user's self-control, and it is difficult to ensure that the pre-planned operation points are consistent with the actual operation points, thus affecting the operation effect. For example, in traditional medical cosmetic operations, users inject beauty liquid into the skin injection points to achieve cosmetic effects such as hydrating and moisturizing the skin, enhancing skin elasticity and removing wrinkles. However, this medical cosmetic operation process completely relies on the user's self-control, and it is easy for the pre-planned skin injection points to be inconsistent with the skin injection points actually identified by the user, which will have a certain impact on the cosmetic effect.

[0003] Therefore, it is necessary to provide a reliable operation control method to improve the operation accuracy of the operation tool. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a job control method, device, equipment and medium.

[0005] In a first aspect, the present disclosure provides an operation control method, which is applied to an electronic device in an operation control system, wherein the operation control system further comprises a mechanical arm, a control module and an end tool are mounted on a flange of the mechanical arm, and the method comprises:

[0006] Obtaining a target pose of a planned operation point on the surface of a target object in a base coordinate system of the robotic arm;

[0007] For a current planned operation point among the planned operation points, based on a first target posture of the current planned operation point and a second target posture of a previous planned operation point, controlling the robot arm to move from the previous planned operation point to the current planned operation point;

[0008] The control module controls the end tool to move from the current planned operation point to a preset operation depth to perform operation along an inward direction perpendicular to the surface of the target object.

[0009] In a second aspect, the present disclosure provides an operation control device, an electronic device configured in an operation control system, wherein the operation control system further comprises a mechanical arm, a control module and an end tool are mounted on a flange of the mechanical arm, and the device comprises:

[0010] An acquisition module, used for acquiring a target pose of a planned operation point on the surface of a target object in a base coordinate system of the robotic arm;

[0011] A control module, for controlling the robot arm to move from the last planned operation point to the current planned operation point based on a first target posture of the current planned operation point and a second target posture of the last planned operation point;

[0012] The operation module is used to control the end tool to move from the current planned operation point to a preset operation depth to perform operation along a direction perpendicular to the surface of the target object inward through the control module.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When one or more programs are executed by one or more processors, the one or more processors implement the method provided in the first aspect.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method provided in the first aspect is implemented.

[0018] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0019] A method, device, equipment and medium for controlling an operation disclosed in an embodiment of the present invention are applied to an electronic device in an operation control system. The operation control system also includes a manipulator, and a control module and an end tool are installed on the flange of the manipulator. The method includes: obtaining the target posture of the planned operation point on the surface of the target object in the base coordinate system of the manipulator; for the current planned operation point in the planned operation points, based on the first target posture of the current planned operation point and the second target posture of the previous planned operation point, controlling the manipulator to move from the previous planned operation point to the current planned operation point; controlling the end tool to move from the current planned operation point to a preset operation depth for operation by the control module along a direction perpendicular to the surface of the target object inward. Thus, the manipulator can be controlled to move on the surface of the target object based on the target postures corresponding to the two adjacent planned operation points, and the end tool can be controlled to move and operate at each planned operation point. In this way, there is no need to rely on the self-control of the user, and the pre-planned operation point is consistent with the actual operation point, thereby improving the accuracy and safety of the operation control. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 A system block diagram of a job control system provided by an embodiment of the present disclosure;

[0023] Figure 2 A flowchart of a job control method provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of the process of S210 provided in an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of the process of S220 provided in an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of the process of S220 provided in an embodiment of the present disclosure;

[0027] Figure 6 A logic diagram of a job control method provided by an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the structure of a job control device provided by an embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0032] To facilitate understanding of the job control method, Figure 1 The system block diagram of the operation control system is shown in FIG. Figure 1 As shown, the operation control system includes: an electronic device 100, a camera 200 and a mechanical arm 300. A control module 310 and an end tool 320 are mounted on the flange of the mechanical arm 300. Optionally, the control module 310 may be a high-precision motor.

[0033] Specifically, the operation control method includes: first, the user marks multiple planned operation points on the surface of the target object; then, the camera 200 performs image acquisition on the planned operation points on the surface of the target object to obtain image data, and sends the image data to the electronic device 100 through a control protocol / Internet protocol (TCP / IP) communication method; then, the electronic device 100 performs image processing on the image data to determine the target posture of the planned operation point in the base coordinate system based on the image processing result and other data; then, the electronic device 100 sends the target posture of the planned operation point in the base coordinate system to the robot 300 through a TCP / IP communication method, so as to control the robot 300 to move from the last planned operation point to the current planned operation point based on the first target posture of the current planned operation point and the second target posture of the last planned operation point; further, the electronic device 100 sends a control instruction to the control module 310 through a controller area network bus (CAN bus) communication method, so as to control the end tool 320 through the control module 310 to move from the current planned operation point to a preset operation depth to perform the operation along a direction perpendicular to the surface of the target object inward.

[0034] In this embodiment, continue to see Figure 1 The operation control system further includes a force sensor 330. The force sensor 330 is installed at one end of the end tool 320, and the other end of the end tool 320 is used for operation.

[0035] Specifically, the electronic device 100 can make a normal based on the current planned operation point on the surface of the target object, and set a point in the normal direction as the target point corresponding to the current planned operation point; then, the electronic device 100 controls the robotic arm 300 to move from the previous planned operation point to the target point corresponding to the current planned operation point, and starts the force sensor 330 by controlling the TCP / IP communication method; then, in the process of the robotic arm 300 moving from the target point along the path close to the current planned operation point, force sensing data is obtained through the force sensor 300, so as to control the robotic arm 300 to move from the target point to the current planned operation point based on the force sensing data.

[0036] Furthermore, after the end tool 320 completes the operation at the preset operating depth, the electronic device 100 continues to control the end tool 320 through the control module 310 to first retract to the current planned operating point, and then controls the robotic arm to retract from the current planned operating point to the target point to start the next operation.

[0037] In this way, the robot arm can be controlled to move on the surface of the target object based on the target postures corresponding to two adjacent planned operation points, and the end tool can be controlled to move and operate at each planned operation point, without relying on user self-control.

[0038] In order to improve the working accuracy of the working tools, the following Figures 2 to 6 The operation control method provided by the embodiment of the present disclosure is described. In the embodiment of the present disclosure, the operation control method can be executed by an electronic device in the operation control system. Figure 1 As shown, the operation control system also includes a robotic arm, on the flange of which a control module and an end tool are mounted. The electronic equipment may include a tablet computer, a desktop computer, a laptop computer and other devices with communication functions, and may also include a virtual machine or a device simulated by a simulator.

[0039] Figure 2 A flowchart of a job control method provided by an embodiment of the present disclosure is shown.

[0040] like Figure 2 As shown, the job control method may include the following steps.

[0041] S210: Obtain the target position and posture of the planned operation point on the surface of the target object in the base coordinate system of the robot arm.

[0042] In this embodiment, when the electronic device controls the operation of the end tool, it first obtains multiple planned operation points on the surface of the target object, and obtains the target posture of each planned operation point in the base coordinate system of the robot arm, so as to automatically control the end tool to perform the operation based on the target posture of each planned operation point.

[0043] The target object surface refers to the outer surface of the work object. For example, in medical cosmetic work, the target object surface refers to the face. For another example, in bone grinding work, the target object surface refers to the bone surface.

[0044] Among them, the planned operation point refers to the operation point calibrated in advance by the user in order to complete the operation, so that the robot arm drives the end tool to operate on the surface of the target object according to the planned operation point. For example, in medical cosmetic operations, the planned operation point refers to the injection point calibrated in advance on the face. For another example, in bone grinding operations, the planned operation point refers to the grinding point calibrated in advance on the bone surface.

[0045] The target posture refers to the posture matrix of the planned operation point in the base coordinate system. Optionally, the electronic device determines the target posture of the planned operation point in the base coordinate system when calibrating the planned operation point in advance, and can also determine the target posture of the planned operation point in the base coordinate system based on the collected data and other calibration data after adjusting the posture of the robot arm.

[0046] S220: For a current planned operation point among the planned operation points, based on a first target posture of the current planned operation point and a second target posture of a previous planned operation point, control the robot arm to move from the previous planned operation point to the current planned operation point.

[0047] In this embodiment, a specific explanation is given with reference to any current planned operation point among the planned operation points. When it is necessary to operate at the current planned operation point, it is necessary to control the robot arm to move from the previous planned operation point to the current planned operation point, then the robot arm is controlled to move in combination with the first target posture of the current planned operation point and the second target posture of the previous planned operation point, so as to ensure that the robot arm moves accurately and safely from the previous planned operation point to the current planned operation point.

[0048] For example, in medical cosmetic surgery, when it is necessary to inject cosmetic liquid at the current planned injection point (i.e., the current planned operation point), based on the first target posture of the current planned injection point and the second target posture of the previous planned injection point (i.e., the previous planned operation point), the robot arm is controlled to move from the previous planned injection point to the current planned injection point, so as to inject cosmetic liquid at the current planned injection point. For another example, in bone grinding surgery, when it is necessary to grind at the current planned grinding point (i.e., the current planned operation point), based on the first target posture of the current planned grinding point and the second target posture of the previous planned grinding point (i.e., the previous planned operation point), the robot arm is controlled to move from the previous planned grinding point to the current planned grinding point, so as to grind the bone at the current planned grinding point.

[0049] It can be understood that when the current planned operation point is the first planned operation point, there is no need to consider the second target posture of the previous planned operation point. Only based on the first target posture of the current planned operation point, the robot arm is controlled to move from the previous planned operation point to the current planned operation point.

[0050] In this way, the robot arm can be automatically controlled to move between two adjacent planned operation points according to the target postures corresponding to the two adjacent planned operation points, without relying on the user's self-control, so that the planned operation point is consistent with the operation point actually reached by the robot arm, which is conducive to improving the operation control accuracy.

[0051] S230, controlling the end tool through the control module to move from the current planned operation point to a preset operation depth in an inward direction perpendicular to the surface of the target object to perform the operation.

[0052] In this embodiment, after controlling the robot arm to move to the currently planned operation point, the electronic device sends instructions to the control module so as to control the end tool to perform the operation through the control module.

[0053] The preset working depth refers to the working depth of the end tool inside the surface of the target object. Optionally, the preset working depth can be predetermined based on experience in different working scenarios. Exemplarily, the preset working depth includes but is not limited to values ​​such as 3mm and 5mm.

[0054] Among them, the specific implementation method of S230 includes but is not limited to the following methods: through the control module, the end tool is controlled to move from the current planned operation point to a preset operation depth at a preset speed along a direction perpendicular to the surface of the target object inward; at the preset operation depth, the end tool is controlled by the control module to operate according to the preset value.

[0055] The preset speed refers to the moving speed of the end tool in a direction perpendicular to the surface of the target object and inward.

[0056] The preset value refers to the operating value of the end tool at the preset operating depth.

[0057] For example, in medical cosmetic surgery, when the electronic device controls the robot arm to move to the currently planned injection point, the control module continues to control the injection needle (i.e., the end tool) to penetrate the face to a certain depth (i.e., the preset operation depth) at a preset speed, and then at this depth, the control module continues to control the injection needle to inject cosmetic liquid at a specific dose (preset value). For another example, in bone grinding surgery, when the electronic device controls the robot arm to move to the currently planned grinding point, the control module continues to control the drill bit (i.e., the end tool) to penetrate the bone surface to a certain depth (i.e., the preset operation depth) at a preset speed, and then at this depth, the control module continues to control the drill bit to grind the bone at a specific strength (preset value).

[0058] In this way, the control module can be used to accurately control the movement speed and operation value of the end tool to perform operations at a preset operating depth, thereby improving the accuracy and safety of operation control and avoiding problems such as operation control deviation and safety hazards due to viewing angle deviation and unstable operation during manual operations by users.

[0059] A method for controlling an operation disclosed in an embodiment of the present invention is applied to an electronic device in an operation control system. The operation control system also includes a manipulator, and a control module and an end tool are installed on the flange of the manipulator. The method includes: obtaining the target posture of the planned operation point on the surface of the target object in the base coordinate system of the manipulator; for the current planned operation point in the planned operation point, based on the first target posture of the current planned operation point and the second target posture of the last planned operation point, controlling the manipulator to move from the last planned operation point to the planned operation point; controlling the end tool to move from the planned operation point to a preset operation depth for operation in a direction perpendicular to the surface of the target object inward through the control module. Thus, the manipulator can be controlled to move on the surface of the target object based on the target postures corresponding to the two adjacent planned operation points, and the end tool can be controlled to move and operate at each planned operation point. In this way, there is no need to rely on the user's self-control, and the pre-planned operation point is consistent with the actual operation point, thereby improving the accuracy and safety of operation control.

[0060] In another embodiment of the present disclosure, Figure 1 The illustrated operation control system also includes a camera, and S210 is explained in detail.

[0061] Figure 3 A schematic diagram of the process of S210 provided in an embodiment of the present disclosure is shown.

[0062] like Figure 3 As shown, S210 may specifically include the following steps.

[0063] S310: Acquire image data of a planned operation point on the surface of a target object, wherein the image data is collected by a camera.

[0064] In this embodiment, after the user marks the planned operation point on the surface of the target object in advance, the camera collects image data of the surface of the target object and sends the image data to the electronic device so that the electronic device processes the image data.

[0065] S320: Determine the target position and posture of the planned operation point in the base coordinate system based on the image data and the pre-acquired calibration data.

[0066] In this embodiment, the electronic device can combine the image data and the pre-acquired calibration data to specifically determine the target pose of the planned operation point in the base coordinate system to ensure accurate determination of the target pose of each planned operation point.

[0067] The specific implementation method of S320 includes but is not limited to the following methods:

[0068] Perform image processing on the image data to determine the first pose data of the planned operation point in the camera coordinate system; obtain the second pose data of the camera in the base coordinate system and the third pose data of the end tool in the flange coordinate system; determine the pose data of the flange in the base coordinate system based on the first pose data, the second pose data and the third pose data, and use the pose data of the flange in the base coordinate system as the target pose.

[0069] Specifically, the electronic device can use principal component analysis, gradient descent algorithm and other methods to perform image processing on the image data to determine the first pose data, and obtain calibration data such as the first pose data and the second pose data, and then multiply the first pose data, the second pose data and the third pose data to determine the pose data of the flange in the base coordinate system, that is, to obtain the target pose.

[0070] Optionally, the position data of the flange in the base coordinate system can be determined by:

[0071]

[0072] in, is the position data of the flange in the base coordinate system, is the second pose data of the camera in the base coordinate system, It is the first position data of the planned operation point in the camera coordinate system. It is the inverse matrix of the third pose data of the end tool in the flange coordinate system.

[0073] It should be noted that the specific order of determining the first pose data and obtaining the second pose data and the third pose data is not limited.

[0074] In this way, the image data of the planned operation point on the surface of the target object is collected by the camera to assist in determining the target position and posture of the planned operation point in the base coordinate system, thereby improving the accuracy of the target position and posture of the planned operation point, which is beneficial to improving the accuracy of operation control.

[0075] In another embodiment of the present disclosure, during the actual operation, the surface of the target object is not absolutely flat, and there may be obstacles that hinder the movement of the robot arm. In this case, the robot arm needs to be controlled to move around the obstacles, thereby improving the smoothness and safety of the operation control process. Based on this, S220 is explained in detail.

[0076] Figure 4 A schematic diagram of the process of S220 provided in an embodiment of the present disclosure is shown.

[0077] like Figure 4 As shown, S220 may specifically include the following steps.

[0078] S410: Connect the last planned operation point and the current planned operation point to obtain a candidate route for controlling the movement of the robot arm.

[0079] In this embodiment, the electronic device directly connects the last planned operation point and the current planned operation point to form a line segment, and the line segment is used as a candidate route for controlling the movement of the robot arm.

[0080] S420: Determine whether there are any obstacles on the candidate route that hinder the movement of the robot arm.

[0081] It is understandable that, since the surface of the target object is not absolutely flat, there may be obstacles on the candidate route that hinder the movement of the robot arm. In this case, if the robot arm moves directly from the last planned operation point to the current planned operation point, the obstacle will hinder the movement of the robot arm, or the movement of the robot arm will have an adverse effect on the surface of the target object. In this regard, the electronic device needs to determine whether there is an obstacle on the candidate route. If so, execute S430 to determine a new route, otherwise, execute S460.

[0082] In this embodiment, the electronic device can determine whether there is an obstacle point on the candidate route that hinders the movement of the robot arm based on the coordinate data of each point on the candidate route in the base coordinate system. Specifically, first, obtain the first Z coordinate of the last planned operation point on the candidate route in the base coordinate system, the second Z coordinate of the current planned operation point in the base coordinate system, and the third Z coordinates corresponding to other points in the base coordinate system; then, if there is a third Z coordinate on the candidate route that is larger than the maximum value of the first Z coordinate and the second Z coordinate, it is determined that there is an obstacle point on the candidate route.

[0083] For example, in medical cosmetic operations, if the last planned operation point is an injection point on the left face, and the currently planned injection point is an injection point on the right face, after the injection needle injects perfect liquid into the injection point on the left face, the robotic arm needs to drive the injection needle from the injection point on the left face to the injection point on the right face. At this time, the electronic device needs to determine whether there are obstacle points (such as the tip of the nose) on the candidate route formed by the injection points on the left face and the injection points on the right face. If there are obstacle points, and directly controlling the movement of the robotic arm will cause the face to be scratched, there is a major safety hazard. Therefore, a new route needs to be determined to avoid such safety hazard.

[0084] S430: Determine a fitting point on the candidate route based on the first target pose and the second target pose.

[0085] In this embodiment, the first target posture refers to the posture matrix of the current planned work point in the base coordinate system, specifically expressed as P1 (X1, Y1, Z1), and the second target posture refers to the posture matrix of the previous planned work point in the base coordinate system, specifically expressed as P2 (X2, Y2, Z2). The electronic device determines the fitting point P3 on the candidate route based on P1 and P2.

[0086] Optionally, the fitting point includes but is not limited to the midpoint, one-third point, one-quarter point, etc. of the current planned operation point and the previous planned operation point.

[0087] S440, performing curve fitting on the last planned operation point, the current planned operation point and the fitting point to generate a target route for controlling the movement of the robot arm.

[0088] In this embodiment, the electronic device uses a fitting algorithm to perform curve fitting on the current planned operation point, the previous planned operation point and the fitting point, thereby generating an arc. The arc can bypass the obstacle points on the surface of the target object and obtain the target route. When the movement of the robotic arm is controlled based on the target route, there will be no situation of touching the obstacle points, thereby improving the safety of the robotic arm's movement process.

[0089] S450: According to the target route, the robot arm is controlled to move from the last planned operation point to the current planned operation point.

[0090] In this embodiment, the electronic device controls the movement of the robotic arm based on the target route, so that the robotic arm can bypass obstacles on the surface of the target object, thereby avoiding safety hazards.

[0091] In this way, in the actual operation process, it is necessary to determine whether there is an obstacle point between the paths formed by two adjacent operation points. If so, it is necessary to determine a new path to control the movement of the robotic arm so that the robotic arm can bypass the obstacle point when moving, thereby improving the smoothness and safety of the operation control process.

[0092] S460: According to the candidate route, control the robot arm to move from the last planned operation point to the current planned operation point.

[0093] It is understandable that if the electronic device determines that there is no obstacle point on the candidate route, the movement of the robotic arm is directly controlled based on the candidate route without additionally determining a new route, so as to save computing resources.

[0094] In another embodiment of the present disclosure, the operation control system further includes a force sensor, which is installed at one end of the end tool, and the other end of the end tool is used for operation. Based on this, S220 is explained in detail.

[0095] Figure 5 A schematic diagram of the process of S220 provided in an embodiment of the present disclosure is shown.

[0096] like Figure 5 As shown, S220 may specifically include the following steps.

[0097] S510. Based on the first target posture and the second target posture, control the robot arm to move from the last planned operation point to the target point corresponding to the current planned operation point, wherein the target point is a preset point on the normal formed by the current planned operation point on the surface of the target object.

[0098] In order to further improve the safety of the target object surface, the electronic device first controls the robotic arm to move from the previous planned operation point to the target point corresponding to the current planned operation point based on the first target posture and the second target posture, that is, controls the robotic arm to move above the current planned operation point, and then controls the robotic arm to continue moving in the direction close to the current planned operation point based on the data collected by the force sensor.

[0099] Optionally, the target point may be a preset point such as 3 mm or 5 mm above the currently planned operation point.

[0100] S520, start the force sensor.

[0101] In this embodiment, when the electronic device detects that the robotic arm moves to the target point, it sends a start instruction to the force sensor to start the force sensor to collect data.

[0102] S530. When the robot arm moves from the target point to the currently planned operation point, force sensing data is acquired through the force sensor.

[0103] In this embodiment, after the electronic device activates the force sensor, it sends a certain distance (for example, 1 cm) to the robotic arm, thereby continuing to control the robotic arm to move downward from the target point to approach the currently planned operation point, and during the movement of the robotic arm, force sensing data is collected through the force sensor.

[0104] S540: Based on the force sensing data, control the robot arm to move from the target point to the currently planned operation point.

[0105] In this embodiment, the electronic device filters the force sensing data in consideration of environmental noise, and controls the robot arm to move from the target point to the currently planned operation point based on the filtered force sensing data.

[0106] Among them, the specific implementation method of S540 includes but is not limited to the following methods: if the force sensing data is equal to the preset force threshold, it is determined that the robotic arm has moved from the target point to the current planned operation point, and the robotic arm is controlled to stop moving; if the force sensing data is less than the preset force threshold, it is determined that the robotic arm has not moved from the target point to the current planned operation point, and the robotic arm is controlled to continue moving until the force sensing data is equal to the preset force threshold.

[0107] The preset force threshold refers to the force applied to the surface of the target object.

[0108] In this way, in the process of the robot arm moving from the last planned operation point to the current planned operation point, the robot arm is first controlled to move from the last planned operation point to the target point corresponding to the current planned operation point, and then the force sensing data collected by the force sensor is used to control the end tool to move between the target point and the current planned operation point, thereby achieving the effect of fine-grained control of the movement of the robot arm and improving the safety of the robot arm's movement process.

[0109] In this embodiment, after executing S230, the method also includes: through the control module, controlling the end tool to move to the current planned working point according to the preset working depth along the direction perpendicular to the surface of the target object; based on the first target posture, controlling the robotic arm to move from the current planned working point to the target point.

[0110] Specifically, after the end tool completes the operation at the preset operating depth of the currently planned operating point, the electronic device continues to control the end tool to move outward to the preset operating depth through the control module, and controls the robotic arm to retract to the target point according to the first target posture of the currently planned operating point.

[0111] In this way, for any planned operation point on the surface of the target object, the robotic arm can be controlled to first move to the target point above the planned operation point, and then move downward from the target point to the preset operation depth of the planned operation point to perform the operation. After the operation is completed, the robotic arm is controlled to retreat from the planned operation point to the target point, waiting to start the next operation.

[0112] In yet another embodiment of the present disclosure, the execution logic of the job control method is explained as a whole.

[0113] Figure 6 A logical schematic diagram of a job control method provided by an embodiment of the present disclosure is shown.

[0114] like Figure 6 As shown, the operation control method may specifically include the following steps.

[0115] S610: The camera collects image data of the planned operation points on the surface of the target object.

[0116] S620: The electronic device determines the target position and posture of the planned operation point in the base coordinate system based on the image data and the pre-acquired calibration data.

[0117] The pre-acquired calibration data include the second pose data of the camera in the base coordinate system and the third pose data of the end tool in the flange coordinate system.

[0118] Specifically, the electronic device first determines the first pose data of the planned operation point in the camera coordinate system based on the image data, and then determines the target pose in combination with the second pose data and the third pose data.

[0119] Furthermore, the electronic device may continue to execute S630 or S690 according to actual needs.

[0120] S630: The electronic device controls the robot arm to move from the last planned operation point to the target point corresponding to the current planned operation point based on the first target position of the current planned operation point and the second target position of the last planned operation point.

[0121] Optionally, the target point may be 5 mm above the currently planned operation point, which is not limited here.

[0122] S640: The electronic device turns on the force sensor and obtains force sensing data collected by the force sensor during the process of the robot arm moving from the target point to the currently planned operation point.

[0123] Specifically, the electronic device can send a moving distance (eg, 1 cm, 1.5 cm, etc.) to the robotic arm to control the robotic arm to move downward from the target point and, at the same time, obtain force sensing data.

[0124] S650: The electronic device determines whether the force sensing data is equal to a preset force threshold.

[0125] Specifically, the electronic device first filters the force sensing data, and then compares the filtered force sensing data with a preset force threshold. If the force sensing data is equal to the preset force threshold, S660 is executed; otherwise, S650 is returned to be executed.

[0126] S660: The electronic device determines that the robot arm has moved from the target point to the currently planned operation point, and controls the robot arm to stop moving.

[0127] S670, the electronic device controls the end tool through the control module to move downward to a preset working depth at the currently planned working point to perform the operation.

[0128] Specifically, the electronic device controls the end tool to move at a preset speed through the control module, and continues to control the end tool to operate according to a preset value at a preset operating depth through the control module.

[0129] S680, the electronic device controls the end tool to retreat from the preset working depth to the current planned working point through the control module, and then continues to control the robot arm to retreat from the current planned working point to the target point to start the next operation.

[0130] Optionally, the control logic of the next operation may be specifically referred to S630 to S680, which will not be described in detail here.

[0131] S690: The electronic device determines whether there is an obstacle point that hinders the movement of the robot arm in the candidate route formed by the last planned operation point and the current planned operation point.

[0132] Specifically, the electronic device first connects the last planned operation point and the current planned operation point to obtain a candidate route for controlling the movement of the manipulator, and then determines whether there is an obstacle point on the candidate route based on the posture matrix of each point on the candidate route in the base coordinate system. If there is an obstacle point on the candidate route, execute S691, otherwise, return to execute S630 to control the manipulator to move from the last planned operation point to the target point corresponding to the current planned operation point according to the candidate route.

[0133] S691. The electronic device fits a target route for controlling the movement of the robotic arm.

[0134] Specifically, the electronic device first determines a fitting point on the candidate route based on the first target posture and the second target posture, and then performs curve fitting on the current planned operation point, the previous planned operation point and the fitting point to generate the target route.

[0135] Furthermore, after the electronic device generates the target route, it returns to execute S630 to control the robot arm to move from the last planned operation point to the target point corresponding to the current planned operation point according to the target route.

[0136] The embodiment of the present disclosure also provides a work control device for implementing the above work control method, the device is configured in an electronic device in a work control system, the work control system also includes a mechanical arm, and a control module and an end tool are installed on the flange of the mechanical arm. Figure 7 In the embodiment of the present disclosure, the operation control device may be an electronic device. The electronic device may include a tablet computer, a desktop computer, a laptop computer, or other device with a communication function, or may include a device simulated by a virtual machine or a simulator.

[0137] Figure 7 A schematic structural diagram of a job control device provided by an embodiment of the present disclosure is shown.

[0138] like Figure 7 As shown, the operation control device 700 may include:

[0139] An acquisition module 710 is used to acquire a target pose of a planned operation point on the surface of a target object in a base coordinate system of the robot arm;

[0140] A control module 720 is used for controlling the robot arm to move from the last planned operation point to the current planned operation point based on the first target posture of the current planned operation point and the second target posture of the last planned operation point.

[0141] The operation module 730 is used to control the end tool to move from the current planned operation point to a preset operation depth to perform operation along an inward direction perpendicular to the surface of the target object through the control module.

[0142] A work control device of an embodiment of the present disclosure is an electronic device configured in a work control system. The work control system also includes a mechanical arm, and a control module and an end tool are installed on the flange of the mechanical arm. The method includes: obtaining the target posture of the planned work point on the surface of the target object in the base coordinate system of the mechanical arm; for the current planned work point in the planned work points, based on the first target posture of the current planned work point and the second target posture of the previous planned work point, controlling the mechanical arm to move from the previous planned work point to the current planned work point; controlling the end tool to move from the current planned work point to a preset work depth to perform work in a direction perpendicular to the surface of the target object inward through the control module. Thus, based on the target postures corresponding to the two adjacent planned work points, the mechanical arm can be controlled to move on the surface of the target object, and at each planned work point, the end tool can be controlled to move and work. In this way, there is no need to rely on the user's self-control, and the pre-planned work point is consistent with the actual work point, thereby improving the accuracy and safety of work control.

[0143] In some embodiments of the present disclosure, the operation control system further includes a camera; then, the acquisition module 710 includes:

[0144] A first acquisition unit, configured to acquire image data of a planned operation point on the surface of a target object, wherein the image data is collected by the camera;

[0145] The first determination unit is used to determine the target position and posture of the planned operation point in the base coordinate system based on the image data and pre-acquired calibration data.

[0146] In some embodiments of the present disclosure, the first determining unit is specifically configured to:

[0147] Performing image processing on the image data to determine the first position data of the planned operation point in the camera coordinate system;

[0148] Obtain the second pose data of the camera in the base coordinate system and the third pose data of the end tool in the flange coordinate system;

[0149] Determine the pose data of the flange in the base coordinate system according to the first pose data, the second pose data and the third pose data, and use the pose data of the flange in the base coordinate system as the target pose

[0150] In some embodiments of the present disclosure, the control module 720 includes:

[0151] A second acquisition unit, used to connect the last planned operation point and the current planned operation point to acquire a candidate route for controlling the movement of the robot arm;

[0152] A judging unit, used to judge whether there is an obstacle point on the candidate route that hinders the movement of the robot arm;

[0153] A second determining unit, configured to determine a fitting point on the candidate route based on the first target pose and the second target pose if the obstacle point exists on the candidate route;

[0154] A first generating unit, configured to perform curve fitting on the last planned operation point, the current planned operation point and the fitting point to generate a target route for controlling the movement of the robot arm;

[0155] The first control unit is used to control the robot arm to move from the last planned operation point to the current planned operation point according to the target route.

[0156] In some embodiments of the present disclosure, the control module 720 further includes:

[0157] The first control unit is used to control the robot arm to move from the last planned operation point to the current planned operation point according to the candidate route if the obstacle point does not exist on the candidate route.

[0158] In some embodiments of the present disclosure, the operation control system further includes a force sensor, which is installed at one end of the end tool, and the other end of the end tool is used for operation; then, the control module 720 includes:

[0159] A third control unit is used to control the robot arm to move from the last planned operation point to a target point corresponding to the current planned operation point based on the first target posture and the second target posture, wherein the target point is a preset point on a normal line formed by the current planned operation point on the surface of the target object;

[0160] A starting unit, used for starting the force sensor;

[0161] A third acquisition unit, configured to acquire force sensing data through the force sensor during the movement of the robot arm from the target point along the path close to the currently planned operation point;

[0162] A fourth control unit is used to control the robot arm to move from the target point to the currently planned operation point based on the force sensing data.

[0163] In some embodiments of the present disclosure, the fourth control unit is specifically configured to:

[0164] If the force sensing data is equal to a preset force threshold, determining that the robot arm has moved from the target point to the currently planned operation point, and controlling the robot arm to stop moving;

[0165] If the force sensing data is less than the preset force threshold, it is determined that the robotic arm has not moved from the target point to the currently planned operation point, and the robotic arm is controlled to continue moving until the force sensing data is equal to the preset force threshold.

[0166] In some embodiments of the present disclosure, the device further includes:

[0167] A first movement control module, configured to control the end tool to move to the currently planned operation point according to the preset operation depth through the control module in a direction perpendicular to the surface of the target object and outward;

[0168] The second movement control module is used to control the robot arm to move from the current planned operation point to the target point based on the first target posture.

[0169] In some embodiments of the present disclosure, the operation module 730 includes:

[0170] a fifth control unit, configured to control the end tool to move from the current planned operation point to the preset operation depth along a direction perpendicular to the surface of the target object inward at a preset speed through the control module;

[0171] The operation unit is used to control the end tool to operate according to the preset value at the preset operation depth through the control module.

[0172] It should be noted that Figure 7 The operation control device 700 shown can execute Figures 2 to 6 The various steps in the method embodiment shown in the figure are implemented Figures 1 to 6 The various processes and effects in the method embodiment shown are not described in detail here.

[0173] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0174] like Figure 8As shown, the electronic device may include a processor 801 and a memory 802 storing computer program instructions.

[0175] Specifically, the processor 801 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0176] The memory 802 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 802 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 802 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 802 is a non-volatile solid-state memory. In a particular embodiment, the memory 802 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.

[0177] The processor 801 reads and executes the computer program instructions stored in the memory 802 to perform the steps of the job control method provided in the embodiment of the present disclosure.

[0178] In one example, the electronic device may further include a transceiver 803 and a bus 804. Figure 8 As shown, the processor 801, the memory 802 and the transceiver 803 are connected via a bus 804 and communicate with each other.

[0179] The bus 804 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 804 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0180] The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure. The computer-readable storage medium and the job control method of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the computer-readable storage medium, reference can be made to the embodiment of the above-mentioned job control method.

[0181] This embodiment provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute a job control method. The method includes:

[0182] Obtaining a target pose of a planned operation point on the surface of a target object in a base coordinate system of the robotic arm;

[0183] For a current planned operation point among the planned operation points, based on a first target posture of the current planned operation point and a second target posture of a previous planned operation point, controlling the robot arm to move from the previous planned operation point to the current planned operation point;

[0184] The control module controls the end tool to move from the current planned operation point to a preset operation depth to perform operation along an inward direction perpendicular to the surface of the target object.

[0185] Of course, the storage medium containing computer executable instructions provided by the embodiment of the present disclosure, whose computer executable instructions are not limited to the above method operations, can also execute related operations in the job control method provided by any embodiment of the present disclosure.

[0186] Through the above description of the implementation methods, the technical personnel in the relevant field can clearly understand that the present disclosure can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer cloud platform (which can be a personal computer, a server, or a network cloud platform, etc.) to execute the job control method provided by each embodiment of the present disclosure.

[0187] Note that the above are only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A method for controlling a job, characterized in that: An electronic device applied to an operation control system, wherein the operation control system further comprises a mechanical arm, a control module and an end tool are mounted on a flange of the mechanical arm, and the method comprises: Obtaining a target pose of a planned operation point on the surface of a target object in a base coordinate system of the robotic arm; For a current planned operation point among the planned operation points, based on a first target posture of the current planned operation point and a second target posture of a previous planned operation point, controlling the robot arm to move from the previous planned operation point to the current planned operation point; The control module controls the end tool to move from the current planned operation point to a preset operation depth to perform operation along an inward direction perpendicular to the surface of the target object.

2. The method according to claim 1, characterized in that: The operation control system also includes a camera; Then, the step of obtaining the target position of the planned operation point on the surface of the target object in the base coordinate system of the robotic arm includes: Acquire image data of a planned operation point on the surface of a target object, wherein the image data is collected by the camera; Based on the image data and pre-acquired calibration data, a target position and posture of the planned operation point in the base coordinate system is determined.

3. The method according to claim 2, characterized in that The step of determining the target position of the planned operation point in the base coordinate system based on the image data and the pre-acquired calibration data includes: Performing image processing on the image data to determine the first position data of the planned operation point in the camera coordinate system; Obtain the second pose data of the camera in the base coordinate system and the third pose data of the end tool in the flange coordinate system; The pose data of the flange in the base coordinate system is determined according to the first pose data, the second pose data and the third pose data, and the pose data of the flange in the base coordinate system is used as the target pose.

4. The method according to claim 1, characterized in that: The controlling the robot arm to move from the last planned operation point to the current planned operation point based on the first target posture of the current planned operation point and the second target posture of the last planned operation point includes: Connecting the last planned operation point and the current planned operation point to obtain a candidate route for controlling the movement of the robotic arm; Determine whether there is an obstacle point on the candidate route that hinders the movement of the robotic arm; If the obstacle point exists on the candidate route, determining a fitting point on the candidate route based on the first target pose and the second target pose; Performing curve fitting on the last planned operation point, the current planned operation point and the fitting point to generate a target route for controlling the movement of the robotic arm; According to the target route, the robot arm is controlled to move from the last planned operation point to the current planned operation point.

5. The method according to claim 4, characterized in that Also includes: If the obstacle point does not exist on the candidate route, the robot arm is controlled to move from the last planned operation point to the current planned operation point according to the candidate route.

6. The method according to claim 1, characterized in that The operation control system further comprises a force sensor, the force sensor being mounted at one end of the end tool, and the other end of the end tool being used for operation; Then, based on the first target posture of the current planned operation point and the second target posture of the last planned operation point, controlling the robot arm to move from the last planned operation point to the current planned operation point includes: Based on the first target posture and the second target posture, controlling the robot arm to move from the last planned operation point to the target point corresponding to the current planned operation point, wherein the target point is a preset point on a normal line formed by the current planned operation point on the surface of the target object; activating the force sensor; Acquiring force sensing data through the force sensor during the movement of the robot arm from the target point to the currently planned operation point; Based on the force sensing data, the robot arm is controlled to move from the target point to the currently planned operation point.

7. The method according to claim 6, characterized in that The controlling the robot arm to move from the target point to the currently planned operation point based on the force sensing data includes: If the force sensing data is equal to a preset force threshold, determining that the robot arm has moved from the target point to the currently planned operation point, and controlling the robot arm to stop moving; If the force sensing data is less than the preset force threshold, it is determined that the robotic arm has not moved from the target point to the currently planned operation point, and the robotic arm is controlled to continue moving until the force sensing data is equal to the preset force threshold.

8. The method according to claim 6, characterized in that Also includes: By means of the control module, the end tool is controlled to move to the currently planned operation point according to the preset operation depth in an outward direction perpendicular to the surface of the target object; Based on the first target posture, the robot arm is controlled to move from the currently planned operation point to the target point.

9. The method according to claim 1, characterized in that: The control module controls the end tool to move from the current planned operation point to a preset operation depth to perform the operation along a direction perpendicular to the surface of the target object and inward, including: By means of the control module, the end tool is controlled to move from the currently planned operation point to the preset operation depth at a preset speed along an inward direction perpendicular to the surface of the target object; At the preset working depth, the control module controls the end tool to operate according to the preset value.

10. A work control device, characterized in that: An electronic device configured in an operation control system, wherein the operation control system further comprises a mechanical arm, a control module and an end tool are mounted on a flange of the mechanical arm, and the device comprises: An acquisition module, used for acquiring a target pose of a planned operation point on the surface of a target object in a base coordinate system of the robotic arm; A control module, for controlling the robot arm to move from the last planned operation point to the current planned operation point based on a first target posture of the current planned operation point and a second target posture of the last planned operation point; The operation module is used to control the end tool to move from the current planned operation point to a preset operation depth to perform operation along an inward direction perpendicular to the surface of the target object through the control module.

11. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the method described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the method according to any one of claims 1 to 9.

Citation Information

Patent Citations

  • Automatic injection control device and control method thereof

    CN111991653A

  • Tooth preparation system

    CN112790885A

  • Measurement path planning method based on laser guidance

    CN113246142A

  • Surgical system based on structured light registration, registration control method and storage medium

    CN115252127A

  • Nail hole position positioning method for industrial robot nail screwing

    CN116834000A