Mechanical arm control method and device for camera assembly

Abstract

The invention provides a mechanical arm control method and device for camera assembly. The method comprises the steps that to-be-shot part information and preparation position information of a mechanical arm are obtained; target pose information of the mechanical arm is determined based on the to-be-shot part information; based on the preparation position information and the target pose information of the mechanical arm, the motion path of the mechanical arm is planned; and controlling the mechanical arm to execute the motion path. In the application, the camera is assembled on the mechanical arm, so that the position of the camera with a relatively good view is obtained during an operation, and the mechanical arm is controlled to move to the corresponding position, so that the good view of the operation is obtained, and the visual effect and the execution effect of the operation are improved.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a method and device for controlling a camera-mounted robotic arm. Background Art

[0002] Surgical robot-assisted orthopedic surgery is an important application of orthopedic surgical robots. It completes complex orthopedic surgeries through preoperative planning, intraoperative image acquisition and actual execution by robotic arms.

[0003] However, existing surgical robots generally obtain intraoperative images through fixed camera equipment set at a preset distance during surgery. Since such images are taken at a fixed position, it is difficult to obtain the best field of view of the surgical part. Summary of the invention

[0004] The problem addressed by this application is that it is currently difficult to obtain a good view of the surgical part.

[0005] In order to solve the above problems, the first aspect of the present application provides a robot arm control method for camera assembly, which includes:

[0006] Obtain information about the part to be photographed and the preparation position information of the robotic arm;

[0007] Determine the target position information of the robot arm based on the information of the part to be photographed;

[0008] Plan the motion path of the robot arm based on the robot arm's preparation position information and target position information;

[0009] The robot arm is controlled to execute the motion path.

[0010] The second aspect of the present application provides a manufacturing system of a camera assembly robot arm control method, which includes:

[0011] An information acquisition module, which is used to obtain information about the part to be photographed and information about the preparation position of the robot arm;

[0012] A target determination module, which is used to determine the target position information of the robot arm based on the information of the part to be photographed;

[0013] A path planning module, which is used to plan the motion path of the robot arm based on the preparation position information and target posture information of the robot arm;

[0014] A robot arm control module is used to control the robot arm to execute the motion path.

[0015] The third aspect of the present application provides an electronic device, comprising: a memory and a processor; the memory may be configured to store a program, and the processor is coupled to the memory and is used to execute the program in the memory, so as to:

[0016] Obtain information about the part to be photographed and the preparation position information of the robotic arm;

[0017] Determine the target position information of the robot arm based on the information of the part to be photographed;

[0018] Plan the motion path of the robot arm based on the robot arm's preparation position information and target position information;

[0019] The robot arm is controlled to execute the motion path.

[0020] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the aforementioned camera assembly robotic arm control method.

[0021] In the present application, by installing a camera on a robotic arm, a camera position with a better field of view is obtained during surgery, and the robotic arm is controlled to move to a corresponding position, thereby achieving a good field of view for surgery and improving the visual effect and execution effect of the surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a robotic arm of a robotic arm control method for a camera assembly according to an embodiment of the present application;

[0023] Figure 2 A flowchart of a method for controlling a robotic arm of a camera assembly according to an embodiment of the present application;

[0024] Figure 3 A schematic diagram of a robotic arm control device equipped with a camera according to an embodiment of the present application;

[0025] Figure 4 is an architectural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below in conjunction with the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0027] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0028] The present application embodiment provides the above-mentioned camera assembly robot arm control method, the specific scheme of the method is as follows: Figure 1-Figure 2 As shown, the method can be performed by a camera-mounted robotic arm control device, and the camera-mounted robotic arm control device can be integrated into electronic devices such as computers, servers, computers, server clusters, and data centers. Figure 1 As shown, the method for controlling a robotic arm of the camera assembly comprises:

[0029] S101, obtaining information of the part to be photographed and the preparation position information of the robot arm;

[0030] In the present application, the information of the part to be photographed refers to the information of the target position or area that the camera needs to aim at or photograph, and the information can be three-dimensional scanning information, image information or other forms of information.

[0031] In the present application, the ready position information of the robot arm is the initial position of the robot arm or its standby position and posture data.

[0032] S102, determining target posture information of the robot arm based on the information of the part to be photographed;

[0033] In this application, after knowing the "information of the part to be photographed", the system will calculate the target posture (including position and posture) that the end effector of the robotic arm (i.e., the location of the camera) needs to achieve based on this information.

[0034] Combined with the coordinates of the target part in the workspace, determine the relative position of the end of the robot arm in space. According to the shooting requirements, such as the field of view direction, pitch angle, etc., calculate the direction or angle that the camera needs to maintain.

[0035] S103, planning a motion path of the robotic arm based on the preparation position information and target position information of the robotic arm;

[0036] S104, controlling the robotic arm to execute the motion path.

[0037] In the present application, according to the planned motion trajectory, the robot arm is controlled to move joint by joint to execute the path.

[0038] During the execution phase, the system needs to monitor the real-time status of the robot (position, speed, torque, etc.) to ensure the accuracy and safety of the movement. If errors or interference occur during the execution process, the controller also needs to fine-tune or re-plan the trajectory in real time.

[0039] In the present application, by installing a camera on a robotic arm, a camera position with a better field of view is obtained during surgery, and the robotic arm is controlled to move to a corresponding position, thereby achieving a good field of view for surgery and improving the visual effect and execution effect of the surgery.

[0040] In a specific implementation, the step S102, determining the target posture information of the robot arm based on the information of the part to be photographed, includes:

[0041] Obtaining preoperative planning mapping information of the part to be photographed;

[0042] Based on the information of the part to be photographed and the preoperative planning mapping information, the part to be photographed is divided into a central part and an opening part;

[0043] Based on the preoperative planning mapping information of the central part, the fixed point information is fitted;

[0044] Based on the information of the part to be photographed in the opening, the central ray information is obtained by fitting;

[0045] Based on the fixed point information and the central ray information, the target position information of the robotic arm is determined.

[0046] In the present application, the part to be photographed has preoperative planning information; mapping the preoperative planning information from the preoperative coordinate system to the intraoperative coordinate system is the preoperative planning mapping information.

[0047] In the present application, the specific coordinate systems of the preoperative coordinate system and the intraoperative coordinate system can be determined according to actual needs.

[0048] Preoperative planning mapping information may include: a three-dimensional reconstructed model of the part to be operated or photographed, important anatomical landmarks (such as bony markers, reference fiducials), and other instructions required for alignment or positioning.

[0049] Central part: usually the most important core area in the operation, such as the osteotomy area, osteotomy surface, etc.; the features of this part are not obvious or cannot be observed visually, so auxiliary observation is required through the opening.

[0050] Opening part: It may be the outer edge area adjacent to the central part or surrounding the center, and can also be understood as an open or unfolded part, through which the shooting angle, direction, etc. required by the robotic arm can be determined.

[0051] The fixed point information may be the geometric center point of the central part, or the information of the characteristic mark point. The information may be the fixed point coordinates.

[0052] Geometric center point: The geometric center calculated by an algorithm on a complex structure.

[0053] Characteristic landmarks: For example, in orthopedic surgery, they may be the reference center of a bone plate, the osteotomy center, etc.

[0054] The fitting method of the fixed point may be to fit multiple known preoperative landmarks or surface features, and finally extract a fixed point to guide the positioning of the robotic arm.

[0055] In this application, geometric analysis or feature extraction is performed on the information of the opening part to obtain this ray (usually the main direction of the fitted straight line). This main direction (center ray) will determine the direction of the end effector (camera) of the robot arm to ensure the correct viewing angle of the shooting.

[0056] The target position information of the robot arm is determined based on the fixed point information and the central ray information, which can be specifically: the direction of the camera is opposite to the central ray and points to the fixed point. The optical axis of the camera can be roughly coincident with the ray or maintain a certain angle to achieve the best observation posture.

[0057] Preferably, the distance between the camera and the fixed point is within a preset range to ensure the accuracy of the captured video or image frame.

[0058] In a specific implementation, the central ray information is the best viewing direction information in an unobstructed situation.

[0059] The meaning of unobstructed shooting angle: If the shooting direction of the camera is blocked, it will lead to unclear vision or blind spots. To this end, it is necessary to determine a direction that can be fully observed in the central area and the overlap between this direction and the surrounding obstacles or tissues is minimal. In this way, the calculation of the overlap can be used as a fitting method for the central ray information.

[0060] It should be noted here that the optimal field of view direction is a range-based regulation, which does not mean that there is only one direction, but that the optimal field of view can be regarded as within a certain range of this direction.

[0061] In a specific implementation, the step S103, planning the motion path of the robot arm based on the preparation position information and the target position information of the robot arm, includes:

[0062] Acquire a plurality of preset assumed target position information; each of the assumed target position information corresponds to a plurality of preset paths starting from the assumed preparation position;

[0063] Mapping the plurality of assumed targets as information to the current coordinate system;

[0064] Selecting the closest assumed target position information based on the central ray information;

[0065] One is selected from a plurality of preset paths corresponding to the assumed target position information as the motion path of the robot arm.

[0066] In this application, each assumed target position information generally includes: position: the position coordinates of the end effector of the robotic arm relative to a reference coordinate system (such as the operating table coordinate system, the workbench coordinate system); posture: the orientation of the end effector of the robotic arm (or camera), such as the rotation angle around the X, Y, and Z axes.

[0067] In the present application, for each assumed target position information, several feasible motion trajectories can be planned in advance based on the assumed preparation position and stored.

[0068] In this application, these preset paths are often calculated in advance offline or in a simulation environment to ensure: accessibility: the robot arm will not exceed the joint limit in the path. Safety: the trajectory avoids known obstacles or sensitive areas. Smoothness: the acceleration and acceleration rate of the trajectory are within a reasonable range.

[0069] In this application, during actual execution, the position and posture of the robot arm in the current coordinate system (such as the patient's body relative to the operating room coordinate system, the workpiece relative to the production line coordinate system) may be inconsistent with the assumed scene. Therefore, coordinate transformation or alignment operations are required to complete the mapping.

[0070] In the present application, mapping the information of the plurality of assumed targets to the current coordinate system is to obtain the real-time coordinates and coordinate system of the plurality of assumed targets when the assumed preparation position and the preparation position of the robot arm are overlapped.

[0071] In this application, the closest assumed target position information is selected, where the closest refers to close in position: the distance between the target position and the end space point (or area) defined by the central ray information is the smallest, and close in direction: the angle difference between the direction of the target position (camera optical axis) and the central ray is the smallest.

[0072] In the present application, an optimization function or distance metric can be obtained by combining the two factors of position and orientation, and then the one that best meets the current central ray information requirements is selected from all candidate assumed target positions.

[0073] In this application, by planning in advance and selecting in real time, the amount of data calculation for path planning can be greatly reduced and the speed of path planning can be greatly improved.

[0074] In this application, it should be noted that the pre-planning and temporary selection method cannot guarantee the most perfect path planning, but there is no significant difference between the most perfect path and the sub-perfect path. The impact caused by this defect is far less than the advantage brought by the increased path planning speed.

[0075] In this application, by presetting a typical path offline, only nearest neighbor matching and simple coordinate transformation are required online, which is more efficient than performing large-scale searches or complex real-time planning each time.

[0076] In this application, the selected "assumed target position information" and its corresponding path have been fully verified in the design and testing phases, reducing the unforeseen risks that may be caused by online planning.

[0077] In a specific implementation, the selecting one of the plurality of preset paths corresponding to the assumed target position information as the motion path of the robot arm includes:

[0078] Acquire multiple preset paths corresponding to the assumed target position information;

[0079] Generate a path coverage space according to a preset path;

[0080] Get current obstacle information;

[0081] The preset path is screened based on the obstacle information and the path coverage space to obtain the motion path of the robotic arm.

[0082] In the present application, after determining a piece of assumed target position information that is closest to the current central ray information, a plurality of pre-stored preset paths associated with the target position information may be retrieved.

[0083] In the present application, the path coverage space is the set of areas "swept" by the robot body (including the end effector and various joint parts) in three-dimensional space when the robot arm executes the preset path.

[0084] The path coverage space generation method can be

[0085] Discrete sampling method: The preset trajectory is sampled discretely at equal intervals in time or joint space; for each sampling point, the position of each joint of the robot is substituted into its CAD model or collision model to obtain the volume or boundary occupied by the robot in the three-dimensional space at this time.

[0086] Merge and dilate: Merge the volumes or geometric boundaries corresponding to all sampling points to form a continuous "3D coverage area".

[0087] Obstacle information can be obtained through sensors, or through depth cameras to construct a three-dimensional environment and a spatial model of obstacles.

[0088] In this application, the obstacle information is matched with the coverage space corresponding to each preset path to perform collision detection or overlapping area judgment: Collision detection: If the coverage space overlaps with any obstacle, it means that the preset path will collide with or get too close to the obstacle during execution, thereby determining that the path is infeasible or unsafe. Collision distance measurement: In some situations with high security requirements, it is necessary not only to determine whether there is a collision, but also whether the minimum safety gap meets the requirements. If the distance between a path and an obstacle is less than the set threshold, it will also be considered unsafe. Perform the above collision and safety gap tests on all optional preset paths, eliminate infeasible paths, and finally obtain a set of feasible paths.

[0089] When multiple paths meet the requirements of no collision or safe distance, other factors can be considered for further optimization, such as: path length / movement time: select the path with the shortest execution time or the lowest energy consumption; smoothness: select the path with the smallest acceleration and impact to reduce disturbance to the robot arm and the environment; operational convenience: some paths may have a better posture at the target position, making subsequent tasks (such as shooting and detection) easier to perform.

[0090] In this application, since the preset path has been calculated and verified offline in the system, only coordinate transformation and collision detection need to be performed online, which greatly shortens the path planning time.

[0091] In a specific implementation, the preset path is a path that satisfies a telecentricity constraint with fixed point information as the telecentricity.

[0092] Telecentric constraint: When the camera (or end effector) moves, it always adjusts its posture or position around the fixed point so that the line of sight from the camera to the fixed point remains constant or satisfies specific constraints.

[0093] The generation of the preset path can be as follows: setting the coordinates of a fixed point between the preparation position and the target position; using inverse kinematics combined with the "line of sight" algorithm to calculate the angle interpolation of each joint of the robot point by point; verifying the path process to ensure that the optical axis of the camera always passes through the fixed point and there is no obstacle collision; and recording the trajectory data that meets the telecentric constraint as the preset path.

[0094] In a specific implementation, the plurality of assumed target position information are evenly distributed.

[0095] In this way, typical scenarios and directions can be better covered, avoiding the occurrence of "no available posture" or "unbalanced posture distribution" during online planning.

[0096] An embodiment of the present application provides a camera assembly robot arm control device, which is used to execute the camera assembly robot arm control method described in the above content of the present application. The camera assembly robot arm control device is described in detail below.

[0097] like Figure 3 As shown, the robotic arm control device of the camera assembly includes:

[0098] An information acquisition module 101 is used to acquire information about the part to be photographed and information about the preparation position of the robot arm;

[0099] A target determination module 102 is used to determine target position information of the robot arm based on the information of the part to be photographed;

[0100] A path planning module 103, which is used to plan the motion path of the robot arm based on the preparation position information and the target position information of the robot arm;

[0101] The robot arm control module 104 is used to control the robot arm to execute the motion path.

[0102] In one embodiment, the target determination module 102 is further configured to:

[0103] Acquire preoperative planning mapping information of the part to be photographed; based on the information of the part to be photographed and the preoperative planning mapping information, divide the part to be photographed into a central part and an opening part; based on the preoperative planning mapping information of the central part, fit the fixed point information; based on the information of the part to be photographed of the opening part, fit the central ray information; based on the fixed point information and the central ray information, determine the target posture information of the robotic arm.

[0104] In one implementation, the central ray information is optimal field of view direction information in an unobstructed condition.

[0105] In one implementation, the path planning module 103 is further configured to:

[0106] Acquire multiple preset assumed target position information; each assumed target position information corresponds to multiple preset paths starting from the assumed preparation position; map the multiple assumed target information to the current coordinate system; select the closest assumed target position information based on the central ray information; select one from the multiple preset paths corresponding to the assumed target position information as the movement path of the robot arm.

[0107] In one implementation, the path planning module 103 is further configured to:

[0108] Acquire multiple preset paths corresponding to the assumed target position information; generate a path coverage space according to the preset paths; acquire current obstacle information; and screen the preset paths based on the obstacle information and the path coverage space to obtain the motion path of the robotic arm.

[0109] In one implementation, the preset path is a path that satisfies a telecentricity constraint with fixed point information as the telecentricity.

[0110] In one implementation, the plurality of assumed target bit information are evenly distributed.

[0111] The camera-mounted robotic arm control device provided in the above-mentioned embodiment of the present application has a corresponding relationship with the camera-mounted robotic arm control method provided in the embodiment of the present application. Therefore, the specific content in the system has a corresponding relationship with the camera-mounted robotic arm control method. The specific content can refer to the records in the camera-mounted robotic arm control method, and will not be repeated in this application.

[0112] The camera-mounted robotic arm control device provided in the above-mentioned embodiment of the present application and the camera-mounted robotic arm control method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0113] The above describes the internal functions and structure of the camera-mounted robotic arm control device, such as Figure 4 As shown, in practice, the robot arm control device equipped with the camera can be realized as an electronic device, including: a memory 301 and a processor 303.

[0114] The memory 301 may be configured to store programs.

[0115] In addition, the memory 301 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0116] The memory 301 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The processor 303, coupled to the memory 301, is used to execute the program in the memory 301, so as to:

[0117] Obtain information about the part to be photographed and the preparation position information of the robotic arm;

[0118] Determine the target position information of the robot arm based on the information of the part to be photographed;

[0119] Plan the motion path of the robot arm based on the robot arm's preparation position information and target position information;

[0120] The robot arm is controlled to execute the motion path.

[0121] In one implementation, the processor 303 is further configured to:

[0122] Acquire preoperative planning mapping information of the part to be photographed; based on the information of the part to be photographed and the preoperative planning mapping information, divide the part to be photographed into a central part and an opening part; based on the preoperative planning mapping information of the central part, fit the fixed point information; based on the information of the part to be photographed of the opening part, fit the central ray information; based on the fixed point information and the central ray information, determine the target posture information of the robotic arm.

[0123] In one implementation, the central ray information is optimal field of view direction information in an unobstructed condition.

[0124] In one implementation, the processor 303 is further configured to:

[0125] Acquire multiple preset assumed target position information; each assumed target position information corresponds to multiple preset paths starting from the assumed preparation position; map the multiple assumed target information to the current coordinate system; select the closest assumed target position information based on the central ray information; select one from the multiple preset paths corresponding to the assumed target position information as the movement path of the robot arm.

[0126] In one implementation, the processor 303 is further configured to:

[0127] Acquire multiple preset paths corresponding to the assumed target position information; generate a path coverage space according to the preset paths; acquire current obstacle information; and screen the preset paths based on the obstacle information and the path coverage space to obtain the motion path of the robotic arm.

[0128] In one implementation, the preset path is a path that satisfies a telecentricity constraint with fixed point information as the telecentricity.

[0129] In one implementation, the plurality of assumed target bit information are evenly distributed.

[0130] In this application, Figure 4 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 4 Components shown.

[0131] The electronic device provided in this embodiment is based on the same inventive concept as the camera-mounted robotic arm control method provided in the embodiment of the present application, and has the same beneficial effects as the method adopted, run or implemented by the application stored therein.

[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory (Flash RAM). The memory is an example of a computer-readable medium.

[0136] The present application also provides a computer-readable storage medium corresponding to the camera-mounted robotic arm control method provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the interactive image analysis auxiliary method for 3D aerial imaging provided in any of the aforementioned embodiments.

[0137] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0138] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the interactive image analysis auxiliary method for 3D aerial imaging provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0139] It should be noted that in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0140] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0141] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A camera-equipped robot arm control method, used to control the end tool position of a robot arm equipped with a camera, characterized in that: include: Obtain information about the part to be photographed and the preparation position information of the robotic arm; Determine the target position information of the robot arm based on the information of the part to be photographed; Plan the motion path of the robot arm based on the robot arm's preparation position information and target position information; The robot arm is controlled to execute the motion path.

2. The camera assembly robot control method according to claim 1, characterized in that: The step of determining the target position information of the robot arm based on the information of the part to be photographed includes: Obtaining preoperative planning mapping information of the part to be photographed; Based on the information of the part to be photographed and the preoperative planning mapping information, the part to be photographed is divided into a central part and an opening part; Based on the preoperative planning mapping information of the central part, the fixed point information is fitted; Based on the information of the part to be photographed in the opening, the central ray information is obtained by fitting; Based on the fixed point information and the central ray information, the target position information of the robotic arm is determined.

3. The camera assembly robot control method according to claim 2, characterized in that: The central ray information is the best viewing direction information when there is no obstruction.

4. The camera assembly robot control method according to claim 2, characterized in that: The step of planning the motion path of the robotic arm based on the preparation position information and the target position information of the robotic arm comprises: Acquire a plurality of preset assumed target position information; each of the assumed target position information corresponds to a plurality of preset paths starting from the assumed preparation position; Mapping the plurality of assumed targets as information to the current coordinate system; Selecting the closest assumed target position information based on the central ray information; One is selected from a plurality of preset paths corresponding to the assumed target position information as the motion path of the robot arm.

5. The camera assembly robot control method according to claim 4, characterized in that: The step of selecting one of the plurality of preset paths corresponding to the assumed target position information as the motion path of the robot arm comprises: Acquire multiple preset paths corresponding to the assumed target position information; Generate a path coverage space according to a preset path; Get current obstacle information; The preset path is screened based on the obstacle information and the path coverage space to obtain the motion path of the robotic arm.

6. The camera assembly robot control method according to claim 4, characterized in that: The preset path is a path that satisfies the telecentricity constraint with the fixed point information as the telecentricity.

7. The camera assembly robot arm control method according to claim 4, characterized in that: The plurality of assumed target bit information are evenly distributed.

8. A camera-mounted robotic arm control device, characterized in that: include: An information acquisition module, which is used to obtain information about the part to be photographed and information about the preparation position of the robot arm; A target determination module, which is used to determine the target position information of the robot arm based on the information of the part to be photographed; A path planning module, which is used to plan the motion path of the robot arm based on the preparation position information and target posture information of the robot arm; A robot arm control module is used to control the robot arm to execute the motion path.

9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Obtain information about the part to be photographed and the preparation position information of the robotic arm; Determine the target position information of the robot arm based on the information of the part to be photographed; Plan the motion path of the robot arm based on the robot arm's preparation position information and target position information; The robot arm is controlled to execute the motion path.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the camera assembly robotic arm control method described in any one of claims 1-7.

Images