A method and device for controlling a robotic arm that assembles a camera

Abstract

This application provides a method and apparatus for controlling a robotic arm equipped with a camera. The method includes: acquiring information about the area to be photographed and the robotic arm's ready position information; determining the target pose information of the robotic arm based on the area to be photographed information; planning the motion path of the robotic arm based on the robotic arm's ready position information and the target pose information; and controlling the robotic arm to execute the motion path. In this application, by mounting a camera on a robotic arm, a camera position with a better field of view can be obtained during surgery, and the robotic arm can be controlled to move to the corresponding position, thereby achieving a good field of view during surgery and improving the visualization and execution effects of the surgery.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and more specifically, to a method and apparatus for controlling a robotic arm equipped with a camera. Background Technology

[0002] Surgical robot-assisted orthopedic surgery is an important application of orthopedic surgical robots. It completes complex orthopedic surgeries through preoperative planning, intraoperative image acquisition, and actual execution by robotic arms.

[0003] However, existing surgical robots typically acquire intraoperative images through fixed cameras positioned at a preset distance. Because these images are taken from a fixed location, it is difficult to obtain the best field of view of the surgical area. Summary of the Invention

[0004] The problem addressed by this application is the difficulty in obtaining a good view of the surgical area.

[0005] To address the aforementioned problems, the first aspect of this application provides a method for controlling a robotic arm equipped with a camera, comprising:

[0006] Obtain information about the part to be photographed and the robotic arm's ready position;

[0007] Determine the target pose information of the robotic arm based on the information of the part to be photographed;

[0008] Based on the robotic arm's ready position information and the target pose information, the motion path of the robotic arm is planned;

[0009] Control the robotic arm to execute the motion path.

[0010] The second aspect of this application provides a manufacturing system for a camera-equipped robotic arm control method, comprising:

[0011] The information acquisition module is used to acquire information about the part to be photographed and the robotic arm's ready position information;

[0012] The target determination module is used to determine the target pose information of the robotic arm based on the information of the part to be photographed.

[0013] The path planning module is used to plan the motion path of the robotic arm based on the robotic arm's ready position information and target pose information;

[0014] A robotic arm control module is used to control the robotic arm to execute the motion path.

[0015] A third aspect of this application provides an electronic device, including: a memory and a processor; the memory being configurable to store a program, and the processor being coupled to the memory for executing the program in the memory for:

[0016] Obtain information about the part to be photographed and the robotic arm's ready position;

[0017] Determine the target pose information of the robotic arm based on the information of the part to be photographed;

[0018] Based on the robotic arm's ready position information and the target pose information, the motion path of the robotic arm is planned;

[0019] Control the robotic arm to execute the motion path.

[0020] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the aforementioned robotic arm control method for camera assembly.

[0021] In this application, by mounting a camera on a robotic arm, a camera position with a better field of view can be obtained during surgery, and the robotic arm can be controlled to move to the corresponding position, thereby achieving a good field of view during surgery and improving the visual effect and execution effect of the surgery. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a robotic arm with a camera assembly according to an embodiment of this application;

[0023] Figure 2 This is a flowchart of a robotic arm control method for a camera assembly according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a robotic arm control device equipped with a camera according to an embodiment of this application;

[0025] Figure 4 This is an architectural diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0027] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0028] This application provides a robotic arm control method for camera assembly as described above. The specific solution of this method is as follows: Figures 1-2 As shown, this method can be executed by a robotic arm control device mounted on a camera, which can be integrated into electronic devices such as computers, servers, server clusters, and data centers. Combined with... Figure 1 As shown, the robotic arm control method for the camera assembly includes:

[0029] S101, acquire information about the part to be photographed and the robotic arm's ready position information;

[0030] In this application, the information of the part to be photographed refers to the information of the target position or area that the camera needs to be aimed at or photographed. This information can be three-dimensional scanning information, image information or other forms of information.

[0031] In this application, the ready position information of the robotic arm is the current initial position or standby position and attitude data of the robotic arm.

[0032] S102, Determine the target pose information of the robotic arm based on the information of the part to be photographed;

[0033] In this application, after the "information of the part to be photographed" is known, the system will calculate the target pose (including position and attitude) that the robotic arm end effector (i.e. the location of the camera) needs to achieve based on this information.

[0034] By combining the coordinates of the target area in the workspace, the relative position of the robotic arm's end effector in space is determined. Based on the shooting requirements, such as the field of view direction and pitch angle, the required orientation or angle of the camera is calculated.

[0035] S103, based on the robotic arm's ready position information and target pose information, plans the robotic arm's motion path;

[0036] S104, control the robotic arm to execute the motion path.

[0037] In this application, the robotic arm is controlled to move joint by joint according to the planned motion trajectory, and the path is executed.

[0038] During the execution phase, the system needs to monitor the real-time status of the robotic arm (position, speed, torque, etc.) to ensure the accuracy and safety of the movement. If errors or interference occur during execution, the controller also needs to fine-tune or replan the trajectory in real time.

[0039] In this application, by mounting a camera on a robotic arm, a camera position with a better field of view can be obtained during surgery, and the robotic arm can be controlled to move to the corresponding position, thereby achieving a good field of view during surgery and improving the visual effect and execution effect of the surgery.

[0040] In one specific embodiment, step S102, determining the target pose information of the robotic arm based on the information of the part to be photographed, includes:

[0041] Obtain preoperative planning and mapping information for the area to be photographed;

[0042] Based on the information of the area to be photographed and the preoperative planning mapping information, the area to be photographed is divided into a central part and an opening part;

[0043] Based on the preoperative planning and mapping information of the central part, fixed point information is obtained by fitting.

[0044] Based on the information of the part to be photographed at the opening, the information of the central ray is obtained by fitting.

[0045] Based on the fixed point information and the central ray information, the target pose information of the robotic arm is determined.

[0046] In this application, the area to be photographed has preoperative planning information; mapping this preoperative planning information from the preoperative coordinate system to the intraoperative coordinate system is the preoperative planning mapping information.

[0047] In this application, the specific coordinate systems of the preoperative and intraoperative coordinate systems can be determined based on actual needs.

[0048] Preoperative planning mapping information may include: a three-dimensional reconstruction model of the area to be operated on or photographed, important anatomical landmarks (such as bony markers, reference points), and other instructions required for alignment or localization.

[0049] Central part: This is usually the most important core area during surgery, such as the osteotomy area and osteotomy surface. The features of this part are not obvious or cannot be observed visually, so it is necessary to make an incision for auxiliary observation.

[0050] Opening section: This may be the outer edge area adjacent to or surrounding the center section. It can also be understood as the open or unfolded part. It can be used to determine the shooting angle, orientation, etc. required by the robotic arm.

[0051] Fixed-point information can be the geometric center point of the central part, or information about feature marker points. This information can be the coordinates of the fixed point.

[0052] Geometric center point: The geometric center obtained by algorithm calculation on a complex structure.

[0053] Characteristic landmarks: such as the reference center of a bone plate or the osteotomy center in orthopedic surgery.

[0054] The fitting method for the fixed point can be to fit multiple known preoperative landmarks or surface features and finally extract a fixed point to guide the positioning of the robotic arm.

[0055] In this application, geometric analysis or feature extraction is performed on the information of the opening to obtain this ray (usually the principal direction of the fitted straight line). This principal direction (central ray) will determine the orientation of the robotic arm's end effector (camera) to ensure the correct shooting angle.

[0056] Specifically, based on the fixed point information and the central ray information, the target pose information of the robotic arm is determined. This can be achieved by aligning the camera with the central ray, pointing it towards the fixed point. The camera's optical axis can be roughly aligned with or at a certain angle to the ray to achieve the optimal observation posture.

[0057] Preferably, the distance between the camera and the fixed point is within a preset range to ensure the accuracy of the captured video or image frames.

[0058] In one specific implementation, the central ray information is the optimal field of view direction information under unobstructed conditions.

[0059] The meaning of an unobstructed shooting perspective: If the camera's shooting direction is obstructed, it will lead to an unclear field of view or blind spots. Therefore, it is necessary to determine a direction that can be fully observed in the central area, and that this direction has minimal overlap with surrounding obstructions or tissues. In this way, the calculation of overlap can be used as a method for fitting the central ray information.

[0060] It should be noted that the optimal field of view is a range-based definition, not that there is only one direction, but that the optimal field of view can be considered within a certain range of that direction.

[0061] In one specific embodiment, step S103, based on the robotic arm's ready position information and target pose information, plans the robotic arm's motion path, including:

[0062] Obtain multiple preset hypothetical target position information; each hypothetical target position information corresponds to multiple preset paths starting from the hypothetical preparation position;

[0063] Map the multiple assumed targets as information to the current coordinate system;

[0064] Based on the central ray information, select the closest hypothetical target position information;

[0065] Select one of the multiple preset paths corresponding to the assumed target position information as the motion path of the robotic arm.

[0066] In this application, each assumed target position information typically includes: position: the position coordinates of the robotic arm end effector relative to a certain reference coordinate system (such as the operating table coordinate system or the worktable coordinate system); attitude: the orientation of the robotic arm end effector (or camera), such as the rotation angle around the X, Y, and Z axes.

[0067] In this application, for each assumed target position information, several feasible motion trajectories can be planned in advance based on the assumed preparation position and stored.

[0068] In this application, these preset paths are often calculated in advance offline or in a simulation environment to ensure: Accessibility: the robotic arm will not exceed its joint limits in the path. Safety: the trajectory avoids known obstacles or sensitive areas. Smoothness: the acceleration and rate of change of acceleration of the trajectory are both within a reasonable range.

[0069] In this application, during actual execution, the position and orientation of the robotic arm in the current coordinate system (such as the patient's body relative to the operating room coordinate system, or the workpiece relative to the production line coordinate system) may not be consistent with the assumed scenario. Therefore, coordinate transformation or alignment operations are required to complete the mapping.

[0070] In this application, mapping multiple hypothetical targets to the current coordinate system means that, when the hypothetical preparation position and the robotic arm's preparation position coincide, the real-time coordinates and coordinate system of the multiple hypothetical targets are obtained.

[0071] In this application, the closest assumed target position information is selected. The closest means that the target position is closest in terms of location: the distance between the target position and the end space point (or region) defined by the central ray information is the smallest, and the closest means that the angle difference between the orientation of the target pose (camera optical axis) and the central ray is the smallest.

[0072] In this application, by combining the two factors of location and orientation, an optimization function or distance metric can be obtained, and then the one that best meets the requirements of the current central ray information can be selected from all candidate hypothetical target locations.

[0073] In this application, by planning in advance and selecting in real time, the amount of data calculation for path planning can be significantly reduced, and the speed of path planning can be greatly improved.

[0074] It should be noted in this application that the method of pre-planning and ad-hoc selection cannot guarantee the most perfect path planning. However, there is no significant difference between the most perfect path and the second most perfect path. The impact of this defect is far less than the advantage brought by the improvement of path planning speed.

[0075] In this application, by pre-setting typical paths offline, only nearest neighbor matching and simple coordinate transformation are needed online, which is more efficient than performing large-scale searches or complex real-time planning each time.

[0076] In this application, the selected "assumed target location information" and its corresponding path have been fully verified during the design and testing phases, reducing unforeseen risks that may arise from online planning.

[0077] In one specific implementation, selecting one of a plurality of preset paths corresponding to the assumed target location information as the motion path of the robotic arm includes:

[0078] Obtain multiple preset paths corresponding to the assumed target location information;

[0079] Generate a path coverage space based on the preset path;

[0080] Obtain current obstacle information;

[0081] The motion path of the robotic arm is obtained by filtering the preset path based on obstacle information and the path coverage space.

[0082] In this application, once a hypothetical target position information that is closest to the current central ray information is determined, multiple preset paths associated with that target position information can be retrieved from pre-stored data.

[0083] In this application, the path coverage space is the set of areas "sweeped" by the robotic arm (including the end effector and each joint) in three-dimensional space during the execution of the preset path.

[0084] The path coverage space generation method can be

[0085] Discrete sampling method: The preset trajectory is sampled at equal intervals in time or joint space; for each sampling point, the position of each joint of the robotic arm is substituted into its CAD model or collision model to obtain the volume or boundary occupied by the robotic arm in three-dimensional space at this time.

[0086] Merging and expansion: Merging the volumes or geometric boundaries corresponding to all sampling points to form a continuous "three-dimensional coverage area".

[0087] Obstacle information can be acquired through sensors or through depth cameras to construct a spatial model of the 3D environment and obstacles.

[0088] In this application, obstacle information is matched with the coverage space corresponding to each preset path, and collision detection or overlap area judgment is performed: Collision detection: If the coverage space overlaps with any obstacle, it indicates that the preset path will collide with or get too close to the obstacle during execution, thus determining that the path is infeasible or unsafe. Collision distance measurement: In some situations with high safety requirements, it is necessary not only to determine whether a collision occurs, but also whether the minimum safety clearance requirement is met. If the distance between a path and an obstacle is less than a set threshold, it will also be considered unsafe. The above collision and safety clearance checks are performed on all selectable preset paths, eliminating infeasible paths, and finally obtaining a set of feasible paths.

[0089] If multiple paths meet the requirements of no collision or safe distance, further optimization can be made by taking into account other factors, such as: path length / movement time: select the path with the shortest execution time or the lowest energy consumption; smoothness: select the path with the lowest acceleration and impact to reduce disturbance to the robotic arm and the environment; ease of operation: some paths may have a better posture at the target position, making subsequent tasks (such as shooting and detection) easier to perform.

[0090] In this application, since the preset path has been calculated and verified offline in the system, only coordinate transformation and collision detection need to be performed online, which greatly shortens the path planning time.

[0091] In one specific implementation, the preset path is a path that satisfies the centroid constraint with fixed point information as the centroid.

[0092] Telecentric constraints are where a camera (or end effector) constantly adjusts its attitude or position around a fixed point during movement, so that the line of sight from the camera to the fixed point remains constant or satisfies a specific constraint.

[0093] The generation of the preset path can be achieved by: setting fixed point coordinates between the ready position and the target position; using inverse kinematics combined with the "line of sight" algorithm to calculate the angle interpolation of each joint of the robotic arm point by point; verifying whether the optical axis of the camera always passes through the fixed point and there are no obstacles or collisions during the path verification process; and recording the trajectory data that satisfies the telecentric constraint as the preset path.

[0094] In one specific implementation, the plurality of assumed target bit information are uniformly distributed.

[0095] This allows for better coverage of typical scenarios and directions, avoiding situations such as "no available poses" or "uneven pose distribution" during online planning.

[0096] This application provides a camera-mounted robotic arm control device for executing the camera-mounted robotic arm control method described above. The camera-mounted robotic arm control device will be described in detail below.

[0097] like Figure 3 As shown, the robotic arm control device mounted on the camera includes:

[0098] Information acquisition module 101 is used to acquire information about the part to be photographed and the preparation position information of the robotic arm;

[0099] The target determination module 102 is used to determine the target pose information of the robotic arm based on the information of the part to be photographed.

[0100] The path planning module 103 is used to plan the motion path of the robotic arm based on the robotic arm's ready position information and target pose information.

[0101] The robotic arm control module 104 is used to control the robotic arm to execute the motion path.

[0102] In one implementation, the target determination module 102 is further configured to:

[0103] Obtain preoperative planning and mapping information of the area to be photographed; based on the area to be photographed and the preoperative planning and mapping information, divide the area to be photographed into a central part and an opening part; based on the preoperative planning and mapping information of the central part, fit and obtain fixed point information; based on the area to be photographed information of the opening part, fit and obtain central ray information; based on the fixed point information and the central ray information, determine the target pose information of the robotic arm.

[0104] In one implementation, the central ray information is the optimal field of view direction information under unobstructed conditions.

[0105] In one implementation, the path planning module 103 is further configured to:

[0106] Obtain multiple preset assumed target position information; each assumed target position information corresponds to multiple preset paths starting from the assumed preparation position; map the multiple assumed target position information to the current coordinate system; select the closest assumed target position information based on the central ray information; select one of the multiple preset paths corresponding to the assumed target position information as the motion path of the robotic arm.

[0107] In one implementation, the path planning module 103 is further configured to:

[0108] The system acquires multiple preset paths corresponding to the assumed target location information; generates a path coverage space based on the preset paths; acquires current obstacle information; and filters the preset paths based on the obstacle information and the path coverage space to obtain the motion path of the robotic arm.

[0109] In one implementation, the preset path is a path that satisfies a telecentric constraint with fixed point information as the telecentric.

[0110] In one implementation, the plurality of assumed target bit information is uniformly distributed.

[0111] The camera-equipped robotic arm control device provided in the above embodiments of this application corresponds to the camera-equipped robotic arm control method provided in the embodiments of this application. Therefore, the specific content in this system corresponds to the camera-equipped robotic arm control method. The specific content can be referred to the records in the camera-equipped robotic arm control method, which will not be repeated in this application.

[0112] The camera-equipped robotic arm control device provided in the above embodiments of this application and the camera-equipped robotic arm control method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0113] The above describes the internal functions and structure of the robotic arm control device assembled with the camera, such as... Figure 4 As shown, in practice, the robotic arm control device equipped with the camera can be implemented as an electronic device, including: memory 301 and processor 303.

[0114] Memory 301 can be configured to store a program.

[0115] Additionally, memory 301 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0116] Memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Processor 303, coupled to memory 301, is used to execute programs in memory 301 for:

[0117] Obtain information about the part to be photographed and the robotic arm's ready position;

[0118] Determine the target pose information of the robotic arm based on the information of the part to be photographed;

[0119] Based on the robotic arm's ready position information and the target pose information, the motion path of the robotic arm is planned;

[0120] Control the robotic arm to execute the motion path.

[0121] In one implementation, the processor 303 is further configured to:

[0122] Obtain preoperative planning and mapping information of the area to be photographed; based on the area to be photographed and the preoperative planning and mapping information, divide the area to be photographed into a central part and an opening part; based on the preoperative planning and mapping information of the central part, fit and obtain fixed point information; based on the area to be photographed information of the opening part, fit and obtain central ray information; based on the fixed point information and the central ray information, determine the target pose information of the robotic arm.

[0123] In one implementation, the central ray information is the optimal field of view direction information under unobstructed conditions.

[0124] In one implementation, the processor 303 is further configured to:

[0125] Obtain multiple preset assumed target position information; each assumed target position information corresponds to multiple preset paths starting from the assumed preparation position; map the multiple assumed target position information to the current coordinate system; select the closest assumed target position information based on the central ray information; select one of the multiple preset paths corresponding to the assumed target position information as the motion path of the robotic arm.

[0126] In one implementation, the processor 303 is further configured to:

[0127] The system acquires multiple preset paths corresponding to the assumed target location information; generates a path coverage space based on the preset paths; acquires current obstacle information; and filters the preset paths based on the obstacle information and the path coverage space to obtain the motion path of the robotic arm.

[0128] In one implementation, the preset path is a path that satisfies a telecentric constraint with fixed point information as the telecentric.

[0129] In one implementation, the plurality of assumed target bit information is uniformly distributed.

[0130] In this application, Figure 4 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 4 The components shown.

[0131] The electronic device provided in this embodiment is based on the same inventive concept as the camera-equipped robotic arm control method provided in this application embodiment, and has the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0132] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 a process or multiple processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0135] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0136] This application also provides a computer-readable storage medium corresponding to the camera-equipped robotic arm control method provided in the foregoing embodiments, wherein a computer program (i.e., a program product) is stored thereon. When the computer program is run by a processor, it executes the interactive image analysis assistance method for 3D aerial imaging provided in any of the foregoing embodiments.

[0137] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0138] The computer-readable storage medium provided in the above embodiments of this application and the interactive image analysis assistance method for 3D aerial imaging provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0139] It should be noted that numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0140] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0141] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for controlling a robotic arm equipped with a camera, used to control the position of the end effector of the robotic arm with the camera mounted thereon, characterized in that, include: Obtain information about the part to be photographed and the robotic arm's ready pose information; Determine the target pose information of the robotic arm based on the information of the part to be photographed; Based on the robotic arm's ready pose information and target pose information, the motion path of the robotic arm is planned. Control the robotic arm to execute the motion path; The process of determining the target pose information of the robotic arm based on the information of the part to be photographed includes: Obtain preoperative planning and mapping information for the area to be photographed; Based on the information of the area to be photographed and the preoperative planning mapping information, the area to be photographed is divided into a central part and an opening part; Based on the preoperative planning and mapping information of the central part, fixed point information is obtained by fitting. Based on the information of the part to be photographed at the opening, the information of the central ray is obtained by fitting. Based on the fixed point information and the central ray information, the target pose information of the robotic arm is determined; The opening portion is an open or unfolded portion, through which the required shooting angle and orientation of the robotic arm can be determined; The fixed point information refers to the coordinates of the fixed point. The central ray information is the optimal field of view direction information under unobstructed conditions.

2. The robotic arm control method for camera assembly according to claim 1, characterized in that, The process of planning the motion path of the robotic arm based on its pre-pose and target pose information includes: Obtain multiple preset assumed target pose information; each assumed target pose information corresponds to multiple preset paths starting from the assumed ready pose; Map the multiple assumed target pose information to the current coordinate system; Based on the central ray information, select the closest assumed target pose information; Select one of the multiple preset paths corresponding to the assumed target pose information as the motion path of the robotic arm.

3. The robotic arm control method for camera assembly according to claim 2, characterized in that, The step of selecting one of multiple preset paths corresponding to the assumed target pose information as the motion path of the robotic arm includes: Obtain multiple preset paths corresponding to the assumed target pose information; Generate a path coverage space based on the preset path; Obtain current obstacle information; The motion path of the robotic arm is obtained by filtering the preset path based on obstacle information and the path coverage space.

4. The robotic arm control method for camera assembly according to claim 2, characterized in that, The preset path is a path that satisfies the centroid constraint with fixed point information as the centroid.

5. The robotic arm control method for camera assembly according to claim 2, characterized in that, The multiple assumed target pose information are uniformly distributed.

6. A robotic arm control device equipped with a camera, characterized in that, The robotic arm control method for assembling the camera according to any one of claims 1-5 includes: The information acquisition module is used to acquire information about the part to be photographed and the robotic arm's ready pose information; The target determination module is used to determine the target pose information of the robotic arm based on the information of the part to be photographed. The path planning module is used to plan the motion path of the robotic arm based on the robotic arm's pre-pose information and target pose information. A robotic arm control module is used to control the robotic arm to execute the motion path.

7. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is used to execute the program for implementing the robotic arm control method for camera assembly according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the robotic arm control method for camera assembly as described in any one of claims 1-5.

Images