Robotic arm positioning methods, devices and systems

Abstract

This application relates to a robotic arm positioning method, apparatus, and system. The method includes: acquiring images corresponding to a target physiological structure at multiple robotic arm positions; constructing a three-dimensional model of the target physiological structure based on the acquired images; determining the coordinates of operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model; and, provided that the robotic arm can reach all operation point coordinates, moving the robotic arm to the operation point coordinates to perform imaging of the target physiological structure at each operation point coordinate. This method can quickly and accurately move the robotic arm to the target position without interfering with other equipment, thus effectively improving the positioning efficiency of the robotic arm.

Description

Technical Field

[0001] This application relates to the field of surgical robot technology, and in particular to a method, apparatus and system for positioning a robotic arm. Background Technology

[0002] In traditional techniques, when using a robotic arm to position and take pictures at a specific location, the robotic arm needs to be manually dragged to the appropriate position. This is inconvenient due to the large range of motion of the robotic arm, and the lack of a reference position for the positioning point requires manual determination. Furthermore, once the robotic arm has been dragged to the positioning point, its current posture may interfere with other equipment involved in the imaging operation, resulting in low positioning efficiency. Summary of the Invention

[0003] Therefore, it is necessary to provide a robotic arm positioning method, device, and system that can quickly and accurately move the robotic arm to the target position without interfering with other equipment, thereby effectively improving the positioning efficiency of the robotic arm.

[0004] In a first aspect, this application provides a robotic arm positioning method. The method includes: acquiring images corresponding to a target physiological structure collected at multiple robotic arm positions; constructing a three-dimensional model corresponding to the target physiological structure based on each of the acquired images; determining the coordinates of operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model; and, provided that the robotic arm can reach all of the operation point coordinates, moving the robotic arm to the operation point coordinates to perform imaging of the target physiological structure at each of the operation point coordinates.

[0005] In one embodiment, acquiring images corresponding to a target physiological structure collected at multiple robotic arm points includes: acquiring a positioning image of the target physiological structure; inputting the positioning image into a robotic arm point generation model to acquire each acquisition point corresponding to the target physiological structure; and sequentially moving the robotic arm to each acquisition point to acquire images corresponding to the target physiological structure.

[0006] In one embodiment, after acquiring the collection points corresponding to the target physiological structure, the method further includes: determining whether the robotic arm can reach each of the collection points; and if there are collection points that the robotic arm cannot reach, acquiring the collection points corresponding to the target physiological structure again based on the robotic arm point generation model.

[0007] In one embodiment, determining the coordinates of the operational placement points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model includes: calculating the coordinates of each initial placement point for imaging the target physiological structure based on the three-dimensional model; verifying whether the robotic arm can reach each of the initial placement point coordinates; if the robotic arm can reach each of the initial placement point coordinates, controlling the robotic arm to move to any of the initial placement point coordinates; if the robotic arm is at the initial placement point coordinate, determining whether the initial placement point coordinate meets the imaging requirements; if it meets the imaging requirements, then using the initial placement point coordinate as the operational placement point coordinate; if it does not meet the imaging requirements, then correcting the initial placement point coordinate of the robotic arm until the corrected placement point coordinate meets the imaging requirements, and using the corrected placement point coordinate as the operational placement point coordinate.

[0008] In one embodiment, the method further includes: if the robotic arm cannot reach any of the initial placement point coordinates, returning to the step of calculating the coordinates of each initial placement point for imaging the target physiological structure based on the three-dimensional model, until the robotic arm can reach each of the initial placement point coordinates; or, if the robotic arm cannot reach any of the initial placement point coordinates, calibrating the initial placement point coordinates based on the three-dimensional image display container, until the robotic arm can reach each of the initial placement point coordinates.

[0009] In one embodiment, calibrating the initial placement point coordinates based on the three-dimensional image display container until the robotic arm can reach each of the initial placement point coordinates includes: obtaining the initial placement point coordinates and the position coordinates of the robotic arm in the same coordinate system based on the three-dimensional image display container; and adjusting the initial placement point coordinates based on the coordinate system and the position coordinates of the robotic arm until the robotic arm can reach each of the initial placement point coordinates.

[0010] In one embodiment, the method further includes: if, at any of the operation positioning point coordinates, there is interference from the robot arm pose corresponding to the robot arm when taking the film, adjusting the robot arm pose to obtain an adjusted pose; and taking the film of the target physiological structure based on the adjusted pose and the operation positioning point coordinates.

[0011] In one embodiment, adjusting the pose of the robotic arm to obtain an adjusted pose includes: adjusting the pose of the robotic arm according to preset robotic arm pose data based on the same operation point coordinates to obtain an adjusted pose; determining whether the adjusted pose interferes with filming; if the adjusted pose interferes with filming, returning to the step of adjusting the pose of the robotic arm according to the preset robotic arm pose data to obtain an adjusted pose, until the adjusted pose is obtained.

[0012] Secondly, this application also provides a robotic arm positioning device. The device includes: an image acquisition module for acquiring images corresponding to a target physiological structure acquired at multiple robotic arm positions; a model construction module for constructing a three-dimensional model corresponding to the target physiological structure based on each acquired image; a coordinate determination module for determining the coordinates of each target operation point for imaging the target physiological structure based on the three-dimensional model; and a robotic arm positioning module for moving the robotic arm to the coordinates of each operation point, whereby imaging the target physiological structure is performed at each operation point coordinate, provided that the robotic arm can reach all of the operation point coordinates.

[0013] Thirdly, this application also provides a robotic arm positioning system. The system includes a control center, a robotic arm, and a camera mounted on the robotic arm. The camera on the robotic arm is used to acquire images corresponding to target physiological structures at multiple robotic arm positions. The control center is used to construct a three-dimensional model corresponding to the target physiological structure based on each acquired image. The control center is also used to determine the coordinates of each target operation point for imaging the target physiological structure based on the three-dimensional model. Furthermore, the control center is used to move the robotic arm to the operation operation point coordinates when it is determined that the robotic arm can reach all the operation operation point coordinates.

[0014] The aforementioned robotic arm method, apparatus, and system acquire images corresponding to a target physiological structure at multiple robotic arm positions; construct a three-dimensional model of the target physiological structure based on each acquired image; determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model; and, after confirming that the robotic arm can reach the coordinates of each operation point, move the robotic arm to the coordinates of the operation point to perform imaging of the target physiological structure at each operation point coordinate.

[0015] By using the target physiological structure orientation navigation system control center software, images are acquired directly above the target physiological structure, and several acquisition points are calculated. Then, the robotic arm carrying a structured light camera is moved to these acquisition points to acquire images. The captured images are used to construct a 3D model through algorithms. Finally, the appropriate operation point coordinates for each target operation point are calculated based on the 3D model. Guided by the operation point coordinates, the robotic arm can move to the designated position to take images of the target physiological structure. This method can quickly and accurately move the robotic arm to the target position without interfering with other equipment, thus effectively improving the positioning efficiency of the robotic arm. Attached Figure Description

[0016] Figure 1 This is an application environment diagram of a robotic arm positioning method in one embodiment;

[0017] Figure 2 This is a flowchart illustrating a robotic arm positioning method in one embodiment;

[0018] Figure 3 This is a flowchart illustrating an image acquisition method for a target physiological structure in one embodiment.

[0019] Figure 4 This is a flowchart illustrating the method for re-acquiring each data acquisition point in one embodiment;

[0020] Figure 5 This is a flowchart illustrating the initial placement point coordinate correction method in one embodiment;

[0021] Figure 6 This is a flowchart illustrating the initial placement point coordinate adjustment method in one embodiment;

[0022] Figure 7 This is a flowchart illustrating the initial placement point coordinate adjustment method in another embodiment;

[0023] Figure 8 This is a flowchart illustrating a robotic arm pose adjustment method in one embodiment;

[0024] Figure 9 This is a flowchart illustrating the robotic arm pose adjustment method in another embodiment;

[0025] Figure 10 This is a schematic diagram showing the distribution of various data collection points in one embodiment;

[0026] Figure 11 This is a flowchart illustrating a three-dimensional modeling method in one embodiment;

[0027] Figure 12This is a flowchart illustrating a method for obtaining the coordinates of the operation points corresponding to each target operation point in one embodiment.

[0028] Figure 13 This is a schematic diagram of the display process for adjusting a 3D image display container in one embodiment.

[0029] Figure 14 This is a flowchart illustrating the robot arm pose adjustment logic in one embodiment;

[0030] Figure 15 This is a structural block diagram of a robotic arm positioning device in another embodiment;

[0031] Figure 16 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. Detailed Implementation

[0032] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0033] The robotic arm positioning method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the robotic arm 102 communicates with the control center 104 via a network. A data storage system can store the data that the control center 104 needs to process. The data storage system can be integrated into the control center 104, or it can be located in the cloud or on another network control center. The control center 104 acquires images of the target physiological structure from multiple robotic arm positions at the robotic arm 102; constructs a 3D model of the target physiological structure based on each acquired image; determines the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the 3D model; and moves the robotic arm to the operation point coordinates after confirming that the robotic arm can reach each operation point coordinate to perform imaging of the target physiological structure at each operation point coordinate. The robotic arm 102 can be, but is not limited to, various IoT devices and portable devices, and the control center 104 can be implemented using an independent control center or a control center cluster composed of multiple control centers.

[0034] In one embodiment, such as Figure 2 As shown, a robotic arm positioning method is provided, which can be applied to... Figure 1 Taking the control center in the middle as an example, the following steps are included:

[0035] Step 202: Obtain images of the target physiological structures collected at multiple robotic arm locations.

[0036] Among them, the robotic arm position can be the coordinates of the position of the robotic arm taking pictures of the target object model.

[0037] The target physiological structure can be a part or all of the physiological structure of the target object, such as the human spinal structure or the organ structure of an animal.

[0038] Specifically, a structured light camera is installed at the end effector of a robotic arm. Using a foot pedal on the control console of the directional navigation system, the robotic arm is dragged to a position directly above a specific location on the target object, ensuring the structured light camera is directly facing that location. With the structured light camera at the end effector directly facing the target object, an image is captured at that location, obtaining a localization image of the target physiological structure. This image is then input into a robotic arm position generation model that calculates the acquisition points for the robotic arm. The model calculates the corresponding acquisition points for each target physiological structure. Based on these acquisition points, the robotic arm is then controlled to sequentially move to each acquisition point to capture multiple images of the target physiological structure, thus obtaining multiple images of the target physiological structure.

[0039] In one embodiment, such as Figure 3 As shown, acquiring images of the target physiological structure from multiple robotic arm positions includes the following steps:

[0040] Step 302: Obtain the localization image of the target physiological structure, input the localization image into the robotic arm point generation model, and obtain the collection points corresponding to the target physiological structure.

[0041] Among them, the positioning image can be an image captured by a structured light camera at the end of the robotic arm, used to calculate the coordinates of the points captured in each image of the target physiological structure of the target object.

[0042] Among them, the robotic arm point generation model can be a point generation algorithm for the robotic arm, which can calculate the point coordinates of at least five images based on the positioning images of the target physiological structure.

[0043] The acquisition point can be the coordinates of the point on which the robotic arm acquires images of the target physiological structure of the target object.

[0044] Specifically, clicking "Take Photo" in the control center's console software causes the control center to input the captured image of the target physiological structure into the invoked robotic arm location generation model. The model calculates at least five locations for photographing the target physiological structure, i.e., various acquisition points. These at least five acquisition points are the minimum number of points required for the robotic arm to capture images while ensuring complete subsequent modeling. The approximate distribution of these acquisition points in one embodiment is shown below. Figure 10 As shown.

[0045] Step 304: Move the robotic arm sequentially to each acquisition point to acquire images corresponding to the target physiological structure.

[0046] Specifically, following the preset shooting sequence, stepping on the foot pedal causes the control center to move the robotic arm sequentially to the coordinates corresponding to each acquisition point. When the robotic arm reaches any acquisition point, it takes a picture of the target physiological structure of the target object to obtain the image corresponding to that acquisition point. By completing the shooting at each acquisition point, the image corresponding to the target physiological structure is acquired.

[0047] In this embodiment, the positioning image of the target physiological structure is input into the point generation model of the robotic arm to determine each acquisition point, and the robotic arm is controlled to acquire images of the target physiological structure at each acquisition point. This ensures that there are enough and reasonable images corresponding to the target physiological structure when constructing the three-dimensional model, thus guaranteeing the performance of the three-dimensional model.

[0048] In one embodiment, such as Figure 4 As shown, after acquiring the acquisition points corresponding to the target physiological structure, the process also includes:

[0049] Step 402: Determine whether the robotic arm can reach each data collection point.

[0050] Specifically, after calculating the various collection points corresponding to the target physiological structure through the robotic arm point generation model, the control center calculates the coordinates of each collection point and determines whether the robotic arm can reach the coordinates of each collection point after moving based on the calculation results.

[0051] Step 404: In the case of sampling points that cannot be reached by the robotic arm, the sampling points corresponding to the target physiological structure are reacquired based on the robotic arm sampling point generation model.

[0052] Specifically, if at least one of the coordinates of the acquisition points is inaccessible to the robotic arm, then based on the previous calculation results of the acquisition points corresponding to the target physiological structure, the positioning image of the target physiological structure is re-input into the robotic arm point generation model, and the new acquisition points corresponding to the target physiological structure are obtained again through the calculation of the robotic arm point generation model.

[0053] In this embodiment, by judging whether the robotic arm can reach each acquisition point, and recalculating each acquisition point if it cannot reach any acquisition point, it can ensure that the image of the target physiological structure can be acquired at each acquisition point, thus avoiding the situation where the robotic arm fails to reach the target physiological structure during the acquisition process and improving the efficiency of image acquisition of the target physiological structure.

[0054] Step 204: Construct a three-dimensional model of the target physiological structure based on each acquired image.

[0055] The three-dimensional model can be a model that displays the target physiological structure of the target object, and has three directional vectors.

[0056] Specifically, based on the images acquired at each acquisition point, these images are input into a 3D model construction algorithm. The algorithm then constructs a 3D model corresponding to the target physiological structure of the object. In the directional navigation system console software for the target physiological structure, the main process of 3D model building is as follows: Figure 11 As shown.

[0057] Step 206: Based on the three-dimensional model, determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure.

[0058] The target operation point can be the point where the robotic arm takes a picture of the target physiological structure of the target object, such as the anterior, lateral, entrance, and exit views in medical imaging.

[0059] The coordinates of the operation point can be the coordinate information of the target operation point.

[0060] Specifically, based on the algorithm for calculating the positioning point coordinates of the 3D model, the coordinates of the positioning points corresponding to each target manipulation point for imaging the target physiological structure of the target object are calculated. For example, the target positioning points are anteroposterior, lateral, inlet, and outlet positions, and the coordinates of each manipulation point are the coordinates corresponding to the anteroposterior, lateral, inlet, and outlet positions, respectively. In one embodiment, the main process for calculating the manipulation point coordinates corresponding to each target manipulation point is as follows: Figure 12 As shown.

[0061] In one embodiment, such as Figure 5 As shown, based on the three-dimensional model, the coordinates of the manipulation points corresponding to each target manipulation point for imaging the target physiological structure are determined, including the following steps:

[0062] Step 502: Calculate the coordinates of each initial positioning point for taking images of the target physiological structure based on the three-dimensional model.

[0063] The initial three-dimensional model of the placement point coordinates was used to calculate the placement point coordinates of each target placement point for the target physiological structure, but the robotic arm was not used to verify whether it met the usage conditions.

[0064] Specifically, based on the algorithm for calculating the coordinates of the placement points in the 3D model, the initial placement point coordinates corresponding to each target operation point for taking images of the target physiological structure of the target object are initially calculated, and the initial placement point coordinates are then verified to ensure they meet the usage conditions.

[0065] Step 504: Verify whether the robotic arm can reach the coordinates of each initial pivot point.

[0066] Specifically, with the support of the control center, a 3D image display container is created. Based on the 3D image display container, the 3D model and the coordinates of each initial placement point are displayed. The display process is as follows: Figure 13 As shown. Further verification is performed to determine whether the robotic arm can move to the coordinates of each initial pivot point, and the verification results are output.

[0067] Step 506: If the robotic arm can reach the coordinates of each initial swing point, control the robotic arm to move to any initial swing point coordinate.

[0068] Specifically, if the verification results indicate that the robotic arm can move smoothly to each initial position coordinate, the control center selects any reachable initial position coordinate and drives the robotic arm to move to that initial position coordinate.

[0069] Step 508: When the robotic arm is at the initial position coordinates, determine whether the initial position coordinates meet the requirements for filming.

[0070] Among them, the imaging requirements may be whether the robotic arm meets the requirements for image acquisition of the target physiological structure at the initial positioning point coordinate position.

[0071] Specifically, given that the control center can control the robotic arm to move to the initial positioning point coordinates of the target, it is determined whether the position of the robotic arm at the initial positioning point coordinates of the target can meet the requirements for image acquisition of the target's physiological structure, i.e., whether it can meet the requirements for taking X-rays of the target's physiological structure. For example, it is determined whether the position of the robotic arm at the initial positioning point coordinates of the target deviates from the doctor's expectations.

[0072] Step 510: If the requirements for filming are met, then the coordinates of the initial placement point are used as the coordinates of the operation placement point.

[0073] Specifically, if the position of the robotic arm at the initial placement point coordinates of the target can meet the requirements for image acquisition of the target's physiological structure, then the initial placement point coordinates of the target are determined as the placement point coordinates that have been verified and can be used for image acquisition of the target's physiological structure.

[0074] Step 512: If the imaging requirements are not met, the initial position coordinates of the robotic arm are corrected until the corrected position coordinates meet the imaging requirements. The corrected position coordinates are then used as the operation position coordinates.

[0075] Specifically, if the position of the robotic arm at the initial placement point coordinates of the target cannot meet the requirements for image acquisition of the target's physiological structure, any method can be used to correct the initial placement point coordinates of the target.

[0076] Method 1: Return to the step of "calculating the initial position coordinates of each target operation point corresponding to the target physiological structure that needs to be imaged based on the position coordinate calculation algorithm of the 3D model" until the corrected position coordinates meet the requirements for image acquisition of the target physiological structure. Then the initial position coordinates of the target are determined as the position coordinates that have been verified and can be used as the position coordinates for image acquisition of the target physiological structure.

[0077] Method 2: With the assistance of a 3D image display container, adjust the coordinates of the initial placement point according to the imaging requirements until the corrected placement point coordinates meet the requirements for image acquisition of the target physiological structure. Then, determine the initial placement point coordinates of the target as the placement point coordinates that have been verified and can be used for image acquisition of the target physiological structure.

[0078] Method 3: According to the imaging requirements, the robotic arm is adjusted by a third party to meet the imaging requirements; the coordinate position of the adjusted robotic arm is further determined as a verified positioning point that can be used for image acquisition of the target physiological structure.

[0079] In this embodiment, by verifying that the robotic arm can reach the coordinates of each initial positioning point, the coordinates of the operation positioning point are further determined based on the verification results of whether the coordinates of the robotic arm at any initial positioning point can meet the requirements for radiography. This allows for the correction of initial positioning point coordinates that do not meet the requirements for radiography, ensuring that the radiography of the target physiological structure meets the preset requirements at each operation positioning point coordinate, thereby improving the accuracy of the robotic arm's positioning.

[0080] In one embodiment, such as Figure 6 As shown, the method also includes the following steps:

[0081] Step 602: If the robotic arm cannot reach any of the initial positioning point coordinates, return to the step of calculating the coordinates of each initial positioning point for taking pictures of the target physiological structure based on the three-dimensional model, until the robotic arm can reach each initial positioning point coordinate.

[0082] Specifically, if the control center cannot control the robotic arm to move to one of the initial positioning point coordinates, the process returns to the step of "calculating the initial positioning point coordinates corresponding to each target operation point that needs to be filmed on the target physiological structure of the target object according to the positioning point coordinate calculation algorithm of the 3D model" until the control center can control the robotic arm to move to each initial positioning point coordinate.

[0083] Furthermore, if the control center can control the robotic arm to move to each initial placement point coordinate, it returns to the step of "determining whether the position of the robotic arm at the initial placement point coordinate of the target can meet the requirements for image acquisition of the target physiological structure, that is, whether it can meet the requirements for taking X-rays of the target physiological structure", until each initial placement point coordinate can meet the requirements for taking X-rays, and then each initial placement point coordinate is used as the coordinate of each operation placement point.

[0084] or,

[0085] Step 604: If the robotic arm cannot reach any of the initial swing point coordinates, the initial swing point coordinates are calibrated according to the container displayed in the 3D image until the robotic arm can reach each initial swing point coordinate.

[0086] Among them, the three-dimensional image display container can be used to display the various placement points of the three-dimensional model and the target physiological structure.

[0087] Specifically, if the control center cannot control the robotic arm to move to one of the initial swing point coordinates, the coordinates of each initial swing point are calibrated using a 3D image display container. That is, the coordinates of each initial swing point that the robotic arm cannot move to are adjusted in the 3D image display container until the control center can control the robotic arm to move to each initial swing point coordinate.

[0088] Furthermore, if the control center can control the robotic arm to move to each initial placement point coordinate, it returns to the step of "determining whether the position of the robotic arm at the initial placement point coordinate of the target can meet the requirements for image acquisition of the target physiological structure, that is, whether it can meet the requirements for taking X-rays of the target physiological structure", until each initial placement point coordinate can meet the requirements for taking X-rays, and then each initial placement point coordinate is used as the coordinate of each operation placement point.

[0089] In this embodiment, when the robotic arm cannot reach any initial position coordinate, the coordinates of the unreachable initial position are adjusted until the robotic arm can reach all initial position coordinates. This prioritizes the practicality of each initial position coordinate and avoids failure to take images of the target physiological structure due to the robotic arm's inability to reach any initial position coordinate, thereby improving the efficiency of taking images of the target physiological structure.

[0090] In one embodiment, such as Figure 7 As shown, based on the 3D image of the container, the coordinates of the initial placement point are calibrated until the robotic arm can reach all the initial placement point coordinates. This includes the following steps:

[0091] Step 702: Based on the 3D image display container, obtain the coordinates of each initial placement point and the position coordinates of the robotic arm in the same coordinate system.

[0092] Among them, the position coordinates can be the real-time coordinate information of the robotic arm in three-dimensional space.

[0093] Specifically, with the support of a 3D image display container, the 3D model, the coordinates of each initial placement point, and the position coordinates of the current robotic arm in the 3D space are read, and the 3D model, the coordinates of each initial placement point, and the position coordinates of the current robotic arm in the 3D space are transformed into the same coordinate system for expression.

[0094] Step 704: Based on the coordinate system, adjust the initial swing point coordinates according to the position coordinates of the robotic arm until the robotic arm can reach each initial swing point coordinate.

[0095] Specifically, based on the same coordinate system, the coordinates of the initial placement points that the robotic arms could not reach are adjusted and verified in the display area of ​​the three-dimensional image display container until the robotic arms can reach the coordinates of each initial placement point.

[0096] In this embodiment, by using a three-dimensional image display container to adjust the initial placement point coordinates on the same coordinate system, the requirement that the robotic arm can reach the initial placement point coordinates is met. This provides more adjustment methods for calculating the initial placement point coordinates in the three-dimensional model, allowing the selection of the corresponding adjustment method according to the actual situation, and improving the redundancy of the adjustment of the initial placement point coordinates.

[0097] Step 208: After confirming that the robotic arm can reach the coordinates of each operation point, move the robotic arm to the coordinates of the operation point to take pictures of the target physiological structure at each operation point coordinate.

[0098] Specifically, the system determines whether the robotic arm can reach the coordinates of each operational pivot point. If the determination indicates that the robotic arm can reach all operational pivot point coordinates, the robotic arm is moved sequentially to different operational pivot point coordinates. For any operational pivot point coordinate, the system determines whether the robotic arm's pose at that coordinate interferes with the imaging. If the robotic arm's pose does not interfere with the imaging, the target physiological structure of the target object is imaged at that operational pivot point coordinate. This process continues until imaging of the target physiological structure is completed at all operational pivot point coordinates.

[0099] In one embodiment, after constructing a 3D model of the target physiological structure based on the acquired images, a calibration target needs to be installed at the end of the robotic arm. The calibration target is positioned between the target physiological structure and the C-arm. The calibration target is used to assist in calibrating the target physiological structure, and the C-arm is a C-arm X-ray machine. When it is necessary to take an X-ray of the target physiological structure, the robotic arm is moved to the corresponding operation position coordinates, and then the C-arm is used to take an X-ray of the target physiological structure.

[0100] In one embodiment, such as Figure 8 As shown, the method also includes the following steps:

[0101] Step 802: At any operation point coordinate, if there is a situation where the robot arm pose interferes with the film taking, adjust the robot arm pose to obtain the adjusted pose.

[0102] The robot arm pose can be the position and orientation of the robot arm in a specified coordinate system.

[0103] The adjusted pose can be the new position and posture of the robotic arm in a specified coordinate system, obtained by adjusting the original pose of the robotic arm and eliminating interference from filming.

[0104] Specifically, based on any operation point coordinate, if the robot arm pose has already interfered with the filming, the original robot arm pose is adjusted while maintaining the operation point coordinate to obtain a new robot arm pose as the adjusted pose.

[0105] Step 804: Take an X-ray of the target physiological structure based on the adjusted pose and the coordinates of the operation point.

[0106] Specifically, based on the coordinates of the manipulation point, and according to the adjusted pose of the robotic arm that will no longer interfere with the imaging, the target physiological structure of the object is imaged. If other manipulation point coordinates also interfere with the imaging, the same adjustment method is used to adjust the robotic arm pose until imaging of the target physiological structure is completed at all manipulation point coordinates. In one embodiment, the main process of adjusting the robotic arm pose is as follows: Figure 14 shown

[0107] In this embodiment, by adjusting the pose of the robotic arm that causes interference in the imaging, and then using the adjusted pose to image the target physiological structure, the interference of the robotic arm's pose on the imaging of the target physiological structure can be avoided, thereby improving the imaging quality of the target physiological structure.

[0108] In one embodiment, such as Figure 9 As shown, adjusting the pose of the robotic arm to obtain the adjusted pose includes the following steps:

[0109] Step 902: Based on the same operation point coordinates, adjust the robot arm pose according to the preset robot arm pose data to obtain the adjusted pose.

[0110] Among them, the robotic arm posture data can be various robotic arm pose data that exist in the control center, which can adjust the pose of the robotic arm.

[0111] The adjusted pose is the new position and orientation of the robotic arm in the specified coordinate system obtained by adjusting the original pose of the robotic arm, but the presence of interference is not verified.

[0112] Specifically, based on the same operation point coordinates, one of the preset robotic arm posture data stored in the control center for adjusting the robotic arm posture is selected, and the robotic arm posture is adjusted to obtain the adjusted posture.

[0113] Step 904: Determine whether the adjusted pose interferes with filming.

[0114] Specifically, the adjusted pose is used as the robot arm pose, and the step of "judging whether there is interference in the robot arm pose at the coordinates of the operation point" is executed again to obtain the pose judgment result.

[0115] Step 906: If the adjusted pose interferes with filming, return to the previous step to adjust the robot arm pose according to the preset robot arm pose data to obtain the adjusted pose, until the adjusted pose is obtained.

[0116] Specifically, if the pose determination result indicates that the adjusted pose of the robotic arm does not interfere with the filming, then the adjusted pose is taken as the adjusted pose.

[0117] If the pose assessment result indicates that the adjusted pose of the robotic arm still interferes with filming, the process returns to the step of "selecting one of the preset robotic arm pose data stored in the control center for adjusting the robotic arm pose, adjusting the robotic arm pose to obtain the adjusted pose," until the pose assessment result indicates that the adjusted robotic arm pose does not interfere with filming, at which point the adjusted pose is taken as the adjusted pose. Note that the preset robotic arm pose data used for each re-execution of the pose adjustment process is different.

[0118] In one embodiment, if the preset robotic arm posture data stored in the control center for adjusting the robotic arm posture are all used to adjust the robotic arm posture, and the posture judgment result still indicates that the adjusted posture of the robotic arm still interferes with filming, then a prompt message for manually adjusting the robotic arm posture is displayed on the display device.

[0119] Alternatively, the process can return to the steps of "using the foot pedal of the directional navigation system console to drag the robotic arm directly above a specific position of the target object, so that the structured light camera is facing that specific position. With the structured light camera at the end of the robotic arm facing the specific position of the target object, an image is taken of that specific position to obtain a localization image of the target physiological structure. This image is then input into the robotic arm position generation model, which calculates the acquisition points of the robotic arm. Through the model's calculations, the corresponding acquisition points for the target physiological structure are obtained. Furthermore, based on these acquisition points, the robotic arm is controlled to sequentially move to each acquisition point to take images of the target physiological structure, thereby obtaining multiple images corresponding to the target physiological structure," until the pose determination result indicates that the adjusted pose of the robotic arm has not interfered with the imaging.

[0120] In this embodiment, by using preset robotic arm posture data to adjust the position of the robotic arm, multiple rapid adjustments to the position of the robotic arm can be achieved, which can improve the efficiency of the position adjustment of the robotic arm and ensure that the adjusted position of the robotic arm will not affect the imaging quality of the target physiological structure.

[0121] In the above-mentioned robotic arm positioning, images corresponding to the target physiological structure are acquired at multiple robotic arm positions; a three-dimensional model corresponding to the target physiological structure is constructed based on each acquired image; the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure are determined based on the three-dimensional model; and, if it is determined that the robotic arm can reach the coordinates of each operation point, the robotic arm is moved to the coordinates of the operation point to perform imaging of the target physiological structure at each operation point coordinate.

[0122] By using the target physiological structure orientation navigation system control center software, images are acquired directly above the target physiological structure, and several acquisition points are calculated. Then, the robotic arm carrying a structured light camera is moved to these acquisition points to acquire images. The captured images are used to construct a 3D model through algorithms. Finally, the appropriate operation point coordinates for each target operation point are calculated based on the 3D model. Guided by the operation point coordinates, the robotic arm can move to the designated position to take images of the target physiological structure. This method can quickly and accurately move the robotic arm to the target position without interfering with other equipment, thus effectively improving the positioning efficiency of the robotic arm.

[0123] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0124] Based on the same inventive concept, this application also provides a robotic arm positioning device for implementing the robotic arm positioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of the one or more robotic arm positioning device embodiments provided below can be found in the limitations of the robotic arm positioning method described above, and will not be repeated here.

[0125] In one embodiment, such as Figure 15 As shown, a robotic arm positioning device is provided, including: an image acquisition module 1502, a model construction module 1504, a coordinate determination module 1506, and a robotic arm positioning module 1508, wherein:

[0126] Image acquisition module 1502 is used to acquire images of the target physiological structure collected at multiple robotic arm positions;

[0127] The model building module 1504 is used to build a three-dimensional model of the target physiological structure based on each acquired image.

[0128] The coordinate determination module 1506 is used to determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model.

[0129] The robotic arm positioning module 1508 is used to move the robotic arm to the coordinates of each operation positioning point, so as to take pictures of the target physiological structure at each operation positioning point coordinate, provided that the robotic arm can reach the coordinates of each operation positioning point.

[0130] In one embodiment, the image acquisition module 1502 is further configured to acquire a localization image of the target physiological structure, input the localization image into the robotic arm point generation model, acquire each acquisition point corresponding to the target physiological structure, and move the robotic arm sequentially to each acquisition point to acquire the image corresponding to the target physiological structure.

[0131] In one embodiment, the image acquisition module 1502 is further used to determine whether the robotic arm can reach each acquisition point; if there are acquisition points that the robotic arm cannot reach, the acquisition points corresponding to the target physiological structure are reacquired based on the robotic arm point generation model.

[0132] In one embodiment, the coordinate determination module 1506 is further configured to calculate the coordinates of each initial placement point for imaging the target physiological structure based on the three-dimensional model; verify whether the robotic arm can reach each initial placement point coordinate; if the robotic arm can reach each initial placement point coordinate, control the robotic arm to move to any initial placement point coordinate; if the robotic arm is at the initial placement point coordinate, determine whether the initial placement point coordinate meets the imaging requirements; if the imaging requirements are met, use the initial placement point coordinate as the operation placement point coordinate; if the imaging requirements are not met, correct the initial placement point coordinate of the robotic arm until the corrected placement point coordinate meets the imaging requirements, and use the corrected placement point coordinate as the operation placement point coordinate.

[0133] In one embodiment, the coordinate determination module 1506 is further configured to, if the robotic arm cannot reach any initial positioning point coordinate, return to the step of calculating the coordinates of each initial positioning point for taking pictures of the target physiological structure according to the three-dimensional model, until the robotic arm can reach each initial positioning point coordinate; or, if the robotic arm cannot reach any initial positioning point coordinate, calibrate the initial positioning point coordinate according to the three-dimensional image display container, until the robotic arm can reach each initial positioning point coordinate.

[0134] In one embodiment, the coordinate determination module 1506 is further configured to obtain the coordinates of each initial placement point and the position coordinates of the robotic arm in the same coordinate system based on the three-dimensional image display container; and adjust the initial placement point coordinates according to the position coordinates of the robotic arm based on the coordinate system until the robotic arm can reach each initial placement point coordinate.

[0135] In one embodiment, the robotic arm positioning module 1508 is further configured to adjust the robotic arm pose at any operation positioning point coordinate if there is interference from the robotic arm pose corresponding to the robotic arm during imaging, thereby obtaining an adjusted pose; and to take an image of the target physiological structure based on the adjusted pose and the operation positioning point coordinate.

[0136] In one embodiment, the robotic arm positioning module 1508 is further configured to adjust the robotic arm pose based on the same operation positioning point coordinates and preset robotic arm pose data to obtain the adjusted pose; determine whether the adjusted pose interferes with filming; if the adjusted pose interferes with filming, return to execute the adjustment of the robotic arm pose based on the preset robotic arm pose data to obtain the adjusted pose, until the adjusted pose is obtained.

[0137] The various modules in the aforementioned robotic arm positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0138] In one embodiment, a surgical robot system is provided, the system including a control center, a robotic arm, and a camera mounted on the robotic arm; wherein:

[0139] A camera mounted on the robotic arm is used to acquire images of the target physiological structure at multiple points on the robotic arm.

[0140] The control center is used to construct a three-dimensional model of the target's physiological structure based on the acquired images.

[0141] The control center is also used to determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model;

[0142] The control center is also used to move the robotic arm to the coordinates of each operating point, provided that the robotic arm can reach all the coordinates of the operating point.

[0143] In one embodiment, a computer device is provided, which may be a control center, and its internal structure diagram may be as follows: Figure 16As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores control center data. The network interface communicates with an external robotic arm via a network connection. When the computer program is executed by the processor, it implements a robotic arm positioning method.

[0144] Those skilled in the art will understand that Figure 16 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0145] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0146] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0147] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0149] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0150] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for positioning a robotic arm, characterized in that, The method includes: The robot acquires positioning images of the target physiological structure at multiple robot arm positions, inputs the positioning images into the robot arm position generation model, and obtains each acquisition point corresponding to the target physiological structure; the robot arm is then moved sequentially to each acquisition point to acquire the image corresponding to the target physiological structure. A three-dimensional model corresponding to the target physiological structure is constructed based on the acquired images. Based on the three-dimensional model, determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure; Once it is determined that the robotic arm can reach the coordinates of each of the said operation points, the robotic arm is moved to the coordinates of the said operation points to take images of the target physiological structure at each of the said operation point coordinates.

2. The method according to claim 1, characterized in that, After acquiring the acquisition points corresponding to the target physiological structure, the method further includes: Determine whether the robotic arm can reach each of the data collection points; In cases where there are sampling points that the robotic arm cannot reach, the sampling points corresponding to the target physiological structure are re-acquired based on the robotic arm sampling point generation model.

3. The method according to claim 1, characterized in that, The step of determining the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model includes: Based on the three-dimensional model, calculate the coordinates of each initial placement point for imaging the target physiological structure; Verify whether the robotic arm can reach the coordinates of each of the initial pendulum points; If the robotic arm can reach the coordinates of each of the initial sizing points, control the robotic arm to move to any of the initial sizing point coordinates; When the robotic arm is at the initial position coordinates, determine whether the initial position coordinates meet the requirements for filming. If the imaging requirements are met, the initial placement point coordinates will be used as the operation placement point coordinates. If the imaging requirements are not met, the initial position coordinates of the robotic arm are corrected until the corrected position coordinates meet the imaging requirements, and the corrected position coordinates are used as the operation position coordinates.

4. The method according to claim 3, characterized in that, The method further includes: If the robotic arm cannot reach any of the initial positioning point coordinates, return to the step of calculating the coordinates of each initial positioning point for taking pictures of the target physiological structure based on the three-dimensional model, until the robotic arm can reach each of the initial positioning point coordinates; or, If the robotic arm cannot reach any of the initial placement point coordinates, the initial placement point coordinates are calibrated according to the 3D image of the container until the robotic arm can reach all of the initial placement point coordinates.

5. The method according to claim 4, characterized in that, The step of calibrating the initial placement point coordinates based on the 3D image of the container until the robotic arm can reach each of the initial placement point coordinates includes: Based on the three-dimensional image display container, obtain the coordinates of each of the initial placement points and the position coordinates of the robotic arm in the same coordinate system; Based on the coordinate system, the initial swing point coordinates are adjusted according to the position coordinates of the robotic arm until the robotic arm can reach each of the initial swing point coordinates.

6. The method according to claim 1, characterized in that, The method further includes: If, at any of the operation point coordinates, there is interference between the robot arm pose and the film taking, the robot arm pose is adjusted to obtain the adjusted pose. Based on the adjusted pose and the coordinates of the operation point, the target physiological structure is photographed.

7. The method according to claim 6, characterized in that, The step of adjusting the pose of the robotic arm to obtain the adjusted pose includes: Based on the same operation point coordinates, the robot arm pose is adjusted according to the preset robot arm pose data to obtain the adjusted pose. Determine whether the adjusted pose interferes with filming; If the adjusted pose interferes with filming, the process returns to adjusting the robotic arm pose based on preset robotic arm pose data to obtain the adjusted pose, until the adjusted pose is obtained.

8. A robotic arm positioning device, characterized in that, The device includes: The image acquisition module is used to acquire positioning images of the target physiological structure at multiple robotic arm positions, input the positioning images into the robotic arm position generation model to obtain each acquisition point corresponding to the target physiological structure, and move the robotic arm sequentially to each acquisition point to acquire the image corresponding to the target physiological structure. The model building module is used to construct a three-dimensional model corresponding to the target physiological structure based on the acquired images. The coordinate determination module is used to determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model. The robotic arm positioning module is used to move the robotic arm to the coordinates of each operation positioning point, provided that the robotic arm can reach the coordinates of each operation positioning point, so as to take pictures of the target physiological structure at each operation positioning point coordinate.

9. A surgical robot system, characterized in that, The system includes a control center, a robotic arm, and a camera mounted on the robotic arm. The camera mounted on the robotic arm is used to acquire positioning images of the target physiological structure at multiple robotic arm points. The positioning images are then input into the robotic arm point generation model to obtain each acquisition point corresponding to the target physiological structure. The robotic arm is moved sequentially to each of the acquisition points to acquire images corresponding to the target physiological structure; The control center is used to construct a three-dimensional model corresponding to the target physiological structure based on the acquired images. The control center is also used to determine the coordinates of the operation points corresponding to each target operation point for imaging the target physiological structure based on the three-dimensional model. The control center is also used to move the robotic arm to the coordinates of the operation point when it is determined that the robotic arm can reach the coordinates of each operation point.

Images