Intraoperative robot swab calibration method and system based on three-dimensional laser radar
By carrying three-dimensional lidar on the robotic arm, automatic identification of the swab needle tip and direction is solved, the problem of insufficient accuracy and efficiency of swab sampling equipment in the existing technology is solved, the accuracy and reliability of collection are improved, and medical personnel are reduced labor intensity.
Patent Information
- Application Number
- CN202210644051.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-08
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-06-08
AI Technical Summary
Existing robot swab sampling equipment is difficult to meet the needs of high efficiency, accuracy and reliability in normalized prevention and control, especially in medical environments where medical staff are tired and have high precision requirements, and the equipment is large in size, high in price or cannot automatically identify the oral cavity. Location.
Three-dimensional lidar is used to carry on the robotic arm, and through three-dimensional point cloud acquisition, preprocessing and distance threshold screening, the position and direction of the swab needle tip are automatically identified, so as to realize swab calibration and automatic collection, and reduce medical personnel's operations.
It improves the collection accuracy, efficiency and reliability, reduces the labor intensity of medical staff, adapts to swabs of different sizes and positions, and achieves accurate real-time online calibration.
Smart Images

Figure CN114898071B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of robot data processing technology, and in particular to a three-dimensional laser radar-based intraoperative robot swab calibration method and system. Background Art
[0002] Nucleic acid testing is an important means of diagnosing the new coronavirus, and swabs are currently the main sampling method for diagnosing new coronavirus infection. At present, more and more cities have entered the stage of normalized prevention and control, resulting in an increase in the number of people collected, which has also put forward higher requirements on the efficiency of sampling. However, manual sampling is still the main method at this stage, and medical staff are at risk of infection during the sampling process. Moreover, long-term nucleic acid testing work and high temperatures will cause medical staff to be overworked and affect their physical condition. In addition, the sampling methods of each medical staff are different, which will also affect the sampling effect. Therefore, there is an urgent need for a fully automatic swab sampling robot equipment that can meet the needs of normalized prevention and control and is suitable for large-scale swab sample collection. At this stage, mainstream robots have not been widely used because they cannot meet the sampling needs well. The specific reasons are as follows:
[0003] 1) One is an industrial robot, which moves with the robot by installing a camera on the end effector. Although this hand-eye calibration system can obtain a clear image by moving the camera close to the target object. However, the camera's jitter when moving and the acceleration at the time of taking pictures will cause image smear, which will also have a certain impact on the calibration and target detection accuracy. Since industrial robots mostly do repetitive work, they only require high repetitive accuracy and do not require high absolute accuracy. Medical robots require high absolute accuracy, so ordinary industrial robot systems are not suitable for medical applications.
[0004] 2) Another method is to install a throat swab and an endoscope at the end of the robot arm, and manually operate it through visual monitoring and manual control of the remote control, so that the throat swab touches the ideal part of the posterior pharyngeal wall to collect the specimen. However, this robot is large in size and expensive.
[0005] 3) There is also a fully automatic collection robot. When the user scans the QR code, the robot puts down a disposable mouthpiece. After the user bites the mouthpiece and presses the start button, the robot can find the effective sampling site of the throat swab, collect, collect samples, package, store and disinfect. However, the detection needs to be carried out at the specified position in the fixed window, and the robot cannot automatically identify the actual position of the oral cavity. If the user's position is offset during the sampling process, effective collection cannot be carried out.
[0006] Based on the above shortcomings of the prior art, the present invention proposes an intraoperative robot swab calibration method based on three-dimensional laser radar, which automatically identifies the swab coordinate position through the three-dimensional laser radar carried on the robotic arm, and realizes robot swab calibration and automatic collection. Compared with the prior art, the accuracy, efficiency and reliability of collection are improved, and the access operation of medical staff can be further reduced, reducing labor intensity. Summary of the invention
[0007] In view of this, the embodiments of the present disclosure provide a three-dimensional laser radar-based intraoperative robot swab calibration method and system to at least partially solve the problems existing in the prior art.
[0008] In a first aspect, the present disclosure provides a method for calibrating an intraoperative robot swab based on a three-dimensional laser radar, comprising:
[0009] The swab at the front end of the robot arm is collected by a 3D laser radar carried by the robot arm to obtain original discrete point cloud data related to the swab, so as to pre-process the obtained original discrete point cloud data;
[0010] In the pre-processed point cloud data, the coordinate position of the pixel farthest from the 3D lidar camera is pre-screened by using the distance threshold and set as the coordinate position of the swab needle tip;
[0011] After determining the position of the swab needle tip, the distance threshold of all remaining points in the pre-processed point cloud data is judged to remove isolated points. When all points are processed, the collected points are stored to obtain stored point cloud data.
[0012] A linear relationship calculation is performed on the stored point cloud data to obtain the length, direction and relative position of the swab.
[0013] According to a specific implementation of the embodiment of the present disclosure, the three-dimensional point cloud collection of the swab at the front end of the robot arm by the three-dimensional laser radar carried by the robot arm includes:
[0014] Each frame of point cloud data of the swab is collected by the three-dimensional laser radar set on the robotic arm, and each frame of point cloud data includes point cloud coordinate information.
[0015] According to a specific implementation of the embodiment of the present disclosure, the preprocessing of the acquired original discrete point cloud data includes:
[0016] Establish a binary table through 3D point cloud data;
[0017] The domain range is set, and the collected point cloud data is subjected to binary method judgment in the binary table to screen and obtain the required point cloud data.
[0018] According to a specific implementation of the embodiment of the present disclosure, the preprocessing of the acquired original discrete point cloud data includes:
[0019] The point cloud data filtered by the binary method is subjected to morphological processing of the binary image to form a point cloud data image.
[0020] According to a specific implementation of the embodiment of the present disclosure, the coordinate position of the pixel farthest from the three-dimensional laser radar camera is pre-screened by a distance threshold and set as the coordinate position of the swab needle tip, including:
[0021] Filter out the coordinate position (U0, V0) of the pixel point P0 farthest from the 3D lidar camera
[0022] The collected P0 points are further screened and judged according to the coordinate position relationship of the pixel points in the point cloud data.
[0023] According to a specific implementation of the embodiment of the present disclosure, the collected P0 point is further screened and judged according to the coordinate position relationship of the pixel points of the point cloud data, including:
[0024] If the distance between point P0 and the 3D laser radar camera is R0, and R0 is less than the set distance threshold R, then this coordinate position point is set as the coordinate position of the swab needle tip;
[0025] If R0 is greater than the set distance threshold R, the point cloud data collected this time is discarded, and a new frame of point cloud data is recollected through the 3D lidar camera.
[0026] According to a specific implementation of the embodiment of the present disclosure, after determining the position of the swab needle tip, performing distance threshold judgment on all remaining points of the pre-processed point cloud data includes:
[0027] Select pixel point P i Coordinate position (U i ,V i ), the distance from the 3D LiDAR camera is R i ;
[0028] If R i If the distance is less than the set distance threshold R, the coordinate position will be retained;
[0029] If R i If the distance is greater than the set distance threshold R, the isolated point cloud data will be removed.
[0030] According to a specific implementation of an embodiment of the present disclosure, the performing linear relationship calculation on the stored point cloud data includes:
[0031] Select all points on the upper cross section with coordinates Pu1 , P u2 , P u3 ......P un , the coordinate positions are (U u1 , V u1 )、(U u2 , V u2 )、(U u3 , V u3 )......(U un , V un );
[0032] Select all points on the lower cross section with coordinates P d1 , P d2 , P d3 ......P dn , the coordinate positions are
[0033] (U d1 , V d1 )、(U d2 , V d2 )、(U d3 , V d3 )......(U dn , V dn );
[0034] Take the average value for all points on the upper cross section:
[0035]
[0036] Take the average value for all points on the lower cross section:
[0037]
[0038] If the positions of the two average points P1 and P2 are not in the same straight line with the swab needle tip P0, the data will be discarded;
[0039] If the two average point positions P1 and P2 are on the same straight line as the swab needle tip P0, this direction is determined to be the direction of the swab.
[0040] According to a specific implementation of the embodiment of the present disclosure, the linear relationship calculation is performed on the stored point cloud data to obtain the length, direction and relative posture of the swab, including:
[0041] From the stored point cloud data, filter out the point coordinates P with the smallest distance from the pixel to the camera position e , then the swab length L = P0-P e ;
[0042] By establishing a point linear relationship, the direction of the swab is calculated;
[0043] According to the calculated swab length and swab direction, the relative position and posture of the swab are finally obtained.
[0044] In a second aspect, the present disclosure provides an intraoperative robot swab calibration system based on a three-dimensional laser radar, comprising:
[0045] A robotic arm, wherein the robotic arm is combined together by a plurality of joints;
[0046] A swab, which is fixed on the end of the robotic arm and has a posture that changes with the movement of the robotic arm and is used for real-time online calibration;
[0047] A three-dimensional laser radar, which is installed on the end effector of the robotic arm and moves with the end of the robotic arm, and the relative position between the three-dimensional laser radar and the end of the robotic arm remains unchanged;
[0048] A computing module, wherein the computing module is used to receive the point cloud image data collected by the three-dimensional laser radar, so as to execute the intraoperative robot swab calibration method based on the three-dimensional laser radar in the first aspect or any implementation of the first aspect based on the point cloud image data.
[0049] In the third aspect, the embodiments of the present disclosure also provide a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the intraoperative robot swab calibration method based on three-dimensional lidar in the aforementioned first aspect or any implementation of the first aspect.
[0050] In a fourth aspect, an embodiment of the present disclosure further provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the intraoperative robotic swab calibration method based on three-dimensional lidar in the aforementioned first aspect or any implementation of the first aspect.
[0051] The intraoperative robot swab calibration scheme based on three-dimensional laser radar in the embodiment of the present disclosure includes three-dimensional point cloud acquisition of the swab at the front end of the robot arm through the three-dimensional laser radar carried on the robot mechanical arm, and obtaining the original discrete point cloud data related to the swab, so as to pre-process the acquired original discrete point cloud data; in the point cloud data after pre-processing, the coordinate position of the pixel point farthest from the three-dimensional laser radar camera is pre-screened by the distance threshold, and set as the coordinate position of the swab needle tip; after determining the position of the swab needle tip, the distance threshold is judged for all the remaining points of the point cloud data after pre-processing, and isolated points are eliminated. When all points are processed, the collected points are stored to obtain the stored point cloud data; the linear relationship calculation is performed on the stored point cloud data to obtain the length, direction and relative posture of the swab. Through the processing scheme disclosed in the present disclosure, the swab coordinate position is automatically identified by the three-dimensional laser radar carried on the mechanical arm, and the robot swab calibration and automatic acquisition are realized. Compared with the existing technology, the accuracy, efficiency and reliability of acquisition are improved, and the access operation of medical staff can be further reduced, and the labor intensity can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0053] Figure 1 A system composition diagram provided for an embodiment of the present disclosure;
[0054] Figure 2 A schematic diagram of a 3D laser radar-based intraoperative robot swab calibration method provided in an embodiment of the present disclosure;
[0055] Figure 3 A point cloud data pixel point coordinate position relationship diagram provided by an embodiment of the present disclosure;
[0056] Figure 4 A schematic diagram of pixel point selection for point cloud data provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0057] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0058] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.
[0059] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.
[0060] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.
[0061] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.
[0062] The disclosed embodiment provides a 3D laser radar-based intraoperative robot swab calibration method. The 3D laser radar-based intraoperative robot swab calibration method provided in this embodiment can be performed by a computing device, which can be implemented as software, or as a combination of software and hardware, and the computing device can be integrated in a server, a client, etc.
[0063] See also Figure 1 , Figure 2 , Figure 3 and Figure 4The intraoperative robot swab calibration method based on three-dimensional laser radar in the embodiment of the present disclosure may include the following steps:
[0064] S101, performing three-dimensional point cloud acquisition of a swab at the front end of the robot arm by using a three-dimensional laser radar carried on the robot arm to obtain original discrete point cloud data related to the swab, so as to pre-process the obtained original discrete point cloud data;
[0065] S102, in the pre-processed point cloud data, pre-screening the coordinate position of the pixel point farthest from the three-dimensional laser radar camera by using a distance threshold, and setting it as the coordinate position of the swab needle tip;
[0066] S103, after determining the position of the swab needle tip, a distance threshold is determined for all remaining points of the pre-processed point cloud data to remove isolated points. When all points are processed, the collected points are stored to obtain stored point cloud data;
[0067] S104, performing linear relationship calculation on the stored point cloud data to obtain the length, direction and relative position of the swab.
[0068] Specifically, during implementation, the main scheme of the present invention includes the following aspects:
[0069] The robotic arm is equipped with a 3D laser radar for point cloud data collection
[0070] Since the medical robot system has extremely high requirements for absolute accuracy, if it is equipped with an ordinary camera, the camera shake when the robot arm is moving and the acceleration at the time of taking pictures will cause image smearing, thereby affecting the accuracy of calibration and target detection. In addition to the high absolute accuracy requirements, medical robots also have extremely high requirements for instant response. Therefore, the present invention adopts a three-dimensional lidar camera, which has millimeter-level accuracy and a response time of no more than 33ms. It also has an internal visual processor to reduce motion blur artifacts, thereby greatly reducing the impact of the robot arm movement on the camera during the hand-eye calibration process, making the system have higher precision and accuracy. It can adapt to swabs of different sizes and postures while achieving accurate real-time online calibration.
[0071] In actual applications, the size and length of each swab are different, and the placement is unknown. In order to adapt to swabs of different sizes and postures, this system uses the 3D laser radar carried on the robotic arm to collect 3D point clouds and pre-process the acquired original discrete point cloud data; pre-screen the coordinate position of the pixel farthest from the 3D laser radar camera through the distance threshold and set it as the coordinate position of the swab needle tip; after determining the position of the swab needle tip, the distance threshold is judged for all remaining points to eliminate isolated points; when all points are processed, the collected points are stored and a linear relationship is obtained; finally, the length, direction and relative posture of the swab are calculated.
[0072] See also Figure 1 , this system consists of 3 parts:
[0073] 1. Robotic arm: high-performance multi-axis robotic arm for calculation and control processing;
[0074] 2. 3D laser radar: installed on the end effector of the robotic arm and moves with the robot, and the relative position between the two remains unchanged;
[0075] 3. Swab: fixed on the end of the robotic arm, its position changes with the movement of the robotic arm, and is used for real-time online calibration.
[0076] The flow chart of the method of the present invention is as follows Figure 2 As shown, the specific steps include:
[0077] 1. Collect and obtain point cloud data
[0078] The robotic arm collects a frame of point cloud data through the three-dimensional laser radar, and each point cloud data includes point cloud coordinate information.
[0079] 2. Filter data
[0080] 1) Establish a binary table through 3D point cloud data;
[0081] 2) Set the domain scope, perform binary judgment on the collected point cloud data, and filter to obtain the required point cloud data.
[0082] 3. Graphics processing
[0083] The filtered point cloud data is subjected to morphological processing of binary images.
[0084] 4. Process and obtain the coordinate position of the swab needle tip
[0085] 1) Filter out the coordinate position (U0, V0) of the pixel point P0 farthest from the 3D lidar camera
[0086] 2) The collected P0 points are Figure 3The pixel coordinate position relationship of the point cloud data shown is further screened and judged:
[0087] ① If the distance between point P0 and the 3D LiDAR camera is R0, and R0 is less than the set distance threshold R,
[0088] Then this coordinate position point is set as the coordinate position of the swab needle tip;
[0089] ② If R0 is greater than the set distance threshold R, the point cloud data collected this time is discarded, and a new frame of point cloud data is recollected through the 3D lidar camera.
[0090] 5. Eliminate isolated points
[0091] After the swab needle tip position is determined, the remaining points are judged and screened by the distance threshold:
[0092] 1) Select pixel point P i Coordinate position (U i ,V i ), the distance from the 3D LiDAR camera is R i .
[0093] ① If R i If the distance is less than the set distance threshold R, the coordinate position will be retained;
[0094] ② If R i If the distance is greater than the set distance threshold R, the isolated point cloud data will be removed;
[0095] 2) Repeat the operation in 1) until all the point cloud data in this frame are judged and the remaining point coordinates are stored.
[0096] 6. Establish point linear relationship
[0097] Since the swab is a straight line, we can obtain the direction of the swab by establishing a point linear relationship. Figure 4 As shown, the specific operations are as follows:
[0098] 1) Select the coordinates of all points on each cross section in the upper and lower cross sections, take the average value on the same cross section, and calculate and judge the two average values obtained with the swab needle tip:
[0099] ① Select all point coordinates P on the upper cross section u1 , P u2 , P u3 ......P un , the coordinate positions are (U u1 , V u1 )、(U u2 , V u2 )、(U u3, V u3 )......(U un , V un );
[0100] ②Select all point coordinates P on the lower cross section d1 , P d2 , P d3 ......P dn , the coordinate positions are
[0101] (U d1 , V d1 )、(U d2 , V d2 )、(U d3 , V d3 )......(U dn , V dn );
[0102] ③ Take the average value of all points on the upper cross section:
[0103]
[0104] ④ Take the average value of all points on the lower cross section:
[0105]
[0106] ⑤ If the two average point positions P1 and P2 are not on the same straight line as the swab needle tip P0, the data will be discarded;
[0107] ⑥ If the two average point positions P1 and P2 are on the same straight line as the swab needle tip P0, this direction is determined as the direction of the swab;
[0108] 2) Repeat step 1) until the orientation of the swab is finally obtained.
[0109] 5. Calculate the swab length, direction and relative position
[0110] 1) Calculate the swab length
[0111] From the stored point cloud data, filter out the point coordinates P with the smallest distance from the pixel to the camera position e , then the swab length L = P0-P e ;
[0112] 2) Calculate the swab direction
[0113] According to step 4, by establishing a point linear relationship, the direction of the swab is calculated;
[0114] 3) The relative position of the swab is finally obtained based on the calculated swab length and swab direction.
[0115] According to a specific implementation of the embodiment of the present disclosure, the three-dimensional point cloud collection of the swab at the front end of the robot arm by the three-dimensional laser radar carried by the robot arm includes:
[0116] Each frame of point cloud data of the swab is collected by the three-dimensional laser radar set on the robotic arm, and each frame of point cloud data includes point cloud coordinate information.
[0117] According to a specific implementation of the embodiment of the present disclosure, the preprocessing of the acquired original discrete point cloud data includes:
[0118] Establish a binary table through 3D point cloud data;
[0119] The domain range is set, and the collected point cloud data is subjected to binary method judgment in the binary table to screen and obtain the required point cloud data.
[0120] According to a specific implementation of the embodiment of the present disclosure, the preprocessing of the acquired original discrete point cloud data includes:
[0121] The point cloud data filtered by the binary method is subjected to morphological processing of the binary image to form a point cloud data image.
[0122] According to a specific implementation of the embodiment of the present disclosure, the coordinate position of the pixel farthest from the three-dimensional laser radar camera is pre-screened by a distance threshold and set as the coordinate position of the swab needle tip, including:
[0123] Filter out the coordinate position (U0, V0) of the pixel point P0 farthest from the 3D lidar camera
[0124] The collected P0 points are further screened and judged according to the coordinate position relationship of the pixel points in the point cloud data.
[0125] According to a specific implementation of the embodiment of the present disclosure, the collected P0 point is further screened and judged according to the coordinate position relationship of the pixel points of the point cloud data, including:
[0126] If the distance between point P0 and the 3D laser radar camera is R0, and R0 is less than the set distance threshold R, then this coordinate position point is set as the coordinate position of the swab needle tip;
[0127] If R0 is greater than the set distance threshold R, the point cloud data collected this time is discarded, and a new frame of point cloud data is recollected through the 3D lidar camera.
[0128] According to a specific implementation of the embodiment of the present disclosure, after determining the position of the swab needle tip, performing distance threshold judgment on all remaining points of the pre-processed point cloud data includes:
[0129] Select pixel point P i Coordinate position (U i ,V i ), the distance from the 3D LiDAR camera is R i ;
[0130] If R i If the distance is less than the set distance threshold R, the coordinate position will be retained;
[0131] If R i If the distance is greater than the set distance threshold R, the isolated point cloud data will be removed.
[0132] According to a specific implementation of an embodiment of the present disclosure, the performing linear relationship calculation on the stored point cloud data includes:
[0133] Select all points on the upper cross section with coordinates P u1 , P u2 , P u3 ......P un , the coordinate positions are (U u1 , V u1 )、(U u2 , V u2 )、(U u3 , V u3 )......(U un , V un );
[0134] Select all points on the lower cross section with coordinates P d1 , P d2 , P d3 ......P dn , the coordinate positions are
[0135] (U d1 , V d1 )、(U d2 , V d2 )、(U d3 , V d3 )......(U dn , V dn );
[0136] Take the average value for all points on the upper cross section:
[0137]
[0138] Take the average value for all points on the lower cross section:
[0139]
[0140] If the positions of the two average points P1 and P2 are not in the same straight line with the swab needle tip P0, the data will be discarded;
[0141] If the two average point positions P1 and P2 are on the same straight line as the swab needle tip P0, this direction is determined to be the direction of the swab.
[0142] According to a specific implementation of the embodiment of the present disclosure, the linear relationship calculation is performed on the stored point cloud data to obtain the length, direction and relative posture of the swab, including:
[0143] From the stored point cloud data, filter out the point coordinates P with the smallest distance from the pixel to the camera position e , then the swab length L = P0-P e ;
[0144] By establishing a point linear relationship, the direction of the swab is calculated;
[0145] According to the calculated swab length and swab direction, the relative position and posture of the swab are finally obtained.
[0146] Compared with the above method embodiment, the present invention also provides an intraoperative robot swab calibration system based on three-dimensional laser radar, comprising:
[0147] A robotic arm, wherein the robotic arm is combined together by a plurality of joints;
[0148] A swab, which is fixed on the end of the robotic arm and has a posture that changes with the movement of the robotic arm and is used for real-time online calibration;
[0149] A three-dimensional laser radar, which is installed on the end effector of the robotic arm and moves with the end of the robotic arm, and the relative position between the three-dimensional laser radar and the end of the robotic arm remains unchanged;
[0150] A computing module, wherein the computing module is used to receive the point cloud image data collected by the three-dimensional laser radar, so as to execute the method described in the above embodiment based on the point cloud image data.
[0151] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0152] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".
[0153] It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.
[0154] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A method for intraoperative robot swab calibration based on three-dimensional laser radar, characterized in that: include: The swab at the front end of the robot arm is collected by a 3D laser radar carried by the robot arm to obtain original discrete point cloud data related to the swab, so as to pre-process the obtained original discrete point cloud data; In the pre-processed point cloud data, the coordinate position of the pixel farthest from the 3D lidar camera is pre-screened by using the distance threshold and set as the coordinate position of the swab needle tip; After determining the position of the swab needle tip, the distance threshold of all remaining points in the pre-processed point cloud data is judged to remove isolated points. When all points are processed, the collected points are stored to obtain stored point cloud data. Performing linear relationship calculation on the stored point cloud data to obtain the length, direction and relative position of the swab; After determining the position of the swab needle tip, the distance threshold judgment is performed on all the remaining points of the pre-processed point cloud data, including: selecting the coordinate position (Ui, Vi) of the pixel point Pi, and the distance from the three-dimensional laser radar camera is Ri; if Ri is less than the set distance threshold R, then this coordinate position is retained; if Ri is greater than the set distance threshold R, then this isolated point cloud data is eliminated; The performing linear relationship calculation on the stored point cloud data includes: Select all points on the upper cross section with coordinates P u1 , P u2 , P u3 ......P un , the coordinate positions are (U u1 , V u1 )、(U u2 , V u2 )、(U u3 , V u3 )......(U un , V un ); Select all points on the lower cross section with coordinates P d1 , P d2 , P d3 ......P dn , the coordinate positions are (U d1 , V d1 )、(U d2 , V d2 )、(U d3 , V d3 )......(U dn , V dn ); Take the average value for all points on the upper cross section: Take the average value for all points on the lower cross section: If the positions of the two average points P1 and P2 are not in the same straight line with the swab needle tip P0, the data will be discarded; If the two average point positions P1 and P2 are in the same straight line with the swab needle tip P0, this direction is determined to be the direction of the swab; The linear relationship calculation is performed on the stored point cloud data to obtain the length, direction and relative posture of the swab, including: From the stored point cloud data, the point coordinate Pe with the smallest distance from the pixel to the camera position is selected, then the swab length L = P0-Pe; by establishing a point linear relationship, the direction of the swab is calculated; based on the calculated swab length and swab direction, the relative posture of the swab is finally obtained.
2. The method according to claim 1, characterized in that The three-dimensional point cloud collection of the swab at the front end of the robot arm by the three-dimensional laser radar carried by the robot arm includes: Each frame of point cloud data of the swab is collected by the three-dimensional laser radar set on the robotic arm, and each frame of point cloud data includes point cloud coordinate information.
3. The method according to claim 2, characterized in that The preprocessing of the acquired original discrete point cloud data includes: Establish a binary table through 3D point cloud data; The domain range is set, and the collected point cloud data is subjected to binary method judgment in the binary table to screen and obtain the required point cloud data.
4. The method according to claim 3, characterized in that The preprocessing of the acquired original discrete point cloud data includes: The point cloud data filtered by the binary method is subjected to morphological processing of the binary image to form a point cloud data image.
5. The method according to claim 1, characterized in that The method of pre-screening the coordinate position of the pixel point farthest from the three-dimensional laser radar camera by using the distance threshold and setting it as the coordinate position of the swab needle tip includes: Filter out the coordinate position (U0, V0) of the pixel point P0 farthest from the 3D lidar camera; If the distance between point P0 and the 3D laser radar camera is R0, and R0 is less than the set distance threshold R, then this coordinate position point is set as the coordinate position of the swab needle tip; If R0 is greater than the set distance threshold R, the point cloud data collected this time is discarded, and a new frame of point cloud data is collected again through the 3D lidar camera.
6. A three-dimensional laser radar-based intraoperative robot swab calibration system, characterized in that: include: A robotic arm, wherein the robotic arm is combined together by a plurality of joints; A swab, which is fixed on the end of the robotic arm and has a posture that changes with the movement of the robotic arm and is used for real-time online calibration; A three-dimensional laser radar, which is installed on the end effector of the robotic arm and moves with the end of the robotic arm, and the relative position between the three-dimensional laser radar and the end of the robotic arm remains unchanged; A computing module, wherein the computing module is used to receive the point cloud image data collected by the three-dimensional laser radar, so as to execute the method according to any one of claims 1 to 5 based on the point cloud image data.
