Medical robot vision automatic alignment method, electronic equipment and storage medium

Abstract

The invention belongs to the technical field of visual guidance, and discloses a medical robot visual automatic alignment method, electronic equipment and a storage medium, and the method comprises the steps: recognizing the contours of a plurality of circles in a light limiting cylinder picture, and fitting the contours into a circle; acquiring the center of the circle, and fitting a plane according to the center point coordinate of the circle; the normal vector of the fitting plane is calculated, the normal vector of the plane serves as the alignment posture of the robot, and the robot is aligned on the normal vector; obtaining three-dimensional coordinates of the circle center of the calibration plate under a camera coordinate system according to the obtained circle center image coordinates on the calibration plate; and through the transformation matrix, converting the three-dimensional coordinate of the circle center of the calibration plate under the camera coordinate system into a robot coordinate system, and carrying out secondary alignment on the robot. A machine vision alignment method is adopted, alignment between the robot and the light limiting cylinder is achieved, the alignment precision is high, the corresponding response speed is high, and the alignment efficiency and safety of the medical robot are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual guidance, and in particular relates to a medical robot visual automatic alignment method, electronic equipment and storage medium. Background Art

[0002] During the treatment process of the E-Flash medical robot, the terminal radiation source needs to be aimed at the patient's diseased area for radiotherapy. Manual alignment has poor accuracy and is prone to misoperation, which may cause excess radiation overflow during radiotherapy and harm other non-lesioned parts of the patient. Therefore, automatic alignment technology is needed for alignment.

[0003] In radiotherapy operations, a beam-limiting device of appropriate size and shape is usually used and placed above the tumor or tumor bed to determine the irradiation range. Therefore, the terminal radiation source needs to be aligned with the beam-limiting device.

[0004] The accuracy of traditional laser alignment is easily affected by environmental factors such as light intensity, characteristics of the reflective surface, and distance, resulting in low alignment accuracy. Laser alignment requires the use of laser transmitters and receivers, which increases the complexity and cost of the system; at the same time, laser alignment has high requirements for the target surface, which requires a certain degree of emissivity or transparency, and has high requirements for the beam limiting device, which increases the cost of the beam limiting device. Summary of the invention

[0005] The purpose of the present invention is to provide a medical robot visual automatic alignment method to solve the problems of low alignment accuracy and high requirements on beam limiting devices in existing automatic alignment methods.

[0006] The present invention is achieved through the following technical solutions: The medical robot vision automatic alignment method is used to align the robot with a light-limiting cylinder. A plurality of circles of the same size are arranged on the disk of the light-limiting cylinder, and the circles are evenly distributed along the circumference of the disk. The distance between the center of the circle and the center of the light-limiting cylinder is equal. The automatic alignment method comprises: Preprocess the light-limiting tube images acquired by the binocular vision camera; Identify the contours of multiple circles in the light-limiting tube image and fit them into circles; Get the center of the circle, get the coordinates of the center point of the circle, and fit a plane based on the coordinates of the center point of the circle; Calculate the normal vector of the fitting plane, use the normal vector of the plane as the robot alignment posture, and align the robot on the normal vector so that the robot alignment posture on the normal vector is within the error range; The calibration plate is photographed from different angles using a binocular vision camera to obtain the calibration parameters of the binocular vision camera. The three-dimensional coordinates of the center of the calibration plate in the camera coordinate system are obtained based on the image coordinates of the center of the calibration plate. Through the transformation matrix, the homogeneous coordinates of the center of the calibration plate in the camera coordinate system are converted to the robot coordinate system, and the robot is aligned again.

[0007] In some embodiments of the present invention, the step of determining whether the alignment posture of the robot on the normal vector is within the error range includes: The average value of the center coordinates of multiple circles is taken to obtain the virtual center of the light-limiting cylinder through fitting; Calculate the distance between the virtual center and the real center according to the coordinates of the virtual center of the light limiting cylinder and the real center of the light limiting cylinder; The calculated distance is compared with the set error threshold to determine whether the robot's alignment posture on the normal vector is within the error range.

[0008] In some embodiments of the present invention, during the secondary alignment of the robot, the light-limiting cylinder image is continuously collected, the light-limiting cylinder dot in the image is identified, and the average value is taken together with the light-limiting cylinder dot of the light-limiting cylinder image collected last time as the new light-limiting cylinder dot; The coordinate transformation is performed on the new light-limiting cylinder point, and the homogeneous coordinates of the center of the calibration plate in the robot coordinate system obtained by the coordinate transformation are used to determine whether the robot and the light-limiting cylinder are aligned.

[0009] In some embodiments of the present invention, when the binocular vision camera cannot capture the dot of the light-limiting cylinder, the secondary alignment operation of the robot is terminated.

[0010] In some embodiments of the present invention, during the secondary alignment of the robot, the image of the light limiting tube is continuously collected to identify the center point of the light limiting tube plane in the image. When the difference between the image coordinates of the center point of the light limiting tube plane and the result of the last identification is less than a set threshold, it is determined that the robot's radiation source is aligned with the light limiting tube.

[0011] In some embodiments of the present invention, the binocular vision camera continuously collects the light-limiting tube image at a frame rate of 5-10 fps.

[0012] In some embodiments of the present invention, the step of preprocessing the light-limiting tube image includes performing adaptive filtering on the image, converting the filtered image into a grayscale image, and binarizing the grayscale image using an adaptive threshold method to obtain a binary image.

[0013] In some embodiments of the present invention, the step of processing the fitted circle is further included, including judging the average brightness value and the degree of the center of the fitted circle and eliminating the error circle obtained by fitting.

[0014] In another aspect, the present invention further provides an electronic device, comprising: processor; and, A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to perform the medical robot visual automatic alignment method by executing the executable instructions.

[0015] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the medical robot visual automatic alignment method is implemented.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: The present invention adopts a machine vision alignment method to achieve alignment between the robot and the light-limiting cylinder, with high alignment accuracy and fast corresponding response speed, thereby improving the alignment efficiency and safety of the medical robot, and having low requirements for the light-limiting cylinder, thereby reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 Schematic diagram of the medical robot structure according to an embodiment of the present invention.

[0019] Figure 2 Schematic diagram of the structure of the light-limiting tube according to an embodiment of the present invention.

[0020] Figure 3 Schematic diagram of the arrangement of circles on the light-limiting tube according to an embodiment of the present invention.

[0021] in: 1. Binocular vision camera, 2. Light limiting tube, 3. Circle. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0023] The present invention uses machine vision to achieve automatic alignment. A binocular vision camera 1 is installed on the robot end effector. The binocular vision camera 1 is located on the same horizontal line. Two pictures of the lower limit light tube of the same scene are captured by the left and right cameras, and the image depth information is obtained by using the principle of visual difference.

[0024] The light-limiting cylinder 2 is located below the binocular vision camera and is used for the robot's alignment operation. A circular disk is provided at the end of the light-limiting cylinder 2, and a plurality of high-precision circles 3 are provided on the upper surface of the disk. The circles are evenly distributed on the disk along the circumference, the interval angles between each circle are the same, and the distances between the centers of each circle and the center of the light-limiting cylinder are the same.

[0025] In some embodiments of the present invention, a medical robot vision automatic alignment method includes: Preprocess the light-limiting tube images acquired by the binocular vision camera; Identify the contours of multiple circles in the light-limiting tube image and fit them into circles; Get the center of the circle, get the coordinates of the center point of the circle, and fit a plane based on the coordinates of the center point of the circle; Calculate the normal vector of the fitting plane, use the normal vector of the plane as the robot alignment posture, and align the robot on the normal vector until the robot alignment posture on the normal vector is within the error range; Obtain the pixel coordinates of the center of the light-limiting cylinder, and obtain the three-dimensional coordinates of the center of the light-limiting cylinder in the camera coordinate system according to the calibration parameters of the binocular vision camera; The three-dimensional coordinates of the center of the light-limiting cylinder are converted into the robot coordinate system through the transformation matrix, and the robot is aligned twice.

[0026] In some embodiments, during the secondary alignment of the robot, the light limiting cylinder image is continuously collected, the center of the light limiting cylinder in the image is identified, and the average value is taken together with the center of the light limiting cylinder of the light limiting cylinder image collected last time as the new center of the light limiting cylinder; The coordinates of the new light-limiting cylinder center are transformed and the robot is aligned again.

[0027] In some embodiments, when the binocular vision camera cannot capture the center of the light-limiting cylinder, the secondary alignment operation of the robot is terminated.

[0028] In some embodiments, during the secondary alignment of the robot, the image of the light limiting cylinder is continuously collected to identify the center of the light limiting cylinder in the image. When the difference between the image coordinates of the center of the light limiting cylinder and the result of the last identification is less than a set threshold, the secondary alignment operation of the robot is terminated.

[0029] The alignment method of the present invention is described below in conjunction with specific embodiments.

[0030] After acquiring the light-limiting tube image, the binocular vision camera performs a preprocessing operation on the light-limiting tube image to obtain the contours of each circle in the light-limiting tube image and fit it into a circle; including: Adaptively filter the image to reduce image noise; The filtered image is converted into a grayscale image, and the grayscale image is binarized using the adaptive threshold method to generate a binary image. Each pixel will be determined to be black or white based on the brightness of its surrounding pixels.

[0031] The binary image is processed using a contour detection algorithm to extract the contour information of the circle, and a fitting circle is obtained based on the extracted contour information.

[0032] By judging the average brightness of the center of the fitting circle, the error circle obtained by fitting is eliminated. The method used is: Count the pixel values ​​in the center area, add up all the pixel values ​​and take the average value. If the average brightness value of the area is close to 255, the center area is judged to be white, indicating that the outline of the fitted circle is approximately circular; if the average brightness value of the area is far below 255, the center area is judged to be not entirely white, indicating that the outline of the fitted circle is not circular and needs to be removed.

[0033] Furthermore, the value of the center degree of the fitted circle is used to determine whether the contour of the fitted circle is a circle. The value of the center degree is between 0 and 1, and the closer the value is to 1, the closer it is to a circle. A threshold of the center degree can be set. If the calculated center degree is greater than the set threshold, the contour of the fitted circle is judged to be a circle; if it is less than the set threshold, the contour of the fitted circle is judged not to be a circle and needs to be removed.

[0034] Reference Figure 2and Figure 3 , taking setting n circles on the disk of the light-limiting tube as an example.

[0035] Get the contours of each circle, locate the centers of n circles, get the coordinates of the center points of the circles, fit a circle based on the coordinates of the center points of the circles, and get a fitting plane based on the plane where the fitted circle lies. The plane equation can be expressed as: ; in,( a,b ) is the coordinate of the center point of the fitted circle, r is the radius of the fitted circle.

[0036] Calculate the normal vector of the fitted plane; The normal vector of a plane is a vector perpendicular to the plane. For a point in three-dimensional space, the calculation formula for the plane normal vector is: ; in, is the vector 1 in the three-dimensional space formed by the center point of the light-limiting cylinder, is the vector 2 in the three-dimensional space formed by the center point of the light-limiting cylinder.

[0037] The normal vector of the plane is used as the robot alignment posture, and the robot is aligned on the normal vector. When the robot posture is within the error range, the alignment is stopped.

[0038] The method used to determine whether the robot's alignment posture on the normal vector is within the error range is: Take the average value of the center coordinates of n circles and fit to get the virtual center of the light-limiting tube ; expressed as: ; in,( ) are the coordinates of the center points of each circle, and n is the number of circles.

[0039] According to the coordinates of the virtual center ( ), the center point of the upper limit light cylinder plane of the image is taken as the real center, and the coordinates of the real center are obtained according to the coordinates of the center point of the circle fitted by the center point of the circle, expressed as ( ), the Euclidean distance is used to calculate the distance between the virtual center and the real center, and the error between the virtual center and the real center is verified.

[0040] The distance between the virtual center and the real center is expressed as: .

[0041] This method can control the error between the fitting center and the true center within 0.1 mm.

[0042] After the plane normal vector is aligned with the robot normal vector, the calibration plate is photographed from different angles using a binocular vision camera, and the calibration parameters of the binocular vision camera are obtained through the calibration algorithm.

[0043] Get the image coordinates of the center of the circle on the calibration plate ( ), according to the camera internal parameters obtained by camera calibration, the three-dimensional coordinates of the center of the calibration plate in the camera coordinate system are obtained ( X,Y,Z ), expressed as: ; in, is the inverse matrix of the camera intrinsic parameter matrix, Z is the depth value, is the pixel coordinate of the image principal point.

[0044] Through the transformation matrix, it is converted to a point in the robot coordinate system, expressed as: ; in, R is the rotation matrix, t is the translation matrix, X,Y,Z is the homogeneous coordinate of the center of the calibration plate in the camera coordinate system, is the homogeneous coordinate of the center of the calibration plate in the robot coordinate system.

[0045] After three-dimensional space coordinate conversion, the image coordinates of the center of the calibration plate in the binocular camera coordinate system identified by the calibration algorithm are converted to the robot coordinates for robot alignment.

[0046] During the alignment process, an open-loop control method is adopted. The camera shoots and identifies the light-limiting cylinder dot in real time at a frame rate of 5fps. The light-limiting cylinder dot is a high-precision circle on the light-limiting cylinder. The homogeneous coordinates of the center point of the calibration plate in the robot coordinate system obtained by coordinate conversion are used to detect whether the robot and the light-limiting cylinder are aligned.

[0047] The robot does not rely on position detection feedback, but continuously collects images and takes the average value of the image coordinate data of the light-limiting cylinder dot identified in the last photo as the new light-limiting cylinder dot. When the robot is aligned with the light-limiting cylinder, or the camera cannot capture the light-limiting cylinder dot, the recognition process will stop automatically. At this time, no coordinate averaging or accumulation processing is performed, and the alignment process ends directly.

[0048] In this process, when the calculated image coordinates of the center point of the light limiting cylinder plane differ from the previous result by less than 0.5 pixel, the positioning calculation of the center point of the light limiting cylinder plane is completed, the control ends, and the radiation source is aligned with the light limiting cylinder.

[0049] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A medical robot vision automatic alignment method for aligning a robot with a light-limiting tube, characterized in that: A plurality of circles of the same size are arranged on the disk of the light limiting cylinder, and the circles are evenly distributed along the circumference of the disk, and the distances between the centers of the circles and the center of the light limiting cylinder are equal; the automatic alignment method includes: Preprocess the light-limiting tube image acquired by the binocular vision camera; Identify the contours of multiple circles in the light-limiting tube image and fit them into circles; Get the center of the circle, get the coordinates of the center point of the circle, and fit a plane based on the coordinates of the center point of the circle; Calculate the normal vector of the fitting plane, use the normal vector of the plane as the robot alignment posture, and align the robot on the normal vector so that the robot alignment posture on the normal vector is within the error range; The calibration plate is photographed from different angles using a binocular vision camera to obtain the calibration parameters of the binocular vision camera. The three-dimensional coordinates of the center of the calibration plate in the camera coordinate system are obtained based on the image coordinates of the center of the calibration plate. Through the transformation matrix, the homogeneous coordinates of the center of the calibration plate in the camera coordinate system are converted to the robot coordinate system, and the robot is aligned again.

2. The medical robot vision automatic alignment method according to claim 1, characterized in that: The steps of determining that the robot's alignment posture on the normal vector is within the error range include: The average value of the center coordinates of multiple circles is taken to obtain the virtual center of the light-limiting cylinder through fitting; Calculate the distance between the virtual center and the real center according to the coordinates of the virtual center of the light limiting cylinder and the real center of the light limiting cylinder; The calculated distance is compared with the set error threshold to determine whether the robot's alignment posture on the normal vector is within the error range.

3. The medical robot vision automatic alignment method according to claim 1, characterized in that: During the secondary alignment of the robot, the light-limiting cylinder image is continuously collected, the light-limiting cylinder dot in the image is identified, and the average value is taken together with the light-limiting cylinder dot of the light-limiting cylinder image collected last time as the new light-limiting cylinder dot; The coordinate transformation is performed on the new light-limiting cylinder point, and the homogeneous coordinates of the center of the calibration plate in the robot coordinate system obtained by the coordinate transformation are used to determine whether the robot and the light-limiting cylinder are aligned.

4. The medical robot vision automatic alignment method according to claim 3, characterized in that: When the binocular vision camera cannot capture the dot of the light-limiting cylinder, the secondary alignment operation of the robot ends.

5. The medical robot vision automatic alignment method according to claim 3, characterized in that: During the secondary alignment of the robot, the image of the light limiting cylinder is continuously collected to identify the center point of the light limiting cylinder plane in the image. When the difference between the image coordinates of the center point of the light limiting cylinder plane and the result of the previous identification is less than the set threshold, it is determined that the robot radiation source is aligned with the light limiting cylinder.

6. The medical robot vision automatic alignment method according to claim 4 or 5, characterized in that: The binocular vision camera continuously collects the light-limiting tube image at a frame rate of 5-10fps.

7. The medical robot vision automatic alignment method according to claim 1, characterized in that: The steps of preprocessing the light-limiting tube image include performing adaptive filtering on the image, converting the filtered image into a grayscale image, and binarizing the grayscale image using an adaptive threshold method to obtain a binary image.

8. The medical robot vision automatic alignment method according to claim 1, characterized in that: The method also includes a step of processing the fitted circle, including judging the average brightness value and the degree of the center of the fitted circle and eliminating the error circle obtained by fitting.

9. An electronic device, characterized in that: include: processor; as well as, A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the medical robot vision automatic alignment method described in claims 1-8 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the medical robot vision automatic alignment method described in claims 1-8 is implemented.

Images