A method for automatically locating a developing device from an image

Abstract

The present invention discloses a method for automatically positioning a developing device from an image, which relates to the technical field of medical image processing, and includes the following steps: S1: collecting an original image and performing analytical processing to obtain a layer array; S2: determining a rough boundary of a developing device in the layer array, and selecting an anchor point layer from the layer array; S3: extracting the anchor point center coordinates of several candidate anchor points in the anchor point layer; S4: finely positioning the outer boundary of the developing device in the axial direction according to the anchor point center coordinates; S5: identifying the developing point coordinates of each layer layer by layer according to the anchor point center coordinates in the anchor point layer, fitting the discrete developing point coordinates in the space coordinate system, and constructing a developing point coordinate set; S6: constructing a point cloud of the developing device in the robot coordinate system according to the developing point coordinate set, performing registration calculation on the developing point coordinate set and the point cloud, and obtaining registration parameters. The present invention is helpful for automatically positioning the developing device of a surgical robot in medical images and improving the registration accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more specifically, to a method for automatically locating a developing device from an image. Background Art

[0002] Currently, malignant tumors in parts such as the liver or cranial nerve functional diseases are major diseases that seriously threaten human health and quality of life. As an effective minimally invasive interventional treatment method, thermal ablation technology based on microwave or radio frequency has been widely used in clinical practice in recent years. Among them, achieving high-precision puncture path planning and execution of the ablation needle is the most important requirement and the key to ensuring the treatment effect.

[0003] In clinical practice, manual puncture guided by medical images such as magnetic resonance, CT, and ultrasound is a commonly used technique for high-precision puncture. However, the manual surgical path planning and puncture execution method is time-consuming and laborious, and highly dependent on the doctor's puncture experience and professional skills, resulting in limited clinical applications.

[0004] In recent years, robot-assisted puncture ablation interventional surgery has gradually entered the clinic. It can provide support for the doctor's surgical operation visually, auditorily, and tactilely, and can achieve precise control of surgical instruments in the field of minimally invasive surgery, effectively improving the accuracy and efficiency of puncture surgery and reducing the excessive dependence on the operator's experience.

[0005] And such surgical robots need to be registered before use. This process is usually based on special marker points or developing devices attached to the robot, and the position and posture of the robot in the surgical space coordinate system are determined by optical or imaging means, and then a mapping relationship is established with the coordinate system of the robot itself. Accurate robot registration results are important factors for high-precision surgical path planning and puncture execution, and thus for ensuring the surgical treatment effect.

[0006] Therefore, how to improve the accuracy of robot registration results is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a method for automatically locating a developing device from an image, which helps to automatically locate the developing device of a surgical robot in a medical image and use its spatial position for registration to improve the registration accuracy.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for automatically locating a developing device from an image, comprising the following steps:

[0010] S1: Collect the original image and perform parsing and processing to obtain a layer array;

[0011] S2: determining a rough boundary of the developing device in the layer array, and selecting an anchor layer from the layer array;

[0012] S3: extract several candidate anchor points in the anchor point layer and calculate the anchor point center coordinates of each candidate anchor point;

[0013] S4: finely positioning the outer boundary of the developing device in the axial direction according to the center coordinates of the anchor point;

[0014] S5: identifying the developing point coordinates of each layer layer by layer according to the center coordinates of the anchor points in the anchor point layer, fitting the discrete developing point coordinates in the spatial coordinate system, and constructing a developing point coordinate set;

[0015] S6: constructing a point cloud of the developing device in the robot coordinate system according to the developing point coordinate set, performing registration calculation on the developing point coordinate set and the point cloud, and obtaining registration parameters.

[0016] Preferably, the specific process of S1 includes:

[0017] S11: reading the original image in a preset format;

[0018] S12: Load the data of each layer in the original image, and arrange all the layers in the axial scanning order to form a layer array;

[0019] S13: parsing metadata corresponding to each layer from the original image;

[0020] S14: reordering the layers in the layer array according to the metadata, arranging them in the order of the spatial coordinates of the origin in the layers, and assigning a unique layer number to each layer for subsequent processing.

[0021] Preferably, the preset format includes but is not limited to Dicom or Nifti format; each layer refers to each frame of the original image of the magnetic resonance or CT scan, usually scanned along the axial direction; the metadata includes the origin space coordinates, direction vector, pixel spacing, layer spacing and other parameters of each layer parsed from the Dicom or Nifti file.

[0022] Preferably, the specific process of S2 includes:

[0023] S21: using a cumulative histogram normalization method or a maximum value normalization method to adjust the pixel grayscale distribution range of each layer in the layer array;

[0024] S22: Perform Gaussian filtering or median filtering on the adjusted layer to remove isolated noise points;

[0025] S23: Binarize the filtered layer according to the set grayscale threshold; calculate the total cumulative histogram curve based on the grayscale values of all pixels in all layers, truncate at 5‰ downward from the maximum value in the total cumulative histogram curve, and use the corresponding value as the grayscale threshold. Set the pixels with grayscale values higher than the grayscale threshold in all layers to 255 and those lower than the grayscale threshold to 0 to form a binary image;

[0026] S24: Use the projection method to determine the outer boundary of the developing device in the binarized layer, screen out the candidate layers, construct a candidate set according to the layer numbers corresponding to the candidate layers, and determine the rough boundary of the developing device based on the outer boundary in the candidate layers;

[0027] S25: Select one from the candidate layers as the anchor layer; select the candidate layer corresponding to the layer number at the center position in the candidate set as the anchor layer.

[0028] Preferably, the cumulative histogram normalization method is to calculate the total cumulative histogram curve based on the grayscale values of all pixels in all layers, truncate at 1‰ downward from the maximum value of the total cumulative histogram curve, and then normalize it to the range of 0 - 255; the maximum - minimum normalization method is to calculate the maximum and minimum values of the grayscale values of all pixels in all layers and normalize the maximum and minimum values to the range of 0 - 255.

[0029] Preferably, the specific steps of S24 are as follows:

[0030] S241: In each layer, project horizontally towards the vertical axis to obtain a one - dimensional array as the vertical - axis projection ; the vertical - axis projection corresponding to the nth layer is expressed as:

[0031]

[0032] where N is the total number of layers in the layer array, H is the total pixel height of each layer, and n represents the nth layer;

[0033] S242: In the vertical - axis projection of each layer, find the highest row (the uppermost position) where the vertical - axis projection = 1 as the upper boundary. In each layer, start from the upper boundary and search downward for a preset height L until the lower boundary. When searching downward, move up and down by a set margin delta at the lower boundary to form the upper edge and lower edge of the lower boundary respectively. If in the layer, the upper edge of the lower boundary is the row where the vertical - axis projection = 1, that is, the upper edge of the lower boundary falls into the shaded area on the left - hand vertical axis, , and the lower edge of the lower boundary is the row where the vertical - axis projection = 0, , then this layer is used as a candidate layer, and the layer numbers corresponding to all candidate layers form a candidate set;

[0034] S243: For each candidate layer in the candidate set, project the pixels within the range of rows where the upper boundary and the lower boundary are located vertically onto the horizontal axis to obtain a one-dimensional array as the horizontal axis projection. ; The horizontal axis projection of the m-th candidate layer is expressed as:

[0035]

[0036]

[0037]

[0038] where C represents the candidate set; U represents the upper boundary; B represents the lower boundary; min(C) and max(C) are respectively the largest layer number and the smallest layer number in the candidate set, W is the total pixel width of each layer, and the direction of increasing j value is to the right along the horizontal axis; L represents the preset height.

[0039] S244: In the horizontal axis projection of each layer corresponding to the candidate set, find the column (the rightmost position) where the horizontal axis projection = 1 and is the farthest along the increasing direction of the horizontal axis as the right boundary. Search leftward from the right boundary in each layer for a preset width until reaching the left boundary. When searching leftward, move a set margin delta before and after the left boundary to form the left edge and the right edge of the left boundary respectively. If the right edge of the left boundary is a column where the horizontal axis projection = 1, that is, the right edge of the left boundary falls into the shaded area on the lower horizontal axis, and the left edge of the left boundary is a column where the horizontal axis projection = 0, then retain this layer as a candidate layer; otherwise, delete the layer number corresponding to this layer in the candidate set.

[0040] S245: Sort the layer numbers in the candidate set according to the layer sorting in the layer array, select the candidate layers corresponding to the first layer number and the last layer number after sorting. The cross-sectional spatial positions in the metadata corresponding to the upper boundary, the lower boundary, the left boundary, and the right boundary in the candidate layer are used as the rough boundaries of the developing device in the coronal plane and the sagittal plane directions; the cross-sectional spatial position includes the origin spatial coordinates and the direction vector in the metadata.

[0041] Preferably, the specific process of S3 is as follows:

[0042] S31: Select several developing points in the anchor layer as candidate anchor points, and determine the local detection area for each candidate anchor point.

[0043] S32: Re-binarize each local detection area.

[0044] S33: Use the contour detection algorithm to detect each binarized local detection area, identify the outer border of the contour of the candidate anchor point, and calculate the center point coordinates based on the outer border of the contour as the anchor point center coordinates.

[0045] Preferably, the specific steps of S31 are as follows:

[0046] S311: Preset the local detection area corresponding to each candidate anchor point as a square with side length S;

[0047] S312: Select the first group of consecutive columns with a value of 1 and the last group of consecutive columns with a value of 1 from the horizontal axis projection of the anchor point layer;

[0048] S313: Select all consecutive rows with a value of 1 from the vertical axis projection of the anchor point layer, and make the selected row range intersect with the two groups of column ranges to obtain a number of candidate rectangles;

[0049] S314: In each candidate rectangle, respectively determine the maximum X coordinate value, maximum Y coordinate value, minimum X coordinate value, and minimum Y coordinate value of all pixels with a pixel value of 255, and take the average value of the X coordinate value and Y coordinate value respectively to obtain the candidate center coordinates;

[0050] S315: Expand the candidate center coordinates equidistantly along the ±X and ±Y directions respectively, with the expansion length l = S / 2, to obtain a square with side length S as the local detection area.

[0051] Preferably, the specific steps of S32 are as follows:

[0052] S321: Preset the physical diameter D of each candidate anchor point, and calculate the regional area ratio according to the area of the corresponding local detection area. Then the regional area ratio ;

[0053] S322: Sort all the original pixel values in each local detection area from small to large, perform indexing according to the sorting result, and take the original pixel value at as the rough pixel threshold; represents the adjustment coefficient;

[0054] S323: Set the pixels in the layer that are higher than the rough pixel threshold to 255, and set the pixels lower than the rough pixel threshold to 0 to form a new binary image.

[0055] Preferably, the specific process of S4 includes:

[0056] S41: Construct the total outer contour according to the minimum circumscribed rectangle of the anchor point layer that includes all the anchor point center coordinates, and cut out the original pixel area corresponding to the total outer contour in all the layers to obtain a number of developed areas;

[0057] S42: The width and height of the total outer contour of the anchor point layer are respectively and , then the new regional area ratio , Represents the empirical proportional coefficient;

[0058] S43: Sort all the original pixel values in the development area of the candidate layer corresponding to the candidate set from smallest to largest, index according to the sorting result, and take the pixel value at as the exact pixel threshold; M represents the number of candidate layers;

[0059] S44: Calculate the eigenvalue for the development area of all layers according to the exact pixel threshold. The eigenvalue c is expressed as:

[0060]

[0061] Where, represents the eigenvalue of the nth layer; represents the pixel value at the (i, j) position of the nth layer. If it is greater than the exact pixel threshold, the function g() = 1, otherwise = 0; i represents the row number, j represents the column number; N represents the total number of layers in the layer array;

[0062] S45: Preset the physical length of the developing device in the axial direction, divide the physical length by the layer spacing in the metadata of the original image to obtain the effective number of developing layers K, and K takes an integer;

[0063] S46: Use the effective number of developing layers K as the width of a sliding window, and slide this sliding window from the first layer with a step of 1 frame until the right side of the sliding window reaches the last layer. Calculate the sum of the eigenvalues of the corresponding layers within the sliding window each time it slides to obtain the total eigenvalue;

[0064] S47: Take the layers at the start and end positions of the sliding window corresponding to the maximum total eigenvalue, and the sectional spatial position in the metadata corresponding to them as the outer boundary of the developing device in the axial direction.

[0065] Preferably, the specific steps of S41 are:

[0066] S411: Take the minimum bounding rectangle including all the center coordinates of the anchor points as the center, and expand outward according to the preset blank border to construct the total outer contour of the rectangles of each anchor point in the sagittal and coronal directions;

[0067] S412: Cut out the original pixel area corresponding to the total outer contour in each layer of the layer array as the developing area; if there are N layers, N rectangular developing areas are obtained, and each developing area may include developing points or may be an empty background.

[0068] Preferably, the specific process of S5 is:

[0069] S51: Starting from the center coordinates of each located anchor point in the anchor point layer, extend forward and backward layer by layer respectively to locate the basic development point coordinates of the development points in each layer within the outer boundary range; adjust the anchor point center coordinates according to the structural characteristics of the developing device and the layer spacing to obtain the basic development point coordinates;

[0070] S52: Construct a square with side length S centered on the basic development point coordinates, detect the development point contour in the square, and calculate the center coordinate value as the accurate development point coordinates through the circumscribed rectangle of the development point contour; if the development point contour is not clear, or the difference between the center coordinate value and the basic development point coordinates is greater than the set threshold, delete the corresponding accurate development point coordinates and enter S53, otherwise enter S54;

[0071] S53: Based on the accurate development point coordinates, calculate the linear fitting parameters, and fill in the accurate development point coordinates of all development points in K layers within the outer boundary range by interpolation;

[0072] S54: Arrange the accurate development point coordinates in the order of the layers to form a set of development point coordinates in the space coordinate system.

[0073] Preferably, the specific process of S6 is as follows:

[0074] S61: Collect the machine coordinates of the physical structure of the developing device in the robot coordinate system, sample the machine coordinates of the developing device along the axial direction of the original image acquisition according to the layer spacing between adjacent layers in the layer array, and aggregate the machine coordinates of each sampling into a coordinate subset to obtain several coordinate subsets;

[0075] S62: Arrange all coordinate subsets in the axial acquisition order of the original image to obtain the point cloud of the developing device in the robot coordinate system;

[0076] S63: Register the set of development point coordinates and the point cloud in the robot coordinate system to obtain the registration parameters.

[0077] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a method for automatically positioning a developing device from an image, which is applicable to robot-assisted puncture surgery guided by magnetic resonance or CT images, and can accurately and automatically determine the spatial position and attitude of the developing device from the image and use it for the calculation of registration parameters, thereby providing strong support for robot-assisted puncture ablation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0079] Figure 1 A schematic flow chart of a method for automatically positioning a developing device from an image provided by the present invention;

[0080] Figure 2 A schematic diagram of developing different layers provided by the present invention;

[0081] Figure 3 A schematic diagram of layer projection provided by the present invention;

[0082] Figure 4 A schematic diagram of candidate anchor points and local detection areas provided by the present invention;

[0083] Figure 5 A schematic diagram of a sliding window sliding selection layer provided by the present invention. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] The embodiment of the present invention discloses a method for automatically positioning a developing device from an image, such as Figure 1 As shown, the following steps are included:

[0086] S1: Collect the original image and perform analysis and processing to obtain the layer array;

[0087] S2: determining a rough boundary of the developing device in the layer array, and selecting an anchor layer from the layer array;

[0088] S3: extract several candidate anchor points in the anchor point layer and calculate the anchor point center coordinates of each candidate anchor point;

[0089] S4: finely positioning the outer boundary of the developing device in the axial direction according to the center coordinates of the anchor point;

[0090] S5: identifying the developing point coordinates of each layer layer by layer according to the center coordinates of the anchor points in the anchor point layer, fitting the discrete developing point coordinates in the spatial coordinate system, and constructing a developing point coordinate set;

[0091] S6: Construct a point cloud of the developing device in the robot coordinate system based on the set of developing point coordinates, perform registration calculations on the set of developing point coordinates and the point cloud, and obtain registration parameters.

[0092] Further, the specific process of S1 includes:

[0093] S11: Read the original image in a preset format;

[0094] S12: Load the data of each layer in the original image, arrange all layers in the axial scan order, and form a layer array;

[0095] S13: Parse the metadata corresponding to each layer from the original image;

[0096] S14: Reorder the layers in the layer array according to the metadata, arrange them in the order of the origin space coordinates in the layer, and assign a unique layer number to each layer for subsequent processing.

[0097] Further, the preset format includes but is not limited to Dicom or Nifti format; each layer refers to each frame of the original image of magnetic resonance or CT scan, usually scanned along the axial direction; the metadata includes parameters such as the origin space coordinates, direction vectors, pixel pitch, slice thickness, etc. of each layer parsed from Dicom or Nifti files.

[0098] Further, in this embodiment, the developing device is composed of several hollow columns filled with a developable liquid inside, including a left column, an inclined column, and a right column. There are columns on both sides, and an inclined column at a 45° angle in the middle. The two ends of the inclined column are connected to the columns on both sides, forming an "N" shape as a whole. Therefore, there are 3 anchor points on each side (i.e., the cross-sectional developing area of the hollow column), distributed from top to bottom, located on the left column, the inclined column, and the right column respectively. In the cross-section of the axial scan, it is circular or elliptical (when the column has a certain inclination angle).

[0099] Further, the specific process of S2 includes:

[0100] S21: Adjust the pixel gray level distribution range of each layer in the layer array by using the cumulative histogram normalization method or the min-max normalization method, and brighten the overall darker scanned image to a moderate level;

[0101] The cumulative histogram normalization method is to calculate the total cumulative histogram curve according to all pixel gray level values in all layers, truncate at 1‰ downward from the maximum value of the total cumulative histogram curve, and then normalize it to the range of 0 - 255; the min-max normalization method is to calculate the maximum and minimum values of all pixel gray level values in all layers, and normalize the maximum and minimum values to the range of 0 - 255;

[0102] S22: Perform Gaussian filtering or median filtering on the adjusted layer to remove isolated noise points; the area of the developing device in the image is as Figure 2 shown, Figure 2 which is the development in two different layers (frames) in the layer array, Figure 2 where (a) in it is one frame of the layer, Figure 2 and (b) in it is another frame of the layer. It can be seen that their positions are slightly different, and this difference can effectively avoid confusion in terms of spatial position and attitude when finally calculating the registration parameters;

[0103] S23: Binarize the filtered layer according to the set gray threshold; calculate the total cumulative histogram curve based on the gray values of all pixels in all layers. Truncate at 5‰ downward from the maximum value in the total cumulative histogram curve, and use the corresponding value as the gray threshold. Set the pixels with gray values higher than the gray threshold in all layers to 255 (white), and the pixels lower than the gray threshold to 0 (black), then a binary (only black and white two colors) image can be formed;

[0104] S24: Use the projection method to determine the outer boundary of the developing device in the binarized layer, screen out the candidate layers, and determine the rough boundary according to the outer boundary of the developing device in the candidate layers; as Figure 3 shown in the schematic diagram of using the projection method to determine the outer boundary of the developing device. For easy observation, the black and white colors of the binarized layer are swapped in the schematic diagram. The specific steps are as follows:

[0105] S241: In each layer, project along the horizontal direction towards the vertical axis. Set the rows with any white pixels along the way to 1, as Figure 3 shown in the shaded part of the vertical axis in it, otherwise set to 0, and obtain a one-dimensional array as the vertical axis projection ; the vertical axis projection corresponding to the nth layer is expressed as:

[0106]

[0107] where N is the total number of layers in the layer array, H is the total pixel height of each layer, and n represents the nth layer;

[0108] S242: Find the uppermost position (i.e., the position where the vertical axis projection = 1) in the vertical axis projection of each layer as the upper boundary U, and then search downward in each layer for a preset height (the preset height is the physical height L of the developing device, and L can take a value of 30 mm), search downward to the lower boundary B, and move a set margin delta up and down at the lower boundary when searching downward. Delta can take a value of 5 mm. If the upper edge of the lower boundary in the layer happens to be the row where the vertical axis projection = 1, that is, the upper edge of the lower boundary falls into Figure 3 the shaded area on the left vertical axis in it, and the lower edge of the lower boundary is the row with a value of 0. If so, this layer is used as a candidate layer, and the layer number n corresponding to the candidate layer is placed in the candidate set C; where the upper boundary U is expressed by the formula (assuming that the upward direction along the vertical axis is the direction of increasing i value):

[0109] ;

[0110] S243: For each candidate layer corresponding to the candidate set, project the pixels in the range of the rows where the upper boundary U and the lower boundary B are located vertically onto the horizontal axis to obtain a one-dimensional array as the horizontal axis projection. , Figure 3 In the horizontal axis shadow, the part with a value of 1; the horizontal axis projection of the m-th candidate layer is expressed as:

[0111]

[0112]

[0113]

[0114] where min(C) and max(C) are the largest layer number and the smallest layer number in the candidate set C respectively, W is the total pixel width of each layer, and the rightward direction along the horizontal axis is the direction of increasing j value.

[0115] S244: For each layer corresponding to the candidate set, find the rightmost position (i.e., the position where the horizontal axis projection = 0) in the horizontal axis projection of each layer as the right boundary, and then search leftward in each layer for a preset width (the preset width is the physical width of the developing device) until the left boundary is reached. When searching leftward, move a set margin delta before and after the left boundary. Delta can take a value of 5 mm. If the right edge of the left boundary in the layer is exactly the column where the horizontal axis projection = 1, and the right edge intersects with the Figure 3 shadow area on the lower horizontal axis in the figure, that is, the right edge of the left boundary falls into the shadow area on the lower horizontal axis in the figure, and the left edge of the left boundary is the column with a value of 0, then this layer is retained as a candidate layer; otherwise, the layer number corresponding to this layer in the candidate set is deleted.

[0116] S245: Sort the layer numbers in the candidate set according to the sorting of the layers in the layer array, and select the candidate layers corresponding to the first layer number and the last layer number after sorting. The sectional spatial positions in the metadata corresponding to the upper boundary, lower boundary, left boundary, and right boundary in the candidate layer are used as the rough boundaries of the developing device in the coronal and sagittal directions; the sectional spatial position includes the origin spatial coordinates and the direction vector in the metadata.

[0117] S25: Select one of the candidate layers as the anchor layer; select the candidate layer corresponding to the layer number at the center position in the candidate set as the anchor layer. The development at the center of the developing device is basically the clearest and best distinguishable, with the least interference from factors such as image quality or artifacts, and the position distribution of each developing point is relatively uniform, making it easy to segment and conducive to subsequent recognition.

[0118] The above S24 and S25 are also the verification processes for whether the image is valid and can be used for registration purposes. If the outer boundary of the developing device and the anchor layer cannot be roughly located, it means that the scanned developing image does not meet the requirements, or the necessary developing area is incomplete, and then it is necessary to adjust and rescan to continue the subsequent processing flow.

[0119] Further, taking the axial direction as an example, the section space position can be represented by the spatial coordinates (i.e., Image Position) of the first pixel in the upper left corner of the layer in the original image in Dicom or Nifti format, and the row / column direction vector of this plane (i.e., Image Orientation, or the normal vector perpendicular to this plane calculated by taking their cross product). The section space position includes the origin space coordinates and direction vector in the metadata; the coronal and sagittal positions are the same. The developing device is installed at the end of the robotic arm. During the registration stage, the robotic arm is located above the human body (such as the abdomen) with appropriate blank space left (leaving appropriate movement space for the robotic arm); and there are no other objects in the horizontal left-right direction where the robotic arm (including the developing device) is located. Therefore, in the binary layer corresponding to the original image, the corresponding area is the background color (i.e., black after binarization).

[0120] Further, the specific process of S3 is as follows:

[0121] S31: Select several developing points in the anchor layer as candidate anchors, determine the local detection area for each candidate anchor, and calculate the anchor center coordinates for each candidate anchor according to the local detection area; select the 3 developing points in the leftmost column and the 3 developing points in the rightmost column in the anchor layer as candidate anchors;

[0122] S311: Preset the local detection area corresponding to each candidate anchor as a square with side length S;

[0123] S312: Select the first group of consecutive columns with a value of 1 (j value range) from the horizontal projection of the anchor layer, as shown in Figure 4 for the range of the 3 candidate anchors in the leftmost column, and select the last group, that is, the third group of consecutive columns with a value of 1, as shown in Figure 4 for the range of the 3 candidate anchors in the rightmost column;

[0124] S313: Select all three groups of consecutive rows with a value of 1 (range of i values) from the vertical axis projection of the anchor point layer, and intersect the selected row range with two column ranges to obtain six candidate rectangles;

[0125] S314: In each candidate rectangle, respectively determine the maximum X coordinate value, maximum Y coordinate value, minimum X coordinate value, and minimum Y coordinate value of all white pixels (pixels with a value of 255), and take the average of the X coordinate values and Y coordinate values respectively to obtain the candidate center coordinates;

[0126] S315: Expand the candidate center coordinates equidistantly along the ±X and ±Y directions respectively, with the expansion length l = S / 2, to obtain a square with a side length of S as the local detection area, as shown by the dotted line box outside the developing point in Figure 4 the figure;

[0127] S32: Re-binarize each local detection area;

[0128] S321: Preset the physical diameter D of each candidate anchor point, calculate the area ratio of the region according to the area of the corresponding local detection area, then the area ratio ;

[0129] The developing device is composed of several upright and inclined hollow columns connected together. The axial scan of the upright hollow column shows a circle, and the axial scan of the inclined hollow column shows an ellipse. The physical diameter of the cross-section of the upright hollow column is D. In this embodiment, the difference between the major axis and minor axis lengths of the elliptical cross-section is ignored, and the average value is taken, and it is also calculated according to the physical diameter of D;

[0130] S322: Sort all the original pixel values in each local detection area from smallest to largest (there are pixel points in total), perform indexing according to the sorting result, and take the original pixel value at as the rough pixel threshold; represents an adjustment coefficient between 0 and 1.0, take a certain empirical value, and it can be taken as 0.9;

[0131] S323: Set the pixels in the layer that are higher than the rough pixel threshold to 255, and set the pixels lower than the rough pixel threshold to 0 to form a new binarized image; by calculating the area ratio of the region, a better threshold can be obtained to normalize the pixel values in the local detection area, so that the edge of the developing point after binarization can be detected more accurately;

[0132] S33: Use the contour detection algorithm to detect the candidate anchor points in each re-binarized local detection area, identify the outer border of the contour of the candidate anchor points, and calculate the center point coordinates according to the outer border of the contour as the anchor point center coordinates.

[0133] Further, the contour detection algorithm can directly call functions from the algorithm library, such as findContours() in OpenCV.

[0134] Further, the specific process of S4 includes:

[0135] S41: Construct the total outer contour according to the center coordinates of each anchor point in the anchor layer, and cut out the original pixel regions corresponding to the total outer contour in all layers to obtain several developing regions;

[0136] S411: Taking the minimum bounding rectangle including all anchor point center coordinates as the center, expand outward according to the preset blank border to construct the total outer contour of the rectangle of each anchor point center coordinate in the sagittal and coronal directions;

[0137] S412: Cut out the original pixel regions corresponding to the total outer contour in each layer of the layer array as the developing regions; if there are N layers, N rectangular developing regions are obtained, and each developing region may include developing points or may be an empty background;

[0138] S42: In this embodiment, there are 7 developing points in each candidate layer, and 6 of them are used for registration. The total outer contour of the anchor layer is rectangular, with the width and height being and , then the area ratio of the new region is ; Considering the existence of phenomena such as local blurring or artifacts, the developing edges in the scanned image are not perfect circles or ellipses, so is introduced, where represents the empirical proportionality coefficient, and its value can be between 0.8 and 1.2;

[0139] S43: Sort all the original pixel values in the developing regions of the candidate layers corresponding to the candidate set from small to large, perform indexing according to the sorting result, and take the pixel value as the accurate pixel threshold; M represents the number of candidate layers;

[0140] S44: Calculate the feature value for the developing regions of all layers according to the accurate pixel threshold. The feature value c is expressed as:

[0141]

[0142] where, represents the feature value of the nth layer; Denote the pixel value at the (i, j) position of the nth layer, where i represents the row number and j represents the column number. If it is greater than the exact pixel threshold, then the function g() = 1, otherwise = 0. Therefore, this eigenvalue is equivalent to the count of pixel points whose pixel values in the developed area of the nth layer are greater than the exact pixel threshold, and it can also be regarded as the possibility that the nth layer contains clear, complete, and effective developed points;

[0143] S45: The physical length of the preset developing device in the axial direction. Divide the physical length by the layer spacing (in mm) in the metadata of the original image to obtain the effective number of developed layers K, and K is an integer;

[0144] S46: Use the effective number of developed layers K as the width of a sliding window. Slide this sliding window from the first layer with a step size of 1 frame until the right side of the sliding window reaches the last layer. Calculate the sum of the eigenvalue of the corresponding layers within the sliding window each time it slides to obtain the total eigenvalue;

[0145] S47: Select the layers at the start and end positions of the sliding window corresponding to the maximum total eigenvalue, and the corresponding section spatial position in the metadata as the outer boundary of the developing device in the axial direction; As Figure 5 shown, the layer numbers on both sides of the range of the outer boundary are F(L) and F(R) respectively, and F(R) = F(L) + K - 1.

[0146] Furthermore, the specific process of S5 is as follows:

[0147] S51: Starting from the center coordinates of each located anchor point in the anchor layer, extend layer by layer forward and backward respectively to locate the basic developed point coordinates of the developed points in the adjacent layers within the outer boundary; Adjust the anchor point center coordinates according to the structural characteristics and layer spacing of the developing device to obtain the basic developed point coordinates;

[0148] S52: Construct a square with side length S centered on the basic developed point coordinates. Detect the developed point contour in the square, and calculate the center coordinate value as the exact developed point coordinate through the circumscribed rectangle of the developed point contour; If the developed point contour is not clear, or the difference between the center coordinate value and the basic developed point coordinate is greater than the set threshold, then delete the corresponding exact developed point coordinate and enter S53, otherwise enter S54;

[0149] S53: Based on the exact developed point coordinates, calculate the linear fitting parameters and fill in the exact developed point coordinates of all developed points in the K layers within the outer boundary by interpolation;

[0150] After the above S51 ends, it is not necessarily guaranteed that all the developed points on the K layer can be detected. For example: at the ends where two columns are close to intersecting, their respective developed points may have partial overlap on the section plane, causing the area of their outer frame to exceed a certain threshold and be rejected for output, which can reduce the deviation of the central coordinates of the developed points; or because the local image is not clear, the edge detection function does not output results. Therefore, it is necessary to further perform linear regression on the detected developed point coordinate sets belonging to the same hollow column by the least squares method to obtain fitting parameters, and then perform interpolation to supplement the corresponding K developed point coordinates on the K layer (such operations are performed for each column);

[0151] S54: Arrange the accurate developed point coordinates in the order of the layers to form a set of developed point coordinates in the space coordinate system;

[0152] The specific order is the custom order of the columns to which the developed points belong. There are three developed columns on the left and right respectively, and a total of K layers. Then the set of developed point coordinates contains a total of (3 + 3) × K coordinate values.

[0153] Furthermore, in S51 of this embodiment, if the central coordinates of the anchor point are located on a vertical column, when extending forward and backward, the X and Y coordinates of the central coordinates of the anchor point extended to the adjacent layers remain unchanged, and the basic developed point coordinates are (X, Y); while if the central coordinates of the anchor point are located on an inclined column (for example, the angle with the vertical column is 45°), when extending forward and backward, the X coordinate of the central coordinates of the anchor point extended to the adjacent layers remains unchanged, and the Y coordinate is added or subtracted by one layer spacing, and the central coordinates of the anchor point become (X, Y').

[0154] Furthermore, the specific process of S6 is as follows:

[0155] S61: Collect the machine coordinates of each point on each hollow column of the developing device in the robot coordinate system, sample the machine coordinates of the developing device along the axis of the original image acquisition according to the layer spacing between adjacent layers in the layer array, and aggregate the machine coordinates of each sampling into a coordinate subset to obtain several coordinate subsets;

[0156] S62: Arrange all the coordinate subsets in the acquisition order of the original image axis to obtain the point cloud of the developing device in the robot coordinate system; aggregate the machine coordinates according to the developed points in the layer and arrange them in the layer order;

[0157] S63: Perform registration on the set of developed point coordinates and the point cloud in the robot coordinate system to obtain the registration parameters;

[0158] The set of developing point coordinates of the developing device in the image space coordinate system is registered and calculated with the point cloud in the robot coordinate system to obtain the corresponding registration parameters (i.e., the transformation matrix between the two coordinate systems). Since the number of elements in these two constructed coordinate sets is equal and the sorting methods are the same, the existing matching functions in the algorithm library (such as vtkLandmarkTransform() in VTK) can be directly called to complete the calculation.

[0159] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple. For related parts, reference can be made to the description in the method section.

[0160] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for automatically positioning a developing device from an image, characterized in that: The following steps are involved: S1: Collect the original image and perform analysis and processing to obtain the layer array; S2: determining a rough boundary of the developing device in the layer array, and selecting an anchor layer from the layer array; S3: extract several candidate anchor points in the anchor point layer and calculate the anchor point center coordinates of each candidate anchor point; S4: finely positioning the outer boundary of the developing device in the axial direction according to the center coordinates of the anchor point; S5: identifying the developing point coordinates of each layer layer by layer according to the center coordinates of the anchor points in the anchor point layer, fitting the discrete developing point coordinates in the spatial coordinate system, and constructing a developing point coordinate set; S6: constructing a point cloud of the developing device in the robot coordinate system according to the developing point coordinate set, performing registration calculation on the developing point coordinate set and the point cloud, and obtaining registration parameters.

2. A method for automatically positioning a developing device from an image according to claim 1, characterized in that: The specific process of S1 includes: S11: reading the original image in a preset format; S12: Load the data of each layer in the original image, and arrange all the layers in the axial scanning order to form a layer array; S13: parsing metadata corresponding to each layer from the original image; S14: reordering the layers in the layer array according to the metadata, arranging them in the order of the spatial coordinates of the origin in the layers, and assigning a unique layer number to each layer.

3. A method for automatically positioning a developing device from an image according to claim 2, characterized in that: The specific process of S2 includes: S21: using a cumulative histogram normalization method or a maximum value normalization method to adjust the pixel grayscale distribution range of each layer in the layer array; S22: filtering the adjusted layer; S23: binarizing the filtered layer according to the set grayscale threshold; S24: using a projection method to determine the outer boundary of the developing device in the binarized layer, screening out candidate layers, constructing a candidate set according to the layer numbers corresponding to the candidate layers, and determining a rough boundary of the developing device according to the outer boundary in the candidate layers; S25: Select the candidate layer corresponding to the layer number at the center position in the candidate set as the anchor layer.

4. A method for automatically positioning a developing device from an image according to claim 3, characterized in that: The specific steps of S24 are: S241: In each layer, project horizontally toward the vertical axis to obtain a one-dimensional array as the vertical axis projection. ; The vertical projection corresponding to the nth layer It is expressed as: ； Where N is the total number of layers in the layer array, H is the total pixel height of each layer, and n represents the nth layer; S242: Find the highest row with a vertical axis projection of 1 in the vertical axis projection of each layer as the upper boundary, search downward from the upper boundary to a preset height L in each layer, search to the lower boundary, and move the lower boundary up and down by a set margin delta to form the upper edge and lower edge of the lower boundary respectively when searching downward. If the upper edge of the lower boundary in the layer is a row with a vertical axis projection of 1, and the lower edge of the lower boundary is a row with a vertical axis projection of 0, then this layer is taken as a candidate layer, and the layer numbers corresponding to all candidate layers form a candidate set; S243: For each candidate layer corresponding to the candidate set, project the pixels within the row range where the upper boundary and the lower boundary are located along the vertical direction toward the horizontal axis to obtain a one-dimensional array as the horizontal axis projection ; The horizontal projection of the mth candidate layer It is expressed as: ； ； ； Where C represents the candidate set; U represents the upper boundary; B represents the lower boundary; min(C) and max(C) are the maximum layer number and the minimum layer number in the candidate set, respectively; W is the total pixel width of each layer, and the j value increases to the right along the horizontal axis; L represents the preset height; S244: in the horizontal axis projection of each layer corresponding to the candidate set, find the column farthest along the horizontal axis increasing direction with the horizontal axis projection = 1 as the right boundary, search the preset width from the right boundary to the left in each layer, search to the left boundary, and when searching to the left, move the set margin delta before and after the left boundary to form the left edge and the right edge of the left boundary respectively. If the right edge of the left boundary in the layer is the column with the horizontal axis projection = 1, and the left edge of the left boundary is the column with the horizontal axis projection = 0, then keep this layer as the candidate layer, otherwise delete the layer number corresponding to this layer in the candidate set; S245: sorting the layer numbers in the candidate set according to the layer sorting in the layer array, selecting the candidate layers corresponding to the first layer number and the last layer number after sorting, and the slice space positions in the metadata corresponding to the upper boundary, the lower boundary, the left boundary and the right boundary in the candidate layers are used as the rough boundaries of the developing device in the coronal and sagittal directions.

5. A method for automatically positioning a developing device from an image according to claim 4, characterized in that: The specific process of S3 is: S31: Selecting several development points in the anchor point layer as candidate anchor points, and determining a local detection area for each candidate anchor point; S32: re-binarize each local detection area; S33: Use a contour detection algorithm to detect each binarized local detection area, identify the contour outer frame of the candidate anchor point, and calculate the center point coordinates as the center coordinates of the anchor point according to the contour outer frame.

6. A method for automatically positioning a developing device from an image according to claim 5, characterized in that: The specific steps of S31 are: S311: Preset the local detection area corresponding to each candidate anchor point to be a square with a side length of S; S312: Selecting the first group of column ranges whose values ​​are continuously 1 and the last group of column ranges whose values ​​are continuously 1 from the horizontal projection of the anchor point layer; S313: Select all row ranges with consecutive values ​​of 1 from the vertical axis projection of the anchor point layer, and make the selected row ranges intersect with the two sets of column ranges to obtain a number of candidate rectangles; S314: In each candidate rectangle, the maximum X coordinate value, the maximum Y coordinate value, the minimum X coordinate value, and the minimum Y coordinate value of all pixels whose pixel values ​​are 255 are determined respectively, and the average of the X coordinate value and the Y coordinate value is taken respectively to obtain the candidate center coordinate; S315: Expand the candidate center coordinates outward along the ±X and ±Y directions at equal distances, with an expansion length of l=S / 2, to obtain a square with a side length of S as the local detection area.

7. A method for automatically positioning a developing device from an image according to claim 5, characterized in that: The specific steps of S32 are: S321: Preset the physical diameter D of each candidate anchor point and calculate the ratio of the area of ​​the candidate anchor point to the area of ​​the local detection area ; S322: Sort all original pixel values ​​in each local detection area from small to large, and index them according to the sorting results. The original pixel value at is used as the rough pixel threshold; represents the adjustment coefficient, S represents the side length of the local detection area; S323: Pixels in the layer that are higher than the rough pixel threshold are set to 255, and pixels that are lower than the rough pixel threshold are set to 0, to form a new binary image.

8. A method for automatically positioning a developing device from an image according to claim 3, characterized in that: The specific process of S4 includes: S41: Taking the minimum circumscribed rectangle including the center coordinates of all anchor points in the anchor point layer as the center, expanding outward according to the preset blank border to construct a total outer contour; cutting out the original pixel area corresponding to the total outer contour in each layer of the layer array to obtain a plurality of development areas; S42: The width and height of the total outer contour of the anchor point layer are and , then the new area ratio , represents the empirical proportionality coefficient, and D represents the physical diameter of the candidate anchor point; S43: sort all original pixel values ​​in the development area of ​​the candidate layer corresponding to the candidate set from small to large, and index according to the sorting result. The pixel value at is used as the precise pixel threshold; M represents the number of candidate layers; S44: Calculate characteristic values ​​for the developed areas of all layers according to the precise pixel threshold, and the characteristic value c is expressed as: ； in, Represents the characteristic value of the nth layer; Represents the pixel value at position (i, j) of the nth layer. If it is greater than the precise pixel threshold, the function g()=1, otherwise =0; i represents the number of rows, j represents the number of columns; N represents the total number of layers in the layer array; S45: Preset the physical length of the developing device in the axial direction, and divide the physical length by the layer spacing in the metadata of the original image to obtain the effective developing layer number K; S46: Taking the number of effective developing layers K as the width of a sliding window, using the sliding window to slide from the first layer of the layer array with a step length of 1 frame until the right side of the sliding window reaches the last layer, and calculating the sum of the eigenvalues ​​of the corresponding layers in the sliding window each time sliding to obtain the sum of the eigenvalues; S47: Take the layer of the sliding window start and end positions corresponding to the maximum eigenvalue sum, and the corresponding section space position in the metadata as the outer boundary of the developing device in the axial direction.

9. A method for automatically positioning a developing device from an image according to claim 1, characterized in that: The specific process of S5 is as follows: S51: Starting from the center coordinates of each anchor point located in the anchor point layer, extending forward and backward layer by layer, respectively, to locate the basic development point coordinates of the development points in each layer within the outer boundary range; S52: constructing a square with a side length of S with the basic developing point coordinate as the center, detecting the developing point outline in the square, and calculating the center coordinate value as the precise developing point coordinate through the circumscribed rectangle of the developing point outline; if the developing point outline is not clear, or the difference between the center coordinate value and the basic developing point coordinate is greater than a set threshold, the corresponding precise developing point coordinate is deleted and the process goes to S53, otherwise the process goes to S54; S53: Calculating linear fitting parameters based on the precise development point coordinates, and filling in the precise development point coordinates of all development points in the K-layer layer within the outer boundary range by interpolation; S54: Arranging the precise development point coordinates according to the layer order in the layer array to form a development point coordinate set in a spatial coordinate system.

10. A method for automatically positioning a developing device from an image according to claim 1, characterized in that: The specific process of S6 is as follows: S61: collecting machine coordinates of the developing device in the robot coordinate system, axially sampling the machine coordinates of the developing device according to the layer spacing between adjacent layers in the layer array, aggregating the machine coordinates sampled each time into a coordinate subset, and obtaining a plurality of coordinate subsets; S62: Arrange all coordinate subsets according to the acquisition order of the original images to obtain a point cloud of the developing device in the robot coordinate system; S63: aligning the development point coordinate set and the point cloud in the robot coordinate system to obtain alignment parameters.

Images