Method for automatically positioning developing device from image

Abstract

The invention discloses a method for automatically positioning a developing device from an image, and relates to the technical field of medical image processing, and the method comprises the following steps: S1, collecting an original image, and carrying out analysis processing to obtain a layer array; s2, determining a rough boundary of the developing device in the layer array, and selecting an anchor point layer from the layer array; s3, extracting anchor point center coordinates of a plurality of candidate anchor points in the anchor point layer; s4, finely positioning the outer boundary of the developing device in the axial direction according to the center coordinate of the anchor point; s5, identifying development point coordinates of each layer layer by layer according to center coordinates of anchor points in the anchor point layer, fitting discrete development point coordinates in a space coordinate system, and constructing a development point coordinate set; and S6, constructing a point cloud of the developing device under the robot coordinate system according to the developing point coordinate set, and performing registration calculation on the developing point coordinate set and the point cloud to obtain registration parameters. According to the invention, the automatic positioning of the developing device of the surgical robot in the medical image is facilitated, and the registration precision is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and more particularly to a method for automatically positioning a developing device from an image. Background Art

[0002] At present, malignant tumors in the liver and other parts of the body or functional brain nerve diseases are major diseases that seriously threaten human health and quality of life. Thermal ablation technology based on microwave or radio frequency has been widely used in clinical practice in recent years as an effective minimally invasive interventional treatment method. Among them, the realization of high-precision puncture path planning and execution of the ablation needle is the most important requirement and the key to ensuring the treatment effect.

[0003] In clinical practice, manual puncture guided by medical images such as magnetic resonance imaging, CT, and ultrasound is a common procedure for high-precision puncture. However, manual surgical path planning and puncture execution methods are time-consuming and labor-intensive, and heavily rely on the doctor's puncture experience and professional skills, which limits their clinical application.

[0004] In recent years, robot-assisted puncture and ablation interventional surgery has gradually entered the clinic. It can provide support for doctors' surgical operations in terms of vision, hearing and touch, and can achieve precise control of surgical instruments in the field of minimally invasive surgery, effectively improving the accuracy and efficiency of puncture surgery and reducing excessive reliance on the operator's experience.

[0005] This type of surgical robot needs to be registered before use. This process is usually based on special markers or developing devices attached to the robot. The robot's position and posture in the surgical space coordinate system are determined by optical or imaging means, and then a mapping relationship is established with the robot's own coordinate system. Accurate robot registration results are an important factor in high-precision surgical path planning and puncture execution, thereby ensuring the effectiveness of surgical treatment.

[0006] Therefore, how to improve the accuracy of robot registration results is an urgent problem that technicians in this field need to solve. Summary of the invention

[0007] In view of this, the present invention provides a method for automatically positioning a developing device from an image, which helps to automatically position the developing device of a surgical robot in a medical image, and use its spatial position for registration to improve the registration accuracy.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] A method for automatically positioning a developing device from an image comprises the following steps:

[0010] S1: Collect the original image and perform analysis and processing to obtain the layer array;

[0011] S2: determining a rough boundary of the developing device in the layer array, and selecting an anchor layer from the layer array;

[0012] S3: extract several candidate anchor points in the anchor point layer and calculate the anchor point center coordinates of each candidate anchor point;

[0013] S4: finely positioning the outer boundary of the developing device in the axial direction according to the center coordinates of the anchor point;

[0014] S5: identifying the developing point coordinates of each layer layer by layer according to the center coordinates of the anchor points in the anchor point layer, fitting the discrete developing point coordinates in the spatial coordinate system, and constructing a developing point coordinate set;

[0015] S6: constructing a point cloud of the developing device in the robot coordinate system according to the developing point coordinate set, performing registration calculation on the developing point coordinate set and the point cloud, and obtaining registration parameters.

[0016] Preferably, the specific process of S1 includes:

[0017] S11: reading the original image in a preset format;

[0018] S12: Load the data of each layer in the original image, and arrange all the layers in the axial scanning order to form a layer array;

[0019] S13: parsing metadata corresponding to each layer from the original image;

[0020] S14: reordering the layers in the layer array according to the metadata, arranging them in the order of the spatial coordinates of the origin in the layers, and assigning a unique layer number to each layer for subsequent processing.

[0021] Preferably, the preset format includes but is not limited to Dicom or Nifti format; each layer refers to each frame of the original image of the magnetic resonance or CT scan, usually scanned along the axial direction; the metadata includes the origin space coordinates, direction vector, pixel spacing, layer spacing and other parameters of each layer parsed from the Dicom or Nifti file.

[0022] Preferably, the specific process of S2 includes:

[0023] S21: using a cumulative histogram normalization method or a maximum value normalization method to adjust the pixel grayscale distribution range of each layer in the layer array;

[0024] S22: Perform Gaussian filtering or median filtering on the adjusted layer to remove isolated noise points;

[0025] S23: Binarizing the filtered layer according to the set grayscale threshold; calculating the total cumulative histogram curve according to the grayscale values ​​of all pixels in all layers, truncating the total cumulative histogram curve at 5‰ below the maximum value, taking the corresponding value as the grayscale threshold, setting the pixels in all layers whose grayscale values ​​are higher than the grayscale threshold to 255, and setting the pixels whose grayscale values ​​are lower than the grayscale threshold to 0, thereby forming a binary image;

[0026] S24: using a projection method to determine the outer boundary of the developing device in the binarized layer, screening out candidate layers, constructing a candidate set according to the layer numbers corresponding to the candidate layers, and determining a rough boundary of the developing device according to the outer boundary in the candidate layers;

[0027] S25: Select one from the candidate layers as the anchor layer; select the candidate layer corresponding to the layer number at the center position in the candidate set as the anchor layer.

[0028] Preferably, the cumulative histogram normalization method is to calculate the total cumulative histogram curve based on the grayscale values ​​of all pixels in all layers, truncate it at 1‰ below the maximum value of the total cumulative histogram curve, and then normalize it to the range of 0~255; the extreme value normalization method is to calculate the maximum and minimum values ​​of the grayscale values ​​of all pixels in all layers, and normalize the maximum and minimum values ​​to the range of 0~255.

[0029] Preferably, the specific steps of S24 are:

[0030] S241: In each layer, project horizontally toward the vertical axis to obtain a one-dimensional array as the vertical axis projection. ; The vertical projection corresponding to the nth layer It is expressed as:

[0031]

[0032] Where N is the total number of layers in the layer array, H is the total pixel height of each layer, and n represents the nth layer;

[0033] S242: Find the highest row (the uppermost position) with the vertical axis projection = 1 in the vertical axis projection of each layer as the upper boundary, and search downward from the upper boundary in each layer at a preset height L until the lower boundary is reached. When searching downward, move the lower boundary up and down by a set margin delta to form the upper edge and lower edge of the lower boundary respectively. If the upper edge of the lower boundary in the layer is a row with the vertical axis projection = 1, that is, the upper edge of the lower boundary falls into the shadow area on the left vertical axis, , and the lower edge of the lower boundary is the row with the vertical axis projection = 0, , then this layer is taken as a candidate layer, and the layer numbers corresponding to all candidate layers form a candidate set;

[0034] S243: For each candidate layer corresponding to the candidate set, project the pixels within the row range where the upper boundary and the lower boundary are located along the vertical direction toward the horizontal axis to obtain a one-dimensional array as the horizontal axis projection ; The horizontal projection of the mth candidate layer It is expressed as:

[0035]

[0036]

[0037]

[0038] Where C represents the candidate set; U represents the upper boundary; B represents the lower boundary; min(C) and max(C) are the maximum layer number and the minimum layer number in the candidate set, respectively; W is the total pixel width of each layer, and the j value increases to the right along the horizontal axis; L represents the preset height;

[0039] S244: in the horizontal axis projection of each layer corresponding to the candidate set, find the column farthest along the increasing direction of the horizontal axis (the rightmost position) with the horizontal axis projection = 1 as the right boundary, search the preset width from the right boundary to the left in each layer, search to the left boundary, and move the set margin delta before and after the left boundary to form the left edge and the right edge of the left boundary respectively when searching to the left. If the right edge of the left boundary in the layer is the column with the horizontal axis projection = 1, that is, the right edge of the left boundary falls into the shadow area on the lower horizontal axis, and the left edge of the left boundary is the column with the horizontal axis projection = 0, then keep this layer as the candidate layer, otherwise delete the layer number corresponding to this layer in the candidate set;

[0040] S245: sorting the layer numbers in the candidate set according to the layer sorting in the layer array, selecting the candidate layers corresponding to the first layer number and the last layer number after sorting, and the slice space positions in the metadata corresponding to the upper boundary, the lower boundary, the left boundary and the right boundary in the candidate layers are used as the rough boundaries of the developing device in the coronal and sagittal directions; the slice space positions include the origin space coordinates and the direction vector in the metadata.

[0041] Preferably, the specific process of S3 is:

[0042] S31: Selecting several development points in the anchor point layer as candidate anchor points, and determining a local detection area for each candidate anchor point;

[0043] S32: re-binarize each local detection area;

[0044] S33: Use a contour detection algorithm to detect each binarized local detection area, identify the contour outer frame of the candidate anchor point, and calculate the center point coordinates as the center coordinates of the anchor point according to the contour outer frame.

[0045] Preferably, the specific steps of S31 are:

[0046] S311: Preset the local detection area corresponding to each candidate anchor point to be a square with a side length of S;

[0047] S312: Select the first group of column ranges whose values ​​are continuously 1 from the horizontal projection of the anchor point layer, and select the last group of column ranges whose values ​​are continuously 1;

[0048] S313: Select all row ranges with consecutive values ​​of 1 from the vertical axis projection of the anchor point layer, and make the selected row ranges intersect with the two groups of column ranges to obtain a number of candidate rectangles;

[0049] S314: In each candidate rectangle, the maximum X coordinate value, the maximum Y coordinate value, the minimum X coordinate value, and the minimum Y coordinate value of all pixels whose pixel values ​​are 255 are determined respectively, and the average of the X coordinate value and the Y coordinate value is taken respectively to obtain the candidate center coordinate;

[0050] S315: Expand the candidate center coordinates outward along the ±X and ±Y directions at equal distances, with an expansion length of l=S / 2, to obtain a square with a side length of S as the local detection area.

[0051] Preferably, the specific steps of S32 are:

[0052] S321: Preset the physical diameter D of each candidate anchor point, and calculate the area ratio according to the area of ​​the corresponding local detection area. ;

[0053] S322: Sort all original pixel values ​​in each local detection area from small to large, and index them according to the sorting results. The original pixel value at is used as the rough pixel threshold; represents the adjustment coefficient;

[0054] S323: Pixels in the layer that are higher than the rough pixel threshold are set to 255, and pixels that are lower than the rough pixel threshold are set to 0, to form a new binary image.

[0055] Preferably, the specific process of S4 includes:

[0056] S41: constructing a total outer contour according to the minimum circumscribed rectangle of the anchor point layer including the center coordinates of all anchor points, and cutting out the original pixel areas corresponding to the total outer contour in all layers to obtain a plurality of development areas;

[0057] S42: The width and height of the total outer contour of the anchor point layer are and , then the new area ratio , represents the empirical proportionality coefficient;

[0058] S43: sort all original pixel values ​​in the development area of ​​the candidate layer corresponding to the candidate set from small to large, and index them according to the sorting result. The pixel value at is used as the precise pixel threshold; M represents the number of candidate layers;

[0059] S44: Calculate characteristic values ​​for the developed areas of all layers according to the precise pixel threshold, and the characteristic value c is expressed as:

[0060]

[0061] in, Represents the characteristic value of the nth layer; Represents the pixel value at position (i, j) of the nth layer. If it is greater than the precise pixel threshold, the function g()=1, otherwise =0; i represents the number of rows, j represents the number of columns; N represents the total number of layers in the layer array;

[0062] S45: Preset the physical length of the developing device in the axial direction, and divide the physical length by the layer spacing in the metadata of the original image to obtain the effective developing layer number K, where K is an integer;

[0063] S46: Taking the number of effective developing layers K as the width of a sliding window, using the sliding window to slide from the first layer with a step length of 1 frame until the right side of the sliding window reaches the last layer, and calculating the sum of the eigenvalues ​​of the corresponding layers in the sliding window each time sliding to obtain the sum of the eigenvalues;

[0064] S47: Take the layer of the sliding window start and end positions corresponding to the maximum eigenvalue sum, and the corresponding section space position in the metadata as the outer boundary of the developing device in the axial direction.

[0065] Preferably, the specific steps of S41 are:

[0066] S411: Taking the minimum circumscribed rectangle including the center coordinates of all anchor points as the center, expanding outward according to a preset blank border, and constructing the total outer contour of the rectangle of each anchor point in the sagittal and coronal directions;

[0067] S412: Cut out the original pixel area corresponding to the total outer contour in each layer of the layer array as a development area; if there are N layers, N rectangular development areas are obtained, and each development area may include a development point or may be an empty background.

[0068] Preferably, the specific process of S5 is:

[0069] S51: Starting from the central coordinates of each anchor point located in the anchor point layer, extending forward and backward layer by layer respectively, locating the basic developing point coordinates of the developing points in each layer within the outer boundary range; adjusting the central coordinates of the anchor point according to the structural characteristics of the developing device and the layer spacing to obtain the basic developing point coordinates;

[0070] S52: constructing a square with a side length of S with the basic developing point coordinate as the center, detecting the developing point outline in the square, and calculating the center coordinate value as the precise developing point coordinate through the circumscribed rectangle of the developing point outline; if the developing point outline is not clear, or the difference between the center coordinate value and the basic developing point coordinate is greater than a set threshold, the corresponding precise developing point coordinate is deleted and the process goes to S53, otherwise the process goes to S54;

[0071] S53: Based on the precise development point coordinates, linear fitting parameters are calculated, and the precise development point coordinates of all development points in the K-layer layer within the outer boundary range are filled in by interpolation;

[0072] S54: Arrange the precise development point coordinates according to the order of the layers to form a development point coordinate set in a spatial coordinate system.

[0073] Preferably, the specific process of S6 is:

[0074] S61: collecting machine coordinates of the physical structure of the developing device in the robot coordinate system, sampling the machine coordinates of the developing device in the axial direction of the original image according to the layer spacing between adjacent layers in the layer array, and aggregating the machine coordinates sampled each time into a coordinate subset to obtain a plurality of coordinate subsets;

[0075] S62: Arrange all coordinate subsets according to the axial acquisition order of the original image to obtain a point cloud of the developing device in the robot coordinate system;

[0076] S63: aligning the development point coordinate set and the point cloud in the robot coordinate system to obtain alignment parameters.

[0077] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a method for automatically positioning a developing device from an image, which is suitable for robot-assisted puncture surgery under the guidance of magnetic resonance or CT images. The spatial position and posture of the developing device can be accurately and automatically determined from the image, and used for the calculation of alignment parameters, thereby providing strong support for robot-assisted puncture and ablation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0079] Figure 1 A schematic flow chart of a method for automatically positioning a developing device from an image provided by the present invention;

[0080] Figure 2 A schematic diagram of developing different layers provided by the present invention;

[0081] Figure 3 A schematic diagram of layer projection provided by the present invention;

[0082] Figure 4 A schematic diagram of candidate anchor points and local detection areas provided by the present invention;

[0083] Figure 5 A schematic diagram of a sliding window sliding selection layer provided by the present invention. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0085] The embodiment of the present invention discloses a method for automatically positioning a developing device from an image, such as Figure 1 As shown, the following steps are included:

[0086] S1: Collect the original image and perform analysis and processing to obtain the layer array;

[0087] S2: determining a rough boundary of the developing device in the layer array, and selecting an anchor layer from the layer array;

[0088] S3: extract several candidate anchor points in the anchor point layer and calculate the anchor point center coordinates of each candidate anchor point;

[0089] S4: finely positioning the outer boundary of the developing device in the axial direction according to the center coordinates of the anchor point;

[0090] S5: identifying the developing point coordinates of each layer layer by layer according to the center coordinates of the anchor points in the anchor point layer, fitting the discrete developing point coordinates in the spatial coordinate system, and constructing a developing point coordinate set;

[0091] S6: constructing a point cloud of the developing device in the robot coordinate system according to the developing point coordinate set, performing registration calculation on the developing point coordinate set and the point cloud, and obtaining registration parameters.

[0092] Furthermore, the specific process of S1 includes:

[0093] S11: reading the original image in a preset format;

[0094] S12: Load the data of each layer in the original image, and arrange all the layers in the axial scanning order to form a layer array;

[0095] S13: parsing metadata corresponding to each layer from the original image;

[0096] S14: reordering the layers in the layer array according to the metadata, arranging them in the order of the spatial coordinates of the origin in the layers, and assigning a unique layer number to each layer for subsequent processing.

[0097] Furthermore, the preset formats include but are not limited to Dicom or Nifti formats; each layer refers to each frame of the original image of the magnetic resonance or CT scan, usually scanned along the axial direction; the metadata includes the origin space coordinates, direction vector, pixel spacing, layer spacing and other parameters of each layer parsed from the Dicom or Nifti file.

[0098] Furthermore, the developing device in this embodiment is composed of a plurality of hollow columns filled with developable liquid, including a left vertical column, an inclined column, and a right vertical column, with vertical columns on both sides and an inclined column inclined at a 45° angle in the middle. The two ends of the inclined column are connected to the vertical columns on both sides, forming an "N" shape as a whole. Therefore, the three anchor points on both sides (i.e., the cross-sectional developing area of ​​the hollow column) are distributed from top to bottom, respectively located on the left vertical column, the inclined column, and the right vertical column, and the cross-section in the axial scanning is circular or elliptical (when the vertical column has a certain inclination angle).

[0099] Furthermore, the specific process of S2 includes:

[0100] S21: using a cumulative histogram normalization method or a maximum value normalization method to adjust the pixel grayscale distribution range of each layer in the layer array, so as to brighten the overall dark scanned image to a moderate degree;

[0101] The cumulative histogram normalization method is to calculate the total cumulative histogram curve based on the grayscale values ​​of all pixels in all layers, truncate it at 1‰ below the maximum value of the total cumulative histogram curve, and then normalize it to the range of 0~255; the extreme value normalization method is to calculate the maximum and minimum values ​​of the grayscale values ​​of all pixels in all layers, and normalize the maximum and minimum values ​​to the range of 0~255;

[0102] S22: Perform Gaussian filtering or median filtering on the adjusted layer to remove isolated noise points; the area of ​​the developing device in the image is as follows: Figure 2 As shown, Figure 2 is developed in two different layers (frames) in the layer array. Figure 2 (a) in the middle is a frame layer. Figure 2 (b) is another frame layer. It can be seen that their positions are slightly different. This difference can effectively avoid confusion in spatial position and posture when finally calculating the registration parameters.

[0103] S23: Binarize the filtered layer according to the set grayscale threshold; calculate the total cumulative histogram curve according to the grayscale values ​​of all pixels in all layers, cut off the top at 5‰ below the maximum value in the total cumulative histogram curve, use the corresponding value as the grayscale threshold, set the pixels in all layers whose grayscale values ​​are higher than the grayscale threshold to 255 (white), and set the pixels whose grayscale values ​​are lower than the grayscale threshold to 0 (black), so as to form a binary image (only black and white colors);

[0104] S24: using a projection method to determine the outer boundary of the developing device in the binarized layer, screening out candidate layers, and determining a rough boundary of the developing device according to the outer boundary of the candidate layers; Figure 3 The figure shows a schematic diagram of determining the outer boundary of the developing device by projection method. For the convenience of observation, the black and white colors of the binary layer are displayed interchangeably in the schematic diagram. The specific steps are as follows:

[0105] S241: In each layer, project horizontally toward the vertical axis, and set any white pixel rows along the way to 1, such as Figure 3 The shadow part of the middle vertical axis, otherwise it is set to 0, and a one-dimensional array is obtained as the vertical axis projection ; The vertical projection corresponding to the nth layer It is expressed as:

[0106]

[0107] Where N is the total number of layers in the layer array, H is the total pixel height of each layer, and n represents the nth layer;

[0108] S242: Find the uppermost position (i.e., the position where the longitudinal projection = 1) in the longitudinal projection of each layer as the upper boundary U, and then search downward for a preset height in each layer (the preset height is the physical height L of the developing device, and L can be 30 mm), and search downward to the lower boundary B. When searching downward, move up and down the lower boundary by a set margin delta, and the value of delta can be 5 mm. If the upper edge of the lower boundary in the layer happens to be a row with a longitudinal projection = 1, that is, the upper edge of the lower boundary falls into Figure 3 The shaded area on the left vertical axis, , and the lower edge of the lower boundary is the row of 0, , then this layer is taken as a candidate layer, and the layer number n corresponding to the candidate layer is put into the candidate set C; the upper boundary U is expressed by the formula (assuming that the value of i increases upward along the vertical axis):

[0109] ;

[0110] S243: For each candidate layer corresponding to the candidate set, project the pixels within the row range where the upper boundary U and the lower boundary B are located along the vertical direction toward the horizontal axis to obtain a one-dimensional array as the horizontal axis projection , Figure 3 The horizontal axis shadow is the part with a value of 1; the horizontal axis projection of the mth candidate layer It is expressed as:

[0111]

[0112]

[0113]

[0114] Where min(C) and max(C) are the maximum and minimum layer numbers in the candidate set C, respectively. W is the total pixel width of each layer. The direction of j value increasing to the right along the horizontal axis.

[0115] S244: For each layer corresponding to the candidate set, find the rightmost position (i.e., the position where the horizontal projection = 0) in the horizontal projection of each layer as the right boundary, then search to the left for a preset width (the preset width is the physical width of the developing device) in each layer, and search to the left to the left boundary. When searching to the left, move a set margin delta before and after the left boundary. The value of delta can be 5mm. If the right edge of the left boundary in the layer happens to be the column with the horizontal projection = 1, the right edge is Figure 3 If the shaded area on the lower horizontal axis intersects, that is, the right edge of the left boundary falls into the shaded area on the lower horizontal axis in the figure, and the left edge of the left boundary is a column of 0, then this layer is retained as a candidate layer, otherwise the layer number corresponding to this layer in the candidate set is deleted;

[0116] S245: sorting the layer numbers in the candidate set according to the layer sorting in the layer array, selecting the candidate layers corresponding to the first layer number and the last layer number after sorting, and using the slice space positions in the metadata corresponding to the upper boundary, the lower boundary, the left boundary and the right boundary in the candidate layers as the rough boundaries of the developing device in the coronal and sagittal directions; the slice space positions include the origin space coordinates and the direction vector in the metadata;

[0117] S25: Select one from the candidate layers as the anchor layer; select the candidate layer corresponding to the layer number at the center position in the candidate set as the anchor layer. The development of the center part of the development device is basically the clearest and best distinguished, and is least affected by factors such as image quality or artifacts. In addition, the position distribution of each development point is relatively uniform, easy to segment, and is conducive to subsequent recognition.

[0118] The above S24 and S25 are also the verification process of whether the image is valid and can be used for the purpose of registration. If the outer boundary and anchor point layer of the developing device cannot be roughly located, it means that the scanned developing image does not meet the requirements, or the required developing area is incomplete, and it needs to be adjusted and re-scanned to continue the subsequent processing flow.

[0119] Furthermore, taking the axial direction as an example, the spatial position of the slice can be represented by the spatial coordinates of the first pixel in the upper left corner of the layer in the original image in Dicom or Nifti format (i.e., Image Position), and the row / column direction vector of the plane (i.e., Image Orientation, or the normal vector perpendicular to the plane calculated by cross-producting them). The spatial position of the slice includes the spatial coordinates and direction vector of the origin in the metadata; the same applies to the coronal and sagittal planes. The developing device is installed at the end of the robotic arm. During the registration stage, the robotic arm is located at the upper part of the human body (such as the abdomen) and is appropriately left blank (leaving appropriate movement space for the robotic arm); and there are no other objects in the horizontal left and right directions where the robotic arm (including the developing device) is located, so in the binarized layer corresponding to the original image, the corresponding area is the background color (i.e., black after binarization).

[0120] Furthermore, the specific process of S3 is:

[0121] S31: Select several developing points in the anchor point layer as candidate anchor points, determine the local detection area of ​​each candidate anchor point, and calculate the anchor point center coordinates of each candidate anchor point according to the local detection area; select the 3 developing points in the leftmost column and the 3 developing points in the rightmost column in the anchor point layer as candidate anchor points;

[0122] S311: Preset the local detection area corresponding to each candidate anchor point to be a square with a side length of S;

[0123] S312: Select the first group of consecutive column ranges (j value ranges) that are 1 from the horizontal axis projection of the anchor point layer, such as Figure 4 As shown in the figure, the three candidate anchor point ranges corresponding to the leftmost column are selected, and the last group, that is, the third group of consecutive columns with values ​​of 1, is selected. Figure 4 The ranges of the three candidate anchor points corresponding to the rightmost column are shown;

[0124] S313: Select all three groups of row ranges (i value ranges) whose values ​​are consecutively 1 from the vertical axis projection of the anchor point layer, and make the selected row ranges intersect with the two groups of column ranges to obtain 6 candidate rectangles;

[0125] S314: in each candidate rectangle, the maximum X coordinate value, the maximum Y coordinate value, the minimum X coordinate value, and the minimum Y coordinate value of all white pixels (pixels with a value of 255) are determined respectively, and the average of the X coordinate value and the Y coordinate value is taken respectively to obtain the candidate center coordinate;

[0126] S315: Expand the candidate center coordinates equidistantly along the ±X and ±Y directions respectively, with an expansion length of l=S / 2, to obtain a square with a side length of S as the local detection area, such as Figure 4 The dotted frame outside the middle developing point;

[0127] S32: re-binarize each local detection area;

[0128] S321: Preset the physical diameter D of each candidate anchor point, and calculate the area ratio according to the area of ​​the corresponding local detection area. ;

[0129] The developing device is composed of several vertical and inclined hollow columns connected together. The axial scanning of the vertical hollow column is displayed as a circle, and the axial scanning of the inclined hollow column is displayed as an ellipse. The physical diameter of the cross section of the vertical hollow column is D. In this embodiment, the difference between the major axis and the minor axis length of the ellipse section is ignored, and the average value is taken, and the calculation is also based on the physical diameter D;

[0130] S322: Sort all original pixel values ​​in each local detection area from small to large (there are a total of pixels), index according to the sorting result, and take The original pixel value at is used as the rough pixel threshold; It represents the adjustment coefficient between 0 and 1.0, taking a certain empirical value, which can be 0.9;

[0131] S323: setting pixels in the layer that are higher than the rough pixel threshold to 255, and setting pixels that are lower than the rough pixel threshold to 0, to form a new binary image; by calculating the area ratio, a better threshold can be obtained to normalize the pixel values ​​in the local detection area, so that the edge of the binary development point can be detected more accurately;

[0132] S33: using a contour detection algorithm to detect the candidate anchor points of each re-binarized local detection area, identifying the contour outer frame of the candidate anchor point, and calculating the center point coordinates according to the contour outer frame as the center coordinates of the anchor point.

[0133] Furthermore, the contour detection algorithm can directly call functions from the algorithm library, such as findContours() of OpenCV.

[0134] Furthermore, the specific process of S4 includes:

[0135] S41: constructing a total outer contour according to the center coordinates of each anchor point in the anchor point layer, and cutting out the original pixel areas corresponding to the total outer contour in all layers to obtain a plurality of development areas;

[0136] S411: Taking the minimum circumscribed rectangle including the center coordinates of all anchor points as the center, expanding outward according to a preset blank border, and constructing the total outer contour of the rectangle of the center coordinates of each anchor point in the sagittal and coronal directions;

[0137] S412: Cut out the original pixel area corresponding to the total outer contour in each layer of the layer array as a development area; if there are N layers, N rectangular development areas are obtained, and each development area may include a development point or may be an empty background;

[0138] S42: In this embodiment, the developing device has 7 developing points in each candidate layer, 6 of which are used for registration, and the total outer contour of the anchor point layer is rectangular, with a width and a height of and , then the new area ratio ; Considering the existence of local blur or artifacts, the edge of the image is not a perfect circle or ellipse, so the , It represents the empirical proportionality coefficient, which can be between 0.8 and 1.2;

[0139] S43: sort all original pixel values ​​in the development area of ​​the candidate layer corresponding to the candidate set from small to large, and index them according to the sorting result. The pixel value obtained is used as the precise pixel threshold; M represents the number of candidate layers;

[0140] S44: Calculate characteristic values ​​for the developed areas of all layers according to the precise pixel threshold, and the characteristic value c is expressed as:

[0141]

[0142] in, Represents the characteristic value of the nth layer; Represents the pixel value at position (i, j) of the nth layer, i represents the number of rows, j represents the number of columns. If it is greater than the precise pixel threshold, the function g()=1, otherwise =0. Therefore, this eigenvalue is equivalent to the count of pixel points in the development area of ​​the nth layer whose pixel value is greater than the precise pixel threshold. It can also be regarded as the possibility that the nth layer contains clear, complete, and effective development points.

[0143] S45: Preset the physical length of the developing device in the axial direction, and divide the physical length by the layer spacing (in mm) in the metadata of the original image to obtain the effective developing layer number K, where K is an integer;

[0144] S46: Taking the number of effective developing layers K as the width of a sliding window, using the sliding window to slide from the first layer with a step length of 1 frame until the right side of the sliding window reaches the last layer, and calculating the sum of the eigenvalues ​​of the corresponding layers in the sliding window each time sliding to obtain the sum of the eigenvalues;

[0145] S47: Take the layer of the sliding window start and end positions corresponding to the maximum eigenvalue sum, and the corresponding section space position in the metadata as the outer boundary of the developing device in the axial direction; Figure 5 As shown, the layer numbers on both sides of the outer boundary are F(L) and F(R), and F(R)=F(L)+K-1.

[0146] Furthermore, the specific process of S5 is as follows:

[0147] S51: Starting from the center coordinates of each anchor point located in the anchor point layer, extending forward and backward layer by layer, locating the basic developing point coordinates of the developing points in the adjacent layers within the outer boundary range; adjusting the anchor point center coordinates according to the structural characteristics of the developing device and the layer spacing to obtain the basic developing point coordinates;

[0148] S52: constructing a square with a side length of S with the basic developing point coordinate as the center, detecting the developing point outline in the square, and calculating the center coordinate value as the precise developing point coordinate through the circumscribed rectangle of the developing point outline; if the developing point outline is not clear, or the difference between the center coordinate value and the basic developing point coordinate is greater than a set threshold, the corresponding precise developing point coordinate is deleted and the process goes to S53, otherwise the process goes to S54;

[0149] S53: Based on the precise development point coordinates, linear fitting parameters are calculated, and the precise development point coordinates of all development points in the K-layer layer within the outer boundary range are filled in by interpolation;

[0150] After the above S51 is completed, it is not necessarily guaranteed that all the development points on the K layer can be detected. For example: at the ends of two columns close to the intersection, their respective development points may partially overlap on the section, so that their outer frame area exceeds a certain threshold and is rejected for output, which can reduce the deviation of the center coordinates of the development points; or because the local image is not clear, the edge detection function has no output result. Therefore, it is necessary to further perform linear regression on the development point coordinates set that have been detected and belong to the same hollow column through the least squares method to obtain the fitting parameters, and then interpolate to fill in the corresponding K development point coordinates on the K layer (this operation is performed for each column);

[0151] S54: Arranging the precise development point coordinates according to the order of the layers to form a development point coordinate set in a spatial coordinate system;

[0152] The specific order is the custom order of the columns to which the development points belong. There are three development columns on the left and three on the right, with a total of K layers. The development point coordinate set contains a total of (3+3)×K coordinate values.

[0153] Furthermore, in S51 of the present embodiment, if the center coordinate of the anchor point is located on a vertical column, then when extending forward and backward, the X and Y coordinates of the center coordinate of the anchor point extended to the adjacent layer remain unchanged, and the basic development point coordinate is (X, Y); and if the center coordinate of the anchor point is located on an inclined column (for example, an angle of 45° with the vertical column), then when extending forward and backward, the X coordinate of the center coordinate of the anchor point extended to the adjacent layer remains unchanged, and the Y coordinate is added or subtracted by one layer spacing, and the center coordinate of the anchor point becomes (X, Y').

[0154] Furthermore, the specific process of S6 is as follows:

[0155] S61: collecting the machine coordinates of each point on each hollow column of the developing device in the robot coordinate system, sampling the machine coordinates of the developing device in the axial direction of the original image according to the layer spacing between adjacent layers in the layer array, and aggregating the machine coordinates sampled each time into a coordinate subset to obtain a plurality of coordinate subsets;

[0156] S62: Arrange all coordinate subsets according to the axial acquisition order of the original image to obtain a point cloud of the developing device in the robot coordinate system; collect the machine coordinates according to the developing points in the layer and arrange them according to the layer order;

[0157] S63: aligning the development point coordinate set and the point cloud in the robot coordinate system to obtain alignment parameters;

[0158] The coordinate set of the developing point of the developing device in the image space coordinate system is registered with the point cloud in the robot coordinate system to obtain the corresponding registration parameters (i.e., the transformation matrix between the two coordinate systems). Since the number of elements of the two constructed coordinate sets is equal and the order is consistent, the existing matching function in the algorithm library (such as vtkLandmarkTransform () in VTK) can be directly called to complete the calculation.

[0159] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0160] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for automatically positioning a developing device from an image, characterized in that: The following steps are involved: S1: Collect the original image and perform analysis and processing to obtain the layer array; S2: determining a rough boundary of the developing device in the layer array, and selecting an anchor layer from the layer array; S3: extract several candidate anchor points in the anchor point layer and calculate the anchor point center coordinates of each candidate anchor point; S4: finely positioning the outer boundary of the developing device in the axial direction according to the center coordinates of the anchor point; S5: identifying the developing point coordinates of each layer layer by layer according to the center coordinates of the anchor points in the anchor point layer, fitting the discrete developing point coordinates in the spatial coordinate system, and constructing a developing point coordinate set; S6: constructing a point cloud of the developing device in the robot coordinate system according to the developing point coordinate set, performing registration calculation on the developing point coordinate set and the point cloud, and obtaining registration parameters.

2. A method for automatically positioning a developing device from an image according to claim 1, characterized in that: The specific process of S1 includes: S11: reading the original image in a preset format; S12: Load the data of each layer in the original image, and arrange all the layers in the axial scanning order to form a layer array; S13: parsing metadata corresponding to each layer from the original image; S14: reordering the layers in the layer array according to the metadata, arranging them in the order of the spatial coordinates of the origin in the layers, and assigning a unique layer number to each layer.

3. A method for automatically positioning a developing device from an image according to claim 2, characterized in that: The specific process of S2 includes: S21: using a cumulative histogram normalization method or a maximum value normalization method to adjust the pixel grayscale distribution range of each layer in the layer array; S22: filtering the adjusted layer; S23: binarizing the filtered layer according to the set grayscale threshold; S24: using a projection method to determine the outer boundary of the developing device in the binarized layer, screening out candidate layers, constructing a candidate set according to the layer numbers corresponding to the candidate layers, and determining a rough boundary of the developing device according to the outer boundary in the candidate layers; S25: Select the candidate layer corresponding to the layer number at the center position in the candidate set as the anchor layer.

4. A method for automatically positioning a developing device from an image according to claim 3, characterized in that: The specific steps of S24 are: S241: In each layer, project horizontally toward the vertical axis to obtain a one-dimensional array as the vertical axis projection. ; The vertical projection corresponding to the nth layer It is expressed as: ； Where N is the total number of layers in the layer array, H is the total pixel height of each layer, and n represents the nth layer; S242: Find the highest row with a vertical axis projection of 1 in the vertical axis projection of each layer as the upper boundary, search downward from the upper boundary to a preset height L in each layer, search to the lower boundary, and move the lower boundary up and down by a set margin delta to form the upper edge and lower edge of the lower boundary respectively when searching downward. If the upper edge of the lower boundary in the layer is a row with a vertical axis projection of 1, and the lower edge of the lower boundary is a row with a vertical axis projection of 0, then this layer is taken as a candidate layer, and the layer numbers corresponding to all candidate layers form a candidate set; S243: For each candidate layer corresponding to the candidate set, project the pixels within the row range where the upper boundary and the lower boundary are located along the vertical direction toward the horizontal axis to obtain a one-dimensional array as the horizontal axis projection ; The horizontal projection of the mth candidate layer It is expressed as: ； ； ； Where C represents the candidate set; U represents the upper boundary; B represents the lower boundary; min(C) and max(C) are the maximum layer number and the minimum layer number in the candidate set, respectively; W is the total pixel width of each layer, and the j value increases to the right along the horizontal axis; L represents the preset height; S244: in the horizontal axis projection of each layer corresponding to the candidate set, find the column farthest along the horizontal axis increasing direction with the horizontal axis projection = 1 as the right boundary, search the preset width from the right boundary to the left in each layer, search to the left boundary, and when searching to the left, move the preset margin delta before and after the left boundary to form the left edge and the right edge of the left boundary respectively. If the right edge of the left boundary in the layer is the column with the horizontal axis projection = 1, and the left edge of the left boundary is the column with the horizontal axis projection = 0, then keep this layer as the candidate layer, otherwise delete the layer number corresponding to this layer in the candidate set; S245: sorting the layer numbers in the candidate set according to the layer sorting in the layer array, selecting the candidate layers corresponding to the first layer number and the last layer number after sorting, and the slice space positions in the metadata corresponding to the upper boundary, the lower boundary, the left boundary and the right boundary in the candidate layers are used as the rough boundaries of the developing device in the coronal and sagittal directions.

5. A method for automatically positioning a developing device from an image according to claim 4, characterized in that: The specific process of S3 is: S31: Selecting several development points in the anchor point layer as candidate anchor points, and determining a local detection area for each candidate anchor point; S32: re-binarize each local detection area; S33: Use a contour detection algorithm to detect each binarized local detection area, identify the contour outer frame of the candidate anchor point, and calculate the center point coordinates as the center coordinates of the anchor point according to the contour outer frame.

6. A method for automatically positioning a developing device from an image according to claim 5, characterized in that: The specific steps of S31 are: S311: Preset the local detection area corresponding to each candidate anchor point to be a square with a side length of S; S312: Selecting the first group of column ranges whose values ​​are continuously 1 and the last group of column ranges whose values ​​are continuously 1 from the horizontal projection of the anchor point layer; S313: Select all row ranges with consecutive values ​​of 1 from the vertical axis projection of the anchor point layer, and make the selected row ranges intersect with the two groups of column ranges to obtain a number of candidate rectangles; S314: In each candidate rectangle, the maximum X coordinate value, the maximum Y coordinate value, the minimum X coordinate value, and the minimum Y coordinate value of all pixels whose pixel values ​​are 255 are determined respectively, and the average of the X coordinate value and the Y coordinate value is taken respectively to obtain the candidate center coordinate; S315: Expand the candidate center coordinates outward along the ±X and ±Y directions at equal distances, with an expansion length of l=S / 2, to obtain a square with a side length of S as the local detection area.

7. A method for automatically positioning a developing device from an image according to claim 5, characterized in that: The specific steps of S32 are: S321: Preset the physical diameter D of each candidate anchor point and calculate the ratio of the area of ​​the candidate anchor point to the area of ​​the local detection area ; S322: Sort all original pixel values ​​in each local detection area from small to large, and index them according to the sorting results. The original pixel value at is used as the rough pixel threshold; represents the adjustment coefficient, S represents the side length of the local detection area; S323: Pixels in the layer that are higher than the rough pixel threshold are set to 255, and pixels that are lower than the rough pixel threshold are set to 0, to form a new binary image.

8. A method for automatically positioning a developing device from an image according to claim 3, characterized in that: The specific process of S4 includes: S41: Taking the minimum circumscribed rectangle including the center coordinates of all anchor points in the anchor point layer as the center, expanding outward according to the preset blank border to construct a total outer contour; cutting out the original pixel area corresponding to the total outer contour in each layer of the layer array to obtain a plurality of development areas; S42: The width and height of the total outer contour of the anchor point layer are and , then the new area ratio , represents the empirical proportionality coefficient, and D represents the physical diameter of the candidate anchor point; S43: sort all original pixel values ​​in the development area of ​​the candidate layer corresponding to the candidate set from small to large, and index according to the sorting result. The pixel value at is used as the precise pixel threshold; M represents the number of candidate layers; S44: Calculate characteristic values ​​for the developed areas of all layers according to the precise pixel threshold, and the characteristic value c is expressed as: ； in, Represents the characteristic value of the nth layer; Represents the pixel value at position (i, j) of the nth layer. If it is greater than the precise pixel threshold, the function g()=1, otherwise =0; i represents the number of rows, j represents the number of columns; N represents the total number of layers in the layer array; S45: Preset the physical length of the developing device in the axial direction, and divide the physical length by the layer spacing in the metadata of the original image to obtain the effective developing layer number K; S46: Taking the number of effective developing layers K as the width of a sliding window, using the sliding window to slide from the first layer of the layer array with a step length of 1 frame until the right side of the sliding window reaches the last layer, and calculating the sum of the eigenvalues ​​of the corresponding layers in the sliding window each time sliding to obtain the sum of the eigenvalues; S47: Take the layer of the sliding window start and end positions corresponding to the maximum eigenvalue sum, and the corresponding section space position in the metadata as the outer boundary of the developing device in the axial direction.

9. A method for automatically positioning a developing device from an image according to claim 1, characterized in that: The specific process of S5 is as follows: S51: Starting from the center coordinates of each anchor point located in the anchor point layer, extending forward and backward layer by layer, respectively, to locate the basic development point coordinates of the development points in each layer within the outer boundary range; S52: constructing a square with a side length of S with the basic developing point coordinate as the center, detecting the developing point outline in the square, and calculating the center coordinate value as the precise developing point coordinate through the circumscribed rectangle of the developing point outline; if the developing point outline is not clear, or the difference between the center coordinate value and the basic developing point coordinate is greater than a set threshold, the corresponding precise developing point coordinate is deleted and the process goes to S53, otherwise the process goes to S54; S53: Calculating linear fitting parameters based on the precise development point coordinates, and filling in the precise development point coordinates of all development points in the K-layer layer within the outer boundary range by interpolation; S54: Arranging the precise development point coordinates according to the layer order in the layer array to form a development point coordinate set in a spatial coordinate system.

10. A method for automatically positioning a developing device from an image according to claim 1, characterized in that: The specific process of S6 is as follows: S61: collecting machine coordinates of the developing device in the robot coordinate system, axially sampling the machine coordinates of the developing device according to the layer spacing between adjacent layers in the layer array, aggregating the machine coordinates sampled each time into a coordinate subset, and obtaining a plurality of coordinate subsets; S62: Arrange all coordinate subsets according to the acquisition order of the original images to obtain a point cloud of the developing device in the robot coordinate system; S63: aligning the development point coordinate set and the point cloud in the robot coordinate system to obtain alignment parameters.

Images