Laser positioning line tracking method and tracking system

Abstract

The present invention relates to the technical field of medical positioning, and more specifically, to a laser positioning line tracking method and tracking system. By inputting an image to be detected into a target detection model, a hand-drawn line region of interest and a laser positioning line region of interest can be automatically identified, and data processing is performed on the hand-drawn line region of interest and the laser positioning line region of interest to obtain the coordinate information of the horizontal line segment and the vertical line segment, and then mapped back to the corresponding image to be detected; based on the coordinate information of the vertical line and the horizontal line of the hand-drawn line and the coordinate information of the vertical line and the horizontal line of the laser positioning line, the movement parameters of the treatment bed can be obtained, and the treatment bed is automatically aligned according to the movement parameters. The present invention automatically identifies the positions of the hand-drawn line and the laser positioning line and calculates the translation and rotation errors between the hand-drawn line and the laser positioning line, and controls the automatic positioning of the treatment bed, reduces the position error of the treatment bed, improves the positioning efficiency, and meets the requirements of precision radiotherapy.

Description

Technical Field

[0001] The present invention relates to the technical field of medical positioning, and more specifically, to a laser positioning line tracking method and tracking system. Background Art

[0002] Radiotherapy is one of the three main technologies for cancer treatment, and about one-third of patients need to receive radiotherapy during cancer treatment. With the development of industrial technology and computer equipment, the accuracy of radiotherapy has been continuously improved, and we have entered the era of precision radiotherapy from traditional ordinary radiotherapy. The mechanical accuracy of radiotherapy medical electron linear accelerators can be controlled within 1-2mm, and the dose calculation accuracy of the radiotherapy planning system used for radiotherapy plan optimization and calculation can be controlled within 3%. Therefore, the core issue of continuing to improve the accuracy of precision radiotherapy is not to improve the accuracy of accelerators and planning systems, but to improve the accuracy of individualized quality control related to patients, such as improving the accuracy of patient positioning, improving respiratory movement management, and using individualized positioning devices.

[0003] During the radiotherapy positioning process, a core device that needs to be used is the radiotherapy three-dimensional laser positioning system. The cross-line lasers emitted by lasers at three positions form a three-dimensional virtual coordinate positioning system to simulate the center position of accelerated treatment. During positioning, the laser positioning system of the positioning room is used to mark the patient's tumor center on the fixed body membrane by drawing a line according to the projection position of the laser positioning line, which is called a hand-drawn line. The hand-drawn line is specifically a cross mark. Before radiotherapy, the hand-drawn line on the patient's fixed body membrane is aligned with the laser positioning line in the treatment room to ensure the consistency of the tumor center and the treatment center. The current three-dimensional positioning laser system for radiotherapy only has the function of emitting laser to instruct radiotherapy technicians on positioning and positioning. The following problems are often encountered in clinical practice: 1. Due to the non-rigid movement of the human body, it is difficult to "perfectly" match the hand-drawn lines on the body membrane with the three-dimensional laser positioning lines during positioning. At this time, the positioning technician will adjust the patient's position many times and align the lines by slightly moving the fixed bed or treatment bed. The whole process is time-consuming and labor-intensive; 2. Even after many adjustments to the position of the body or bed, there will still be misalignment. At this time, the technician usually compromises with errors within ±2mm or 3°, which affects the positioning accuracy and then the radiotherapy accuracy, which does not meet the requirements of precision radiotherapy. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of poor positioning accuracy, poor radiotherapy accuracy and low manual positioning efficiency in the prior art, and to provide a laser positioning line tracking method and tracking system, which automatically identify the positions of hand-drawn lines and laser positioning lines and calculate the translation and rotation errors between the hand-drawn lines and the laser positioning lines, and control the automatic positioning of the treatment bed, reduce the position error of the treatment bed, improve the positioning efficiency, and meet the requirements of precise radiotherapy.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A laser positioning line tracking method is provided for aligning a hand-drawn line with a laser positioning line; the method comprises the following steps:

[0007] S10. Acquire a number of image data including hand-drawn lines and laser positioning lines, and perform rectangular annotation on the obtained image data to obtain a training data set;

[0008] S20. Input the training data set in step S10 into the target detection model network framework for iterative training to obtain the final target detection model;

[0009] S30. Take the image to be detected as the input of the final target detection model in step S20 to obtain the hand-drawn region of interest ROI A and laser positioning line region of interest ROI B ;

[0010] S40. The hand-drawn region of interest ROI obtained in step S30 A and laser positioning line region of interest ROI B , crop the image to be detected to obtain the hand-drawn line ROI image and the laser positioning line ROI image;

[0011] S50. Processing the hand-drawn line ROI image and the laser positioning line ROI image in step S40 to obtain a straight line segment group of the hand-drawn line ROI image and a straight line segment group of the laser positioning line ROI image, each straight line segment group includes horizontal line segments and vertical line segments vertically connected to each other;

[0012] S60. Map the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image in step S50 back to the corresponding image to be detected, and obtain: the starting point coordinates, end point coordinates and intersection point coordinates of the vertical line of the hand-drawn line, the starting point coordinates, end point coordinates and intersection point coordinates of the horizontal line of the hand-drawn line, the starting point coordinates, end point coordinates and intersection point coordinates of the vertical line of the laser positioning line, and the starting point coordinates, end point coordinates and intersection point coordinates of the horizontal line of the laser positioning line;

[0013] S70. Based on the coordinates of each starting point, each ending point and each intersection point in step S60, the relative angle displacement difference between the hand-drawn line and the laser positioning line is calculated, thereby calculating the movement parameters of the treatment bed;

[0014] S80. The treatment bed is moved and positioned according to the movement parameters described in step S70 so that the hand-drawn line coincides with the laser positioning line.

[0015] The laser positioning line tracking method of the present invention can automatically identify the hand-drawn line region of interest ROI by inputting the image to be detected into the target detection model A and laser positioning line region of interest ROI B , hand-drawn region of interest ROI A and laser positioning line region of interest ROI B Data processing is performed to obtain the coordinate information of the horizontal and vertical line segments, and then mapped back to the corresponding image to be detected, to obtain the coordinate information of the vertical and horizontal lines of the hand-drawn lines and the coordinate information of the vertical and horizontal lines of the laser positioning lines. Compared with directly processing the image to be detected to obtain the relevant coordinate information, the processing steps are simpler and the recognition results are more accurate; based on the coordinate information of the vertical and horizontal lines of the hand-drawn lines and the coordinate information of the vertical and horizontal lines of the laser positioning lines, the movement parameters of the treatment bed can be obtained, and the treatment bed is automatically aligned according to the movement parameters. The laser positioning line tracking method of the present invention automatically identifies the positions of the hand-drawn lines and the laser positioning lines and calculates the position error between the hand-drawn lines and the laser positioning lines, and controls the automatic positioning of the treatment bed, reduces the position error of the treatment bed, improves the positioning efficiency, and meets the requirements of precision radiotherapy.

[0016] The present invention also provides a laser positioning line tracking system, comprising a laser positioning system for emitting laser positioning lines, a visual camera for capturing a number of image data including hand-drawn lines and laser positioning lines, a treatment bed for patients to lie down and adjust their positions, a diaphragm for fixing the patient's radiotherapy position, and a control system for executing the laser positioning line tracking method as described above, wherein the laser positioning system, the visual camera, and the treatment bed are all connected to the control system, the hand-drawn lines are drawn on the diaphragm, and the diaphragm is connected to the treatment bed.

[0017] The laser positioning line tracking system of the present invention uses a visual camera to capture the image data of the laser positioning line and the hand-drawn line in different positions of the patient, and the image data is input into the control system. The control system recognizes and calculates the movement parameters of the treatment bed, and the treatment bed automatically aligns according to the movement parameters. The laser positioning line tracking system of the present invention automatically identifies the positions of the hand-drawn line and the laser positioning line and calculates the translation and rotation errors between the hand-drawn line and the laser positioning line, and controls the automatic positioning of the treatment bed, reduces the position error of the treatment bed, improves the positioning efficiency, and meets the requirements of precision radiotherapy.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The laser positioning line tracking method and tracking system of the present invention automatically identify the positions of the hand-drawn line and the laser positioning line and calculate the translation and rotation errors between the hand-drawn line and the laser positioning line, and control the automatic positioning of the treatment bed, thereby reducing the position error of the treatment bed, improving the positioning efficiency, and meeting the requirements of precise radiotherapy;

[0020] The laser positioning line tracking method of the present invention first identifies the region of interest and intercepts the region of interest for analysis and identification for the image to be detected, and then maps the analysis and identification result to the image to be detected. Compared with directly analyzing and identifying the image to be detected, it greatly reduces the amount of data required to be processed, reduces the difficulty of data processing, reduces image recognition noise, and improves recognition accuracy, thereby helping to improve the alignment accuracy of treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of the laser positioning line tracking method;

[0022] Figure 2 is a schematic diagram of the structure of the membrane marked with hand-drawn lines;

[0023] Figure 3 This is a schematic diagram of the image of the patient's right side in the YOZ plane;

[0024] Figure 4 This is a schematic diagram of the image of the patient's left side in the YOZ plane;

[0025] Figure 5 It is a schematic diagram of the image of the middle of the patient in the XOY plane;

[0026] Figure 6 Schematic diagram of the laser positioning line tracking system in Example 3. DETAILED DESCRIPTION

[0027] The present invention is further described below in conjunction with specific implementation methods. The accompanying drawings are only used for exemplary descriptions and are only schematic diagrams, not actual drawings, and cannot be understood as limiting this patent; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0028] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0029] Embodiment 1

[0030] This embodiment is an embodiment of the laser positioning line tracking method of the present invention. Figure 1 As shown, the following steps are included:

[0031] S10. Acquire a number of image data including hand-drawn lines and laser positioning lines, and perform rectangular annotation on the obtained image data to obtain a training data set;

[0032] S20. Input the training data set in step S10 into the target detection model network framework for iterative training to obtain the final target detection model;

[0033] S30. Take the image to be detected as the input of the final target detection model in step S20 to obtain the hand-drawn region of interest ROI A and laser positioning line region of interest ROI B ;

[0034] S40. The hand-drawn region of interest ROI obtained in step S30 A and laser positioning line region of interest ROI B , crop the image to be detected to obtain the hand-drawn line ROI image and the laser positioning line ROI image;

[0035] S50. Processing the hand-drawn line ROI image and the laser positioning line ROI image in step S40 to obtain a straight line segment group of the hand-drawn line ROI image and a straight line segment group of the laser positioning line ROI image, each straight line segment group includes horizontal line segments and vertical line segments vertically connected to each other;

[0036] S60. Map the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image in step S50 back to the corresponding image to be detected, and obtain: the starting point coordinates, end point coordinates and intersection point coordinates of the vertical line of the hand-drawn line, the starting point coordinates, end point coordinates and intersection point coordinates of the horizontal line of the hand-drawn line, the starting point coordinates, end point coordinates and intersection point coordinates of the vertical line of the laser positioning line, and the starting point coordinates, end point coordinates and intersection point coordinates of the horizontal line of the laser positioning line;

[0037] S70. Based on the coordinates of each starting point, each ending point and each intersection point in step S60, the relative angle displacement difference between the hand-drawn line and the laser positioning line is calculated, thereby calculating the movement parameters of the treatment bed;

[0038] S80. The treatment bed is moved and positioned according to the movement parameters described in step S70 so that the hand-drawn line coincides with the laser positioning line.

[0039] When the present embodiment is implemented, the image to be detected is input into the target detection model to automatically identify the hand-drawn line region of interest ROI. A and laser positioning line region of interest ROIB , hand-drawn region of interest ROI A and laser positioning line region of interest ROI B Data processing is performed to obtain coordinate information of horizontal and vertical line segments, which are then mapped back to the corresponding image to be detected to obtain coordinate information of the vertical and horizontal lines of the hand-drawn lines and the vertical and horizontal lines of the laser positioning lines. Compared with directly processing the image to be detected to obtain relevant coordinate information, the processing steps are simpler and the recognition results are more accurate. Based on the coordinate information of the vertical and horizontal lines of the hand-drawn lines and the vertical and horizontal lines of the laser positioning lines, the movement parameters of the treatment bed can be obtained, and the treatment bed is automatically aligned according to the movement parameters.

[0040] In this embodiment, before the patient is positioned, three images of image data to be detected are obtained each time, corresponding to the right side of the patient, the left side of the patient, and the middle of the patient, respectively. The orientation of the image obtained by shooting corresponds to the orientation of the laser positioning line emitted by the currently commonly used laser positioning system. Each image data to be detected includes a group of hand-drawn lines and a group of laser positioning lines, each group of hand-drawn lines includes a horizontal line and a vertical line that intersect vertically, and each group of laser positioning lines also includes a horizontal line and a vertical line that intersect vertically.

[0041] The spatial coordinate system is established with the horizontal laser positioning line PL as the X-axis, the vertical laser positioning line QL as the Z-axis, the middle laser positioning line UL as the Y-axis, and the cross intersection of the laser positioning lines as the origin. The image data to be detected are all two-dimensional images. The image corresponding to the right side of the patient is in the YOZ plane, the image corresponding to the left side of the patient is in the YOZ plane, and the image corresponding to the middle of the patient is in the XOY plane.

[0042] In addition, in this embodiment, a diaphragm is used to fix the patient's head, and the position of the hand-drawn line on the diaphragm is as follows: Figure 2 However, it should be noted that the present invention is not limited to positioning the treatment position of the patient's head, but is also applicable to radiotherapy positioning of other parts of the patient. When radiotherapy positioning is performed on other parts, it is only necessary to change the position of the picture taken and obtain the picture data of different orientations of the corresponding part without making other changes.

[0043] In step S10, the rectangle annotation tool labelimg is used to perform rectangle annotation on the hand-drawn lines and laser positioning lines in the image data. The information of the rectangle annotation includes the hand-drawn line label drawn_line, the laser positioning line label Laser_line, the upper left corner coordinates and the lower right corner coordinates of the hand-drawn line annotation rectangle, and the upper left corner coordinates and the lower right corner coordinates of the laser positioning line annotation rectangle. The purpose of step S10 is to obtain a training data set of the target detection model network framework to train the target detection model network framework. The final target detection model, when inputting image data, can automatically identify the hand-drawn line area of ​​interest and the laser positioning line area of ​​interest on the image, thereby narrowing the scope of analysis and identification, reducing the amount of data calculation, and improving the recognition accuracy.

[0044] In step S20, the target detection model network framework can adopt the YOLOv5 network model, but is not limited to the YOLOv5 network model. Other network models that can recognize rectangular annotations in images can be applicable to the present invention.

[0045] In step S30, in order to avoid the influence of data noise, this embodiment identifies the color and line width of the laser positioning line and hand-drawn line on the image to be detected before inputting the image to be detected into the final target detection model. Only the image to be detected whose color and line width meet the requirements is input into the final target detection model to ensure the accuracy of the result prediction. Specifically, the result output by the final target detection model is the ROI of the hand-drawn line of the i-th image to be detected. A And the region of interest ROI of the laser positioning line of the i-th image to be detected B , the region of interest ROI of the hand-drawn line of the i-th image to be detected A The information includes the hand-drawn line label, the coordinates of the upper left corner of the hand-drawn line rectangle p left_up (x i,A0 ,y i,A0 ) and the lower right corner coordinate p right_down (x i,A1 ,y i,A1 ), the region of interest ROI of the laser positioning line of the i-th image to be detected B The information includes the laser positioning line label, the upper left corner coordinates p of the laser positioning line marked rectangle l ' eft_up (x i ' ,A0 ,y i ' ,A0 ) and the lower right corner coordinate p' right_down (x i ' ,A1 ,y i ' ,A1 ). Among them, i=0,1,2, i=0 represents the right side, i=1 represents the left side, and i=2 represents the middle.

[0046] In step S40, a hand-drawn region of interest ROI is obtained from the image to be detected. A and laser positioning line region of interest ROI B The hand-drawn line ROI image and the laser positioning line ROI image are cut out and used as the processing objects in the subsequent steps, which can greatly reduce the difficulty of data processing and the amount of data calculation.

[0047] Step S50 is specifically performed according to the following steps:

[0048] S51. Grayscale the hand-drawn line ROI image and the laser positioning line ROI image;

[0049] S52. Using the Canny contour detection method to perform contour detection on the hand-drawn line ROI image and the laser positioning line ROI image processed in step S51 to obtain the contour of the hand-drawn line ROI image and the contour of the laser positioning line ROI image;

[0050] S53. Use the Hough transform straight line detection method to perform straight line detection on the hand-drawn line ROI image contour and the laser positioning line ROI image contour in step S52, and obtain the starting point coordinates, ending point coordinates and intersection point coordinates of the horizontal and vertical line segments of the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image.

[0051] Among them, the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image are three groups, each straight line segment group includes a horizontal line segment and a vertical line segment, and the horizontal line segment and the vertical line segment intersect vertically. In addition, the Canny contour detection method and the Hough transform straight line detection method used in the above steps are both existing detection methods.

[0052] Specifically, step S53 is performed as follows:

[0053] For the i-th hand-drawn line ROI image, the starting point coordinate d of the j-th line segment can be obtained by Hough transform line detection. ij (x 0ij ,y 0ij ) and the end point coordinates d ij (x 1ij ,y 1ij ) ; A coordinate system is established with the point at the upper left corner of the hand-drawn ROI image as the origin, the horizontal side of the hand-drawn ROI image as the X-axis, and the vertical side of the hand-drawn ROI image as the Y-axis. The straight line vector is expressed as X-axis direction vector Y-axis direction vector

[0054] The angles between the straight line j and the X and Y axis directions can be expressed as:

[0055]

[0056]

[0057] For two straight line segments in the i-th straight line segment group, if r x,ij >r y,ij , then the straight line segment is a vertical line segment, and the coordinates of its starting point are marked as t 0i (x 0i ,y 0i ), the end point coordinates are marked as t 1i (x 1i ,y 1i ); if r x,ij <r y,ij , then the line segment is a horizontal straight line segment, and the coordinates of its starting point are marked as s 0i (x' 0i ,y' 0i ), the end point coordinates are marked as s 1i (x' 1i ,y' 1i ), the above coordinate values ​​can be found at the starting point coordinate d ij (x 0ij ,y 0ij ) and the end point coordinates d ij (x 1ij ,y 1ij ) find the corresponding value, which becomes the starting point and end point determined by the coordinate value, and calculates the intersection point of the hand-drawn line according to the principle of intersection of two straight lines. i,d (x i ,y i );

[0058] Similarly, for the i-th laser positioning line ROI image, the vertical line segment and horizontal line segment of the i-th laser positioning line ROI image are obtained, and the coordinates of the starting point of the vertical line segment are marked as p 0i (u 0i ,v 0i ), the end point coordinates are marked as p 1i (u 1i ,v 1i ), the coordinates of the starting point of the horizontal line segment are marked as q 0i (u' 0i ,v' 0i ), the end point coordinates are marked as q 1i (u' 1i ,v' 1i ), the above coordinate values ​​can be found at the starting point coordinate d ij (x 0ij ,y 0ij ) and the end point coordinates dij (x 1ij ,y 1ij ) find the corresponding value, which becomes the starting point and end point determined by the coordinate value, and calculates the intersection point o of the laser positioning line according to the principle of intersection of two straight lines i,l (u i ,v i ).

[0059] In step S60, the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image in step S50 are mapped back to the corresponding image to be detected. Specifically, the hand-drawn line ROI image and the laser positioning line ROI image are restored to the cropped image to be detected to determine the starting point coordinates, end point coordinates and intersection coordinates of each vertical line and horizontal line in the image to be detected. The coordinates here refer to the coordinate values ​​in the following coordinate system: the image-related coordinates corresponding to the right side of the patient are the coordinate values ​​in the YOZ plane, the image-related coordinates corresponding to the left side of the patient are the coordinate values ​​in the YOZ plane, and the image-related coordinates corresponding to the middle of the patient are the coordinate values ​​in the XOY plane.

[0060] The coordinates of the vertical line, horizontal line and intersection point of the hand-drawn line are expressed as follows:

[0061] The coordinates of the starting point of the vertical line are: T 0i (X 0i ,Y 0i ) = t 0i (x 0i ,y 0i )+p left_up (x i,A0 ,y i,A0 );

[0062] The coordinates of the end point of the vertical line are: T 1i (X 1i ,Y 1i ) = t 1i (x 1i ,y 1i )+p left_up (x i,A0 ,y i,A0 );

[0063] The direction vector of the vertical line is identified as:

[0064] The coordinates of the starting point of the horizontal line are: S 0i (X' 0i ,Y' 0i )=s 0i (x' 0i ,y' 0i )+pleft_up (x i,A0 ,y i,A0 );

[0065] The coordinates of the end point of the horizontal line are: S 1i (X' 1i ,Y' 1i )=s 1i (x' 1i ,y' 1i )+p left_up (x i,A0 ,y i,A0 );

[0066] The direction vector of the horizontal line is identified as:

[0067] Intersection of hand-drawn lines: O i,d (X i ,Y i )=o i,d (x i ,y i )+p left_up (x i,A0 ,y i,A0 );

[0068] The coordinates of the vertical line, horizontal line and intersection point of the laser positioning line are expressed as follows:

[0069] The coordinates of the starting point of the vertical line are: P 0i (U 0i ,V 0i )=p 0i (u 0i ,v 0i )+p' left_up (x' i,A0 ,y' i,A0 );

[0070] The coordinates of the end point of the vertical line are: P 1i (U 1i ,V 1i )=p 1i (u 1i ,v 1i )+p' left_up (x' i,A0 ,y' i,A0 );

[0071] The direction vector of the vertical line is identified as:

[0072] The coordinates of the starting point of the horizontal line are: Q 0i (U' 0i ,V' 0i )=q0i (u' 0i ,v' 0i )+p' left_up (x' i,A0 ,y' i,A0 );

[0073] The coordinates of the end point of the horizontal line are: Q 1i (U' 1i ,V' 1i )=q 1i (u' 1i ,v' 1i )+p' left_up (x' i,A0 ,y' i,A0 );

[0074] The direction vector of the horizontal line is identified as:

[0075] Intersection point of laser positioning line: O i,l (U i ,V i )=o i,l (u i ,v i )+p' left_up (x' i,A0 ,y' i,A0 ).

[0076] In step S70, the coordinates of the intersection center of the hand-drawn line and the intersection center of the laser line are expressed as:

[0077] O i,d (X i ,Y i )=o i,d (x i ,y i )+p left_up (x i,A0 ,y i,A0 )

[0078] O i,l (U i ,V i )=o i,l (u i ,v i )+p' left_up (x' i,A0 ,y' i,A0 )

[0079] Among them, i = 0, 1, 2, i = 0 represents the right side, i = 1 represents the left side, and i = 2 represents the middle;

[0080] The relative angular displacement difference between the hand-drawn line and the laser positioning line is expressed as:

[0081] pR=U0-X0

[0082] qR=V0-Y0

[0083] pL=U1-X1

[0084] qL=V1-Y1

[0085] γ=U2-X2

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] Where pR is the horizontal displacement distance on the right, qR is the vertical displacement distance on the right, pL is the horizontal displacement distance on the left, qL is the vertical displacement distance on the left, γ is the middle displacement distance, αR is the parallel angle of the hand-drawn line on the right, βR is the vertical angle of the hand-drawn line on the right, αL is the parallel angle of the hand-drawn line on the left, βL is the vertical angle of the hand-drawn line on the left, and θ is the angle of the middle hand-drawn line. Figures 3 to 5 shown.

[0092] The parameters obtained in step S70 are all in pixel units. In order to convert pixel units into millimeter units, this embodiment further includes, after step S70, a step of converting the relative angular displacement difference in pixel units into an angular displacement difference in millimeter units and angle units; including:

[0093] By calibrating each visual camera, the pixel of the image to be detected on the right is obtained as d right (mm)×d right (mm), the pixel of the image to be detected on the left is d left (mm)×d left (mm), the pixel of the middle image to be detected is d center (mm)×d center (mm), the calculation formula of the movement parameters of the treatment bed is as follows:

[0094] The displacement in the X direction is: L x =γ*d center (mm)

[0095] The displacement in the Y direction is:

[0096] The displacement in the Z direction is:

[0097] The angle around the X axis is:

[0098] The angle around the Y axis is: X represents the patient’s shoulder width;

[0099] The angle around the Z axis is: λ=θ.

[0100] Among them, the angle around the Y axis is calculated by the X-shoulder width, and there is no need to collect image data of the XOZ plane. Three visual cameras can realize detection, ensuring high tracking accuracy without increasing costs.

[0101] Since the treatment bed is located in the XYZ three-dimensional space coordinate system, in order to realize the movement of the treatment bed in the XYZ three-dimensional space coordinate system, this embodiment also includes the step of converting the calculated movement parameters in the plane coordinate system into movement parameters in the three-dimensional space coordinate system, including:

[0102] L x , L y , L z A three-dimensional space vector composed of a three-dimensional point and the coordinate origin

[0103] The three-dimensional space rotation matrix is ​​calculated according to the following three formulas:

[0104]

[0105]

[0106]

[0107] According to the ZYX rotation order, we can get the rotation matrix M 3×3 =R z (λ)*R y (β)*R x (α);

[0108] After obtaining the rotation matrix, the space vector Perform a rotation transformation to obtain a new space vector The calculation formula is:

[0109]

[0110] The final L x ,L y ,L z ,α,β,λ are the bed movement values ​​of the treatment bed.

[0111] The movable bed in this embodiment is a six-dimensional bed, and the adjustment of each dimension is electrically controlled.

[0112] In addition, during the patient's radiotherapy, this embodiment monitors the position of the hand-drawn line and the laser positioning line in real time, and issues an early warning of changes in the patient's position during treatment. The early warning may be in the form of a sound warning, a light warning, or other early warning forms. At the same time, the position of the hand-drawn line and the laser positioning line before treatment is recorded, and the positioning data is recorded and archived to facilitate historical tracing.

[0113] In summary, after the above steps, the present invention can realize the automatic tracking of the hand-drawn line and the laser positioning line, realize the automatic positioning of the treatment bed, and do not need to manually adjust the patient's position, saving time and effort. The rotation and translation errors can be controlled within 1° and 0.5mm, with high positioning accuracy, meeting the development requirements of precision radiotherapy. Moreover, in this embodiment, three groups of images to be detected can be used to calculate and identify the movement parameters of the treatment bed, and better alignment accuracy can be achieved without increasing costs.

[0114] Embodiment 2

[0115] This embodiment is the second embodiment of a laser positioning line tracking method of the present invention. This embodiment is similar to the first embodiment, except that: each group of images to be detected is four, i=0, 1, 2, i=0 represents the right side, i=1 represents the left side, i=2 represents the middle, i=3 represents the front or back side; the image corresponding to the right side of the patient is in the YOZ plane, the image corresponding to the left side of the patient is in the YOZ plane, the image corresponding to the middle of the patient is in the XOY plane, and the image corresponding to the front or back side of the patient is in the XOZ plane. In this way, the angle around the Y axis can be calculated by referring to the calculation method in the first embodiment and combining the coordinate parameters in the XOZ plane.

[0116] In summary, after the above steps, the present invention can realize the automatic tracking of the hand-drawn line and the laser positioning line, realize the automatic positioning of the treatment bed, and there is no need to manually adjust the patient's position, which saves time and effort. The rotation and translation errors can be controlled within 1° and 0.5mm, and the positioning accuracy is high, meeting the development requirements of precision radiotherapy. Compared with the first embodiment, this embodiment has more accurate calculation results of its movement parameters because it has complete data in the XOY plane, XOZ plane, and YOZ plane, and has higher alignment accuracy than the first embodiment.

[0117] Embodiment 3

[0118] This embodiment is the first embodiment of a laser positioning line tracking system of the present invention, comprising a laser positioning system 1 for emitting laser positioning lines, a visual camera 2 for photographing a plurality of image data including hand-drawn lines and laser positioning lines, a treatment bed 3 for patients to lie down and adjust the patient's position, a diaphragm 4 for fixing the patient's radiotherapy position, and a control system for executing the laser positioning line tracking method as described in the first embodiment, wherein the laser positioning system 1, the visual camera 2, and the treatment bed 3 are all connected to the control system, the hand-drawn lines are drawn on the diaphragm 4, and the diaphragm 4 is connected to the treatment bed 3, as shown in FIG. Figure 6 shown. Figure 6 The middle diaphragm 4 is used to fix the patient's head, but this is only an example and is not intended to be a limitation of the present invention. The present invention can use different diaphragms to fix the corresponding parts of the patient for different treatment parts.

[0119] Among them, there are three groups of laser positioning systems 1 and three groups of visual cameras 2. The positions of the visual cameras 2 correspond to the positions of the laser positioning systems. The three groups of visual cameras respectively shoot the left side, right side, and middle (shooting from top to bottom) of the patient to obtain the images to be detected. The images to be detected are respectively located in the YOZ plane, YOZ plane, and XOY plane.

[0120] The present invention can realize automatic tracking of hand-drawn lines and laser positioning lines, realize automatic positioning of the treatment bed, and do not need to manually adjust the patient's position, saving time and effort. The rotation and translation errors can be controlled within 1° and 0.5mm, with high alignment accuracy, meeting the development requirements of precision radiotherapy. Moreover, in this embodiment, three sets of visual cameras can be used to calculate and identify the movement parameters of the treatment bed, and better alignment accuracy can be achieved without increasing costs.

[0121] Embodiment 4

[0122] This embodiment is the second embodiment of a laser positioning line tracking system of the present invention. This embodiment is similar to the third embodiment, except that there are three groups of laser positioning systems and four groups of visual cameras. The visual cameras are arranged on the mechanical structure of the laser positioning system. The four groups of visual cameras respectively shoot the left side, right side, middle (shooting from top to bottom), and front side (shooting from the patient's head to feet) of the patient to obtain images to be detected. The images to be detected are respectively located in the YOZ plane, YOZ plane, XOY plane and XOZ plane.

[0123] The present invention can realize automatic tracking of hand-drawn lines and laser positioning lines, realize automatic positioning of the treatment bed, and do not need to manually adjust the patient's position, saving time and effort. The rotation and translation errors can be controlled within 1° and 0.5mm, with high alignment accuracy, meeting the development requirements of precision radiotherapy. Compared with Example 3, this embodiment has more accurate calculation results of its movement parameters because it has complete data in the XOY plane, XOZ plane, and YOZ plane, and has higher alignment accuracy than Example 3.

[0124] In the specific contents of the above-mentioned specific implementation methods, the various technical features can be combined in any non-contradictory manner. In order to make the description concise, not all possible combinations of the above-mentioned technical features are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A laser positioning line tracking method for aligning a hand-drawn line with a laser positioning line; characterized in that: The following steps are involved: S10. Acquire a number of image data including hand-drawn lines and laser positioning lines, and perform rectangular annotation on the obtained image data to obtain a training data set; S20. Input the training data set in step S10 into the target detection model network framework for iterative training to obtain the final target detection model; S30. Take the image to be detected as the input of the final target detection model in step S20 to obtain the hand-drawn region of interest ROI A and laser positioning line region of interest ROI B ; S40. The hand-drawn region of interest ROI obtained in step S30 A and laser positioning line region of interest ROI B , crop the image to be detected to obtain the hand-drawn line ROI image and the laser positioning line ROI image; S50. Processing the hand-drawn line ROI image and the laser positioning line ROI image in step S40 to obtain a straight line segment group of the hand-drawn line ROI image and a straight line segment group of the laser positioning line ROI image, each straight line segment group includes horizontal line segments and vertical line segments vertically connected to each other; S60. Map the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image in step S50 back to the corresponding image to be detected, and obtain: the starting point coordinates, end point coordinates and intersection point coordinates of the vertical line of the hand-drawn line, the starting point coordinates, end point coordinates and intersection point coordinates of the horizontal line of the hand-drawn line, the starting point coordinates, end point coordinates and intersection point coordinates of the vertical line of the laser positioning line, and the starting point coordinates, end point coordinates and intersection point coordinates of the horizontal line of the laser positioning line; S70. Based on the coordinates of each starting point, each ending point and each intersection point in step S60, the relative angle displacement difference between the hand-drawn line and the laser positioning line is calculated, thereby calculating the movement parameters of the treatment bed; S80. The treatment bed is moved according to the movement parameters described in step S70 so that the hand-drawn line coincides with the laser positioning line; Step S50 is performed according to the following steps: S51. Grayscale the hand-drawn line ROI image and the laser positioning line ROI image; S52. Using the Canny contour detection method to perform contour detection on the hand-drawn line ROI image and the laser positioning line ROI image processed in step S51 to obtain the contour of the hand-drawn line ROI image and the contour of the laser positioning line ROI image; S53. Perform straight line detection on the hand-drawn line ROI image contour and the laser positioning line ROI image contour in step S52 using the Hough transform straight line detection method to obtain the starting point coordinates, ending point coordinates and intersection point coordinates of the horizontal and vertical line segments of the straight line segment group of the hand-drawn line ROI image and the straight line segment group of the laser positioning line ROI image; Step S53 is performed as follows: For the i-th hand-drawn line ROI image, the starting point coordinate d of the j-th line segment can be obtained by Hough transform line detection. ij (x 0ij ,y 0ij ) and the end point coordinates d ij (x 1ij ,y 1ij ) ; A coordinate system is established with the point at the upper left corner of the hand-drawn ROI image as the origin, the horizontal side of the hand-drawn ROI image as the X-axis, and the vertical side of the hand-drawn ROI image as the Y-axis. The straight line vector is expressed as X-axis direction vector Y-axis direction vector The angles between the straight line vector of the jth line segment and the X and Y axis directions can be expressed as: For two straight line segments in the i-th straight line segment group, if r x,ij >r y,ij , then the straight line segment is a vertical line segment, and the coordinates of its starting point are marked as t 0i (x 0i ,y 0i ), the end point coordinates are marked as t 1i (x 1i ,y 1i ); if r x,ij <r y,ij , then the line segment is a horizontal straight line segment, and the coordinates of its starting point are marked as s 0i (x' 0i ,y' 0i ), the end point coordinates are marked as s 1i (x' 1i ,y' 1i ), and calculate the intersection point o of the hand-drawn line according to the principle of intersection of two straight lines i,d (x i ,y i ); Similarly, for the i-th laser positioning line ROI image, the vertical line segment and horizontal line segment of the i-th laser positioning line ROI image are obtained, and the coordinates of the starting point of the vertical line segment are marked as p 0i (u 0i ,v 0i ), the end point coordinates are marked as p 1i (u 1i ,v 1i ), the coordinates of the starting point of the horizontal line segment are marked as q 0i (u' 0i ,v' 0i ), the end point coordinates are marked as q 1i (u' 1i ,v' 1i ), and calculate the intersection point o of the laser positioning line based on the principle of intersection of two straight lines i,l (u i ,v i ).

2. The laser positioning line tracking method according to claim 1, characterized in that: In step S10, the rectangle annotation tool labelimg is used to perform rectangular annotation on the hand-drawn lines and laser positioning lines in the image data. The rectangle annotation information includes the hand-drawn line label drawn_line, the laser positioning line label Laser_line, the upper left corner coordinates and lower right corner coordinates of the hand-drawn line annotation rectangle, and the upper left corner coordinates and lower right corner coordinates of the laser positioning line annotation rectangle.

3. The laser positioning line tracking method according to claim 1, characterized in that: In step S30, the region of interest ROI of the hand-drawn line of the i-th image to be detected is A The information includes the hand-drawn line label, the coordinates of the upper left corner of the hand-drawn line rectangle p left_up (x i,A0 ,y i,A0 ) and the lower right corner coordinate p right_down (x i,A1 ,y i,A1 ), the region of interest ROI of the laser positioning line of the i-th image to be detected B The information includes the laser positioning line label, the upper left corner coordinate p' of the laser positioning line marked rectangle left_up (x' i,A0 ,y' i,A0 ) and the lower right corner coordinate p' right_down (x' i,A1 ,y' i,A1 ).

4. The laser positioning line tracking method according to claim 3, characterized in that: In step S60, the coordinates of the vertical line, the horizontal line and the intersection point of the hand-drawn line are expressed as follows: The coordinates of the starting point of the vertical line are: T 0i (X 0i ,Y 0i ) = t 0i (x 0i ,y 0i )+p left_up (x i,A0 ,y i,A0 ); The coordinates of the end point of the vertical line are: T 1i (X 1i ,Y 1i ) = t 1i (x 1i ,y 1i )+p left_up (x i,A0 ,y i,A0 ); The direction vector of the vertical line is identified as: The coordinates of the starting point of the horizontal line are: S 0i (X' 0i ,Y' 0i )=s 0i (x' 0i ,y' 0i )+p left_up (x i,A0 ,y i,A0 ); The coordinates of the end point of the horizontal line are: S 1i (X' 1i ,Y' 1i )=s 1i (x' 1i ,y' 1i )+p left_up (x i,A0 ,y i,A0 ); The direction vector of the horizontal line is identified as: The coordinates of the intersection of the hand-drawn lines are marked as: O i,d (X i ,Y i )=o i,d (x i ,y i )+p left_up (x i,A0 ,y i,A0 ); The coordinates of the vertical line, horizontal line and intersection point of the laser positioning line are expressed as follows: The coordinates of the starting point of the vertical line are: P 0i (U 0i ,V 0i )=p 0i (u 0i ,v 0i )+p' left_up (x' i,A0 ,y' i,A0 ); The coordinates of the end point of the vertical line are: P 1i (U 1i ,V 1i )=p 1i (u 1i ,v 1i )+p' left_up (x' i,A0 ,y' i,A0 ); The direction vector of the vertical line is identified as: The coordinates of the starting point of the horizontal line are: Q 0i (U' 0i ,V' 0i )=q 0i (u' 0i ,v' 0i )+p' left_up (x' i,A0 ,y' i,A0 ); The coordinates of the end point of the horizontal line are: Q 1i (U' 1i ,V' 1i )=q 1i (u' 1i ,v' 1i )+p' left_up (x' i,A0 ,y' i,A0 ); The direction vector of the horizontal line is identified as: The coordinates of the intersection of the laser positioning line are marked as: i,l (U i ,V i )=o i,l (u i ,v i )+p' left_up (x' i,A0 ,y' i,A0 ).

5. The laser positioning line tracking method according to claim 4, characterized in that: In step S70, i=0, 1, 2, i=0 represents the right side, i=1 represents the left side, and i=2 represents the middle; the relative angular displacement difference between the hand-drawn line and the laser positioning line is expressed as: pR=U0-X0 qR=V0-Y0 pL=U1-X1 qL=V1-Y1 γ=U2-X2 Where pR is the horizontal displacement distance on the right, qR is the vertical displacement distance on the right, pL is the horizontal displacement distance on the left, qL is the vertical displacement distance on the left, γ is the middle displacement distance, αR is the parallel angle of the hand-drawn line on the right, βR is the vertical angle of the hand-drawn line on the right, αL is the parallel angle of the hand-drawn line on the left, βL is the vertical angle of the hand-drawn line on the left, and θ is the angle of the middle hand-drawn line.

6. The laser positioning line tracking method according to claim 5, characterized in that: After step S70, the step of converting the relative angular displacement difference in pixel units into the angular displacement difference in millimeter units and angle units is also included; including: By calibrating each visual camera, the pixel of the first image to be detected is d right (mm)×d right (mm), the pixel of the second image to be detected is d left (mm)×d left (mm), the pixel of the third image to be detected is d center (mm)×d center (mm), the calculation formula of the movement parameters of the treatment bed is as follows: The displacement in the X direction is: L x =γ*d center (mm) The displacement in the Y direction is: The displacement in the Z direction is: The angle around the X axis is: The angle around the Y axis is: X represents the patient’s shoulder width; The angle around the Z axis is: λ=θ.

7. The laser positioning line tracking method according to claim 6, characterized in that: The method further includes the step of converting the calculated movement parameters in the plane coordinate system into movement parameters in the three-dimensional space coordinate system, including: L x , L y , L z A three-dimensional space vector composed of a three-dimensional point and the coordinate origin The three-dimensional space rotation matrix is ​​calculated according to the following three formulas: According to the ZYX rotation order, we can get the rotation matrix M 3×3 =R z (λ)*R y (β)*R x (α); After obtaining the rotation matrix, the space vector Perform a rotation transformation to obtain a new space vector The calculation formula is: The final L x ,L y ,L z ,α,β,λ are the bed movement values ​​of the treatment bed.

8. A laser positioning line tracking system, characterized in that: The invention comprises a laser positioning system for emitting laser positioning lines, a visual camera for capturing a plurality of image data including hand-drawn lines and laser positioning lines, a treatment bed for patients to lie down and adjust their positions, a diaphragm for fixing the patient's radiotherapy position, and a control system for executing the laser positioning line tracking method as claimed in any one of claims 1 to 7, wherein the laser positioning system, the visual camera, and the treatment bed are all connected to the control system, the hand-drawn lines are drawn on the diaphragm, and the diaphragm is connected to the treatment bed.

Images