Body surface breathing movement tracking system and method based on position perception mark

Abstract

The invention provides a body surface respiratory movement tracking system and method based on a position sensing mark, which are used in the field of radiotherapy, and are characterized in that a visual mark generated based on a visual position sensing coding technology is arranged on clothes worn by a patient; detection and three-dimensional reconstruction of body surface marks are achieved through a binocular vision system to obtain body surface point clouds under respiratory movement, dimension reduction and fusion of body surface feature point clouds are achieved through PCA for the body surface point clouds at different moments, and body surface respiratory signals capable of being used for tumor movement tracking are extracted.

Description

Technical Field

[0001] This invention belongs to the field of image-guided precision radiotherapy, specifically relating to a body surface respiratory motion tracking system and method based on position-aware markers. Background Technology

[0002] Radiation therapy is an important cancer treatment method. Its purpose is to maximize the local dose to the tumor, minimize damage to surrounding normal tissues and organs and complications, and improve the cure rate of tumors. Respiratory movements can induce changes in tumor location in the chest and abdomen of up to 3 cm, increasing the uncertainty of tumor location and the likelihood of complications. Therefore, it is necessary to monitor the tumor location during radiation therapy.

[0003] Currently, tumor tracking is divided into direct tracking and indirect tracking. Direct tracking typically uses relevant medical imaging technologies, such as CT, MRI, and X-rays, to directly locate the tumor. However, this method suffers from high radiation levels and low temporal resolution, making continuous monitoring of the tumor's location throughout the radiotherapy process impossible. Indirect methods involve tracking respiratory motion, establishing a correlation model between respiratory and tumor motion, and then tracking respiratory motion intraoperatively based on this model to achieve real-time tumor tracking. Current methods include electromagnetic tracking, optical tracking, and visual tracking. Electromagnetic tracking places electromagnetic markers near the tumor, offering good motion correlation but carries invasive risks and has lower positioning accuracy. Optical surface tracking is divided into markerless (e.g., the C-RAD optical surface tracking system) and marked (e.g., the CyberKnife respiratory detection system). Markerless surface tracking has low temporal resolution and poor accuracy, while marked tracking has fewer feature points and is insufficient for representing respiratory motion on the body surface. In summary, current commercial surface tracking systems mainly suffer from high costs, invasive risks, and a trade-off between high temporal resolution and a large number of feature points.

[0004] Therefore, a surface respiratory motion tracking system with high temporal resolution, a large number of feature points, and no invasive risks is needed to quickly and robustly extract respiratory signals for indirect tumor motion tracking and improve the effectiveness of radiotherapy. Summary of the Invention

[0005] The purpose of this invention is to propose a body surface respiratory motion tracking system and method based on position-aware markers, for detecting body surface respiratory motion and extracting respiratory signals for tumor motion tracking during radiotherapy.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A body surface respiratory motion tracking system based on position-aware markers, the system being used for tumor motion tracking in radiotherapy, comprising visual position-aware markers, a binocular visual tracking system, and a respiratory feature extraction system; The visual position sensing marker is a flexible marker set on the surface of clothing, which is used to deform with human breathing movements; The binocular vision tracking system is used to detect visual position perception markers in order to obtain the three-dimensional position of visual feature points under different respiratory movements, thereby constructing a body surface feature point cloud. The respiratory feature extraction system is used to perform temporal analysis on the feature point cloud of the body surface and extract respiratory feature signals that characterize respiratory movements on the body surface.

[0007] Preferably, the visual position perception mark is formed by alternating black and white triangular blocks. The visual position perception mark is divided into a triangular region composed of n×n black and white triangular blocks. Each triangular region contains a number of dots, and the number m of dots in each triangular region satisfies: m < n². The dots are distributed at the center of the corresponding black and white triangular blocks, and the color of the dots is the opposite of the black and white color of the triangular block.

[0008] Preferably, the visual position perception marker uses the intersection of adjacent black and white triangular blocks as visual feature points, and a composite hexagon formed by ten triangular blocks as a self-identification unit. Each visual feature point and each self-identification unit has a unique identifier number; wherein, the uniqueness of each self-identification unit is determined by the distribution of dots in the ten triangular blocks inside.

[0009] Preferably, the visual position sensing marker is printed onto clothing using an ink printing process. The clothing is used to cover the patient's chest to upper abdomen area for surface tracking.

[0010] Preferably, the visual positioning system includes two industrial cameras of the same model, which together form a stereo vision system device and are fixed on a camera gimbal bracket.

[0011] A method for tracking respiratory motion on the body surface based on location-aware markers, wherein the method is implemented using any of the above-mentioned body surface respiratory motion tracking systems, and specifically includes the following steps: S1: Acquire binocular images of the body surface containing visual position-aware markers; S2: Preprocess the acquired images and obtain the region of interest image; S3: Extract and identify self-identifying units and visual feature points in the region of interest image; S4: Based on binocular vision, visual feature points with the same identification number on the body surface are matched and reconstructed in three dimensions (using triangulation method) to obtain the feature point cloud of the body surface. S5: Analyze the feature point cloud of the body surface in time series and extract respiratory feature signals that characterize respiratory movements on the body surface.

[0012] Preferably, the image preprocessing method includes variance-based region of interest extraction, image contrast enhancement, and median filtering to further shorten the detection time and improve the detection effect. The variance-based region of interest extraction is specifically as follows: Calculate the variance of pixel coordinates of visual feature points P var Compared with the average P ave The starting point of the region of interest in the image is set to P ave - K × P var The image of interest is 2× K × P var , where K is the coefficient of the variance between the image region and the pixel values ​​of the feature points, used to set the size of the region of interest based on the number of feature points and the size of the marker pattern.

[0013] Preferably, the step of extracting and identifying visual feature points with unique identifiers in the region of interest image is as follows: The preprocessed image is sequentially processed by performing intersection point extraction, intersection point thinning, and intersection point recognition. Cross-point extraction: Candidate cross-points are extracted using the Hessian matrix, and redundant candidate points within the same cross-point region are suppressed by using non-maximum values; Cross-point refinement: Candidate cross-points are located based on image gradient information and hyperplane model. Quadratic surface fitting is used to calculate the direction information of the two edges at the cross-point, and corner templates are generated according to the direction. The correlation score between the corner template and the local area of ​​the candidate cross-point is calculated to eliminate false cross-points. Intersection point recognition: The Delaunay triangulation algorithm is used to divide the false intersection points into triangular meshes; inverted color dot detection is performed on the triangular mesh area: the pixel value difference between the center and corner of each triangle is detected, and the presence of inverted color dots inside the triangle is determined by a preset difference threshold. The presence of dots is recorded as 1, and the absence of dots is recorded as 0; the pre-imported position-aware marker field matrix is ​​matched with the feature matrix obtained by inverted color dot detection, and the unique ID corresponding to each visual feature point is obtained by bit-by-bit comparison and pattern recognition algorithm, so as to achieve accurate recognition of position-aware markers.

[0014] Preferably, step S5 is as follows: A three-dimensional position information dataset of visual feature points located on the body surface at multiple moments is constructed in the spatiotemporal dimension. Principal component analysis is performed on the dataset to extract the main motion direction and displacement change of each visual feature point under respiratory motion. After extracting the displacement of each ID point by PCA, it can be found that the motion change trend of each point is basically consistent, and the motion displacement of each ID in its main motion direction is approximately proportional. Therefore, this body surface motion trend under respiratory motion is called respiratory feature. Based on this, a respiratory feature extraction method based on feature point amplitude and point cloud fusion is proposed. By fusing the displacement amplitude of multiple visual feature points in the main motion direction, respiratory features that characterize the overall body surface respiratory motion are obtained.

[0015] Preferably, the step of obtaining respiratory features characterizing overall body surface respiratory motion by fusing the displacement amplitudes of multiple visual feature points in the main motion direction specifically involves: Pre-calculate the standard deviation of displacement of multiple visual feature points within a preset time window (e.g., 100s); The respiratory feature signal characterizing the overall body surface respiratory movement is obtained by removing the current position of multiple visual feature points and averaging the values ​​after calculating the pre-calculated displacement standard deviation.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a surface respiratory motion tracking system with high temporal resolution, a large number of feature points, and no invasive risks. It can quickly and reliably extract respiratory signals for indirect tumor motion tracking, thereby improving the effectiveness of radiotherapy. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall structure of the hardware components of the body surface tracking system based on position-aware markers according to the present invention. Figure 2 This is a schematic diagram illustrating the principle of the body surface tracking system based on position-aware markers of the present invention. Figure 3 This is a schematic diagram of the visual position-aware markers in the body surface tracking system based on position-aware markers of the present invention; Figure 4 This is a schematic diagram of clothing with visual position-aware markers in the body surface tracking system based on position-aware markers of the present invention; Figure 5 This is a flowchart illustrating the visual feature recognition process for clothing according to the present invention. Figure 6 This is a flowchart of the present invention for identifying clothing features with visual position-aware markers; Figure 7 This is a schematic diagram showing the main motion direction and main motion amplitude of the feature point in this invention; Figure 8The respiratory curve is a respiratory curve that characterizes the respiratory movement of the entire body surface according to the present invention. Figure 9 This is a diagram illustrating the location-aware marker generation process of the present invention; Figure 10 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] The following is in conjunction with the appendix Figure 1-10 The technical solution of the present invention will be described in detail below.

[0019] To address the issue of surface tracking during radiotherapy, embodiments of this disclosure provide a method and system for surface respiratory motion tracking for precision radiotherapy.

[0020] like Figure 1 As shown in the embodiments of this disclosure, a surface respiratory motion tracking system based on position-aware markers is provided to provide surface motion data for radiotherapy and stable respiratory motion signals for tumor tracking. This visual tracking and positioning system includes: a control host 1, a binocular visual tracking system 2, clothing with visual position-aware markers 3, and visual position-aware markers 4. Specifically: the clothing with visual position-aware markers 3 is used to attach the visual markers to the surface of the chest and abdomen; the visual position-aware markers 4 are flexible markers for surface tracking; the binocular visual system 2 is used to acquire images with the markers; and the control host 1 is used to preprocess the images, perform feature recognition and matching, feature reconstruction, and draw respiratory curves.

[0021] The working principle of this visual surface tracking system is as follows: After camera calibration, the binocular visual tracking system acquires camera parameters. During the operation, the patient wears clothing with position-aware markers. The visual tracking system acquires images and performs image feature detection and matching. Through triangulation, the three-dimensional position of the features is recovered to obtain the surface feature point cloud. Based on the surface point cloud data at different times, the main motion direction of the same ID feature points in respiratory motion is extracted. The displacement of the ID in the main motion direction is calculated. All identified feature point displacements are fused to extract respiratory feature signals, which are then displayed on the control host's operating screen.

[0022] like Figure 3 As shown, the visual position perception mark is formed by alternating black and white triangular blocks. The visual position perception mark is divided into a triangular region composed of n×n black and white triangular blocks. Each triangular region contains a number of dots, and the number m of dots in each triangular region satisfies: m < n². The dots are distributed at the center of the corresponding black and white triangular blocks, and the color of the dots is the opposite of the black and white color of the triangular block they belong to.

[0023] The visual position perception marker uses the intersection of adjacent black and white triangular blocks as visual feature points and a composite hexagon formed by ten triangular blocks as a self-identifying unit. Each visual feature point and each self-identifying unit has a unique identifier number. The uniqueness of each self-identifying unit is determined by the distribution of dots in the ten internal triangular blocks. The designed marker has two significant characteristics: rotation invariance and no repetition in the grid.

[0024] The binocular vision tracking system 2 includes two identical monocular industrial cameras, which constitute a binocular industrial camera module. These two identical monocular industrial cameras are fixed to an external bracket. Specifically, in this embodiment, the two monocular industrial cameras are fixed to a camera bracket, and their absolute and relative positions remain unchanged.

[0025] The self-recognizing visual markers are auxiliary tracking tools of this visual tracking system. The binocular visual tracking system 2 acquires surface feature point clouds under respiratory movements by recognizing the visual position perception markers 4. Figure 5 As shown, the patient wore clothing with position-aware markers during the procedure. The process of acquiring the body surface point cloud consisted of camera grayscale image acquisition, image preprocessing, feature detection, and feature matching. After matching, the features were reconstructed in three dimensions to obtain... Figure 6 The surface feature point cloud shown.

[0026] After performing principal component analysis (PCA) on feature points at different times, the main motion method of the feature points can be obtained. Mapping the position of the feature points to the main motion axis yields the motion displacement, such as... Figure 7 As shown, the arrows indicate the main motion direction of feature points with different IDs, and the length of the arrows represents the standard deviation of the main motion displacement. Then, a breathing feature extraction method based on the fusion of feature point amplitude and point cloud is used to obtain the following... Figure 8 The respiratory signal curve shown.

[0027] The control host 1 is an industrial control computer with an operating screen. When the visual body surface tracking system provides respiratory signals for radiotherapy surgery, it displays the tracking results on the operating screen of the control host 1. In the preparation stage before radiotherapy, the control host 1 is used to complete camera calibration; during the radiotherapy stage, the control host 1 is used to complete image preprocessing, real-time detection and localization of visual position perception markers, reconstruction of body surface motion point clouds, and extraction of respiratory features, plotting the curve of respiratory features versus time on the operating screen of the control host 1.

[0028] Through the above methods, the body surface respiratory motion tracking system based on position-aware markers provided in this embodiment of the present disclosure achieves the goal of body surface motion tracking and meets the requirements of real-time performance. For the convenience of doctors' observation, the system displays the results on the operation screen of the control host.

[0029] This disclosure also provides a computer program comprising: computer-executable instructions, which, when executed, are used to implement a surface respiratory motion tracking system for precision radiotherapy according to an embodiment of this disclosure.

[0030] This disclosure has now been described in detail with reference to the accompanying drawings. Based on the above description, those skilled in the art should have a clear understanding of this disclosure.

[0031] It should be noted that implementations not shown or described in the accompanying drawings or the main text of the specification are all forms known to those skilled in the art and are not described in detail. Furthermore, the definitions of the elements above are not limited to the various specific structures, shapes, or methods mentioned in the embodiments, and those skilled in the art can easily modify or substitute them.

[0032] Of course, this disclosure may also include other parts as needed, which will not be elaborated here as they are not relevant to the innovation of this disclosure. Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the disclosure, in the above description of exemplary embodiments of this disclosure, various features of this disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the aspects of the disclosure consist of fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this disclosure.

[0033] Furthermore, similar or identical parts are referred to by the same reference numerals in the accompanying drawings or description. Technical features in the various embodiments exemplified in the specification can be freely combined to form new solutions without conflict. Additionally, each claim can stand alone as an embodiment, or the technical features in the various claims can be combined to form new embodiments. Moreover, elements or implementations not shown or described in the drawings are forms known to those skilled in the art. Furthermore, while this document provides examples of parameters containing specific values, it should be understood that the parameters need not be exactly equal to the corresponding values, but can approximate the corresponding values ​​within acceptable error tolerances or design constraints.

[0034] Unless there are technical obstacles or contradictions, the various embodiments described above in this disclosure can be freely combined to form other embodiments, all of which are within the protection scope of this disclosure.

[0035] Although this disclosure has been described in conjunction with the accompanying drawings, the embodiments disclosed in the drawings are intended to illustrate preferred embodiments of this disclosure and should not be construed as limiting the disclosure. The dimensions in the drawings are merely illustrative and should not be construed as limiting the disclosure.

[0036] While some embodiments of the general concept of this disclosure have been shown and described, those skilled in the art will understand that changes may be made to these embodiments without departing from the principles and spirit of the general concept of this disclosure, the scope of which is defined by the claims and their equivalents.

[0037] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this disclosure. It should be understood that the above descriptions are merely specific embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A body surface respiratory motion tracking system based on position-aware markers, characterized in that, The system is used for tumor motion tracking in radiotherapy, including visual position-aware markers, a binocular visual tracking system, and a respiratory feature extraction system; The visual position sensing marker is a flexible marker set on the surface of clothing, which is used to deform with human breathing movements; The binocular vision tracking system is used to detect visual position perception markers in order to obtain the three-dimensional position of visual feature points under different respiratory movements, thereby constructing a body surface feature point cloud. The respiratory feature extraction system is used to perform temporal analysis on the feature point cloud of the body surface and extract respiratory feature signals that characterize respiratory movements on the body surface.

2. The body surface respiratory motion tracking system based on position-aware markers according to claim 1, characterized in that, The visual position perception mark is formed by alternating black and white triangular blocks. The visual position perception mark is divided into a triangular region composed of n×n black and white triangular blocks. Each triangular region contains a number of dots, and the number m of dots in each triangular region satisfies: m < n². The dots are distributed at the center of the corresponding black and white triangular blocks, and the color of the dots is the opposite of the black and white color of the triangular block they belong to.

3. A body surface respiratory motion tracking system based on position-aware markers according to claim 2, characterized in that, The visual position perception marker uses the intersection of adjacent black and white triangular blocks as visual feature points and a composite hexagon formed by ten triangular blocks as a self-identification unit. Each visual feature point and each self-identification unit has a unique identifier number; the uniqueness of each self-identification unit is determined by the distribution of dots in the ten triangular blocks inside.

4. A body surface respiratory motion tracking system based on position-aware markers according to claim 2, characterized in that, The visual positional awareness markers are printed onto clothing using an ink printing process, and the clothing is used to cover the patient's chest to upper abdomen area.

5. A body surface respiratory motion tracking system based on position-aware markers according to claim 1, characterized in that, The visual positioning system includes two industrial cameras of the same model. The two industrial cameras form a stereo vision system device and are fixed on the camera gimbal bracket.

6. A method for tracking respiratory motion on the body surface based on position-aware markers, characterized in that, The method is implemented using the body surface respiratory motion tracking system as described in any one of claims 1 to 5, and specifically includes the following steps: S1: Acquire binocular images of the body surface containing visual position-aware markers; S2: Preprocess the acquired images and obtain the region of interest image; S3: Extract and identify self-identifying units and visual feature points in the region of interest image; S4: Based on binocular vision, visual feature points with the same identification number on the body surface are matched and reconstructed in three dimensions to obtain a feature point cloud of the body surface. S5: Analyze the feature point cloud of the body surface in time series and extract respiratory feature signals that characterize respiratory movements on the body surface.

7. The method for tracking body surface respiratory motion based on position-aware markers according to claim 6, characterized in that, Image preprocessing methods include variance-based region of interest extraction, image contrast enhancement, and median filtering. The variance-based region of interest extraction is detailed below: Calculate the variance of pixel coordinates of visual feature points P var Compared with the average P ave The starting point of the region of interest in the image is set to P ave - K × P var The image of interest is 2× K × P var , where K is the coefficient of the variance between the image region and the pixel values ​​of the feature points, used to set the size of the region of interest based on the number of feature points and the size of the marker pattern.

8. The method for tracking body surface respiratory motion based on position-aware markers according to claim 6, characterized in that, The extraction and identification of visual feature points with unique identifiers in the region of interest image is as follows: Candidate intersection points are extracted using the Hessian matrix, and redundant candidate points within the same intersection point region are suppressed by using non-maximum values. Candidate intersections are located based on image gradient information and hyperplane model. Quadratic surface fitting is used to calculate the direction information of the two edges at the intersection. Corner templates are generated based on the direction. The correlation score between the corner template and the local area of ​​the candidate intersection is calculated to eliminate false intersections. The Delaunay triangulation algorithm is used to divide the false intersections into triangular meshes. Inverted-color dot detection is then performed on the triangular mesh regions: the pixel value difference between the center and corners of each triangle is detected, and a preset difference threshold is used to determine whether inverted-color dots exist inside the triangles; the presence of a dot is recorded as 1, and the absence of a dot is recorded as 0. The pre-imported position-aware marker field matrix is ​​matched with the feature matrix obtained from the inverted-color dot detection. A unique ID corresponding to each visual feature point is obtained through bit-by-bit comparison and pattern recognition algorithms, achieving accurate identification of position-aware markers.

9. The method for tracking body surface respiratory motion based on position-aware markers according to claim 6, characterized in that, S5 is specifically as follows: A three-dimensional position information dataset of visual feature points located on the body surface at multiple times is constructed in the spatiotemporal dimension. Principal component analysis is performed on the dataset to extract the main motion direction and displacement change of each visual feature point under respiratory motion, i.e., the first principal component projection. By fusing the displacement amplitude of multiple visual feature points in the main motion direction, respiratory features that characterize the overall respiratory motion of the body surface are obtained.

10. The method for tracking body surface respiratory motion based on position-aware markers according to claim 9, characterized in that, The method of obtaining respiratory characteristics representing overall body surface respiratory motion by fusing the displacement amplitudes of multiple visual feature points in the main motion direction is as follows: Pre-calculate the standard deviation of displacement of multiple visual feature points within a preset time window; The respiratory feature signal characterizing the overall body surface respiratory movement is obtained by removing the current position of multiple visual feature points and averaging the values ​​after calculating the pre-calculated displacement standard deviation.

Images