Respiration monitoring method, gating control method, positioning method, system and equipment

Abstract

The invention provides a respiration monitoring method, a gating control method, a positioning method, a system and equipment, and belongs to the technical field of medical equipment. The method comprises the following steps: sending near-infrared random speckles to a measured target through a near-infrared speckle projection module; acquiring a plurality of body surface images of the measured target subjected to near-infrared random speckle projection in real time through a binocular image acquisition module; determining a body surface three-dimensional point cloud of the measured target according to the plurality of body surface images; and according to the body surface three-dimensional point cloud, determining a respiration curve of the measured target. The near-infrared random speckles are projected to the body surface of the detected target, so that non-contact measurement and sensing are realized, additional devices do not need to be added on the body of the detected target, and the comfort of the detected target in the whole detection process is improved. Near-infrared random speckles and binocular stereoscopic vision are fused, so that the reliability of the body surface three-dimensional point cloud of the measured target is enhanced, and an accurate respiratory gating signal is provided for medical imaging equipment or radiotherapy equipment.

Description

Technical Field

[0001] This application belongs to the field of medical device technology, and in particular relates to a respiratory monitoring method, gating control method, positioning method, system and device. Background Technology

[0002] When imaging patients using medical imaging equipment (such as computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET), the patient's breathing movements can cause the imaging target area (e.g., chest or abdomen) to shift, leading to decreased image quality and affecting the doctor's diagnosis. When treating tumors with radiation therapy (RT), the patient's breathing can cause movement of the tumor and other organs, resulting in low-dose radiation to the tumor target area or high-dose radiation to non-target areas. Respiratory gating technology refers to collecting the patient's respiratory signals through respiratory monitoring equipment, generating a respiratory signal curve, and triggering medical imaging equipment to acquire images or triggering radiation therapy equipment to irradiate the target area within a specific respiratory interval (e.g., end-expiration), thereby improving image quality or reducing radiation exposure to healthy tissues.

[0003] Traditional methods for acquiring respiratory signal curves require the addition of extra devices to the patient, such as abdominal pressure bands or markers, to obtain the patient's respiratory motion signals. However, adding these extra devices can cause discomfort to the patient. Summary of the Invention

[0004] The purpose of this application is to provide a respiratory monitoring method, gating control method, positioning method, system and device, which aims to solve the problem of patient discomfort caused by the need to add additional devices to the patient in traditional respiratory signal curve acquisition methods.

[0005] This application provides a respiratory monitoring method, including:

[0006] Near-infrared random speckle is sent to the target under test through a near-infrared speckle projection module;

[0007] Multiple surface images of the target under test are acquired in real time by the binocular image acquisition module after being projected by the near-infrared random speckle pattern.

[0008] Based on the multiple body surface images, determine the three-dimensional point cloud of the tested target's body surface;

[0009] The respiratory curve of the target under test is determined based on the three-dimensional point cloud of the body surface.

[0010] In one embodiment, the step of determining the three-dimensional point cloud of the target surface based on the plurality of surface images includes:

[0011] Using a human body recognition model, the chest and abdomen regions are extracted from each of the body surface images to obtain multiple chest and abdomen images;

[0012] Based on the multiple chest and abdominal images, the three-dimensional point cloud of the chest and abdomen of the target under test is determined.

[0013] In one embodiment, the step of determining the three-dimensional point cloud of the chest and abdomen of the target object based on the plurality of chest and abdominal images includes:

[0014] The multiple chest and abdominal images are subjected to epipolar correction and distortion correction to obtain multiple corrected chest and abdominal images.

[0015] Binocular stereo matching is performed on the plurality of corrected chest and abdomen images to obtain the image parallax of the plurality of corrected chest and abdomen images;

[0016] The calibration parameters of the binocular image acquisition module (120) are obtained, and the multiple corrected chest and abdomen images are triangulated according to the calibration parameters and the image parallax to obtain the three-dimensional point cloud of the chest and abdomen of the target under test.

[0017] In one embodiment, the calibration parameters are obtained by performing binocular calibration on the binocular image acquisition module (120).

[0018] In one embodiment, the step of determining the respiratory curve of the target based on the three-dimensional point cloud of the body surface includes:

[0019] Three-dimensional feature points are extracted from the three-dimensional point cloud of the chest and abdomen to obtain multiple respiratory feature points of the target under test;

[0020] The respiratory curve of the target under test is determined based on the multiple respiratory feature points.

[0021] This application provides a gating control method, including:

[0022] Near-infrared random speckle is sent to the target under test through a near-infrared speckle projection module;

[0023] Multiple surface images of the target under test are acquired in real time by the binocular image acquisition module after being projected by the near-infrared random speckle pattern.

[0024] Based on the multiple body surface images, determine the three-dimensional point cloud of the tested target's body surface;

[0025] Based on the three-dimensional point cloud of the body surface, the respiratory curve of the target under test is determined;

[0026] Based on the breathing curve, a trigger signal is sent to the medical device to trigger the medical device to acquire an image of the target under test or to emit a ray beam toward the target under test.

[0027] In one embodiment, the step of sending a trigger signal to the medical device based on the respiratory curve includes:

[0028] The respiratory cycle of the target under test is determined based on the peaks and troughs of the respiratory curve.

[0029] The trigger signal is sent to the medical device during the peak-to-valley and trough-to-valley phases of the respiratory cycle.

[0030] This application provides a placement method, including:

[0031] Near-infrared random speckle is sent to the target under test through a near-infrared speckle projection module;

[0032] Multiple surface images of the target under test are acquired in real time by the binocular image acquisition module after being projected by the near-infrared random speckle pattern.

[0033] Based on the multiple body surface images, determine the three-dimensional point cloud of the tested target's body surface;

[0034] The target under test is positioned based on the three-dimensional point cloud of the body surface and the historical three-dimensional point cloud of the body surface.

[0035] This application provides a respiratory gating system, including:

[0036] A respiratory gating device, including a near-infrared speckle projection module and a binocular image acquisition module;

[0037] The near-infrared speckle projection module is used to send near-infrared random speckle to the target being tested;

[0038] The binocular image acquisition module includes a first image acquisition module and a second image acquisition module spaced apart. The near-infrared speckle projection module is spaced between the first image acquisition module and the second image acquisition module. The first image acquisition module is used to acquire a first surface image of the target after being projected by the near-infrared random speckle projection in real time. The second image acquisition module is used to acquire a second surface image of the target after being projected by the near-infrared random speckle projection in real time.

[0039] A control module, connected to the near-infrared speckle projection module, is used to control the near-infrared speckle projection module to send near-infrared random speckle towards the target under test. The control module is also connected to the first image acquisition module and the second image acquisition module, respectively, and is used to control the first image acquisition module and the second image acquisition module to acquire, in real time, the first surface image and the second surface image of the target under test after being projected by the near-infrared random speckle projection. The control module is further used to determine the three-dimensional point cloud of the target's surface based on the first surface image and the second surface image, and to determine the respiratory curve of the target based on the three-dimensional point cloud, and to send a trigger signal to the medical device based on the respiratory curve.

[0040] This application provides a medical device including at least one respiratory gating system as described in the above embodiments.

[0041] In one embodiment, this application provides a medical device, including the imaging gantry of the respiratory gating system and imaging device described in the above embodiments;

[0042] The respiratory gating device of the respiratory gating system is located inside the aperture of the imaging gantry.

[0043] In one embodiment, this application provides a medical device including a plurality of respiratory gating systems and a radiotherapy head of a radiation device as described in the above embodiments;

[0044] The multiple respiratory gating devices of the multiple respiratory gating systems are respectively arranged around the radiotherapy head to acquire surface images of the target from multiple angles in order to determine the respiratory curve of the target.

[0045] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0046] The respiratory monitoring method provided in this application sends near-infrared random speckle patterns to the target through a near-infrared speckle projection module. These speckle patterns are projected onto the target's surface, enabling non-contact measurement and sensing. This eliminates the need for additional devices on the target, improving comfort throughout the detection process. The surface texture information of the target is enhanced after near-infrared random speckle projection, and multiple surface images based on binocular stereo vision are acquired in real time through a binocular image acquisition module. This enhances the reliability of the three-dimensional point cloud of the surface, providing more accurate depth and spatial information for determining the respiratory curve. By fusing near-infrared random speckle with binocular stereo vision, the respiratory monitoring method enhances the reliability of the three-dimensional point cloud of the target's surface, thereby improving the accuracy of the respiratory curve and providing more accurate respiratory gating signals for medical imaging or radiotherapy equipment. Furthermore, the near-infrared random speckle projected onto the target by this method is invisible to the human eye and will not affect the target or physician, further improving the target's comfort throughout the detection process. Attached Figure Description

[0047] Figure 1 A flowchart illustrating the steps of the respiratory monitoring method provided in this application;

[0048] Figure 2 A schematic diagram of binocular stereo 3D reconstruction of a polyhedron under both near-infrared random speckle projection and near-infrared random speckle projection conditions, provided in one embodiment of this application;

[0049] Figure 3 A flowchart illustrating the steps of the gating control method provided in this application;

[0050] Figure 4 A flowchart illustrating the steps of the placement method provided in this application;

[0051] Figure 5 This is a schematic diagram of the respiratory gating system provided in this application;

[0052] Figure 6 This is a schematic diagram of the imaging system provided in this application;

[0053] Figure 7 A schematic diagram of the radiation system provided in this application. Detailed Implementation

[0054] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0055] It should be noted that when a component is referred to as being "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as being "connected to" another component, it can be directly connected to or indirectly connected to that other component.

[0056] It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0057] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0058] Please see Figure 1 This application provides a respiratory monitoring method, comprising:

[0059] Step S10: Near-infrared random speckle is sent to the target under test through the near-infrared speckle projection module 110;

[0060] Step S20: The binocular image acquisition module 120 acquires multiple surface images of the target after near-infrared random speckle projection in real time.

[0061] Step S30: Determine the three-dimensional point cloud of the target surface based on multiple surface images;

[0062] Step S40: Determine the breathing curve of the target based on the three-dimensional point cloud of the body surface.

[0063] In this embodiment, the near-infrared speckle projection module 110 includes an infrared light source and optical elements (such as diffractive optical elements, diffusers, gratings, etc.) for scattering or diffracting light into speckles, capable of projecting near-infrared random speckles. The target being tested can be a patient. Near-infrared random speckles are randomly distributed light spot patterns. After the near-infrared random speckles are projected onto the surface of the target being tested, the scattering and reflection characteristics of light change due to the differences in the microstructure and texture of the surface, thereby forming a unique speckle pattern that can enhance the surface texture information of the target being tested.

[0064] The surface texture information of the target under test is enhanced after near-infrared random speckle projection. This enhanced surface texture information is then acquired in real-time by the binocular image acquisition module 120, resulting in multiple surface images. These multiple images contain the enhanced surface texture information, increasing the number of pixels (also called corresponding points) in the same physical space across different surface images. This allows for a more accurate representation of the target's surface texture features, which are then acquired by the binocular image acquisition module 120. Therefore, based on these multiple surface images, multiple corresponding points can be extracted quickly and accurately, significantly improving the success rate of binocular stereo matching and resulting in a more accurate 3D point cloud of the target's surface.

[0065] Multiple body surface images can characterize the respiratory movement information of the target, accurately reflecting its body surface contour. Furthermore, by analyzing and processing these images, more features about the target's body surface texture can be obtained, enhancing the perception and recognition of this texture. This leads to the acquisition of a three-dimensional point cloud of the target's body surface, allowing for more precise recording of surface changes caused by respiration. Therefore, based on the three-dimensional point cloud, the target's respiratory curve can be determined more accurately, enabling the monitoring of respiratory signals.

[0066] Therefore, the respiratory monitoring method provided in this application sends near-infrared random speckle patterns to the target through the near-infrared speckle projection module 110. These near-infrared random speckle patterns are projected onto the target's surface, achieving non-contact measurement and sensing. This eliminates the need for additional devices on the target, improving comfort throughout the detection process. The surface texture information of the target is enhanced after near-infrared random speckle projection, and multiple surface images based on binocular stereo vision are acquired in real-time by the binocular image acquisition module 120. This enhances the reliability of the three-dimensional point cloud of the surface, providing more accurate depth and spatial information for determining the respiratory curve. By fusing near-infrared random speckle with binocular stereo vision, the respiratory monitoring method provided in this application enhances the reliability of the three-dimensional point cloud of the target's surface, thereby improving the accuracy of the respiratory curve and providing more accurate respiratory gating signals for medical imaging or radiotherapy equipment. Meanwhile, the near-infrared random speckle projected by the respiratory monitoring method provided in this application onto the target is invisible to the human eye and will not affect the target or the physician, further improving the comfort of the target throughout the detection process.

[0067] In one embodiment, before step S20, which involves real-time acquisition of multiple surface images of the target after near-infrared random speckle projection by the binocular image acquisition module 120, the respiratory monitoring method further includes:

[0068] Step S11: Perform binocular calibration on the binocular image acquisition module 120 to obtain the calibration parameters of the binocular image acquisition module 120.

[0069] In this embodiment, the binocular image acquisition module 120 includes two cameras. The binocular image acquisition module 120 is calibrated using a calibration board to obtain the intrinsic and extrinsic parameters of the two cameras. The two cameras in the binocular image acquisition module 120 acquire images of the target from different angles. The calibration parameters include intrinsic and extrinsic parameters. Intrinsic parameters include focal length, principal point position, and distortion coefficients. Extrinsic parameters include the relative position and orientation of the two cameras.

[0070] In one embodiment, a calibration plate of known size is placed at different positions and angles, and multiple images of the calibration plate are captured by two cameras. A calibration algorithm is then used to analyze these images to calculate the internal and external parameters.

[0071] Therefore, by using the calibrated binocular image acquisition module 120, multiple surface images of the target object after near-infrared random speckle projection can be acquired in real time, resulting in left and right surface images. The left and right surface images have overlapping areas, which can be used for subsequent feature point matching.

[0072] In one embodiment, step S30, the step of determining the three-dimensional point cloud of the target surface based on multiple surface images, includes:

[0073] Step S310: Using a human body recognition model, the chest and abdomen regions are extracted from each body surface image to obtain multiple chest and abdomen images.

[0074] Step S320: Determine the three-dimensional point cloud of the chest and abdomen of the target object based on multiple chest and abdominal images.

[0075] In this embodiment, in step S310, the human body recognition model is obtained by training a human body recognition algorithm. The human body recognition algorithm can be the YOLO object detection algorithm, specifically YOLOv3, YOLOv4, YOLOv5, etc. A dataset containing human body annotations is constructed. The human body annotation dataset includes bounding box coordinates and category labels. The category labels include the categories of different parts of the human body, such as chest, abdomen, head, arms, etc. The human body recognition algorithm is trained using the human body annotation dataset to obtain a trained model, i.e., the human body recognition model in step S310. Furthermore, in step S310, the human body recognition model is used to locally extract the body surface image acquired by the binocular image acquisition module 120, extracting the chest and abdomen regions from the body surface image to obtain a chest and abdomen image. The chest and abdomen image can be understood as an image of the patient's chest and abdomen. When the patient breathes, the chest and abdomen will rise and fall.

[0076] In step S320, multiple chest and abdominal images extracted from multiple body surface images using the human body recognition model are used to reconstruct the three-dimensional point cloud of the target's chest and abdomen. Through steps S310 and S320 in this embodiment, the body surface images acquired by the binocular image acquisition module 120 are further processed to extract local regions, allowing for more precise localization of the desired areas, extraction of target data, and filtering out redundant data. Only the target data is processed in subsequent steps. Therefore, performing subsequent binocular stereo matching based on the target data formed from the chest and abdomen three-dimensional point cloud reduces the computational load of binocular stereo matching, improves data processing speed, and lowers hardware requirements.

[0077] In one embodiment, step S320, the step of determining the three-dimensional point cloud of the chest and abdomen of the target object based on multiple chest and abdominal images, includes:

[0078] Step S321: Perform epipolar correction and distortion correction on multiple chest and abdominal images to obtain multiple chest and abdominal surface images;

[0079] Step S322: Perform binocular stereo matching on multiple corrected chest and abdomen images to obtain the image parallax of multiple corrected chest and abdomen images;

[0080] Step S323: Based on the calibration parameters and image parallax, perform triangulation on multiple calibrated chest and abdomen images to obtain a three-dimensional point cloud of the chest and abdomen.

[0081] In this embodiment, step S321, adjusting the corresponding epipolar lines in multiple chest and abdominal images to the same horizontal line for epipolar correction simplifies the stereo matching process in subsequent steps, reduces computational complexity, and improves the efficiency of binocular stereo matching. Epipolar correction involves geometrically transforming multiple chest and abdominal images so that the epipolar lines of corresponding points in the multiple images lie on the same horizontal line, thereby reducing the search range of corresponding points from two dimensions to one dimension and reducing computational complexity. In one embodiment, the geometric relationship between multiple chest and abdominal images is determined using the fundamental matrix and homography matrix, and then corresponding transformations are performed to achieve epipolar correction of multiple chest and abdominal images.

[0082] Distortion correction processing of multiple chest and abdominal images can eliminate distortions caused by lens optical characteristics and other factors, resulting in more accurate images. In one embodiment, distortion correction is performed on multiple chest and abdominal images using a camera calibration method. The internal parameters and distortion coefficients of the binocular image acquisition module 120 are calculated using a calibration plate of known size. Based on the internal parameters and distortion coefficients, each pixel in the image is reverse-mapped to eliminate distortion.

[0083] In step S322, by performing binocular stereo matching on multiple calibrated chest and abdomen images, the positional relationship of corresponding points in the multiple calibrated chest and abdomen images is obtained, thereby obtaining the image disparity. In one embodiment, corresponding pixels are found by comparing the color, texture, gradient, and other features of pixels in multiple calibrated chest and abdomen images. Then, the disparity is calculated based on the corresponding pixels, which can also be understood as the difference in the horizontal position of corresponding points in the image. In one embodiment, binocular stereo matching is performed on multiple calibrated chest and abdomen images using a region-based matching algorithm or a feature-based matching algorithm.

[0084] In step S323, triangulation can be understood as calculating the position of the target in three-dimensional space based on the geometric relationship of binocular stereo vision, using the internal parameters, external parameters, and image disparity of the binocular image acquisition module 120. The depth of each pixel is calculated using the formula Z = bf / d, where b is the baseline distance between the two cameras, f is the focal length, and d is the image disparity. The image coordinates of the pixels are converted to camera coordinates based on the intrinsic parameter matrix of the binocular image acquisition module 120. The camera coordinates are then converted to world coordinates based on the extrinsic parameter matrix of the binocular image acquisition module 120, thereby obtaining the three-dimensional point cloud of the chest and abdomen of the target.

[0085] In one embodiment, after step S321 and before step S322, step S320, the step of determining the three-dimensional point cloud of the chest and abdomen of the target object based on multiple chest and abdominal images, further includes:

[0086] Step S3211: Extract and match feature points from multiple corrected chest and abdominal images to obtain corresponding feature point pairs in the multiple corrected chest and abdominal images.

[0087] In this embodiment, feature point detection is performed on multiple calibrated chest and abdomen images to extract image feature points with uniqueness and stability. In one embodiment, feature point extraction is performed using the Scale-Invariant Feature Transform (SIFT) algorithm and the Speeded Up Robust Features (SURF) algorithm. The extracted feature points in the multiple calibrated chest and abdomen images are compared, and matching is performed using information such as the color, texture, and shape of the feature points to obtain corresponding feature point pairs in the multiple calibrated chest and abdomen images. Further, using the corresponding feature point pairs obtained in step S311, stereo matching is performed to obtain the image disparity of the multiple calibrated chest and abdomen images.

[0088] Therefore, in step S320, the multiple chest and abdominal images contain enhanced surface texture information. The multiple corrected chest and abdominal images obtained after epipolar correction and distortion correction also contain enhanced surface texture information, increasing the number of pixels (also called corresponding points) in the same physical space in different corrected chest and abdominal images, thus more accurately reflecting the surface texture features of the chest and abdomen of the target. Furthermore, when extracting and matching feature points from multiple corrected chest and abdominal images, multiple pixels (also called corresponding points) in the same physical space can be extracted more quickly and accurately, obtaining feature point pairs and achieving binocular stereo matching. Thus, the respiratory monitoring method provided in this application fuses near-infrared random speckle with binocular stereo vision, enhancing the reliability of the three-dimensional point cloud of the chest and abdomen of the target, improving the matching rate of binocular stereo matching, and making the three-dimensional point cloud of the chest and abdomen of the target more accurate.

[0089] Please see Figure 2 In one embodiment, Figure 2 (a) represents a schematic diagram of an image without near-infrared random speckle projection. Figure 2 Image (b) shows a schematic diagram of the image obtained after binocular stereo 3D reconstruction of a polyhedron without near-infrared random speckle projection. It can be seen that a complete polyhedron cannot be reconstructed, especially against a pure white background, where it is completely impossible to reconstruct.

[0090] Figure 2 (c) represents a schematic diagram of an image with near-infrared random speckle projection. Figure 2 Image (d) represents a schematic diagram of the image obtained after binocular stereo 3D reconstruction of a polyhedron under near-infrared random speckle projection. This demonstrates that by projecting near-infrared random speckle onto the polyhedron, texture features are added, enabling more accurate reconstruction of the polyhedron, and both the background wall and the foreground cabinet can be completely reconstructed.

[0091] In one embodiment, step S40, determining the respiratory curve of the target based on the three-dimensional point cloud of the body surface, includes:

[0092] Step S410: Extract three-dimensional feature points from the three-dimensional point cloud of the chest and abdomen to obtain multiple respiratory feature points of the target under test.

[0093] Step S420: Determine the breathing curve of the target based on multiple breathing feature points.

[0094] In this embodiment, the three-dimensional point cloud of the chest and abdomen obtained in step S323 contains a large number of data points. By extracting three-dimensional feature points from the chest and abdomen three-dimensional point cloud and removing noise and useless points, representative key feature points can be extracted from the original chest and abdomen three-dimensional point cloud. In one embodiment, considering the different undulation characteristics of the chest and abdomen surface during breathing of different test targets, 30 to 50 feature points on the chest and abdomen surface (which can also be understood as feature points with a relatively large range of surface changes during breathing of the test target) can be selected as the extracted three-dimensional feature points, that is, the multiple breathing feature points of the test target in step S410. The mean value of the coordinates of the multiple extracted breathing feature points on the Z-axis (which can also be understood as the direction perpendicular to the examination bed) is calculated as the exhalation curve to generate the breathing curve of the test target.

[0095] By extracting 3D feature points from the 3D point cloud of the chest and abdomen in this embodiment, useful feature points can be extracted to establish a respiratory curve, while other useless data is removed, which reduces the amount of computation and data processing time, thus improving computational efficiency. Furthermore, after extracting 3D feature points from the 3D point cloud of the chest and abdomen, the reduced data volume makes the point cloud data easier to transfer and store between different devices, saving storage space and reducing storage costs.

[0096] In one embodiment, the method for extracting three-dimensional feature points from the three-dimensional point cloud of the chest and abdomen includes the Histogram of Orientations (SHOT), the Point Feature Histogram (PFH), the Fast Point Feature Histogram (FPFH), and the Viewpoint Feature Histogram (VFH).

[0097] Please see Figure 3 This application provides a gating control method, comprising:

[0098] Step S10: Near-infrared random speckle is sent to the target under test through the near-infrared speckle projection module 110;

[0099] Step S20: The binocular image acquisition module 120 acquires multiple surface images of the target after near-infrared random speckle projection in real time.

[0100] Step S30: Determine the three-dimensional point cloud of the target surface based on multiple surface images;

[0101] Step S40: Determine the respiratory curve of the target based on the three-dimensional point cloud of the body surface;

[0102] Step S50: Based on the breathing curve, a trigger signal is sent to the medical device to trigger the medical device to acquire an image of the target under test or to emit a ray beam to the target under test.

[0103] In this embodiment, the descriptions of steps S10 to S40 can be found in the descriptions of the above embodiments. In step S50, based on the respiratory curve of the target being tested, a trigger signal can be sent to the medical device at a specific respiratory interval (e.g., end-expiration) of the target being tested to control the medical device to acquire images of the chest and abdomen of the target being tested. At the specific respiratory interval of the target being tested (e.g., end-expiration), the lung inflation volume is at its maximum, and the lesion is displayed most clearly. This can reduce the impact of respiratory motion on image quality, improve image clarity and accuracy, thereby effectively reducing respiratory motion artifacts and improving the detection rate of lesions.

[0104] In step S50, based on the respiratory curve of the target, the irradiation position of the X-ray can be adjusted in real time to ensure that the X-ray is always aimed at the tumor site, reducing the radiation dose to surrounding normal tissues and effectively minimizing the impact of respiratory motion on treatment, thus improving the accuracy and safety of treatment. Therefore, the gating control method provided in this embodiment fuses near-infrared random speckle with binocular stereo vision, enhancing the reliability of the three-dimensional point cloud of the target's surface and obtaining a highly accurate respiratory curve. Based on this highly accurate respiratory curve, personalized treatment can be performed according to the respiratory characteristics of different targets, reducing the impact of respiratory motion on imaging examinations and radiotherapy, and improving image clarity and accuracy.

[0105] In one embodiment, step S50, the step of sending a trigger signal to the medical device based on the respiratory curve, includes:

[0106] Step S510: Determine the respiratory cycle of the target based on the peaks and troughs of the respiratory curve.

[0107] Step S520: During the peak-to-valley and trough-to-valley phases of the respiratory cycle, a trigger signal is sent to the medical device.

[0108] In this embodiment, the range of the peak-to-valley period can be from 30% of the peak value to the peak value. The range of the trough-to-valley period can be from 30% of the trough value to the trough value. Within the peak-to-valley and trough-to-valley periods of the respiratory cycle, the target device reaches a relatively stable range after exhaling as much gas as possible during respiration. Sending a trigger signal to the medical device within the peak-to-valley and trough-to-valley periods of the respiratory cycle can better reduce the impact of respiratory motion on imaging examinations and radiotherapy.

[0109] Please see Figure 4 This application provides a placement method, including:

[0110] Step S10: Near-infrared random speckle is sent to the target under test through the near-infrared speckle projection module 110;

[0111] Step S20: The binocular image acquisition module 120 acquires multiple surface images of the target after near-infrared random speckle projection in real time.

[0112] Step S30: Determine the three-dimensional point cloud of the target surface based on multiple surface images;

[0113] Step S40': Position the target under test based on the three-dimensional point cloud of the body surface and the historical three-dimensional point cloud of the body surface.

[0114] In this embodiment, the descriptions of steps S10 to S30 can be found in the descriptions of the above embodiments. In step S40', during a medical examination or treatment, each three-dimensional point cloud of the target's body surface corresponds to a specific position and posture. By comparing the current three-dimensional point cloud with historical three-dimensional point clouds, the position and posture of the target are adjusted so that the current three-dimensional point cloud coincides with the historical three-dimensional point cloud, thereby aligning the target's position and posture with the historical position and posture, thus achieving proper positioning of the target. Therefore, during a medical examination or treatment, the target's body is placed in the required position and posture, ensuring that the area to be examined is accurately within the field of view of the imaging device, obtaining clear and accurate images to meet the needs of the examination or treatment, and ensuring accuracy, safety, and effectiveness.

[0115] Please see Figure 5 This application provides a respiratory gating system. The respiratory gating system includes a respiratory gating device 10 and a control module 20. The respiratory gating device 10 includes a near-infrared speckle projection module 110 and a binocular image acquisition module 120. The near-infrared speckle projection module 110 is used to send near-infrared random speckle to the target being tested. The binocular image acquisition module 120 includes a first image acquisition module 121 and a second image acquisition module 122 spaced apart. The near-infrared speckle projection module 110 is spaced between the first image acquisition module 121 and the second image acquisition module 122. The first image acquisition module 121 is used to acquire a first surface image of the target being tested after near-infrared random speckle projection in real time. The second image acquisition module 122 is used to acquire a second surface image of the target being tested after near-infrared random speckle projection in real time.

[0116] The control module 20 is connected to the near-infrared speckle projection module 110. The control module 20 controls the near-infrared speckle projection module 110 to send near-infrared random speckle projections to the target. The control module 20 is also connected to the first image acquisition module 121 and the second image acquisition module 122, respectively, and controls the first image acquisition module 121 and the second image acquisition module 122 to acquire, in real time, the first and second surface images of the target after near-infrared random speckle projection. The control module 20 is also used to determine the three-dimensional point cloud of the target's surface based on the first and second surface images. The control module 20 is also used to determine the respiratory curve of the target based on the three-dimensional point cloud. The control module 20 is also used to send a trigger signal to the medical device based on the respiratory curve.

[0117] In this embodiment, the respiratory gating device 10 includes a near-infrared speckle projection module 110, a first image acquisition module 121, and a second image acquisition module 122. The descriptions of the near-infrared speckle projection module 110 and the binocular image acquisition module 120 can be found in the descriptions in the above embodiments. The method steps executed by the control module 20 can also be found in the descriptions in the above embodiments. The near-infrared speckle projection module 110 projects near-infrared random speckles, invisible to the human eye, onto the surface of the target. The first image acquisition module 121 and the second image acquisition module 122 have overlapping fields of view, acquiring first and second surface images, and transmitting them to the control module 20. The control module 20 processes the first and second surface images to obtain a three-dimensional point cloud of the target's surface, thereby obtaining the target's respiratory signal. In one embodiment, the control module 20 can be a computer, including a high-performance graphics card, for rapid data processing.

[0118] The respiratory gating system provided in this application fuses near-infrared random speckle with binocular stereo vision, enhancing the reliability of the three-dimensional point cloud on the surface of the target, thereby improving the accuracy of the respiratory curve, controlling image acquisition or radiation emission, improving image quality, and reducing the side effects of radiotherapy.

[0119] This application provides a medical device including at least one respiratory gating system as described in the above embodiments.

[0120] Please see Figure 6 This application provides a medical device. The medical device can be an imaging system. The imaging system includes an imaging gantry 310 comprising the respiratory gating system and the imaging device described in the above embodiments. The respiratory gating device 10 of the respiratory gating system is disposed inside the aperture of the imaging gantry 310.

[0121] In this embodiment, the imaging device can be CT, MRI, PET, etc. The imaging device includes an imaging gantry 310. The imaging gantry 310 encloses and forms a detection space. The target being measured is a patient 320, located on an examination bed 330, and can move within the detection space enclosed by the imaging gantry 310. A respiratory gating device 10 is disposed inside the aperture of the imaging gantry 310, located within the detection space. The imaging field of view of the respiratory gating device 10 can completely cover the chest and abdomen of the patient 320 on the examination bed 330. Furthermore, based on the respiratory curve obtained by the respiratory gating system, a trigger signal is sent to the imaging device to trigger the imaging device to acquire an image of the target being measured, which can reduce image artifacts and improve the image quality acquired by the imaging device.

[0122] Please see Figure 7 This application provides a medical device. The medical device can be a radiation system. The radiation system includes multiple respiratory gating systems as described in the above embodiments and a radiation therapy head 410 of the radiation device. Multiple respiratory gating devices 10 of the multiple respiratory gating systems are respectively disposed around the radiation therapy head 410, for acquiring surface images of the target from multiple angles to determine the respiratory curve of the target.

[0123] In this embodiment, the radiation device can be a radiation therapy (RT) device. The radiation device includes a radiation therapy head 410. Multiple respiratory gating devices 10 are arranged around the radiation therapy head 410 to image the target from different angles, avoiding obstruction by the radiation therapy head 410. In one embodiment, the multiple respiratory gating devices 10 can be suspended from the ceiling of the radiotherapy room where the radiation device is located, respectively positioned on the left, right, and top sides of the radiation therapy head 410, ensuring that the imaging field of view of the multiple respiratory gating devices 10 can cover the chest and abdomen of the target regardless of the position of the radiation therapy head 410. Furthermore, based on the respiratory curves obtained by the multiple respiratory gating systems, a trigger signal is sent to the radiation device to trigger the radiation device to emit a radiation beam towards the target, adjusting the irradiation position of the radiation in real time so that the radiation is always aimed at the tumor site, reducing the radiation dose to surrounding normal tissues, and effectively reducing the impact of respiratory movements on treatment, thereby improving the accuracy and safety of treatment.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0130] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0131] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A respiratory monitoring method, characterized in that, include: Near-infrared random speckle is sent to the target under test through the near-infrared speckle projection module (110); Multiple surface images of the target under test are acquired in real time by the binocular image acquisition module (120) after being projected by the near-infrared random speckle pattern; Based on the multiple body surface images, determine the three-dimensional point cloud of the tested target's body surface; The respiratory curve of the target under test is determined based on the three-dimensional point cloud of the body surface.

2. The respiratory monitoring method as described in claim 1, characterized in that, The step of determining the three-dimensional point cloud of the target surface based on the plurality of surface images includes: Using a human body recognition model, the chest and abdomen regions are extracted from each of the body surface images to obtain multiple chest and abdomen images; Based on the multiple chest and abdominal images, the three-dimensional point cloud of the chest and abdomen of the target under test is determined.

3. The respiratory monitoring method as described in claim 2, characterized in that, The step of determining the three-dimensional point cloud of the chest and abdomen of the target object based on the multiple chest and abdominal images includes: The multiple chest and abdominal images are subjected to epipolar correction and distortion correction to obtain multiple corrected chest and abdominal images. Binocular stereo matching is performed on the plurality of corrected chest and abdomen images to obtain the image parallax of the plurality of corrected chest and abdomen images; The calibration parameters of the binocular image acquisition module (120) are obtained, and the multiple corrected chest and abdomen images are triangulated according to the calibration parameters and the image parallax to obtain the three-dimensional point cloud of the chest and abdomen.

4. The respiratory monitoring method as described in claim 3, characterized in that, The calibration parameters are obtained by performing binocular calibration on the binocular image acquisition module (120).

5. The respiratory monitoring method as described in claim 3, characterized in that, The step of determining the respiratory curve of the target based on the three-dimensional point cloud of the body surface includes: Three-dimensional feature points are extracted from the three-dimensional point cloud of the chest and abdomen to obtain multiple respiratory feature points of the target under test; The respiratory curve of the target under test is determined based on the multiple respiratory feature points.

6. A gating control method, characterized in that, include: Near-infrared random speckle is sent to the target under test through the near-infrared speckle projection module (110); Multiple surface images of the target under test are acquired in real time by the binocular image acquisition module (120) after being projected by the near-infrared random speckle pattern; Based on the multiple body surface images, determine the three-dimensional point cloud of the tested target's body surface; Based on the three-dimensional point cloud of the body surface, the respiratory curve of the target under test is determined; Based on the breathing curve, a trigger signal is sent to the medical device to trigger the medical device to acquire an image of the target under test or to emit a ray beam toward the target under test.

7. The gating control method as described in claim 6, characterized in that, The step of sending a trigger signal to the medical device based on the respiratory curve includes: The respiratory cycle of the target under test is determined based on the peaks and troughs of the respiratory curve. The trigger signal is sent to the medical device during the peak-to-valley and trough-to-valley phases of the respiratory cycle.

8. A placement method, characterized in that, include: Near-infrared random speckle is sent to the target under test through the near-infrared speckle projection module (110); Multiple surface images of the target under test are acquired in real time by the binocular image acquisition module (120) after being projected by the near-infrared random speckle pattern; Based on the multiple body surface images, determine the three-dimensional point cloud of the tested target's body surface; The target under test is positioned based on the three-dimensional point cloud of the body surface and the historical three-dimensional point cloud of the body surface.

9. A respiratory gating system, characterized in that, include: The respiratory gating device (10) includes a near-infrared speckle projection module (110) and a binocular image acquisition module (120); The near-infrared speckle projection module (110) is used to send near-infrared random speckle to the target under test; The binocular image acquisition module (120) includes a first image acquisition module (121) and a second image acquisition module (122) spaced apart. The near-infrared speckle projection module (110) is spaced between the first image acquisition module (121) and the second image acquisition module (122). The first image acquisition module (121) is used to acquire a first surface image of the target after being projected by the near-infrared random speckle projection in real time. The second image acquisition module (122) is used to acquire a second surface image of the target after being projected by the near-infrared random speckle projection in real time. The control module (20) is connected to the near-infrared speckle projection module (110) and is used to control the near-infrared speckle projection module (110) to send near-infrared random speckle to the target under test. The control module (20) is connected to the first image acquisition module (121) and the second image acquisition module (122) respectively and is used to control the first image acquisition module (121) and the second image acquisition module (122) to acquire the first body surface image and the second body surface image of the target under test after being projected by the near-infrared random speckle in real time. The control module (20) is also used to determine the three-dimensional point cloud of the body surface of the target under test according to the first body surface image and the second body surface image, and to determine the breathing curve of the target under test according to the three-dimensional point cloud of the body surface, and to send a trigger signal to the medical device according to the breathing curve.

10. A medical device, characterized in that, Includes at least one respiratory gating system as described in claim 9.

Images