Radiation therapy motion monitoring method and system based on a portable electronic device

Abstract

The application discloses a radiotherapy motion monitoring method and system based on a portable electronic device, relates to the technical field of medical treatment, and is applied to a motion monitoring application program.The method comprises the following steps: receiving body surface depth data collected by a depth sensor of the portable electronic device for a target object; in response to a reference surface collection instruction initiated by a user at a target time, storing a three-dimensional space point cloud corresponding to the body surface depth data collected by the portable electronic device at the target time as a reference surface; during radiotherapy, continuously receiving body surface depth data collected by the portable electronic device, and storing the collected body surface depth data as a motion surface; and determining body position change information of the target object during radiotherapy according to the difference between the motion surface and the reference surface.The application solves the technical problem that the existing surface-guided radiotherapy motion monitoring system has high hardware costs due to the dependence on special optical equipment.

Description

Technical Field

[0001] This application relates to the field of medical technology, and more specifically, to a method and system for monitoring motion in radiotherapy based on a portable electronic device. Background Technology

[0002] In radiotherapy, surface-guided radiation therapy (SGRT) improves positioning accuracy and intra-fractional motion monitoring by continuously monitoring the target's position on the body surface.

[0003] In related technologies, surface-guided radiotherapy mainly relies on specialized optical equipment such as laser scanning systems, stereoscopic vision systems, time-of-flight technology, or structured light imaging technology, including dedicated fixed depth cameras. The hardware costs of these specialized optical devices are high, and the subsequent maintenance and upgrade costs are also significant. Furthermore, these systems are complex, requiring precise installation and calibration, as well as rigorous quality assurance procedures, demanding high levels of user expertise and a long learning curve.

[0004] The high cost and complex system architecture of surface-guided radiotherapy technology limit its widespread application in small and medium-sized hospitals and resource-scarce areas. Therefore, how to implement surface-guided radiotherapy using consumer-grade electronic devices has become a pressing technical problem to be solved. Summary of the Invention

[0005] This application provides a method and system for motion monitoring in radiotherapy based on a portable electronic device, which at least solves the technical problem of high hardware costs in existing surface-guided radiotherapy motion monitoring systems due to their reliance on dedicated optical equipment.

[0006] According to one aspect of the embodiments of this application, a method for motion monitoring in radiotherapy based on a portable electronic device is provided, applied to a motion monitoring application installed on a portable electronic device including a depth sensor, such as a smartphone, tablet, or smart wearable device. The method includes: receiving surface depth data collected by the portable electronic device from a target object via the depth sensor; and, in response to a reference surface acquisition command initiated by a user at a target time, storing the three-dimensional spatial point cloud corresponding to the surface depth data collected by the portable electronic device at the target time as a reference surface, including: displaying the surface depth data collected by the portable electronic device on a display interface of the portable electronic device; determining a sub-region of the surface defined by the region of interest (ROI) in the display interface in response to a user's adjustment operation on the ROI frame; storing the three-dimensional spatial point cloud within the ROI frame as the reference surface; continuously receiving surface depth data collected by the portable electronic device during radiotherapy and storing the collected surface depth data as a motion surface; and determining the positional change information of the target object during radiotherapy based on the difference between the motion surface and the reference surface.

[0007] According to another aspect of the embodiments of this application, a radiotherapy motion monitoring system based on a portable electronic device is also provided, comprising: a portable electronic device having a depth sensor, the portable electronic device including a smartphone, tablet computer or smart wearable device; and a motion monitoring application installed and running on the portable electronic device and configured to perform the above-described radiotherapy motion monitoring method based on a portable electronic device.

[0008] According to another aspect of the embodiments of this application, a radiotherapy motion monitoring device based on a portable electronic device is also provided. The device includes: a data receiving unit for receiving surface depth data collected by the portable electronic device via a depth sensor targeting a target object, the portable electronic device including a smartphone, tablet, or smart wearable device; a reference data processing unit for, in response to a user's reference surface acquisition command initiated at a target time, storing the three-dimensional spatial point cloud corresponding to the surface depth data collected by the portable electronic device at the target time as a reference surface, including: displaying the surface depth data collected by the portable electronic device on the display interface; determining a sub-region of the surface defined by the region of interest in the display interface in response to the user's adjustment operation on the region of interest box; and storing the three-dimensional spatial point cloud within the region of interest box as the reference surface; a motion data processing unit for continuously receiving surface depth data collected by the portable electronic device during radiotherapy and storing the collected surface depth data as a motion surface; and a position information determination unit for determining the positional change information of the target object during radiotherapy based on the difference between the motion surface and the reference surface.

[0009] In this embodiment, the motion monitoring application is installed on a portable electronic device (e.g., a smartphone) containing a depth sensor. The radiotherapy motion monitoring method based on this portable electronic device can directly reuse the built-in depth sensor to collect body surface depth data, eliminating the need to purchase dedicated optical hardware such as laser scanning systems, stereo vision systems, time-of-flight sensors, or structured light imaging equipment. Furthermore, by responding to a user's command to acquire a reference surface at a target time, the body surface depth data at that target time is stored as a reference surface. This facilitates the dynamic definition of the reference surface, eliminating the need for complex spatial calibration and fixed installation positions using dedicated optical equipment. Subsequently, by continuously receiving and storing body surface depth data as a motion surface during radiotherapy, and determining positional change information based on the difference between the motion surface and the reference surface, a closed-loop real-time monitoring and comparison process can be established. After acquiring body surface data using the depth sensor built into a portable electronic device, motion monitoring can be completed through calculations within the application, such as differential analysis between the reference surface and the moving surface. The entire monitoring process can be completed without relying on the high-precision and high-stability hardware configuration unique to dedicated optical equipment. Instead, it relies on the logical judgment and data processing in this method to compensate for the insufficient sensor accuracy of consumer electronic devices (such as smartphones), reducing the dependence on dedicated hardware. This solves the technical problem of high hardware costs caused by the reliance on dedicated optical equipment in existing surface-guided radiotherapy motion monitoring systems. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0011] Figure 1 This is a schematic diagram of an optional method for monitoring motion in radiotherapy based on a portable electronic device, according to an embodiment of this application.

[0012] Figure 2 This is a flowchart of an optional method for monitoring motion in radiotherapy based on a portable electronic device, according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of an optional motion monitoring device for radiotherapy based on a portable electronic device, according to an embodiment of this application. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0017] According to the embodiments of this application, a motion monitoring application can be used as the execution subject of the radiotherapy motion monitoring method based on a portable electronic device in the embodiments of this application. The system can be a software system or an embedded system combining software and hardware. Of course, the execution subject of the method in the embodiments of this application can also be other forms of execution subject, such as devices, equipment, etc. It should be known by those skilled in the art that this application does not particularly limit the specific form of the execution subject.

[0018] According to an embodiment of this application, a method embodiment for motion monitoring in radiotherapy based on a portable electronic device is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0019] Figure 1 This is a schematic diagram of a radiotherapy motion monitoring method based on a portable electronic device according to an embodiment of this application, as shown below. Figure 1 As shown, the method, applied to a motion monitoring application installed on a portable electronic device containing a depth sensor, includes the following steps:

[0020] Step S101: Receive surface depth data of the target object collected by a portable electronic device through a depth sensor.

[0021] For example, surface depth data can refer to a pixelated distance image formed by a depth sensor measuring the distance between a target object and the depth sensor. The grayscale value or numerical value of each pixel can represent the distance from the corresponding measurement point to the depth sensor. The target object can be, but is not limited to, a patient or a bionic model; the following explanation uses a patient as an example. The depth sensor can include a light detection and ranging sensor, a time-of-flight sensor, or a structured light sensor. The motion monitoring application receives surface depth data collected by the depth sensor from a portable electronic device, enabling the application to acquire three-dimensional spatial information of the target object's surface, providing raw data input for subsequent establishment of a reference surface and real-time motion monitoring. By receiving surface depth data, the motion monitoring application can autonomously obtain surface geometry information without relying on external dedicated equipment, which helps reduce system hardware costs and simplify the deployment process.

[0022] In some embodiments, the motion monitoring application continuously receives each frame of depth map data output by the depth sensor in a streaming manner by calling the native depth sensor application programming interface of the portable electronic device's operating system. The depth sensor can collect surface depth data at a preset frame rate, and the motion monitoring application triggers a data processing flow once each frame of depth map data is received by registering a callback function. This embodiment helps to ensure the real-time performance of surface depth data and can meet the low-latency requirements for motion monitoring during radiotherapy.

[0023] In other embodiments, the motion monitoring application requests user access to the camera and depth sensor before receiving surface depth data. Once authorized, the application activates the depth sensor's preview mode, displaying a depth map or color-coded distance image in real-time on the portable electronic device's display. The user can adjust the depth sensor's exposure parameters or depth range through the display to optimize data quality for different skin tones or surface features. Receiving parameter-optimized surface depth data improves the signal-to-noise ratio and stability of the depth data, thereby enhancing the accuracy of subsequent surface registration and motion monitoring.

[0024] Step S102: In response to the reference surface acquisition command initiated by the user at the target time, the three-dimensional spatial point cloud corresponding to the body surface depth data acquired by the portable electronic device at the target time is stored as the reference surface.

[0025] For example, the reference surface acquisition command can refer to the operation signal triggered by the user through the user interface of the motion monitoring application to start acquiring a reference surface. The target time can refer to the specific time point corresponding to when the user triggers the reference surface acquisition command. The three-dimensional spatial point cloud can refer to the set of multiple spatial points with three-dimensional coordinates obtained after coordinate transformation of the body surface depth data acquired by the depth sensor of the portable electronic device. In response to the reference surface acquisition command initiated by the user at the target time, the motion monitoring application stores the three-dimensional spatial point cloud corresponding to the body surface depth data acquired by the portable electronic device at the target time as the reference surface, so that the application can use the body surface model of the target object in an ideal positioning state or a breath-holding state as a reference for subsequent motion monitoring. By storing the reference surface, the motion monitoring application can provide a fixed spatial benchmark for subsequent real-time comparative analysis of the motion surface and the reference surface, which is beneficial to improving the accuracy and consistency of body position change detection.

[0026] In some embodiments, after receiving a reference surface acquisition command, the motion monitoring application can first preprocess the surface depth data output by the depth sensor at the current moment, including removing outliers, smoothing noise, and filling holes. The motion monitoring application converts the preprocessed depth data into a point cloud in a unified three-dimensional spatial coordinate system and saves the point cloud as a reference surface file in the local memory of the portable electronic device. After the motion monitoring application completes the storage, it displays a prompt message on the user interface indicating that the reference surface is ready. This embodiment helps to ensure the quality and integrity of the reference surface data.

[0027] In other embodiments, before receiving a reference surface acquisition command, the motion monitoring application allows the user to adjust the region of interest (ROI) bounding box in depth map mode, selecting a specific sub-region of the body surface for tracking. When the user initiates the reference surface acquisition command, the motion monitoring application only stores the 3D point cloud within the sub-region of the ROI bounding box as the reference surface, ignoring areas outside the bounding box. After storing the reference surface, the motion monitoring application renders it as a translucent red surface on the real-time depth image. This embodiment helps reduce interference from irrelevant areas such as the bed surface or surrounding equipment, allowing focus on key clinically important body surface areas such as the chest or abdomen, thereby improving the targeting and computational efficiency of motion monitoring.

[0028] Step S103: During radiotherapy, the body surface depth data collected by a portable electronic device is continuously received, and the collected body surface depth data is stored as a moving surface.

[0029] For example, the radiotherapy process can refer to the entire time period from the completion of the target object's positioning to the end of treatment. During this time period, the target object's body surface may change position due to respiration, heartbeat, or body movement. The motion surface can refer to the three-dimensional spatial point cloud obtained by converting the real-time acquired body surface depth data and used for comparison with a reference surface. During radiotherapy, the motion monitoring application continuously receives body surface depth data acquired by a portable electronic device through a depth sensor at a preset sampling frequency, and stores the three-dimensional spatial point cloud corresponding to each frame of acquired body surface depth data as the motion surface. By continuously receiving and storing the motion surface, the motion monitoring application can obtain the spatial information of the target object's current body posture in real time, which is beneficial for providing data support for subsequent difference analysis with the reference surface, thereby facilitating the dynamic monitoring of the target object's positional changes during treatment.

[0030] In some embodiments, the motion monitoring application receives body surface depth data output from a depth sensor in a continuous stream. Upon receiving each frame of body surface depth data, the motion monitoring application immediately stores the corresponding 3D spatial point cloud as the current motion surface, overwriting the previously stored motion surface. By maintaining a single motion surface variable, the motion monitoring application can reduce storage overhead and improve data processing efficiency. This embodiment facilitates high-frequency motion surface updates, meeting the real-time tracking requirements of rapid respiratory movements of a target object.

[0031] In other embodiments, the motion monitoring application stores the 3D spatial point cloud corresponding to each frame of received body surface depth data as a motion surface queue in chronological order. This queue retains motion surface data from the most recent few time points. In subsequent body position change analysis, the motion monitoring application can select the frame of motion surface with the smallest difference from the reference surface as a representative, or calculate the average position of multiple frames of motion surfaces as the current body position. This embodiment helps smooth out jitter caused by instantaneous noise from the depth sensor, improving the stability and anti-interference capability of motion monitoring, and is particularly suitable for monitoring scenarios in areas with weak body surface signals or low-light environments.

[0032] Step S104: Determine the positional changes of the target object during radiotherapy based on the difference between the moving surface and the reference surface.

[0033] For example, the difference between the motion surface and the reference surface can refer to the three-dimensional spatial offset of each spatial point on the motion surface relative to the corresponding position point on the reference surface. Postural change information can be used to characterize the changes in body surface position of the target object during radiotherapy caused by respiration, heartbeat, or involuntary body movements. The motion monitoring application spatially registers the real-time acquired motion surface with a pre-stored reference surface, calculates the overall rigid body transformation matrix of the motion surface relative to the reference surface, and extracts the translation and rotation angles in multiple degrees of freedom directions from this transformation matrix as postural change information. By determining the postural change information, the motion monitoring application can quantify the real-time displacement and rotation of the target object's body surface relative to the reference positioning, which helps provide users with objective motion monitoring data, thereby assisting in determining whether beam intervention or repositioning is necessary.

[0034] In some embodiments, the motion monitoring application may employ a rigid registration algorithm to perform point cloud matching between the moving surface and a reference surface, searching for a target rigid body transformation matrix that minimizes the distance metric between the moving surface and the reference surface. The motion monitoring application then extracts the spatial displacement information of the moving surface relative to the reference surface from the target rigid body transformation matrix. This spatial displacement information includes translation along the transverse, longitudinal, and vertical axes, as well as rotation angles about these three axes. The motion monitoring application outputs this spatial displacement information as body position change information to the user interface. This embodiment facilitates the quantitative presentation of the six-degree-of-freedom motion state of the target object, meeting the clinical need for quantitative assessment of positioning accuracy.

[0035] In other embodiments, before determining postural change information, the motion monitoring application first extracts point cloud data from the corresponding sub-regions of the body surface from the moving surface and the reference surface based on the region of interest (ROI) bounding box set by the user in the depth map mode. The motion monitoring application only performs registration and difference calculation on the point cloud within the ROI bounding box, ignoring the area outside the bounding box. After calculating the postural change information within the ROI, the motion monitoring application displays the displacement and rotation components in numerical and graphical form on the user interface. This embodiment can focus on key body surface areas of clinical interest, which helps to reduce the interference of non-target area movements such as involuntary limb swaying on the monitoring results, thereby improving the targeting and computational efficiency of motion monitoring.

[0036] This application embodiment utilizes a portable electronic device and its built-in depth sensor to replace the expensive dedicated optical camera, which significantly reduces the hardware cost of the surface-guided radiotherapy motion monitoring system, making it easier to promote its application in small and medium-sized hospitals and resource-scarce areas.

[0037] In some optional embodiments, storing the three-dimensional spatial point cloud corresponding to the body surface depth data collected by the portable electronic device at a target time as a reference surface includes: displaying the body surface depth data collected by the portable electronic device in the display interface of the portable electronic device; determining the body surface sub-region defined by the region of interest in the display interface in response to the user's adjustment operation of the region of interest box; and storing the three-dimensional spatial point cloud within the region of interest box as a reference surface.

[0038] For example, a region of interest (ROI) can refer to a rectangular visual frame that a user manually drags or resizes on the display interface of a portable electronic device, used to delineate a sub-region from complete body surface depth data. The sub-region can refer to the local body surface area covered by the ROI, such as the thoracic or abdominal region of the target object. The motion monitoring application displays real-time body surface depth data acquired by the depth sensor on the display interface of the portable electronic device. The user can select the sub-region of body surface requiring focused monitoring by adjusting the size and position of the ROI. In response to the user's adjustment, the motion monitoring application stores the 3D point cloud within the sub-region of body surface defined by the ROI as a reference surface. By allowing users to customize the ROI, the motion monitoring application can focus the monitoring range on clinically critical areas, such as the thoracic region affected by respiration, helping to eliminate interference from the bed surface, fixation devices, or other non-target body surfaces, thereby improving the representativeness of the reference surface and the specificity of subsequent motion monitoring.

[0039] Optionally, the motion monitoring application can provide two visualization modes: depth map mode and surface imaging mode. In depth map mode, users can define a region of interest (ROI) by adjusting a box (e.g., a red rectangle) on the portable electronic device's display interface. This ROI box is used to select specific sub-regions of the body surface that need to be tracked. In surface imaging mode, the motion monitoring application renders a 3D surface model of the target object reconstructed from depth sensor data in real time. Users can adjust the depth range, surface rendering transparency, and the smoothness of the depth data, such as one-dimensional or two-dimensional smoothing, to optimize the visualization effect under different body surface features. The depth map mode and surface imaging mode can be displayed independently or in parallel, and users can switch between views according to monitoring needs. By providing two different view modes, the motion monitoring application can meet users' different needs for target area selection and body surface morphology observation, which helps to improve operational flexibility and visualization adaptability.

[0040] In some embodiments, the motion monitoring application displays a color-coded depth image in a depth map mode interface, with different colors representing different depth values. The user adjusts the position and size of a red rectangle using a two-finger gesture or a slider; the area within the rectangle is the selected sub-region of the body surface. The motion monitoring application calculates the number of point clouds within the rectangle in real time and displays this number on the interface. When the number of point clouds within the rectangle reaches a preset minimum threshold, the motion monitoring application allows the user to trigger a reference surface acquisition command and stores the 3D spatial point cloud within the rectangle as a reference surface. This embodiment avoids registration failures caused by insufficient effective point clouds within the rectangle, thus ensuring the quality of the reference surface.

[0041] In other embodiments, the motion monitoring application displays a 3D surface model reconstructed from body surface depth data in surface imaging mode. The user can directly draw a polygonal region of interest on the 3D model, for example, by plotting points on a touchscreen. The motion monitoring application determines a defined sub-region of the body surface based on the user-drawn polygonal outline, extracts the 3D point cloud within this sub-region, and renders the sub-region with a highlight color. After user confirmation, the motion monitoring application stores only the point cloud within this sub-region as a reference surface. This embodiment allows the user to freely define regions of interest of arbitrary shapes, enabling more precise fitting of the anatomical morphology of the target body surface and improving the accuracy of subsequent registration.

[0042] In some optional embodiments, the method further includes: determining a temporal resolution based on the size of the region of interest (ROI) bounding box, wherein the temporal resolution is used to characterize the update frequency of the moving surface; the larger the ROI bounding box, the more point clouds the surface sub-region contains, and the lower the temporal resolution is set; the smaller the ROI bounding box, the fewer point clouds the surface sub-region contains, and the higher the temporal resolution is set.

[0043] For example, temporal resolution can refer to the frequency at which a motion monitoring application updates the moving surface, i.e., the rate at which the motion monitoring application receives and processes body surface depth data frames from the depth sensor per unit time. The size of the region of interest (ROI) box determines the number of point clouds contained in the body surface sub-region: the larger the ROI box area, the wider the spatial range covered by the body surface sub-region, and the more 3D spatial point clouds there are; the smaller the ROI box area, the fewer the point clouds. The motion monitoring application determines the temporal resolution based on the ROI box size: the larger the ROI box, the lower the temporal resolution set by the motion monitoring application, i.e., reducing the update frequency of the moving surface; the smaller the ROI box, the higher the temporal resolution set by the motion monitoring application, i.e., increasing the update frequency of the moving surface. By establishing an inverse correlation between the ROI box size and temporal resolution, the motion monitoring application can reduce the data processing volume of registration calculations in scenarios with a large number of point clouds to maintain a stable computational load; and increase the update rate in scenarios with a small number of point clouds to obtain a faster response speed. This setting is beneficial for achieving a balance between computational efficiency and real-time performance under different monitoring ranges, enabling the motion monitoring application to adaptively adjust the data processing frequency according to the monitoring area selected by the user.

[0044] In some embodiments, the motion monitoring application monitors the number of point clouds within the body surface sub-region corresponding to the currently adjusted region of interest (ROI) in real time. The motion monitoring application sets the temporal resolution to multiple levels based on a preset threshold range for the number of point clouds. When the number of point clouds exceeds a first threshold, the motion monitoring application sets the temporal resolution to the first level, such as a lower update frequency; when the number of point clouds is below a second threshold, the motion monitoring application sets the temporal resolution to the second level, such as a higher update frequency; when the number of point clouds is between the first and second thresholds, the motion monitoring application sets the temporal resolution to an intermediate level. This embodiment achieves adaptive adjustment of the temporal resolution through a discretized level approach, which simplifies the control logic and ensures stable operation of the motion monitoring application under different monitoring ranges.

[0045] In other embodiments, the motion monitoring application provides a temporal resolution selection control in the surface imaging mode settings interface, which is linked to the adjustment operation of the region of interest (ROI) bounding box. After the user manually sets the target temporal resolution, the motion monitoring application automatically calculates the maximum allowable number of point clouds based on the target temporal resolution. When the user expands the ROI bounding box, if the number of point clouds within the box exceeds the maximum allowable value, the motion monitoring application automatically performs uniform downsampling on the point clouds within the box, ensuring that the number of downsampled point clouds meets the computational load requirements corresponding to the target temporal resolution. This embodiment allows users to actively select the response speed according to clinical monitoring needs, while ensuring computational feasibility at high temporal resolution through the downsampling mechanism, which is beneficial for meeting the differentiated temporal resolution requirements of different dynamic monitoring scenarios.

[0046] The embodiments of this application utilize an intuitive motion monitoring application, which helps to reduce complex system installation, calibration, and quality assurance procedures, enabling ordinary users to quickly master and use it.

[0047] In some optional embodiments, the positional change information of the target object during radiotherapy is determined based on the difference between the moving surface and the reference surface, including: using a rigid registration algorithm to perform point cloud matching between the moving surface and the reference surface to determine the target rigid body transformation matrix that minimizes the distance metric between the moving surface and the reference surface; extracting the spatial displacement information of the moving surface relative to the reference surface from the target rigid body transformation matrix, and using the spatial displacement information as the positional change information.

[0048] For example, a rigid registration algorithm can refer to a mathematical method that aligns two point clouds through rotation and translation transformations, assuming that the target object's surface undergoes only overall rigid body motion without deformation during monitoring. Distance metrics can be used to quantify the registration error between the moving surface and the reference surface, such as the sum of squared Euclidean distances between point pairs or the distance from a point to a plane. The target rigid body transformation matrix can refer to the optimal combination of rotation matrices and translation vectors that minimizes the distance metric between the moving surface and the reference surface. The motion monitoring application uses a rigid registration algorithm to match the point clouds of the moving surface and the reference surface, determining the target rigid body transformation matrix that minimizes the distance metric between them. The motion monitoring application extracts the spatial displacement information of the moving surface relative to the reference surface from the target rigid body transformation matrix and uses this spatial displacement information as body position change information. Through rigid registration, the motion monitoring application can quantify the overall spatial offset of the target object's surface from the reference position to the current moment, which is beneficial for providing users with objective and quantitative body position change data and assisting in determining whether the target object has deviated from the treatment position.

[0049] Alternatively, a non-rigid registration algorithm can be used instead of a rigid registration algorithm. Non-rigid registration algorithms can capture local deformations of the target object's surface, such as chest and abdominal deformations caused by respiration, thus providing more refined motion information. The motion monitoring application uses a non-rigid registration algorithm to match the moving surface with the reference surface, calculates the local displacement field of each region of the body surface, and uses this displacement field as information on body position changes. This embodiment is suitable for scenarios requiring monitoring of body surface deformation, but may increase computational load and affect real-time performance.

[0050] In some embodiments, the motion monitoring application uses the sum of squared point-to-point distances as a distance metric. In each registration iteration, the application searches for the nearest neighbor on the reference surface for each point on the moving surface, calculating the sum of squared distances between all nearest point pairs as the registration error. The application solves for the rigid body transformation matrix by minimizing this error, and then performs another nearest-neighbor search between the transformed moving surface and the reference surface, iterating repeatedly until the error converges or a preset number of iterations is reached. The application extracts the translation vector (displacement along the three coordinate axes) and the rotation matrix (corresponding Euler angles) from the final rigid body transformation matrix. This embodiment employs a classic point cloud registration strategy, which is beneficial for obtaining stable and repeatable results for calculating body position changes.

[0051] In other embodiments, the motion monitoring application can use point-to-plane distance as a distance metric. The application first calculates the normal vector of each point on the reference surface, i.e., the direction vector perpendicular to the tangent plane of the body surface. Then, during registration, it calculates the perpendicular distance from a point on the moving surface to the tangent plane containing the corresponding point on the reference surface, and uses the sum of the squares of these perpendicular distances as the registration error. Compared to point-to-point distance, point-to-plane distance allows points on the moving surface to slide along the tangent plane direction, which is beneficial for obtaining faster convergence and smoother registration results in regions with large changes in body surface curvature. The motion monitoring application extracts the spatial displacement information of the moving surface relative to the reference surface from the optimized rigid body transformation matrix. This embodiment can reduce the number of iterations while maintaining registration accuracy, thus improving the computational efficiency of real-time motion monitoring.

[0052] In some optional embodiments, the rigid registration algorithm implements each iteration through the following steps executed sequentially: Step 1, detect the nearest point pair between the moving surface and the reference surface; Step 2, calculate the rigid body transformation matrix that minimizes the sum of squared distances between the nearest point pairs; Step 3, apply the rigid body transformation matrix to the moving surface to obtain the updated moving surface; Step 4, using the updated moving surface as the current moving surface, repeat steps 1 to 3 until the convergence condition is met, and then use the last obtained rigid body transformation matrix as the target rigid body transformation matrix. The convergence condition is: the change amplitude of the rigid body transformation matrix obtained by two consecutive iterations is lower than a preset threshold, or the preset maximum number of iterations is reached.

[0053] For example, the nearest point pair can refer to the pair formed by each point on the moving surface and the nearest point on the reference surface. The sum of squared distances can refer to the cumulative sum of the squared Euclidean distances between all nearest point pairs, used to quantify the registration error between the two point clouds. The rigid body transformation matrix can refer to the transformation parameters containing rotation and translation components, used to transform the moving surface to the coordinate system of the reference surface. Convergence criteria can be used to determine when the iterative process terminates, including when the change in the transformation matrix between two consecutive iterations is less than a preset threshold or when a preset maximum number of iterations is reached. The motion monitoring application iteratively performs four steps: nearest point detection, rigid body transformation matrix calculation, and transformation application, progressively optimizing the alignment between the moving surface and the reference surface, so that the registration error gradually decreases after each iteration. This iterative approximation method is beneficial for obtaining high-precision rigid body transformation parameters through successive optimization without explicit solution, thereby improving the accuracy of body position change information calculation.

[0054] In some embodiments, the motion monitoring application uses spatial indexing to accelerate nearest-point search in step one. The motion monitoring application first constructs a spatial index structure such as a KD-tree or octree for the reference surface. In each iteration, for each point on the moving surface, the motion monitoring application quickly finds the nearest point on the reference surface using the spatial index, avoiding traversing all point pairs. In step two, the motion monitoring application uses singular value decomposition to solve the rigid body transformation matrix, minimizing the sum of squared distances between nearest point pairs. In step three, the motion monitoring application applies the solved rigid body transformation matrix to all points on the moving surface to obtain an updated moving surface. The motion monitoring application repeats the above steps, stopping the iteration when the rotation angle change of the transformation matrix calculated in two consecutive iterations is less than a preset threshold (e.g., 0.01 degrees) and the translation change is less than a preset threshold (e.g., 0.01 millimeters). This embodiment, by accelerating the nearest-point search and using strict convergence conditions, helps to improve computational efficiency while ensuring registration accuracy.

[0055] In some embodiments, the motion monitoring application introduces weighting coefficients in step two to suppress the influence of outliers (such as isolated points) on the registration results. The motion monitoring application first calculates the weight value of each point pair based on the distance between the nearest point pairs; the farther the point pair, the lower the weight, and the closer the point pair, the higher the weight. When solving the rigid body transformation matrix, the motion monitoring application minimizes the weighted sum of squared distances. After each iteration, the motion monitoring application removes point pairs whose distance exceeds a preset outlier threshold, retaining only reliable point pairs for the registration calculation in the next iteration. The motion monitoring application repeats the above process until a preset maximum number of iterations (e.g., 50) is reached or the transformation converges. This embodiment helps to mitigate the negative impact of outliers caused by depth sensor noise or local occlusion on the body surface on registration accuracy, thereby improving the reliability of motion monitoring.

[0056] In some optional embodiments, the spatial displacement information includes displacement components of the moving surface relative to a reference surface in multiple degrees of freedom directions, wherein the multiple degrees of freedom directions include translation along X mutually orthogonal axial directions and rotation angles about X axial directions, where X is an integer greater than 2.

[0057] For example, the degree of freedom can refer to the dimensions of independent movement of the target object in three-dimensional space, including translation along X mutually orthogonal axes and rotation around these X axes. X can be an integer greater than 2; for example, X equals 3, corresponding to translation along the horizontal, vertical, and triangular axes, and rotation around each axis, for a total of six degrees of freedom. Spatial displacement information can refer to the specific displacement values ​​of the moving surface relative to the reference surface in these degrees of freedom directions, such as translation along a certain axis or rotation around a certain axis. The motion monitoring application decomposes the spatial displacement information into displacement components in multiple degrees of freedom directions, which can comprehensively describe the positional and posture changes of the target object's body surface in three-dimensional space. This is beneficial for providing users with more complete postural deviation data, thereby assisting in determining whether to adjust the target object's posture or suspend treatment. By outputting displacement information in multiple degrees of freedom, the motion monitoring application can meet the clinical needs for refined monitoring of the target object's motion state.

[0058] In some embodiments, the motion monitoring application decomposes spatial displacement information into translational components on three mutually orthogonal axes (such as the X-axis, Y-axis, and Z-axis) and rotational components about these three axes. The translational components represent the distance the moving surface travels in the left-right, front-back, and up-down directions, respectively; the rotational components represent the roll, pitch, and yaw angles of the moving surface, respectively. The motion monitoring application displays the current values ​​of these six components in real-time in numerical form on the user interface and compares each component with a preset clinically acceptable threshold. When any component exceeds the corresponding threshold, the motion monitoring application triggers an alarm. This embodiment helps users intuitively understand the specific deviations of the target object in each degree of freedom direction, quickly locate the source of motion, such as whether it is lateral displacement or rotation of the target object's body, thereby improving intervention efficiency.

[0059] In other embodiments, after extracting spatial displacement information, the motion monitoring application integrates the displacement components in each degree of freedom direction into the overall displacement vector magnitude (i.e., the square root of the sum of the squares of all translational components) and the overall rotation angle magnitude (i.e., the square root of the sum of the squares of all rotational components), and displays the trends of the overall displacement and overall rotation over time as curves on the interface. Simultaneously, the motion monitoring application retains independent data for each component, allowing users to view detailed analysis reports. This embodiment provides a concise overview of the motion state without losing detail, facilitating users to quickly determine whether a target object is within a safe range of motion, while also meeting the need for in-depth analysis of anomalies in specific components.

[0060] By using real-time rigid registration between the reference surface and the moving surface, six-degree-of-freedom positional change information can be provided, which can improve the accuracy of static positioning verification and dynamic respiratory motion monitoring, meeting clinical requirements.

[0061] In some optional embodiments, after determining the positional changes of the target object during radiotherapy, the method further includes: controlling the display screen of a portable electronic device to display an overlay view of a moving surface and a reference surface; wherein the moving surface is rendered in a first color, the reference surface is rendered in a second color, and there is a semi-transparent overlay effect between the rendered reference surface and the rendered moving surface, and the first color and the second color are different colors.

[0062] For example, an overlay view can refer to a visualization displayed on a portable electronic device screen that simultaneously renders and semi-transparently overlays a motion surface and a reference surface in the same three-dimensional coordinate system. A first color can be used to render the motion surface, and a second color can be used to render the reference surface. The first and second colors are different colors, for example, the first color is green and the second color is red, and the rendered motion surface and reference surface have a semi-transparent overlay effect. The motion monitoring application controls the display screen of the portable electronic device to show the overlay view of the motion surface and reference surface, allowing users to distinguish the spatial relationship between the current body surface position and the reference positioning through color. The semi-transparent overlay effect helps to observe the degree of overlap and the direction of deviation between the motion surface and the reference surface. By providing a visual overlay view, the motion monitoring application can intuitively present information on changes in body position, which helps users quickly determine whether the target object has deviated from the reference position, thus providing a visual basis for whether treatment needs to be paused or repositioned.

[0063] In some embodiments, the motion monitoring application renders the motion surface as a semi-transparent point cloud or mesh of a first color, and a reference surface as a semi-transparent point cloud or mesh of a second color. The motion monitoring application receives body surface depth data collected by a depth sensor in real time and updates the motion surface while keeping the reference surface fixed. The motion monitoring application periodically redraws the motion surface and the reference surface in an overlay view, allowing the relative positional differences between the two surfaces to be dynamically presented over time. This embodiment helps users quickly identify the direction and magnitude of body positional shifts through visual observation, improving the intuitiveness of motion monitoring.

[0064] In other embodiments, the motion monitoring application provides transparency adjustment controls in the overlay view, allowing users to adjust the semi-transparency levels of a first color and / or a second color. In response to the transparency adjustment command, the motion monitoring application re-renders the moving surface and the reference surface, enabling users to customize the overlay effect according to visual needs, allowing for clearer differentiation of the boundary between the moving surface and the reference surface in complex surface areas or under low-contrast conditions. This embodiment facilitates adaptation to different user visual preferences and display needs under varying lighting environments, improving the adaptability and usability of the visualization interface.

[0065] In some optional embodiments, after determining the positional changes of the target object during radiotherapy, the method further includes: controlling the display screen of a portable electronic device to display, in the form of a graph, the overall displacement magnitude and the dynamic curves of the individual translational and rotational components changing over time in the positional change information.

[0066] For example, the overall displacement magnitude can refer to the combined translational distance of the moving surface relative to the reference surface in three-dimensional space, such as the scalar value obtained by taking the square root of the sum of the squares of each translation component. Each translation component can refer to the displacement value along mutually orthogonal axes, and each rotational component can refer to the rotation angle value around the corresponding axis. The dynamic curve can refer to a continuously changing line or dotted graph plotted with time as the horizontal axis and displacement magnitude or angle value as the vertical axis. The motion monitoring application controls the display screen of the portable electronic device to display the overall displacement magnitude and the dynamic curves of each translational and rotational component over time in the form of a curve graph. This allows users to observe the historical trend and current state of the body position change, and intuitively understand the degree of deviation of the moving surface relative to the reference surface and its change over time. Through the dynamic curve graph, the motion monitoring application can provide users with a quantified historical record of body position changes, which is helpful in judging motion trends such as whether the displacement is gradually increasing and in assessing the body position stability of the target object.

[0067] In some embodiments, the motion monitoring application divides the display screen into multiple sub-graph regions to display the overall displacement magnitude curve, three translational component curves, and three rotational component curves, respectively. Each sub-graph region has an independently set vertical axis scale range, and all sub-graph regions share the same horizontal time axis. The motion monitoring application updates the curves in a scrolling manner, with the latest data point displayed at the right end of the curve, and historical data points that are out of display range automatically scrolling to the left. The motion monitoring application marks the value of the current data point and the corresponding preset threshold line on the curve, such as a warning line for overall displacement exceeding 3 mm. This embodiment allows users to simultaneously monitor changes in body position in multiple dimensions, quickly locate the specific direction of exceeding limits, and improve the efficiency of motion monitoring.

[0068] In other embodiments, the motion monitoring application provides interactive controls in the graph display interface, allowing users to select specific time points on the graph by clicking or dragging on the touchscreen. In response to the user's selection, the motion monitoring application highlights the data point corresponding to that time point and simultaneously overlays a three-dimensional view of the motion surface and reference surface at that time point on the interface, enabling users to view the actual body surface offset morphology at moments of significant positional change. The motion monitoring application also supports exporting curve data within the selected time range to a file, such as in a comma-separated value format.

[0069] Optionally, the motion monitoring application can also provide data recording functionality. In response to a user-triggered recording command, the application exports the translational and rotational component data (including timestamps and values ​​for each component) of the collected postural change information in a structured file format and stores it in the memory of the portable electronic device. The exported file format can be comma-separated value format, facilitating offline analysis or quality assurance recording using data analysis software. The motion monitoring application can also provide recording control buttons on the user interface, such as a start recording button and a stop recording button. After the user clicks the start recording button, the application continuously appends real-time postural change data to the file until the user clicks the stop recording button. This embodiment is beneficial for users to save motion data during treatment for subsequent analysis or retrospective quality assurance.

[0070] In some optional embodiments, after determining the positional change information of the target object during radiotherapy, the method further includes: when the positional change information triggers a preset condition, outputting a control signal for controlling the treatment beam through a motion monitoring application; wherein the preset condition includes: any translational component in the positional change information exceeds a preset threshold corresponding to the translational component, or any rotational component exceeds a preset threshold corresponding to the rotational component.

[0071] For example, preset conditions can refer to one or more pre-defined judgment rules used to determine whether the positional change of the target object reaches a level requiring intervention. Preset thresholds can refer to the permissible upper limits set for each displacement component (including each translational component and each rotational component), such as the permissible upper limits for translational components and rotational components. Control signals can refer to command signals used to instruct the treatment beam generation device to perform beam holding or beam recovery. When the motion monitoring application detects that the positional change information triggers the preset conditions, i.e., any translational component in the positional change information exceeds the preset threshold corresponding to that translational component, or any rotational component exceeds the preset threshold corresponding to that rotational component, the motion monitoring application outputs a control signal to control the treatment beam. By automatically detecting whether the positional change exceeds the limits and outputting control signals, the motion monitoring application can promptly trigger beam interruption or alarms, which helps prevent target area irradiation deviation caused by excessive positional shift of the target object, thereby improving the safety of radiotherapy.

[0072] Optionally, the positional change information output by the motion monitoring application (e.g., real-time displacement values ​​of each translational and rotational component) can be sent to the linear accelerator's control system via a standard communication interface (e.g., User Datagram Protocol communication or shared memory). When the motion monitoring application detects that the positional change information triggers a preset condition, it outputs a beam hold signal to the linear accelerator control system; when the positional change information falls below a preset threshold, it outputs a beam recovery signal. This embodiment facilitates automated beam gating without manual intervention, thus improving treatment efficiency and safety.

[0073] In some embodiments, the motion monitoring application maintains a preset threshold for each displacement component and compares the current value of each displacement component with its corresponding preset threshold after each calculation of body position change information. When the motion monitoring application detects that the current value of any displacement component exceeds the corresponding preset threshold, it immediately sends a beam holding control signal to the treatment beam generating device through its output interface. When all displacement components in subsequent body position change information fall back below their corresponding preset thresholds, the motion monitoring application automatically sends a beam recovery control signal. This embodiment can automatically resume treatment after the target object's body position has returned to a safe range, which helps reduce the number of manual interventions and improves treatment efficiency.

[0074] In other embodiments, the motion monitoring application first confirms the duration of the over-limit state before outputting the control signal. The motion monitoring application only outputs the beam hold control signal when it detects that a displacement component exceeds a corresponding preset threshold, and the over-limit state persists for more than a preset duration threshold (e.g., 0.5 seconds). Simultaneously with outputting the beam hold control signal, the motion monitoring application displays an over-limit warning message on the portable electronic device's screen, such as highlighting the name and current value of the over-limit component, and recording the timestamp and displacement value of the over-limit event. This embodiment helps avoid false triggering caused by transient noise or brief physiological fluctuations, improves the stability and reliability of beam control, and provides records for post-event analysis.

[0075] In some optional embodiments, after determining the positional changes of the target subject during radiotherapy, the method further includes: controlling a portable electronic device to establish a two-way communication link with a remote monitoring device; pushing the current display interface content of the portable electronic device to the remote monitoring device in the form of streaming media, so that the remote monitoring device displays a visual interface synchronized with the portable electronic device; receiving control commands from the remote monitoring device; wherein the control commands include beam control commands manually triggered by the user after observing data on the remote monitoring device; and outputting control signals for controlling the treatment beam in response to the beam control commands.

[0076] For example, a two-way communication link can refer to a network connection established between a portable electronic device and a remote monitoring device for bidirectional data transmission, such as a connection established based on a local area network or a point-to-point wireless protocol. Streaming media format can refer to the real-time encoding and transmission of the display interface content of the portable electronic device as a continuous sequence of image frames, allowing the remote monitoring device to synchronously play the same image. The visualization interface on the remote monitoring device can include an overlay view of the moving surface and a reference surface, a dynamic graph of body position change information, and numerical displays of various displacement components. The motion monitoring application controls the establishment of a two-way communication link between the portable electronic device and the remote monitoring device, and pushes the current display interface content of the portable electronic device to the remote monitoring device in streaming media format, enabling the remote monitoring device to display a visualization interface synchronized with the portable electronic device. The motion monitoring application receives control commands from the remote monitoring device, including beam control commands manually triggered by the user after observing data on the remote monitoring device. In response to the beam control commands, the motion monitoring application outputs control signals for controlling the therapeutic beam. By pushing the interface content to a remote monitoring device, the motion monitoring application allows users to observe the target's positional changes in real time from a location far from the treatment room, and intervene in the treatment process by remotely sending beam control commands. This helps reduce the user's radiation exposure in the treatment room while maintaining real-time control over the motion monitoring.

[0077] Optionally, the streaming media transmission for remote monitoring can employ a real-time messaging protocol to push the display interface and data of the portable electronic device to a local server. The remote monitoring device retrieves the data stream from the server via a browser or client, enabling a synchronized visual interface display with the portable electronic device. This embodiment improves cross-platform compatibility.

[0078] In some embodiments, the motion monitoring application automatically discovers the remote monitoring device via a local network, for example, using a zero-configuration network protocol, and establishes a bidirectional communication link with the remote monitoring device. The motion monitoring application encodes the current display interface into a video stream at a preset frame rate and pushes it to the remote monitoring device via a real-time messaging protocol. Simultaneously, the motion monitoring application continuously receives control commands from the remote monitoring device via the bidirectional communication link. When the motion monitoring application receives a beam hold command, it immediately outputs a control signal to the therapeutic beam generator. This embodiment utilizes a local network to achieve low-latency streaming media delivery, which helps ensure the real-time nature of remote monitoring and the timeliness of control response.

[0079] In other embodiments, the motion monitoring application dynamically adjusts streaming parameters based on the data reception status of the remote monitoring device during the streaming process. The motion monitoring application monitors the remote monitoring device's receive buffer and network latency. When network conditions are poor, the application automatically reduces the streaming frame rate or image resolution to ensure video continuity and timely delivery of control commands. The application also overlays current streaming status indicators, such as connection status icons and latency indicators, onto the streaming interface. This embodiment helps maintain basic remote monitoring functions in environments with unstable networks.

[0080] Optionally, when deploying a motion monitoring system, a portable electronic device can be fixed to the treatment head or treatment bed in the treatment room via a bracket (e.g., an electronic flashlight adapter), allowing the depth sensor's field of view to cover the target object's body surface area. The portable electronic device and the remote monitoring device automatically discover compatible devices on the local network and establish a bidirectional communication link via a zero-configuration network protocol (e.g., the Bonjour protocol). After the motion monitoring application controls the connection between the portable electronic device and the remote monitoring device, the user can choose between a single-device standalone mode (using only the portable electronic device for surface acquisition and motion monitoring) or a master-control / monitoring mode (the portable electronic device acts as the master device for data acquisition, and the remote monitoring device acts as the monitoring device for displaying a synchronized interface) as needed. This embodiment simplifies device deployment and network configuration processes, reduces system installation complexity, and improves the ease of use of remote monitoring functions.

[0081] In some optional embodiments, the remote monitoring device also independently displays a virtual threshold indicator; wherein, the virtual threshold indicator dynamically presents the current value of each displacement component in the body position change information and the relative ratio of the corresponding preset threshold in the form of a simulated dashboard or progress bar, based on the body position change information sent by the motion monitoring application.

[0082] For example, a virtual threshold indicator can be a graphical component independent of the main visualization area on the display interface of a remote monitoring device. It visually presents the relative proportion of the current value of each displacement component in the body position change information to its corresponding preset threshold. A simulated dashboard format can refer to a semi-circular pointer scale similar to a car speedometer, where the pointer position indicates the percentage of the current value relative to the preset threshold. A progress bar format can refer to a horizontal or vertical bar graph, where the fill length ratio reflects the size of the current value relative to the preset threshold. The motion monitoring application sends body position change information to the remote monitoring device. Based on the received body position change information, the remote monitoring device dynamically presents the relative proportion of the current value of each displacement component to its corresponding preset threshold in the form of a simulated dashboard or progress bar. Through the virtual threshold indicator, the remote monitoring device can provide users with a more intuitive warning of exceeding limits, facilitating quick judgment in a remote monitoring environment whether the target object's position is approaching or exceeding the safe range, thereby assisting in decisions on whether to manually send beam control commands.

[0083] In some embodiments, the motion monitoring application packages the current value of each displacement component, its corresponding preset threshold, and the component identifier, and sends them to the remote monitoring device via a bidirectional communication link. The remote monitoring device generates an independent analog dashboard indicator for each displacement component. The dashboard's scale ranges from 0 to 1.2 times the preset threshold, with the pointer pointing to the scale position corresponding to the current value. When the current value exceeds the preset threshold, the indicator dial area changes color, for example, from green to red, and flashes as a warning. The remote monitoring device also provides dashboard scaling and arrangement settings, allowing users to view the status of all displacement components simultaneously on a single screen. This embodiment provides an intuitive threshold comparison through an analog dashboard, which helps users quickly identify and react to out-of-limit components in emergency situations.

[0084] In other embodiments, the remote monitoring device displays the relative proportion of each displacement component in the form of a progress bar. Each displacement component corresponds to a horizontal progress bar, the total length of which corresponds to a preset threshold, and the fill length corresponds to the absolute value of the current value. When the current value is less than the preset threshold, the fill color is green; when the current value reaches more than 80% of the preset threshold, the fill color turns yellow as a warning; when the current value exceeds the preset threshold, the fill color turns red, and the value exceeding the threshold is displayed at the end of the progress bar. The remote monitoring device also provides a comprehensive situation indicator, displaying the maximum relative proportion value among all components and the corresponding component name. This embodiment, through the color zoning of the progress bar and comprehensive indication, helps users quickly grasp the overall offset status and critical components, reducing the time spent checking multiple dashboards one by one and improving the efficiency of remote monitoring.

[0085] In some alternative embodiments, the portable electronic device is any one of a smartphone, tablet computer, or smart wearable device.

[0086] For example, a smartphone can refer to a mobile phone terminal that integrates light detection and ranging sensors, time-of-flight sensors, or structured light sensors; a tablet computer can refer to a portable large-screen computing device equipped with a depth sensor; and a smart wearable device can refer to smart hardware that is worn on the body and has depth sensing capabilities, such as head-mounted or wrist-worn devices. Limiting portable electronic devices to any of the above types allows motion monitoring applications to utilize readily available consumer-grade hardware to perform radiotherapy motion monitoring, avoiding the use of expensive dedicated optical equipment. This significantly reduces the hardware cost and deployment threshold of the system, while also facilitating migration and application between different device models.

[0087] Alternatively, the portable electronic device can also be a mobile device without a built-in depth sensor. This mobile device communicates with an external depth sensor via a wired connection (e.g., a Universal Serial Bus interface) or a wireless connection (e.g., Bluetooth), the external depth sensor being a time-of-flight sensor or a structured light sensor. The motion monitoring application receives surface depth data collected by the external depth sensor through this connection, achieving the same surface data acquisition function as with a built-in depth sensor.

[0088] The solution based on portable electronic devices can be easily integrated with linear accelerators of different brands and models, and is especially suitable for intra-segment motion monitoring in enclosed rack systems. The device installation location is flexible and the deployment is simple.

[0089] In some embodiments, the portable electronic device is a smartphone. The smartphone has a built-in depth sensor, such as a front-facing TrueDepth camera or a rear-facing light detection and ranging scanner, and a motion monitoring application is installed and runs on the smartphone. The user fixes the smartphone in a suitable location within the treatment room, such as mounting it on the treatment head or beside the bed using a bracket, and uses the smartphone's depth sensor to collect real-time surface depth data of the target object. This embodiment leverages the widespread availability and high-performance computing capabilities of smartphones to achieve motion monitoring at a lower cost, while utilizing the smartphone's built-in high-resolution screen and wireless communication module to facilitate the display of a visual interface and communication with remote devices.

[0090] In other embodiments, the portable electronic device is a smart wearable device, such as smart glasses or a smart head-mounted display. This smart wearable device has a built-in depth sensor, such as a time-of-flight camera or a structured light module, which the user can wear directly on their head, with a motion monitoring application running on it. The smart wearable device can collect surface depth data of a target object from multiple angles as the user's head moves, without the need for additional support. This embodiment further improves the flexibility and vantage point coverage of data acquisition, making it particularly suitable for scenarios requiring surface monitoring at different angles, while maintaining device portability and low deployment complexity.

[0091] For example, Figure 2 A flowchart illustrating a radiotherapy motion monitoring method based on a portable electronic device according to an embodiment of this application is shown. First, the device is deployed and connected by fixing the portable electronic device, which includes a depth sensor, in the treatment room and establishing a communication link with a remote monitoring device. Next, application initialization and view mode invocation are performed. In depth map mode, the user can adjust the red box to define the region of interest (ROI) for selecting a specific sub-region of the body surface to be tracked; this is called ROI box adjustment. In surface imaging mode, the user can adjust the depth range, transparency, and data smoothing to optimize the visualization of the 3D surface. The two view modes can be executed in parallel. After completing the mode settings, the system enters the reference surface acquisition stage. When the target object is in a reference position such as the initial setup, the user triggers a fixed surface acquisition command, and the system saves the 3D spatial point cloud corresponding to the current body surface depth data as the reference surface. Subsequently, the system enters the real-time motion monitoring and six-degree-of-freedom calculation stage. The system continuously acquires real-time depth data to generate a moving surface and uses an iterative nearest-point rigid registration algorithm to match the moving surface with the reference surface, calculating the six-degree-of-freedom displacement, including translation and rotation angles. During the visualization feedback and data recording phase, the system displays the moving surface and reference surface (e.g., in red) in an overlay view, and displays displacement and rotation curves in real time. It also supports recording and exporting data as structured files. Finally, in the remote monitoring and treatment intervention phase, the remote monitoring device displays the interface in real time. The user observes whether the displacement exceeds a preset threshold (e.g., 3mm). If it does, the user manually triggers the beam hold command, and the system outputs a control signal to pause the treatment.

[0092] According to another aspect of the embodiments of this application, a radiotherapy motion monitoring system based on a portable electronic device is also provided, comprising: a portable electronic device including a depth sensor; and a motion monitoring application installed and running on the portable electronic device and configured to perform the above-described radiotherapy motion monitoring method based on a portable electronic device.

[0093] For example, a portable electronic device can refer to an electronic terminal device with independent computing capabilities, a built-in depth sensor, and easy portability, such as a smartphone, tablet, or smart wearable device. A depth sensor can refer to a sensor used to acquire distance information from the surface of a target object, such as a light detection and ranging sensor, a time-of-flight sensor, or a structured light sensor. The motion monitoring application is installed and runs on the portable electronic device and can be configured to perform the aforementioned portable electronic device-based radiotherapy motion monitoring method. By deploying the motion monitoring application on a portable electronic device containing a depth sensor, the system can utilize readily available consumer-grade hardware to perform radiotherapy motion monitoring without the need for expensive dedicated optical equipment, significantly reducing system hardware costs and simplifying the deployment process while maintaining the functional integrity of motion monitoring.

[0094] In some embodiments, the radiotherapy motion monitoring system includes a smartphone (as a portable electronic device) and a motion monitoring application installed on the smartphone. The motion monitoring application acquires real-time body surface depth data by calling the depth sensor application programming interface of the smartphone's operating system, and performs reference surface storage, motion surface updating, and positional change information calculation locally on the phone. The motion monitoring application uses the smartphone's display to show a real-time overlay view of the motion surface and the reference surface, as well as the positional change curve. This embodiment facilitates independent motion monitoring with a single device, reduces additional hardware support, and allows for rapid deployment and use in resource-constrained radiotherapy departments or primary healthcare institutions.

[0095] In other embodiments, the radiotherapy motion monitoring system may include a tablet computer with a built-in depth sensor and a motion monitoring application installed on the tablet. The tablet computer is fixed in the treatment room by an adjustable stand, and its large screen size allows users to more clearly observe the overlay view of the body surface and curve data. In addition to performing calculations of body position change information, the motion monitoring application also pushes the display interface content to another remote monitoring device in the control room, such as a second tablet computer or computer, via a local network, facilitating visual synchronization between the treatment room and the control room. This embodiment utilizes the large screen of the tablet computer to enhance the user's observation experience, while the remote push function meets the user's need for remote monitoring in the control room, further reducing the user's radiation exposure risk.

[0096] In some optional embodiments, the system further includes a remote monitoring device that is communicatively connected to a portable electronic device; wherein: the portable electronic device is used to push the current display interface content to the remote monitoring device in the form of streaming media; the remote monitoring device receives and displays a visual interface synchronized with the portable electronic device; the remote monitoring device is also used to generate beam control commands in response to manual operations by the user and send them to the portable electronic device; wherein, after receiving the beam control commands from the remote monitoring device, the portable electronic device outputs control signals for controlling the therapeutic beam.

[0097] For example, the remote monitoring device can refer to an independent display and control terminal located in a different physical space from the portable electronic device, such as a tablet computer, computer, or dedicated monitoring screen. Streaming media can refer to the real-time encoding and transmission of the portable electronic device's display interface content as a continuous sequence of image frames, allowing the remote monitoring device to synchronously play the same images. Beam control commands can refer to command signals manually triggered by the user through the remote monitoring device to pause or resume the therapeutic beam. The motion monitoring application controls the portable electronic device to push the current display interface content to the remote monitoring device in streaming media format, enabling the remote monitoring device to display a visual interface synchronized with the portable electronic device. The motion monitoring application receives beam control commands from the remote monitoring device, which are manually triggered by the user after observing the data displayed on the remote monitoring device. In response to the beam control commands, the motion monitoring application outputs control signals to control the therapeutic beam. By introducing a remote monitoring device and synchronously pushing the display interface content, the motion monitoring application allows users to monitor the target's positional changes in real time without entering the treatment room and intervene in the treatment process by remotely sending beam control commands, which helps reduce the user's radiation exposure while maintaining real-time control over the motion.

[0098] In some embodiments, the remote monitoring device is a tablet or computer with a companion monitoring application installed. The motion monitoring application automatically discovers the remote monitoring device and establishes a two-way communication link via the local network. The motion monitoring application encodes the display content of the portable electronic device into a video stream at a preset frame rate and pushes it to the remote monitoring device via a real-time messaging protocol. The monitoring application on the remote monitoring device decodes and displays a synchronized visual interface. The display interface of the remote monitoring device provides virtual buttons, such as a beam hold button and a beam recovery button. When the user clicks a virtual button, the remote monitoring device generates a corresponding beam control command and sends it to the motion monitoring application. After receiving the command, the motion monitoring application outputs a control signal to the therapeutic beam generating device. This embodiment utilizes a local network to achieve low-latency remote monitoring and manual control, which helps ensure the timeliness and accuracy of user remote intervention.

[0099] In other embodiments, the remote monitoring device is a browser client, reducing the need for dedicated applications. The motion monitoring application, through its built-in web server functionality, pushes the display content of the portable electronic device to the remote monitoring device in a Hypertext Transfer Protocol (HTTP) streaming format. The remote monitoring device accesses the network address provided by the motion monitoring application via a browser to display a visual interface synchronized with the portable electronic device. The user triggers beam control commands via buttons on the browser page or keyboard shortcuts; these commands are sent back to the motion monitoring application through the web interface. The motion monitoring application outputs control signals in response to the received commands. This embodiment eliminates the need for pre-installed software on the remote monitoring device, simplifying the deployment process, improving cross-platform compatibility, and facilitating rapid activation of remote monitoring functionality in temporary use or device-constrained scenarios.

[0100] See Figure 3 According to another aspect of the embodiments of this application, a radiotherapy motion monitoring device based on a portable electronic device is also provided. The device includes: a data receiving unit for receiving surface depth data collected by the portable electronic device using a depth sensor targeting a target object; a reference data processing unit for storing the three-dimensional spatial point cloud corresponding to the surface depth data collected by the portable electronic device at the target time as a reference surface in response to a reference surface acquisition command initiated by a user at a target time; a motion data processing unit for continuously receiving surface depth data collected by the portable electronic device during radiotherapy and storing the collected surface depth data as a motion surface; and a position information determination unit for determining the positional change information of the target object during radiotherapy based on the difference between the motion surface and the reference surface.

[0101] Optionally, the reference data processing unit includes: a display subunit for displaying body surface depth data collected by the portable electronic device on the display interface of the portable electronic device; a region determination subunit for determining a sub-region of the body surface defined by the region of interest in the display interface in response to the user's adjustment operation on the region of interest box; and a storage subunit for storing the three-dimensional spatial point cloud within the region of interest box as the reference surface.

[0102] Optionally, the motion data processing unit further includes a temporal resolution adjustment subunit, configured to determine the temporal resolution based on the size of the region of interest (ROI) bounding box, wherein the temporal resolution characterizes the update frequency of the moving surface; the larger the ROI bounding box, the more point clouds the body surface sub-region contains, and the lower the temporal resolution is set; the smaller the ROI bounding box, the fewer point clouds the body surface sub-region contains, and the higher the temporal resolution is set.

[0103] Optionally, the body position information determination unit includes: a registration subunit, used to perform point cloud matching between the moving surface and the reference surface using a rigid registration algorithm to determine a target rigid body transformation matrix that minimizes the distance metric between the moving surface and the reference surface; and a decomposition and extraction subunit, used to decompose and extract the spatial displacement information of the moving surface relative to the reference surface from the target rigid body transformation matrix, and use the spatial displacement information as the body position change information.

[0104] Optionally, the registration subunit includes: a nearest point detection module for detecting the nearest point pairs between the moving surface and the reference surface; a transformation calculation module for calculating the rigid body transformation matrix that minimizes the sum of squared distances between the nearest point pairs; a transformation application module for applying the rigid body transformation matrix to the moving surface to obtain an updated moving surface; and an iteration control module for repeatedly calling the nearest point detection module, the transformation calculation module, and the transformation application module with the updated moving surface as the current moving surface until a convergence condition is met, and then using the last obtained rigid body transformation matrix as the target rigid body transformation matrix. The convergence condition is: the change amplitude of the rigid body transformation matrix obtained from two consecutive iterations is lower than a preset threshold, or the preset maximum number of iterations is reached.

[0105] Optionally, the spatial displacement information extracted by the decomposition extraction subunit includes displacement components in multiple degrees of freedom directions of the moving surface relative to the reference surface, wherein the multiple degrees of freedom directions include translation along X mutually orthogonal axes and rotation angles about the X axes, where X is an integer greater than 2.

[0106] Optionally, the radiotherapy motion monitoring device based on a portable electronic device further includes: a display control unit for controlling the display screen of the portable electronic device to display an overlay view of the motion surface and the reference surface; wherein the motion surface is rendered in a first color, the reference surface is rendered in a second color, and there is a semi-transparent overlay effect between the rendered reference surface and the rendered motion surface, and the first color and the second color are different colors.

[0107] Optionally, the radiotherapy motion monitoring device based on a portable electronic device further includes: a graph display unit for controlling the display screen of the portable electronic device to display, in the form of a graph, the overall displacement magnitude and the dynamic curves of the translational and rotational components changing over time in the positional change information.

[0108] Optionally, the radiotherapy motion monitoring device based on a portable electronic device further includes: a beam control unit, used to output a control signal for controlling the treatment beam through the motion monitoring application when the body position change information triggers a preset condition; wherein the preset condition includes: any translation component in the body position change information exceeds a preset threshold corresponding to the translation component, or any rotation component exceeds a preset threshold corresponding to the rotation component.

[0109] Optionally, the radiotherapy motion monitoring device based on a portable electronic device further includes: a communication control unit for controlling the establishment of a two-way communication link between the portable electronic device and the remote monitoring device; a push unit for pushing the current display interface content of the portable electronic device to the remote monitoring device in the form of streaming media, so that the remote monitoring device displays a visual interface synchronized with the portable electronic device; an instruction receiving unit for receiving control instructions from the remote monitoring device; wherein the control instructions include beam control instructions manually triggered by the user after observing data on the remote monitoring device; and an instruction response unit for responding to the beam control instructions and outputting control signals for controlling the treatment beam.

[0110] Optionally, the remote monitoring device may also independently display a virtual threshold indicator; wherein, the virtual threshold indicator dynamically presents the relative ratio of the current value of each displacement component in the body position change information to the corresponding preset threshold in the form of a simulated dashboard or progress bar, based on the body position change information sent by the motion monitoring application.

[0111] Optionally, the portable electronic device can be any one of a smartphone, tablet computer, or smart wearable device.

[0112] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, which stores a computer program, wherein when the computer program is executed, the device on which the computer-readable storage medium is located performs the above-described method for monitoring motion in radiotherapy based on a portable electronic device.

[0113] According to another aspect of the embodiments of this application, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to perform the above-described method for monitoring motion in radiotherapy based on a portable electronic device.

[0114] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the above-described method for monitoring motion in radiotherapy based on a portable electronic device.

[0115] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0116] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0121] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for monitoring motion in radiotherapy based on a portable electronic device, characterized in that, The method, applicable to a motion monitoring application installed on a portable electronic device including a smartphone, tablet, or wearable device, comprises: Receives surface depth data of the target object collected by the portable electronic device via a depth sensor; In response to a user's instruction to acquire a reference surface at a target time, the three-dimensional spatial point cloud corresponding to the body surface depth data acquired by the portable electronic device at the target time is stored as a reference surface. This includes: displaying the body surface depth data acquired by the portable electronic device on the display interface of the portable electronic device; determining the body surface sub-region defined by the region of interest in the display interface in response to the user's adjustment operation on the region of interest box; and storing the three-dimensional spatial point cloud within the region of interest box as the reference surface. During radiotherapy, the portable electronic device continuously receives surface depth data and stores the collected surface depth data as a moving surface. Based on the difference between the moving surface and the reference surface, the positional changes of the target object during radiotherapy are determined.

2. The method according to claim 1, characterized in that, The method further includes: The temporal resolution is determined based on the size of the region of interest (ROI) bounding box, where the temporal resolution characterizes the update frequency of the moving surface. The larger the ROI bounding box, the more point clouds the surface sub-region contains, and the lower the temporal resolution is set. Conversely, the smaller the ROI bounding box, the fewer point clouds the surface sub-region contains, and the higher the temporal resolution is set.

3. The method according to claim 1, characterized in that, Based on the difference between the moving surface and the reference surface, the positional changes of the target object during radiotherapy are determined, including: A rigid registration algorithm is used to perform point cloud matching between the moving surface and the reference surface to determine the target rigid body transformation matrix that minimizes the distance metric between the moving surface and the reference surface. The spatial displacement information of the moving surface relative to the reference surface is extracted from the target rigid body transformation matrix, and this spatial displacement information is used as the body position change information.

4. The method according to claim 3, characterized in that, The rigid registration algorithm implements each iteration through the following steps executed sequentially: Step 1: Detect the closest point pair between the moving surface and the reference surface; Step 2: Calculate the rigid body transformation matrix that minimizes the sum of squared distances between the closest point pairs; Step 3: Apply the rigid body transformation matrix to the moving surface to obtain the updated moving surface; Step four: Using the updated motion surface as the current motion surface, repeat steps one to three until the convergence condition is met. Then, use the last obtained rigid body transformation matrix as the target rigid body transformation matrix. The convergence condition is: the change amplitude of the rigid body transformation matrix obtained by two consecutive iterations is lower than a preset threshold, or the preset maximum number of iterations is reached.

5. The method according to claim 3, characterized in that, The spatial displacement information includes displacement components of the moving surface relative to the reference surface in multiple degrees of freedom directions, wherein the multiple degrees of freedom directions include translation along X mutually orthogonal axes and rotation angles about the X axes, where X is an integer greater than 2.

6. The method according to claim 1, characterized in that, After determining the positional changes of the target subject during radiotherapy, the method further includes: The display screen of the portable electronic device controls the display of an overlay view of the moving surface and the reference surface; wherein the moving surface is rendered in a first color, the reference surface is rendered in a second color, and there is a semi-transparent overlay effect between the rendered reference surface and the rendered moving surface, and the first color and the second color are different colors.

7. The method according to claim 1, characterized in that, After determining the positional changes of the target subject during radiotherapy, the method further includes: The display screen of the portable electronic device is controlled to show the overall displacement and the dynamic curves of the translation and rotation components over time in the body position change information in the form of a graph.

8. The method according to claim 1, characterized in that, After determining the positional changes of the target subject during radiotherapy, the method further includes: When the detected change in body position triggers a preset condition, the motion monitoring application outputs a control signal for controlling the therapeutic beam. The preset conditions include: any translation component in the body position change information exceeds the preset threshold corresponding to the translation component, or any rotation component exceeds the preset threshold corresponding to the rotation component.

9. The method according to claim 1, characterized in that, After determining the positional changes of the target subject during radiotherapy, the method further includes: The portable electronic device is controlled to establish a two-way communication link with the remote monitoring device; The current display content of the portable electronic device is pushed to the remote monitoring device in the form of streaming media, so that the remote monitoring device displays a visual interface synchronized with the portable electronic device; Receive control commands from the remote monitoring device; wherein the control commands include beam control commands manually triggered by the user after observing data on the remote monitoring device; In response to the beam control command, a control signal for controlling the treatment beam is output.

10. The method according to claim 9, characterized in that, The remote monitoring device also independently displays a virtual threshold indicator; wherein, the virtual threshold indicator dynamically presents the relative ratio of the current value of each displacement component in the body position change information to the corresponding preset threshold in the form of a simulated dashboard or progress bar, based on the body position change information sent by the motion monitoring application.

11. A radiotherapy motion monitoring system based on a portable electronic device, characterized in that, include: A portable electronic device, wherein the portable electronic device is equipped with a depth sensor, and the portable electronic device includes a smartphone, tablet computer, or smart wearable device; A motion monitoring application, which is installed and runs on the portable electronic device and is configured to perform the method of claim 1.

12. The system according to claim 11, characterized in that, The system also includes a remote monitoring device, which is communicatively connected to the portable electronic device; wherein: The portable electronic device is used to push the current display interface content to the remote monitoring device in the form of streaming media; The remote monitoring device receives and displays a visual interface synchronized with the portable electronic device; the remote monitoring device is also used to generate beam control commands in response to manual operations by the user and send them to the portable electronic device; wherein, after receiving the beam control commands from the remote monitoring device, the portable electronic device outputs control signals for controlling the treatment beam.

13. A radiotherapy motion monitoring device based on a portable electronic device, characterized in that, The device includes: A data receiving unit is used to receive surface depth data of the target object collected by the portable electronic device through a depth sensor. The portable electronic device includes a smartphone, tablet computer, or smart wearable device. A reference data processing unit is configured to, in response to a reference surface acquisition command initiated by a user at a target time, store the three-dimensional spatial point cloud corresponding to the body surface depth data acquired by the portable electronic device at the target time as a reference surface, including: displaying the body surface depth data acquired by the portable electronic device on the display interface of the portable electronic device; determining the body surface sub-region defined by the region of interest frame in response to the user's adjustment operation on the display interface; and storing the three-dimensional spatial point cloud within the region of interest frame as the reference surface. The motion data processing unit is used to continuously receive body surface depth data collected by the portable electronic device during radiotherapy, and to store the collected body surface depth data as a motion surface. The body position information determination unit is used to determine the body position change information of the target object during radiotherapy based on the difference between the motion surface and the reference surface.

Images