An augmented reality simulation system and method integrated with adepth-sensing device for modeling radiographic positioning and exposure parameters

Abstract

The invention relates to a depth-sensing-device-integrated augmented reality simulation system and method for modeling radiographic positioning and exposure parameters. This system comprises a depth-sensing device, interface, server, database, X-ray tube, and position database, enabling the simultaneous transfer of an individual's movements in a real environment into 3D models in a virtual X-ray room. By matching the individual's position in the real environment with information stored in the position database, the system determines correct imaging positions. The system is then integrated into a real X-ray machine, allowing X-rays to be sent to the patient only when the correct positions are achieved, thereby improving the accuracy of radiographic imaging.

Description

[0001] AN AUGMENTED REALITY SIMULATION SYSTEM AND METHOD INTEGRATED WITH A DEPTH-SENSING DEVICE FOR MODELING RADIOGRAPHIC POSITIONING AND EXPOSURE PARAMETERS

[0002] Field of invention:

[0003] The invention relates to a depth-sensing-device-integrated augmented reality (AR) simulation system and method for modeling radiographic positioning and exposure parameters. This system comprises a depth-sensing device, interface, server, database, X-ray tube, and position database, enabling the simultaneous transfer of individuals' movements in a real environment into 3D models in a virtual X-ray room. By matching the individual's position in the real environment with the position information stored in the database, the system determines correct imaging positions. The system is then integrated into a real X-ray machine, allowing X-rays to be sent to the patient only when the correct positions are achieved, thereby improving the accuracy of radiographic imaging.

[0004] Background of invention:

[0005] In recent years, the use of Al-assisted virtual, augmented, and mixed reality applications in education has been rapidly expanding, particularly at the international level. In specialized fields such as medical education, scenario-based simulations, especially those focusing on medical imaging techniques and radiographic positioning, are showing significant development. In traditional educational methods, students first receive theoretical knowledge and then gain experience through practical application whenever possible. However, AR technology offers these two stages in parallel, making educational processes more efficient by supporting practical application opportunities. Medical imaging technicians and students can experience patient positioning in a virtual environment and optimize imaging parameters thanks to AR simulations. This allows students to instantly identify and correct their mistakes, resulting in a faster learning process.

[0006] Radiographic positioning is a critical process for obtaining accurate medical images. In this process, the correct positioning of both the patient and the equipment directlyaffects the quality of the images. However, positioning errors made by inexperienced users can lead to degradation in image quality and result in misdiagnosis.

[0007] In recent years, AR has become more integrated with medical imaging techniques. Technical details such as imaging parameters, patient doses, and device angles are vital for obtaining an accurate image. Augmented reality-assisted simulations enable the precise adjustment of these parameters. Users experiment with various parameter settings in a virtual environment, refining their approach to achieve optimal results. This is highly beneficial for medical imaging techniques students, providing a more effective learning process that combines theoretical knowledge with practical application.

[0008] One of the primary advantages of using AR in education is its cost efficiency and accessibility, whereas traditional simulation systems generally require expensive equipment. For example, some companies offer virtual reality applications with annual access fees; however, these fees typically require a high budget.

[0009] In conclusion, the integration of AR technologies into medical imaging education emerges as a significant innovation. It is clear that these technologies, particularly the modeling of radiographic positioning and imaging parameters, improve the accuracy of medical imaging and make education processes more efficient. Augmented reality-assisted systems are expected to contribute to improving the quality of healthcare services, leading to more accurate diagnoses and enhanced patient care.

[0010] Although various proposals and implementations have been developed for X-ray simulation systems in the prior art, these developments remain insufficient. Some applications relating to inventions developed for this purpose are given below.

[0011] The patent application numbered JP2021043106A relates to a method of image generation based on a simulated member for conducting radiation transmission tests. This method aims to evaluate the accuracy and effectiveness of measurements taken via a radiation transmission testing device by generating simulated test data that can replace real tests. Furthermore, this system includes validation, selection, and evaluation phases, thus enabling more efficient and reliable testing processes.Using training data generation methods to prepare and analyze test data enables tests to be conducted more quickly and cost-effectively.

[0012] The patent application numbered US10568602B2 describes an application involving the acquisition of quantitative data using sensors and cameras to accurately position the X-ray source or detector and monitor the patient's position. The obtained data can be compared with anatomical atlases to ensure accurate positioning according to the patient's body structure. Sensors and cameras are used to determine the patient's anatomical regions. Collimators in the X-ray machine system ensure that X-rays are directed only to the area to be imaged, based on the determined area information. This method is a technological approach that aims to perform medical imaging with reduced radiation exposure. In the current state of the art, the following are required: i) development of a "software development kit" for the Azure Kinect (AK) device, ii) a position database in which the correct parameters for radiographic examinations are determined in the software, and ii) an AK integrated AR simulation (RdntgenXSim) system and method that models radiographic positioning and imaging parameters, including the ability to process radiological images.

[0013] In conclusion, due to the aforementioned drawbacks and the inadequacy of existing solutions, an improvement in the relevant technical field is necessary.

[0014] Aim of the invention:

[0015] The primary aim of the invention is to integrate radiographic positioning information into the X-ray room using an AR environment, thereby ensuring accurate imaging in clinical settings. The aim is to create a more efficient and safer radiographic imaging process by enabling object-environment interaction with the depth-sensing device.

[0016] Another aim of the invention is its application in medical imaging education. The invention, which is continuously improved based on user feedback, enhances the effectiveness of education by assisting radiography students in learning correct radiographic positioning.

[0017] Another aim of the invention is to offer a more comprehensive and accessible solution by employing a more cost-effective depth-sensing device instead ofexpensive equipment, thereby offering an economical option for educational institutions and clinics.

[0018] Another aim of the invention is to minimize radiation dose by ensuring correct positioning on the first attempt, thereby creating a safer process in clinical environments.

[0019] Another aim of the invention is to provide high-quality radiographic images free of artifacts, enabling more precise assessments and informed clinical decisions. This enables patients to receive more effective treatment.

[0020] Another aim of the invention is to enable patients to receive faster treatment by facilitating quicker and more accurate images in clinical settings through the integration of this technology into the X-ray room.

[0021] Explanation of the figures:

[0022] Figure 1: The drawing illustrates the design of 'An Augmented Reality Simulation System Integrated With A Depth-Sensing Device For Modeling Radiographic Positioning And Exposure Parameters'.

[0023] Reference numbers:

[0024] 1: Depth-Sensing Device

[0025] 2: Interface

[0026] 3: Server

[0027] 4: Database

[0028] 5: X-ray tube

[0029] 6: Position Database

[0030] Descriptiion of the invention:The invention is an AR simulation system and method that models radiographic positioning and exposure parameters, and integrates a depth-sensing device. This system includes a depth-sensing device (1), interface (2), server (3), database (4), X-ray tube (5), and position database (6), enabling the simultaneous transfer of the movements of individuals in a real environment into 3D models in a virtual X-ray room. By matching the individual's position in the real environment with the position information stored in the database, the system determines the correct imaging positions. This system is then integrated into a real X-ray machine, allowing X-rays to be sent to the patient only when the correct positions are achieved, thereby improving the accuracy of radiographic imaging.

[0031] The depth-sensing device (1) is an Azure Kinect device configured to detect patient movement and positioning and to transmit this information to the interface (2). The interface (2) is configured to display the patient's movements and positions detected by the Azure Kinect device. The server (3) is configured to enable the processing and analysis of digital images using artificial intelligence, as well as secure operation and data sharing over a network. The database (4) is configured to store data. The X-ray tube (5) is configured to emit X-rays toward an area to be imaged in accordance with data received from a computer.

[0032] The invention provides users with real-time guidance, enabling the selection of correct angles and distances. This reduces error rates in clinical settings and improves the quality of the images obtained. The invention, which employs AR technology integrated with depth-sensing devices, can be configured to serve multiple users simultaneously at a lower cost. The one-time installation costs of AK devices are more affordable than those of traditional systems, thereby providing significant cost advantages for educational institutions.

[0033] The use of the invention, supported by AR and artificial intelligence technologies, in education is expected to: i) contribute to the quality of life of radiology technician candidates by preventing unnecessary radiation exposure during their practical training; ii) contribute to the quality of life of the community by providing more competent healthcare services through the training of qualified radiology technicians; and iii) contribute to the improvement of the education level through a higher quality associate degree program.The invention utilizes a depth-sensing device (1) and its Body Tracking SDK to capture the patient's body position data in real-time. This data is processed into a position database supported by advanced artificial neural networks (ANN) and additional anatomical points.

[0034] This position database (6) is created by defining parameters required for correct positioning during a scan, including patient orientation in anterior-posterior or lateral directions, angular positioning (including orientations of approximately 30°-45°) in standing or supine positions, distance from the X-ray tube, and the positions and angles of the hands, arms, and feet. These definitions are established for at least 90 radiographic examinations.

[0035] The invention enables the completion of the simulation process steps and subsequent integration into a real X-ray machine, allowing X-rays to be delivered to the patient. This approach improves the accuracy of radiographic images, facilitating the development of more effective and accurate diagnostic methods.

[0036] The invention enables accurate imaging in clinical environments by integrating radiographic positioning information into the X-ray room with an AR environment. The invention also enables interaction between objects and environments using a depthsensing device (1), creating a more efficient and safer radiographic imaging process.

[0037] The invention is utilized in the field of medical imaging education and is continuously being improved based on user feedback. This enhances the effectiveness of education and assists radiography students in learning correct positioning techniques.

[0038] The invention offers a more comprehensive and accessible solution by employing a cost-effective depth-sensing device (1) instead of expensive equipment, thereby offering an economical option for educational institutions and clinics.

[0039] The invention ensures accurate positioning on the first attempt, minimizing radiation dose and creating a safer process in clinical environments. Integration into X-ray rooms enables faster and more accurate imaging, allowing patients to receive quicker treatment.The invention enables more precise assessments and informed clinical decisions by providing high-quality radiographic images free of artifacts. This, in turn, enables more effective treatment of patients.

[0040] A depth-sensor-integrated augmented reality simulation method, which models radiographic positioning and imaging parameters, includes the following steps:

[0041] • Simulation created by the developer,

[0042] • Integration of the simulation created by the developer into the X-ray room via computer and Azure Kinect device integration,

[0043] • Implementation of the simulation integrated into the X-ray room by the developer.

[0044] The process step of “simulation created by the developer” for the Azure Kinect integrated augmented reality simulation system, which models radiographic positioning and imaging parameters, includes the following steps:

[0045] • using a real-time 3D development platform and 3D modeling and animation software, the virtual environment is constructed by designing the patient table, X-ray tube, control panel, and X-ray imaging equipment in the virtual room, and by generating six adult patient models — three male and three female — classified as underweight, normal weight, and overweight according to World Health Organization data;

[0046] • transferring 3D models created with 3D modeling and animation software to a virtual environment; and

[0047] • creating a patient record module for entering patient data (weight, height, age, gender), a radiographic protocol module for selecting and applying imaging protocols, an examination selection module for choosing which examination to perform, an imaging module for performing X-rays, and an archive module for recording and monitoring imaging results.

[0048] The "implementing the simulation integrated into the X-ray room" procedure, which is a process step for the Azure Kinect integrated augmented reality control system that models radiographic rolling and imaging, includes the following steps:• admitting the patient to the imaging room and information including procedure name, height, weight, age, and gender into a patient record module via an interface;

[0049] • calculating body mass index values via the interface;

[0050] • displaying, via the interface, one of six different adult patient models selected in accordance with the body mass index;

[0051] • selecting an imaging method by the system or the user;

[0052] • positioning the patient by the user;

[0053] • adjusting an imaging position;

[0054] • verifying, by an artificial intelligence, that the patient position corresponds to data stored in a position database in which positions are defined

[0055] • retrieving a corresponding radiograph image from an image archive module comprising radiographs for six different models and at least 90 radiographic examinations; and

[0056] • displaying the retrieved radiograph image via the interface.

[0057] Within the process step of the Azure Kinect integrated augmented reality simulation system, which models radiographic positioning and imaging parameters, and which is integrated into the X-ray room, the 'selection of the imaging method by the system or user' process step involves selecting the desired radiographic examination from the examination selection module and selecting the predefined parameters.

[0058] The process step of "implementation of the simulation integrated into the X-ray room by the developer" sub-step, "the patient is positioned by the user", for the Azure Kinect integrated augmented reality simulation system, which models radiographic positioning and imaging parameters, includes the following steps:

[0059] • positioning the patient in front of the depth-sensing device;

[0060] • processing data received by the depth sensing device on a server and transferring a 3D patient model to a virtual environment; and • simulating the patient model as a 3D avatar in the virtual environment.The process step of "implementation of the simulation integrated into the X-ray room by the developer" sub-step, "the imaging position is adjusted", for the Azure Kinect integrated augmented reality simulation system, which models radiographic positioning and imaging parameters, includes the following steps:

[0061] • a user simultaneously monitoring a patient's real-time position in a virtual environment;

[0062] • the user viewing anatomical details, including a musculoskeletal system and internal organ structures via an avatar;

[0063] • the user moving, rotating, and zooming the avatar to adjust a correct imaging position and angle; and

[0064] • adjusting a distance and angle relative to an X-ray tube.

[0065] The process step of " implementation of the simulation integrated into the X-ray room by the developer" sub-step, "using artificial intelligence, checking that the patient's position matches the data in the position database where positions are defined ", for the Azure Kinect integrated augmented reality simulation system, which models radiographic positioning and imaging parameters, includes the following steps:

[0066] • determining, by an artificial intelligence, whether the patient’s position corresponds to predefined data in the position database, including patient posture (standing, sitting, or lying), imaging orientation (anterior, posterior, or lateral), imaging angles, and positions and angles of body limbs, including hands, arms, and feet;

[0067] • when the patient’s position does not correspond to the predefined data, generating an alert and providing audio and / or visual guidance to correct the patient’s position, and upon correction, activating an “Imaging Possible” indication;

[0068] • when the patient’s position corresponds to the predefined data, directly activating the “Imaging Possible” indication; and

[0069] • adjusting the X-ray tube to a corresponding position and angle, activating the X-ray tube, and performing the scan.

Claims

CLAIMS1. The device comprises an integrated augmented reality simulation system for modeling radiographic positioning and imaging parameters using a depthsensing device characterized by:• a depth sensing device (1) that detects patient movements and position and transmits the detected data to an interface;• an interface (2) that displays the patient movements and positions detected by the depth sensing device (1);• a server (3) that enables the processing and analysis of digital images with artificial intelligence, secure operation, and data sharing over the network;• a database (4) where the data is stored;• an X-ray tube (5) that sends X-rays to the area to be imaged by adjusting the correct position and angle with the data coming from the computer;• a position database (6) that contains all parameters, such as the patient's standing or lying position, front, back, or lateral orientation, distance to the X-ray tube (5), and the position and angle of their hands, arms, and feet.

2. A depth-sensing-device-integrated augmented reality simulation system that models radiographic positioning and imaging parameters according to Claim 1, characterized in that the depth-sensing device (1) contains an Azure Kinect device.

3. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters, characterized in that it comprises the following steps:• creation of the simulation by the developer;• integration of the simulation created by the developer into the X-ray room through computer and depth-sensing-device integration;• application of the simulation integrated into the X-ray room by the developer.

4. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3, characterized in that the method comprises the following features, and wherein process steps for creating the simulation, performed by a developer, include:• creating the basis of the virtual environment with a real-time 3D development platform and 3D modeling and animation software, and creating the device equipment and six different adult patient models; • transferring the 3D model created with 3D modeling and animation software to the virtual environment;• creating the patient record module, radiographic protocol module, examination determination module, imaging module, and archive module.

5. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the virtual environment is formed using a realtime 3D development platform and 3D modeling and animation software, and in that the simulation creation process includes the use of 3D modeling and animation tools, such as Blender, to create the equipment, as well as six different adult patient models.

6. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the virtual environment is formed using a realtime 3D development platform and 3D modeling and animation software, and in that the simulation creation process includes generating radiographic equipment comprising a patient table, X-ray tube, control panel, and X-ray positioning device, as well as six different adult patient models.

7. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the virtual environment is formed using a real-time 3D development platform and 3D modeling and animation software, and in that the simulation creation process includes a total of six different adultpatient models (3 men and 3 women) categorized as underweight, normal weight, and overweight, in accordance with World Health Organization data.

8. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the simulation creation process includes the creation of a patient record module in which patient data such as weight, height, age, and gender are entered, and further includes the creation of a radiographic protocol module, examination determination module, imaging module, and archive module.

9. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the simulation creation process includes the creation of a radiographic protocol module in which imaging protocols are selected and applied, and further includes the creation of a patient record module, radiographic protocol module, examination selection module, imaging module, and archive module.

10. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the simulation creation process includes the creation of an examination selection module in which the examination to be performed is selected, and further incudes the creation of a patient record module, radiographic protocol module, examination selection module, imaging module and archive module.

11. A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the simulation creation process includes the creation of an imaging module in which an X-ray image is taken and further includes the creation of a patient record module, radiographic protocol module, examination determination module, imaging module, and archive module.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 4, characterized in that the simulation creation process includes the creation of an archive module in which imaging results are recorded and monitored, and further includes the creationg of a patient record module, radiographic protocol module, examination determination module, imaging module, and archive module.A depth-sensing-device-integrated augmented reality simulation method that models the radiographic positioning and imaging parameters in accordance with Claim 3, characterized in that the simulation is integrated into an X-ray room by the developer, and in that the application of the simulation comprises the following steps:• introducing a patient into an imaging room and entering imaging information including height, weight, age, and gender into a patient record module via an interface;• calculating body mass index values via the interface;• displaying, via the interface, one of six different adult patient models selected in accordance with the body mass index;• selecting an imaging method by the system or the user;• positioning the patient by the user;• adjusting an imaging position;• verifying, by an artificial intelligence, that the patient position corresponds to data stored in a position database in which positions are defined• retrieving a corresponding radiograph image from an image archive module comprising radiographs for six different models and at least 90 radiographic examinations; and• displaying the retrieved radiograph image via the interface.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that during application of the simulation, imaging information and patient data including height, weight, age, and gender areentered into a patient record module via an interface when the patient is brought into the imaging room.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that the method is configured to retrieve radiographs from an image archive module containing radiographs for radiographic examinations, and includes six different models and at least 90 radiographic examinations.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that an imaging method selection step performed by a system or user involves selecting a desired radiographic examination from an examination determination module and selecting predefined imaging parameters associated with the selected examination.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that a patient positioning step performed by a user comprises the following steps:• positioning the patient in front of the depth-sensing device;• processing data received by the depth sensing device on a server and transferring a 3D patient model to a virtual environment; and• simulating the patient model as a 3D avatar in the virtual environment. A depth-sensing-device-integrated augmented reality simulation method that models the radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that an imaging position adjustment process step comprises:• a user simultaneously monitoring a patient's real-time position in a virtual environment;• the user viewing anatomical details, including a musculoskeletal system and internal organ structures via an avatar;• the user moving, rotating, and zooming the avatar to adjust a correct imaging position and angle; and• adjusting a distance and angle relative to an X-ray tube.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that the method comprises the following steps:• checking, using artificial intelligence, whether a patient's position matches data stored in a predefined position database;• providing a warning when the patient's position does not match the data in the predefined position database, providing audio / textual guidance, and activating an "Imaging Ready" message when correct imaging parameters are determined according to the guidance, or activating the "Imaging Ready" message when the patient's position matches the data in the predefined position database; and• adjusting an X-ray tube to a correct position and angle according to data received from a computer, operating the X-ray tube, and performing imaging.A depth-sensing-device-integrated augmented reality simulation method that models radiographic positioning and imaging parameters according to Claim 3 or Claim 13, characterized in that the method provides a warning when a patient's position does not match data in a predefined position database, provides audio / textual guidance, and activates an "Imaging Ready" message when correct imaging parameters are determined according to the guidance, or activates the "Imaging Ready" message when the patient's position matches the data in the defined position database, wherein the process step includes data relating to standing, sitting, or lying positions, front, back, or lateral imaging angles, and positions and angles of body limbs including hands, arms, and feet.

Images