Synthetic data generation for training artificial intelligence models
The system generates synthetic data using 3D models to train AI models for camera-based imaging, overcoming data acquisition challenges and enabling efficient training of AI models for patient positioning and alignment, thus reducing time and resource consumption.
Patent Information
- Application Number
- PCT/US2025/038361
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-30
- Filing Date
- 2025-07-18
- Publication Date
- 2026-03-05
AI Technical Summary
Acquiring extensive image data for training deep learning models in camera-based imaging workflows is impractical due to efforts involved, privacy concerns, regulatory restrictions, and unavailability of certain populations or medical imaging accessories, limiting the variety and realism of training data.
A system and method for generating synthetic data using 3D models of humans and medical imaging accessories in different poses and environments, employing rendering and training components to create realistic images for training deep learning models to detect patient positioning and alignment with medical equipment.
Enables efficient training of AI models for camera-based imaging workflows without physical setups, reducing time and resource consumption, addressing privacy and regulatory issues, and simulating diverse patient demographics and accessory interactions.
Smart Images

Figure US2025038361_05032026_PF_FP_ABST
Abstract
Description
SYNTHETIC DATA GENERATION FOR TRAINING ARTIFICIAL INTELLIGENCE MODELSCROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to India Provisional Patent Application No. 202441065493 filed on August 30, 2024, entitled “SYSTEM AND METHOD FOR SYNTHETIC DATA GENERATION FOR A CAMERA-BASED IMAGING WORKFLOW.” The entireties of the aforementioned application are incorporated by reference herein.TECHNICAL FIELD
[0002] This application relates to the field of generating data for developing artificial intelligence (Al) applications, and more particularly to systems and methods for synthetic data generation for camera-based imaging workflows.BACKGROUND
[0003] Camera-based imaging workflows involve the automatic detection of human anatomical regions (e.g., head and neck, spine, chest, etc.), imaging poses (e.g., supine, prone, lateral, sitting, standing, etc.), and imaging workflow related accessories (e.g., magnetic resonance imaging (MRI) coils, electrocardiogram (ECG) leads, pads, etc.) via deep learning models. Training a deep learning models to perform such automatic detection can involve generating a large variety of images such as images displaying patients in different imaging poses and orientations, images displaying patients with different medical conditions (e.g., missing limbs, extreme scoliosis, etc.), images displaying relative placements of medical imaging accessories with respect to the patients and the deformations that the medical imaging accessories undergo as a result of the interactions with the subjects.However, acquiring such extensive image data can be impractical due to the effort involved in the process, privacy concerns, unavailability of image data for certain population types (e.g., pediatric patients), regulatory restrictions, the physical unavailability of certain types of medical imaging accessories, and so on.SUMMARY
[0004] The following presents a summary to provide a basic understanding of one or more embodiments of the invention. This summary is not intended to identify key or critical elements or delineate any scope of the different embodiments or any scope of the claims. Itssole purpose is to present concepts in a simplified form as a prelude to the more detailed description that is presented later. In one or more embodiments described herein, systems, computer-implemented methods, apparatus and / or computer program products are described that facilitate adaptation of Al models across a range of computer vision applications and use cases in the healthcare domain.
[0005] According to an embodiment, a system is provided. The system can comprise a memory that can store computer-executable components. The system can further comprise a processor that can execute the computer-executable components stored in the memory, where the computer-executable components can comprise an execution component that can execute a rendering queue comprising a set of imaging scenes representing different medical imaging setups, where execution of the rendering queue can generate a set of finished images. The computer-executable components can further comprise a training component that can train a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, where the deep learning model can be employable to detect patient positioning within a camera-based imaging workflow. The computer-executable components can further comprise an imaging component that can employ the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0006] According to another embodiment, a computer-implemented method is provided. The computer-implemented method can comprise executing, by a system operatively coupled to a processor, a rendering queue comprising a set of imaging scenes representing different medical imaging setups, where the executing can generate a set of finished images. The computer-implemented method can further comprise training, by the system, a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, where the deep learning model is employable to detect patient positioning within a camera-based imaging workflow. The computer-implemented system can further comprise employing, by the system, the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0007] According to yet another embodiment, a computer program product for synthetic data generation for artificial intelligence (Al) models employable in camera-based imaging workflows is provided. The computer program product can comprise a non- transitory computer readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to execute a renderingqueue comprising a set of imaging scenes representing different medical imaging setups, where execution of the rendering queue can generate a set of finished images. The program instructions can be further executable by the processor to cause the processor to train a deep learning model by employing the set of finished images as ground truth data for training the deep learning model, where the deep learning model is employable to detect patient positioning within a camera-based imaging workflow. The program instructions can be further executable by the processor to cause the processor to employ the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.BRIEF DESCRIPTION TO THE DRAWINGS
[0008] One or more embodiments are described below in the Detailed Description section with reference to the following drawings:
[0009] FIG. 1 illustrates a block diagram of an example, non-limiting system that can generate synthetic data for training Al models employable in camera-based imaging workflows, in accordance with one or more embodiments described herein.
[0010] FIG. 2 illustrates another block diagram of an example, non-limiting system that can generate synthetic data for training Al models employable in camera-based imaging workflows, in accordance with one or more embodiments described herein.
[0011] FIG. 3 illustrates example, non-limiting image sets that show images of different articulating humanoids, in accordance with one or more embodiments described herein.
[0012] FIG. 4 illustrates an example, non-limiting image set that shows images of different three-dimensional (3D) computer-aided design (CAD) models of medical imaging accessories, in accordance with one or more embodiments described herein.
[0013] FIG. 5 illustrates an example, non-limiting image set that shows images of a flexible MRI coil, in accordance with one or more embodiments described herein.
[0014] FIG. 6 illustrates example, non-limiting image sets that show images of different MRI coils and coil components, in accordance with one or more embodiments described herein.
[0015] FIG. 7 illustrates an example non-limiting image set that shows images of MRI scanners located within different imaging environments, in accordance with one or more embodiments described herein.
[0016] FIG. 8 illustrates diagrams of an example non-limiting medical imaging setup that shows different camera positions and viewing directions of virtual cameras employable to generate synthetic images, in accordance with one or more embodiments described herein.
[0017] FIG. 9 illustrates an example, non-limiting image set that shows an MRI coil positioned on different patients situated on an MRI table, in accordance with one or more embodiments described herein.
[0018] FIG. 10 illustrates a flow diagram of an example, non-limiting workflow that can simulate humans and medical imaging accessories, equipment and environments by employing 3D CAD models and 2D graphics, in accordance with one or more embodiments described herein.
[0019] FIG. 11 illustrates a block diagram of an example, non-limiting system employable for synthetic data generation for a camera-based imaging workflow, in accordance with one or more embodiments described herein.
[0020] FIG. 12 illustrates a flow diagram of an example, non-limiting method for synthetic data generation for a camera-based imaging workflow, in accordance with one or more embodiments described herein.
[0021] FIG. 13 illustrates flow diagrams of example, non-limiting methods that can generate and employ synthetic data to train Al models employable in camera-based imaging workflows, in accordance with one or more embodiments described herein.
[0022] FIG. 14 illustrates a block diagram of an example, non-limiting operating environment in which one or more embodiments described herein can be facilitated.
[0023] FIG. 15 illustrates an example networking environment operable to execute various implementations described herein.DETAILED DESCRIPTION
[0024] The following detailed description is merely illustrative and is not intended to limit embodiments and / or application or uses of embodiments. Furthermore, there is no intention to be bound by any expressed or implied information presented in the preceding Background or Summary sections, or in the Detailed Description section.
[0025] One or more embodiments are now described with reference to the drawings, wherein like referenced numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a more thorough understanding of the one or more embodiments. It isevident, however, in various cases, that the one or more embodiments can be practiced without these specific details.
[0026] In the following specification and the claims, reference will be made to a number of terms, which shall be defined to have the following meanings.
[0027] The singular forms “a”, “an”, and “the” include plural references unless the context clearly dictates otherwise.
[0028] As used herein, the terms “software” and “firmware” are interchangeable and include any computer program stored in memory for execution by devices that include, without limitation, mobile devices, clusters, personal computers, workstations, clients, and servers.
[0029] As used herein, the term “computer” and related terms, e.g., “computing device”, “computer system” “processor”, “controller” are not limited to integrated circuits referred to in the art as a computer, but broadly refers to at least one microcontroller, microcomputer, programmable logic controller (PLC), application specific integrated circuit, and other programmable circuits, and these terms are used interchangeably herein.
[0030] Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about” and “substantially”, are not to be limited to the precise value specified. In at least some instances, the approximating language may correspond to the precision of an instrument for measuring the value. Here and throughout the specification and claims, range limitations may be combined and / or interchanged, such ranges are identified and include all the sub-ranges contained therein unless context or language indicates otherwise.
[0031] As used herein, the terms “systems”, “devices” and “apparatuses are interchangeable and include components, sub-components, sub-systems that include, without limitation, the medical imaging devices.
[0032] Hospitals, doctors and paramedical staff are increasingly relying on digitally obtaining, processing, storing and retrieving the medical records of a subject (e.g., patient). The medical records are generated during investigations of the subject via methods and techniques including a variety of imaging techniques, documents such as pathology reports, scanning by employing various medical imaging systems like ultrasound, MRI (also known as magnetic resonance (MR)), computed tomography (CT) systems and other radiological investigations. Further, the medical records generated via the various investigations can bestored in different formats. For example, medical images are stored in a format known as Digital Imaging and Communications in Medicine (DICOM) that is different than the format employed for storing pathology reports. Imaging data forms a critical part of medical records, particularly in enabling Al-based analysis. For example, in medical imaging, AI- based deep learning techniques are widely employed for faster and accurate identification and analysis of features comprised within an image. Deep learning is a class of machine learning techniques that employs representation learning methods that allow a machine to ingest raw data and determine the representations needed for data classification. Deep learning ascertains structure in data sets by employing backpropagation algorithms. Deep learning machines can utilize a variety of multilayer architectures and algorithms. While machine learning, for example, involves an identification of features to be employed in training a network, deep learning can process raw data to identify features of interest without external identification. The raw data may be image data or video data obtained via medical imaging techniques such as ultrasound, X-ray, CT, MRI, etc.
[0033] Obtaining images of a subject by employing medical imaging equipment involves positioning the subject in an imaging position and employing medical imaging equipment (X-ray scanner, CT scanner, MRI machine / scanner, etc.) and medical imaging accessories to capture an image. For example, in MRI, a subject can be positioned on the table of an MRI machine, and a variety of imaging coils (also known as MRI coils, MR coils or body coils) can be wrapped around the subject. Further, the subject can be positioned on the MR table according to a scanning protocol, since proper alignment of the subject on the MR table and alignment of the imaging coils over the subject play a crucial role in obtaining high-quality MR images of the subject. Thereafter, a camera can be employed to detect positioning of the subject on MR table and to detect positioning of the MR coils around the body of the subject. That is, the camera can be employed to ensure proper positioning (e.g., accurate alignment) of the subject on the MR table and proper positioning (e.g., accurate alignment) of the MR coils around the body of the subject. For example, the camera can employ an artificial intelligence (Al) model (also known as an Al module) that can be trained on camera images of subjects generated via MRI scanning to detect alignment of a subject with respect to the MRI machine and to detect alignment of the imaging coils with respect to the subject. Medical imaging workflows that employ cameras are known as camera-based imaging workflows (also known as camera-assisted imaging workflows or camera- based / assisted image acquisition workflows). However, obtaining image data to generate Al models that can be employed within camera-based workflows can be challenging.
[0034] Specifically, camera-based imaging workflows involve the automatic detection of human anatomical regions, body poses, medical imaging accessories (e.g., MR coils, ECG leads, pads, respiratory elbows, etc.) related to different medical imaging workflows via Al models (e.g., machine learning models, deep learning models, etc.). Training a deep learning model by employing image data involves generating (e.g., via a camera-based imaging workflow or another technique) a large variety of images such as images that vary in terms of patients’ poses, orientations, medical conditions (e.g., a missing limb, extreme scoliosis, etc.), relative placements of medical imaging accessories with respect to patients, the deformation that a medical imaging accessory undergoes while touching a patient, etc. It may not always be possible to physically acquire image data that can capture variations applicable to different imaging modalities (e.g., CT, MRI, etc.) such as, for example, variations of the human body, MRI coils, the alignment of coils around a body and positioning of the body on an MR table in case of MRI, and so on. Moreover, it may not always be possible to acquire such images in a hospital setting to train a deep learning model for camera automation / camera-based automation. Mimicking a hospital setting for data collection can involve simulating a medical imaging setup along with human volunteers, and volunteer studies can limit the variety of data that can be collected in terms of patient demography, habitus, clothing, ethnicity, as well as imaging workflow artifacts such as variations in accessory design, shape, color, etc. Privacy concerns, unavailability of certain types of populations (e.g., pediatric patients, etc.), regulatory restrictions, and physical unavailability of different types of medical imaging accessories can further limit such a data collection effort.
[0035] In a similar imaging domain, computer vision technology can identify objects such as humans in the field of view of a camera to generate image data, and Al models can be developed by employing such image data. Obtaining images or videos via computer vision involves arranging the subjects to be imaged in desired positions, obtaining a valid consent from every subject and physically obtaining the images at a predetermined location. This method has shortcomings in terms of the physical environment (e.g., light, visibility, shadow, locations, availability of human beings at a particular location, etc.), and the amount of data that can be obtained by this method is limited.
[0036] Thus, a method of synthetic data generation that can address the above described restrictions in relation to data collection can be highly valuable and desirable within the medical imaging industry.
[0037] Various embodiments of the present disclosure can be implemented to produce a solution to these problems. Embodiments described herein include systems, computer-implemented methods, and computer program products that can generate and employ synthetic data to train Al models to automatically detect patient positioning, human anatomical regions and medical imaging accessories within camera-based imaging workflows.
[0038] For example, in various embodiments, a synthetic data generation model is provided. The synthetic data generation model can comprise a data generation component, a scene generation component, a rendering component, an execution component, a training component and an imaging component. In one or more embodiments, the data generation component can generate a first library comprising first three-dimensional (3D) models (i.e., articulating humanoids) that can represent humans having different demographics (e.g., age, gender, etc.), different physical characteristics (e.g., physical builds, weights, heights, etc.), different appearances (e.g., hair styles, apparel, clothing, etc.), and assuming different imaging poses (e.g., supine, prone, decubitus, etc.). The data generation component can further generate, based on the first library, a second library comprising second 3D models that represent different medical imaging accessories (e.g., MRI coils (also known as MR coils), ECG leads, pads, etc.) associated with different medical imaging equipment and corresponding to different medical imaging modalities (e.g., MRI, CT, positron emission tomography (PET), X-ray, etc.), wherein the different medical imaging accessories can be represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models. The data generation component can further generate a third library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment. The data generation component can further generate a fourth library comprising two-dimensional (2D) graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects can be represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments can be represented as high dynamic range images (HDRIs).
[0039] In one or more embodiments, scene generation component can generate a set of imaging scenes based on the first library, the second library, the third library and the fourth library, wherein the set of imaging scenes can comprise multi-dimensional representations of the different medical imaging setups. To generate an imaging scene, scene generation component can first select a scanning protocol. Based on the scanning protocol, the scene generation component can generate a patient setup by selecting, according to the scanningprotocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library. Further, scene generation component can add, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings of virtual cameras employable to render the imaging scene, to the patient setup. The imaging scene thus generated can represent a first imaging scene comprising 3D CAD models of a human assuming an imaging pose defined by the scanning protocol, medical imaging accessories corresponding to the imaging pose, medical imaging equipment and a medical imaging environment suitable for the scanning protocol, 2D graphics that can add further details to the imaging scene, and virtual cameras defined by the scene generation component. Based on the first imaging scene, the scene generation component can generate additional scenes, for example, by varying parameters of the first imaging scene and / or by repeating the process of generating the first imaging scene.
[0040] In one or more embodiments, the rendering component can add the set of imaging scenes generated by scene generation component to a rendering queue.
[0041] In one or more embodiments, the execution component can execute the rendering queue to render the set of imaging scenes. As a result, a set of finished images can be generated.
[0042] In one or more embodiments, the training component can evaluate respective finished images comprised within the set of finished images for accuracy and quality. Based on the evaluation, training component can reject the finished images that do not meet defined accuracy levels and defined quality levels to generate ground truth data. Thereafter, training component can train a deep learning model on the ground truth data, wherein the deep learning model can be employed within a camera-based imaging workflow to automate the camera and capture images of patients.
[0043] In one or more embodiments, the imaging component can employ the deep learning model in the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment, detect anatomical regions and / or detect medical imaging accessories within the camera-based imaging workflow.
[0044] In one or more embodiments, one or more operations performed by the different components comprised within the synthetic data generation model can be automated to enable an automated synthetic data generation and model training workflow.
[0045] In various embodiments, elements described with reference to the above disclosed system can be embodied in different forms such as a computer-implementedmethod, a computer program product, or another form. Additional aspects and embodiments of the above-described system are detailed with reference to the drawings.
[0046] Contrary to existing techniques that employ synthetic data to train Al models within domains other than medical imaging, the embodiments of the present disclosure can employ synthetic data to train Al models that can enable camera automation within camerabased imaging workflows. Additionally, the embodiments of the present disclosure can be beneficially employed across different medical imaging modalities (e.g., MRI, CT, PET, X- ray, ultrasound, etc.) that employ cameras to assess patient setups and across product lines for different diagnostic radiology workflows. The embodiments of the present disclosure can provide benefits to customers and patients by generating a variety of training data employable to train Al models without relying on physical medical imaging setups, thereby conserving time and resources, potentially circumventing regulatory restrictions associated with volunteer categories such as pediatric populations that can be challenging to image, and simulating rare cases of body deformities without recruiting the appropriate subjects.Further, the methods and techniques described in the present disclosure can potentially address privacy issues, issues with data accumulation and preservation, and issues with data variety in connection with training Al models for camera-based imaging workflows. For example, the methods and techniques described in the present disclosure can generate image data capturing a variety of patient sizes and body shapes, medical imaging accessories (e.g., MR coils, ECG leads, pads, etc.) and medical imaging accessory shapes, medical imaging environments, different alignments of patients with respect to different medical imaging equipment (e.g., positioning of patients’ bodies over different medical imaging platforms), interactions of patients’ bodies with different medical imaging accessories etc., and the image data can be employed to generate a machine learning model (also known as machine learning module) for faster and efficient medical imaging workflows.
[0047] The embodiments depicted in one or more figures described herein are for illustration only, and as such, the architecture of embodiments is not limited to the systems, devices and / or components depicted therein, nor to any particular order, connection and / or coupling of systems, devices and / or components depicted therein. For example, in one or more embodiments, the non-limiting systems described herein, such as non-limiting system 100 as illustrated at FIG. 1, and / or systems thereof, can further comprise, be associated with and / or be coupled to one or more computer and / or computing-based elements described herein with reference to an operating environment, such as the operating environment 1400 illustrated at FIG. 14. For example, non-limiting system 100 can be associated with, such asaccessible via, a computing environment 1400 described below with reference to FIG. 14, such that aspects of processing can be distributed between non-limiting system 100 and the computing environment 1400. In one or more described embodiments, computer and / or computing-based elements can be used in connection with implementing one or more of the systems, devices, components and / or computer-implemented operations shown and / or described in connection with FIG. 1 and / or with other figures described herein.
[0048] FIG. 1 illustrates a block diagram of an example, non-limiting system 100 that can generate synthetic data for training Al models employable in camera-based imaging workflows, in accordance with one or more embodiments described herein.
[0049] Non-limiting system 100 and / or the components of non-limiting system 100 can be employed to use hardware and / or software to solve problems that are highly technical in nature (e.g., related to Al models in medical imaging, synthetic data generation, automating medical imaging workflows, etc.), that are not abstract and that cannot be performed as a set of mental acts by a human. Further, some of the processes performed may be performed by specialized computers for carrying out defined tasks related to synthetic data generation for training Al models. Non-limiting system 100 and / or components of nonlimiting system 100 can be employed to solve new problems that arise through advancements in technologies mentioned above, computer architecture, and / or the like. Non-limiting system 100 can provide technical improvements to camera-based imaging workflow systems that employ Al models by improving the speed of generating the Al models and by generating accurate and robust Al models.
[0050] For example, embodiments of the present disclosure can synthetically generate a variety of data employable to train Al models for camera automation within camera-based imaging workflows, wherein the data can be generated without employing physical medical imaging setups to perform volunteer scans and capture image data. For example, in one or more embodiments, different 3D models representing humans can interact with different 3D models of medical imaging accessories in different 3D medical imaging workflow environments comprising the different 3D models of the medical imaging accessories and of medical imaging equipment to generate data variations. The data variations can be employed to train robust Al models and algorithms that can enable automation within camera-based imaging workflows. Some example of such data variations can include, without being limited to, variations in 3D models of humans (e.g., in terms of demography, habitus, apparel, etc.), variations in 3D models of imaging accessories (e.g., in terms of structures, placements, imaging poses, appearances, etc.) and environmental variability (e.g., in terms of indoor areaappearances, lighting, distance from the camera, etc.). In addition to generation variations in the synthetic data, the process of data synthesis described in one or more embodiments can ensure that the synthetic data is realistic. Employing synthetic data to train Al models can obviate the need for volunteer studies and reduce the time and resources that can be otherwise consumed in generating the Al models. That is, model development times can be faster, which can reduce the time to launch models in the market.
[0051] In one or more embodiments, non-limiting system 100 can comprise system 102. Discussion turns briefly to processor 104, memory 106 and bus 108 of system 102. For example, in one or more embodiments, system 102 can comprise processor 104 (e.g., computer processing unit, microprocessor, classical processor, and / or like processor). In one or more embodiments, a component associated with system 102, as described herein with or without reference to the one or more figures of the one or more embodiments, can comprise one or more computer and / or machine readable, writable and / or executable components and / or instructions that can be executed by processor 104 to enable performance of one or more processes defined by such component s) and / or instruction(s).
[0052] In one or more embodiments, system 102 can comprise a computer-readable memory (e.g., memory 106) that can be operably connected to processor 104. Memory 106 can store one or more computer and / or machine readable, writable and / or executable components and / or instructions that, when executed by processor 104, can enable performance of one or more operations defined by such component(s) and / or instruction(s). For example, in one or more embodiments, memory 106 can store computer-executable instructions that, upon execution by processor 104, can cause processor 104 and / or one or more other components of system 102 (e.g., synthetic data generation model 110, data generation component 202, scene generation component 204, rendering component 206, execution component 208, training component 210 and imaging component 212) to perform one or more actions. In one or more embodiments, memory 106 can store computerexecutable components (e.g., synthetic data generation model 110, data generation component 202, scene generation component 204, rendering component 206, execution component 208, training component 210 and imaging component 212). In one or more embodiments, one or more of the components of system 102 can reside in the cloud, and / or can reside locally in a local computing environment (e.g., at a specified location(s)).
[0053] System 102 and / or a component thereof as described herein, can be communicatively, electrically, operatively, optically and / or otherwise coupled to one another via bus 108. Bus 108 can comprise one or more of a memory bus, memory controller,peripheral bus, external bus, local bus, and / or another type of bus that can employ one or more bus architectures. One or more of these examples of bus 108 can be employed. In one or more embodiments, non-limiting system 100 can be coupled (e.g., communicatively, electrically, operatively, optically and / or like function) to one or more external systems (e.g., a non-illustrated electrical output production system, one or more output targets, an output target controller and / or the like), sources and / or devices (e.g., classical computing devices, communication devices and / or like devices), such as via a network.
[0054] In one or more embodiments, system 102 can comprise synthetic data generation model 110. As illustrated in FIG. 2, synthetic data generation model 110 can comprise data generation component 202, scene generation component 204, rendering component 206, execution component 208, training component 210 and imaging component 212. In one or more embodiments, synthetic data generation model 110 can generate synthetic data that can be employed to train Al models for different tasks. In a non-limiting example, the synthetic data can be employed to train an Al model to automatically detect patient positioning, anatomical regions and medical imaging accessories within camera-based imaging workflows. For example, in one or more embodiments, execution component 208 can execute a rendering queue comprising a set of imaging scenes (i.e., the set of imaging scenes 130) representing different medical imaging setups, wherein execution of the rendering queue can generate a set of finished images (i.e., the set of finished images 132). In one or more embodiments, training component 210 can train deep learning model 134 by employing the set of finished images 132 as ground truth data to train deep learning model 134, wherein deep learning model 134 can be employed to detect patient positioning within a camera-based imaging workflow. In one or more embodiments, imaging component 212 can employ deep learning model 134 within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0055] Specifically, in one or more embodiments, data generation component 202 can generate first library 122 comprising first 3D models that represent humans having different demographics and assuming different imaging poses. Data generation component 202 can generate first library 122 to simulate patients with different demographics, body characteristics and appearances. For example, the first 3D models can be 3D CAD models that represent patients that vary in terms of genders, ages, stature (e.g., tall, medium, short), body mass index (BMI) values, physical builds (e.g., lean versus obese builds), facial appearances, hairstyles, accessories, clothing, eyewear (e.g., sunglasses). Additionally, the patients can be simulated as assuming different imaging poses defined by scanning protocols(e.g., standard scanning protocols such as brain MRI, CT Angiography (CTA), cardiac PET, spine series, etc.) corresponding to different medical imaging modalities (e.g., CT, MRI, PET, X-ray, etc.). In medical imaging, a scanning protocol (also known as an imaging protocol) is a predefined set of instructions that specify how to acquire medical images, including patient positioning, scan parameters, and imaging sequences. Accordingly, patient examinations are defined by scanning protocols. For example, a medical practitioner can recommend an MRI of the spine for a patient experiencing back pain. For spine imaging via MRI, the concerned MRI technician / operator can select a scanning protocol optimized for spine imaging, wherein the scanning protocol can define how the patient is to be positioned for the imaging, the MRI coil to be employed for the imaging, placement of the MRI coil, etc. Thus, in one or more embodiments, to generate synthetic data that can be employed to train an Al model for camera-based MRI workflows, 3D models of humans assuming imaging poses defined by the relevant scanning protocols are desirable.
[0056] In one or more embodiments, the first 3D models can be articulating humanoids. In the context of 3D CAD models, articulating humanoids are 3D human-shaped models with movable joints that simulate realistic body movements (e.g., rotating arms, bending knees) for design, simulation, or ergonomic analysis. As a result, data generation component 202 can generate a variety of different articulating humanoids by changing the pose and appearance of a single articulating humanoid. For example, data generation component 202 can position the hands, legs, head, etc. of an articulating humanoid in different poses followed by applying different visual characteristics to the visible areas of the articulating humanoid that represent external body parts, changing the apparel and clothing of the articulating humanoid, and so on. Thus, hundreds of combinations of 3D articulating humanoids simulating a variety of patients can be generated.
[0057] In one or more embodiments, data generation component 202 can further generate, based on first library 122, second library 124 comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories can be represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models. For example, the second 3D models can be CAD models of medical imaging accessories such as MRI coils, ECG leads, pads, respiratory elbows, patient positioning aids, radiation shields, gating devices, etc. employed in different medical imaging modalities (e.g., CT, MRI, PET, X-ray, etc.). Additionally, the medical imaging accessories can be simulated in their deformed states. For example, the second 3D models can comprisemultiple CAD models of an MRI coil in different deformed states depending on the patient that the MRI coil can be wrapped around. As illustrated in FIG. 9, a single MRI coil can assume five different shapes by being wrapped around five different patients. That is, the manner in which the MRI coil is wrapped around a patient, the portions of the MRI coil that remain visible after wrapping the MRI coil around the patient, and other such features can be a function of the patient since MRI coils can be flexible. Moreover, MRI coils are available in a variety of designs, and an MRI coil can be employed to scan different anatomical regions of a patient by placing the MRI coil on different parts of the patient’s body. Thus, second library 124 can comprise a large variety of CAD models of different medical imaging accessories in their deformed states. The CAD models comprised within second library 124 can also be articulating. In one or more embodiments, second library 124 can also comprise CAD models of the different medical imaging accessories in their undeformed states.
[0058] In one or more embodiments, data generation component 202 can employ first library 122 to generate second library 124. For example, upon generating the first 3D models that simulate a variety of patients assuming different imaging poses defined by different scanning protocols (i.e., imaging poses employable for image acquisition via different medical imaging modalities), data generation component 202 can map a CAD model of a medical imaging accessory to different CAD models comprised within first library 122. As a result, multiple CAD models of the medical imaging accessory in different deformed states can be generated. Additionally, the CAD models comprised within second library 124 can be based on different standard scanning protocols since the CAD models comprised within first library 122 and employed to generate second library 124 can be articulating humanoids assuming imaging poses defined by different standard scanning protocols.
[0059] In one or more embodiments, data generation component 202 can further generate third library 126 comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment. For example, the third 3D models can be CAD models of medical imaging equipment such as MRI scanners, CT scanners, etc. and the related environments. For example, the third 3D models can comprise CAD models of different brands and configurations of MRI equipment and the corresponding bays. In the context of MRI, a bay refers to a designated physical area or room that houses an MRI scanner and the related equipment.
[0060] A camera-based MRI workflow typically comprises cameras that can be directed at a patient to be imaged, and the patient can be positioned on the MRI bed.However, different camera-based MRI workflows can have variations in the surroundings of the MRI machine, lighting employed within the bay, a color temperature of the lights employed within the bay, other objects within the bay, etc. For example, FIG. 7 illustrates an MRI scanner and MRI table in different imaging environments. Thus, an Al model trained on CAD models that represent, for example, different MRI scanners within a single environmental setting can generate inaccurate predictions if employed within a camera-based MRI workflow being executed in a different environment setting. For example, if an AI- based computer vision algorithm is trained only on CAD models representing scenes such as that illustrated in the first image (on the top left comer) in FIG. 7, and the Al-based computer vision algorithm is employed within a camera-based MRI workflow at a hospital where the appearance of the bay and the MRI setup has some variations, the Al-based computer vision algorithm can fail to generate accurate predictions. Thus, in one or more embodiments, it is desirable to generate CAD models of different medical imaging devices in different imaging environments to ensure that an Al model employable within camera-based imaging workflows can be trained to detect variations in medical imaging equipment from different original equipment manufacturers (OEMs), the lighting, imaging environments and reflections on different surfaces surrounding different medical imaging equipment, and so on.
[0061] In one or more embodiments, data generation component 202 can further generate fourth library 128 comprising 2D graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects can be represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments can be represented as HDRIs. A bump map is a grayscale image that can be employed to simulate surface detail on a 3D model (e.g., 3D CAD objects) by altering the way light interacts with the surface of the 3D model, without changing the actual geometry of the 3D model. An HDRI image is an image with a wide range of brightness levels, employed in 3D rendering to create realistic lighting and reflections. HDRIs are 2D images that represent real-world 3D environments from which they are acquired. The 2D data can be generated by embossing the bump maps onto different 2D images to generate 3D appearances.
[0062] The 2D graphics comprised within fourth library 128 can be employed as backgrounds within the set of imaging scenes 130, and the set of imaging scenes 130 can be rendered to generate the set of finished images 132 that can be employed to train Al models for camera-based imaging workflows. For example, in one or more embodiments, 3D CAD models of imaging environments (e.g., a room with a Reinforced Cement Concrete (RCC)column, a door behind an MRI scanner, etc.) can be generated and the imaging environments can be rendered by adding the 3D CAD models into one or more imaging scenes comprised within the set of imaging scenes 130. The finished images generated upon rendering the set of imaging scenes 130 can be employed to train Al models for camera-based imaging workflows. However, in some scenarios, 3D CAD models of imaging environments may be unavailable or challenging to generate. In such scenarios, HDRI images of 3D medical imaging environments generated by data generation component 202 can be employed as background images within the one or more imaging scenes, creating the perception that the imaging environments are modeled via CAD to simulate the appearance of a 3D medical imaging environment. Similarly, in one or more embodiments, bump maps of 3D objects, combined with color application and texture mapping, can be employed to simulate 3D objects within the one or more imaging scenes. The HDRI images and bump maps can also be employed to enhance features associated with the 3D objects and 3D environments. In brand simulations, many of these elements can be manipulated within an object’s environment, allowing for a wide range of effects beyond the 3D CAD geometry of an object’s surface.
[0063] In one or more embodiments, data generation component 202 can automatically generate first library 122, second library 124, third library 126 and fourth library 128 based on input data 120, wherein input data 120 can comprise data having different data formats. For example, an end entity (e.g., hardware, software, machine, Al, neural network and / or user) can input image data (e.g., input data 120) comprising images of a woman taken from different angles into system 102 via the user interface (UI) of a computing device (e.g., desktop computer, laptop, tablet, smartphone, etc.). The image data can be accessed by data generation component 202 as a prompt based on which, data generation component 202 can automatically generate a 3D CAD model representing the woman and add the 3D CAD model to first library 122. Similarly, the end entity (e.g., hardware, software, machine, Al, neural network and / or user) can input text data (e.g., input data 120) comprising measurements of a medical imaging accessory into system 102 via the UI of the computing device (e.g., desktop computer, laptop, tablet, smartphone, etc.). Data generation component 202 can access the text data as a prompt based on which, data generation component can automatically generate a 3D CAD model of the medical imaging accessory and further generate additional 3D CAD models showing the medical imaging accessory in different deformed states, in accordance with one or more embodiments described herein. In general, data generation component 202 can automatically generate the3D CAD models comprised within first library 122, second library 124, and third library 126 and the 2D graphics comprised within fourth library 128 based on data having different formats (e.g., image data, text data, voice prompts, 3D scan data, structured metadata, patient profiles, etc.). In one or more embodiments, data generation component 202 can employ techniques and / or tools known in the art, such as CAD software and tools, to generate the 3D CAD models comprised within first library 122, second library 124, and third library 126, and to generate the 2D graphics comprised within fourth library 128.
[0064] In one or more embodiments, the CAD models comprised within first library 122, second library 124 and third library 126 as well as the 2D graphics comprised within fourth library 128 can be parametric, such that data generation component 202 can employ an existing CAD model or existing 2D graphics to generate a new CAD model or new 2D graphics, respectively. For example, a first CAD model of an MRI coil can have a parametric design, such that data generation component 202 can automatically modify the parameters of the first CAD model to generate a second CAD model of the MRI coil, wherein the second CAD model can represent the MRI coil in a different deformed state as compared to that represented by the first CAD model. In a different example, an articulating humanoid comprised within first library 122 representing a woman can be parametric such that data generation component 202 can automatically modify a few parameters of the articulating humanoid to generate a new articulating humanoid representing a man with a similar physical build as the woman. Similarly, data generation component 202 can automatically modify a 3D CAD model of a medical imaging accessory based on the articulating humanoid representing the woman and generate a new 3D CAD model of the medical imaging accessory applicable to the articulating humanoid representing the man by employing the physical measurements of the man to modify one or more parameters of the 3D CAD model of the medical imaging accessory. In general, data generation component 202 can perturb the structure and appearance of a 3D CAD model to generate a new 3D CAD model. In some embodiments, the parameters of the 3D CAD models of the medical imaging accessories can be tied to the imaging poses of the articulating humanoids comprised within first library 122 such that data generation component 202 can modify the parameters of the articulating humanoids to automatically scale the parameters of the corresponding 3D CAD models of the medical imaging accessories.
[0065] In one or more embodiments, scene generation component 204 can generate (e.g., compose) the set of imaging scenes 130, wherein the set of imaging scenes 130 can comprise multi-dimensional (e.g., 2D, 3D, etc.) representations of different medical imagingsetups. To generate an imaging scene comprised within the set of imaging scenes 130, scene generation component 204 can select a scanning protocol (e.g., a standard scanning protocol such as brain MRI, CTA, cardiac PET, spine series, etc.). Thereafter, scene generation component 204 can generate a patient setup by selecting, according to the scanning protocol, a 3D model of a human from first library 122 and one or more 3D models of medical imaging accessories from second library 124. For example, based on the imaging pose and the corresponding patient anatomy of the articulating humanoid selected by scene generation component 204, scene generation component 204 can select a matching medical imaging accessory from second library 124. Further, scene generation component 204 can add, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from third library 126, 2D graphics selected from fourth library 128 and camera locations and camera settings (i.e., of virtual cameras) employable to render the imaging scene, to the patient setup to generate the imaging scene. FIG. 8 illustrates camera positions and viewing directions of virtual cameras that can be defined by scene generation component 204 to generate the set of imaging scenes 130. The imaging scene thus generated can represent a first imaging scene comprised within the set of imaging scenes 130, and the first imaging scene can be employed to generate additional imaging scenes comprised within the set of imaging scenes 130. For example, scene generation component 204 can generate the additional imaging scenes by modifying one or more parameters or details associated with the first imaging scene. In one or more embodiments, to generate the set of imaging scenes 130, scene generation component 204 can position 3D CAD models of medical imaging accessories (e.g., MRI coils, ECG leads, pads, respiratory elbows, etc.) at various anatomical locations on the articulating humanoids to mimic the actual images / appearances of the medical imaging accessories when employed within camera-based imaging workflows.
[0066] In one or more embodiments, rendering component 206 can automatically add the set of imaging scenes 130 to the rendering queue. Stated differently, rendering component 206 can automatically generate the rendering queue based on the set of imaging scenes 130.
[0067] In one or more embodiments, upon generation of the rendering queue by rendering component 206, execution component 208 can automatically execute the rendering queue, wherein execution of the rendering queue can generate the set of finished images 132. In other words, execution component 208 can render the set of imaging scenes 130 to generate the set of finished images 132. In one or more embodiments, execution component 208 can employ techniques known in the art to render the set of imaging scenes 130. Forexample, execution component 208 can employ a rendering software connected to a render farm to execute the rendering queue. In one or more embodiments, execution component 208 can automatically save the set of finished images 132 in a portable network graphics (.png) format, a raw image data (.raw) format or other suitable formats in a storage system (e.g., memory 106 or another storage). The finished images comprised within the set of finished images 132 can be described as 2D projections of 3D data. In one or more embodiments, execution component 208 can also perform post-processing and / or photo-editing (e.g., to adjust the viewing direction, resolution, file formats, etc.) on respective finished images comprised within the set of finished images 132, prior to the set of finished images 132 being employed by training component 210 to train deep learning model 134.
[0068] In one or more embodiments, employing the set of imaging scenes 130 to generate the set of finished images 132 can eliminate a task of generating the ground truth data via a medical imaging device, thereby conserving time and resources and reducing model development times.
[0069] In one or more embodiments, the set of finished images 132 can comprise images in different formats, depending on the cameras defined by scene generation component 204 to compose the set of imaging scenes 130, and scene generation component 204 can simulate different types of cameras. For example, a normal visible light camera / Red, Green and Blue (RBG) camera can generate 2D color photographs, a depth sensing camera can generate depth profiles / 3D images of objects / estimates of depth maps instead of color photographs, resulting in a 3D point cloud of a 3D space (e.g., an imaging scene), an infrared camera can generate an infrared signature of objects, black light cameras can generate fluorescent black light images, etc. Thus, based on the camera identification (ID) (e.g., RGB cameras, depth cameras, infrared cameras, etc.) defined by scene generation component 204, the set of finished images 132 can comprise data or images having identical or different formats (e.g., 2D images, 3D images, infrared footprints, blue light images, fluorescent black light images, etc.). For example, in an embodiment, the set of finished images 132 can comprise only RGB images, only depth images, and so on, whereas in other embodiments, the set of finished images 132 can comprise a combination of images such as RGB images, depth images, and infrared images, infrared images and color images, or other combinations of images. In one or more embodiments, rendering component 206 can employ tools to interchange one or more finished images comprised within the set of finished images 132 from one format to another format to generate different types of images.
[0070] In one or more embodiments, the set of finished images 132 can drive the training of deep learning model 134. For example, in some embodiments, deep learning model 134 can be trained on different types of images, such that deep learning model 134 can employ color images to recognize objects, materials, etc. within a medical imaging setup. For example, deep learning model 134 can identify whether an object is a person, a body tissue, a synthetic part such as a mattress or MRI bed, and so on. For example, skin color within a color image can indicate to deep learning model 134 that an object captured by the color image is an exposed area of the human body versus a non-huma part. Similarly, deep learning model 134 can be trained on depth images, such that deep learning model 134 can identify the shapes of different 3D objects. Deep learning model 134 can also be trained on infrared images, such that deep learning model 134 can identify the body temperature of a patient or patient body movement within a camera-based imaging workflow, and so on. In some embodiments, deep learning model 134 can be trained on a combination of color images, depth images and infrared images or on another combination of different types of images.
[0071] In one or more embodiments, training component 210 can automatically train (e.g., generate, retrain, update, etc.) deep learning model 134 based on the set of finished images 132. To train deep learning model 134, training component 210 can evaluate respective finished images comprised within the set of finished images 132 for accuracy and quality, and training component 210 can reject, based on the evaluation, finished images comprised within the set of finished images 132 that do not meet defined accuracy levels and defined quality levels, to generate the ground truth data. For example, an end entity (e.g., hardware, software, machine, Al, neural network and / or user) can define, via the UI of a computing device (e.g., desktop computer, laptop, tablet, smartphone, etc.), accuracy levels (e.g., accuracy metrics or degrees of accuracy such as 95 percent (%) accuracy, etc.) and quality levels (e.g., quality metrics such as low resolution, realism etc.) employable by training component 210 to evaluate different finished images comprised within the set of finished images 132. Training component 210 can determine whether the accuracy and quality of a finished image are greater than or equal to the defined accuracy levels and defined quality levels corresponding to the finished image. If so, training component 210 can include the finished image in the ground truth data. If not, training component 210 can exclude the finished image from the ground truth data.
[0072] In one or more embodiments, training component 210 can also assess the accuracy and quality of different finished images comprised within the set of finished images132 (i.e., the rendered images) by evaluating whether the different medical imaging setups illustrated by the finished images are realistic, because in some scenarios, the set of finished images 132 generated by execution component 208 by rendering the set of imaging scenes 130 can comprise unrealistic images. Training component 210 can also evaluate other factors to further assess the quality of the different finished images. For example, due to the ambient lighting defined by scene generation component 204 for some of the imaging scenes comprised within the set of imaging scenes 130, the corresponding finished images comprised within the set of finished images 132 can display a high amount of glare or reflection on a patient’s body and the finished images may lack clarity. As previously described, training component 210 can exclude low-quality and unrealistic finished images from the ground truth data employable to train deep learning model 134 because including such finished images in the ground truth data can cause deep learning model 134 to perform inaccurately. For example, deep learning model 134 can fail to accurately detect MRI coils or patients being imaged if it is trained on poor quality ground truth data. Thus, training component 210 can perform a quality assurance step to discard poor quality data from the training data employable to train deep learning model 134. In one or more embodiments, upon detecting that certain finished images comprised within the set of finished images 132 have accuracy and quality below the defined accuracy and quality levels, training component 210 can provide feedback to scene generation component 204 and execution component 208 about the low-accuracy and low-quality finished images, and the feedback can be employed by scene generation component 204 and execution component 208 to generate imaging scenes and finished images of higher accuracy and quality.
[0073] Upon generating the ground truth data, training component 210 can train deep learning model 134 on the ground truth data. Stated differently, training component 210 can generate deep learning model 134 based on the ground truth data. As stated elsewhere herein, in one or more embodiments, deep learning model 134 can be employable to detect patient positioning within a camera-based imaging workflow. Deep learning model 134 can be further employable to automatically detect anatomical regions and medical imaging accessories within the camera-based imaging workflow. It should be appreciated that in one or more embodiments, the set of finished images 132 can be employed by training component 210 to train Al models other than deep leaning models, and different algorithms can be trained for different applications.
[0074] As stated elsewhere herein, in one or more embodiments, imaging component 212 can employ deep learning model 134 to automatically evaluate patient alignment, detectanatomical regions and / or detect medical imaging accessories within the camera-based imaging workflow. For example, in an embodiment, deep learning model 134 can be trained on finished images that illustrate hundred different MRI coils and twenty different patients. When employed by imaging component 212 within the camera-based imaging workflow, deep learning model 134 can automatically detect the MRI coils and imaging positions illustrated by the finished images as well as MRI coils and imaging positions approximately similar to those illustrated by the finished images. That is, deep learning model 134 can be trained to detect similarities between the finished images and real-life medical imaging scenarios, such that despite being trained on finished images based on specific 3D CAD models, deep learning model 134 can detect objects that can be slightly different from the 3D CAD models. Thus, deep learning model 134 can comprise built-in automation.
[0075] In one or more embodiments, an Al model (e.g., deep learning model 134) trained on finished images based on the 3D models comprised within first library 122, second library 124 and third library 126 as well as the 2D graphics comprised within fourth library 128 can detect a patient being imaged via a medical imaging equipment, recognize a posture of the patient, detect an examination setup, detect the medical imaging equipment and accessories employed within the corresponding camera-based imaging workflow, automatically identify whether a scanning protocol has been selected or which scanning protocol has been selected for the camera-based imaging workflow, and generate intelligent predictions that can enable the camera-based imaging workflow to be meaningfully automated. For example, deep learning model 134 can automate the camera (e.g., MRI camera, CT camera, etc.) within a camera-based imaging workflow, such that the camera can automatically capture images of patients once a desirable medical imaging setup is detected by deep learning model 134. In addition to automating image capture based on detected medical imaging setups, deep learning model 134 can also streamline other aspects of the camera-based imaging workflow. For example, deep learning model 134 can automatically verify patient positioning accuracy, suggest adjustments in real time, and trigger system readiness checks prior to image acquisition. Deep learning model 134 can also anticipate and preconfigure scanner settings, such as field of view, slice thickness, or contrast usage, based on recognized examination context. In some embodiments, deep learning model 134 can be employed within different camera-based imaging workflows for Al landmarking and positioning.
[0076] It should be appreciated that in one or more embodiments of the present disclosure, the synthetic data generated by synthetic data generation model 110 can refer to3D CAD models comprised within first library 122, second library 124, and / or third library 126, 2D graphics comprised within fourth library 128, the set of imaging scenes 130 and / or the set of finished images 132. Additionally, the synthetics data can be equivalent to that produced by a variety of medical imaging equipment. Additional embodiments of the present disclosure are discussed with reference to the subsequent figures.
[0077] FIG. 2 illustrates another block diagram of an example, non-limiting system 200 that can generate synthetic data for training Al models employable in camera-based imaging workflows, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0078] As described with reference to FIG. 1, synthetic data generation model 110 can comprise data generation component 202, scene generation component 204, rendering component 206, execution component 208, training component 210 and imaging component 212.
[0079] With continued reference to FIG. 1, in one or more embodiments, synthetic data generation model 110 can employ suitable image properties and data diversity to generate the synthetic data to ensure that the Al models trained on the synthetic data are robust. For example, in one or more embodiments, data generation component 202 can employ appropriate 3D modeling techniques to generate the articulating humanoids comprised within first library 122 and the 3D CAD models of medical imaging accessories comprised within second library 124 to ensure that the 3D CAD models of humans and medical imaging accessories assume the desired shapes and forms for the simulated medical imaging studies (i.e., the set of imaging scenes 130) prior to the set of imaging scenes 130 being rendered. For example, MRI coils can have flexible as well as inflexible configurations, and data generation component 202 can ensure that the flexible and inflexible MRI coils are modeled accordingly. For example, data generation component 202 can ensure that the 3D CAD models generated for inflexible MRI coils do not have deformed shapes. Likewise, data generation component 202 can accurately capture non-rigid deformations of medical imaging accessories according to their fit on the surface of the human body. In this regard, FIG. 4 illustrates the use of inflexible MRI coils, whereas FIG. 5 illustrates the use of flexible MRI coils.
[0080] In one or more embodiments, to achieve diversity of images within the set of finished images 132, scene generation component 204 can develop and employ a suitable inclusion-exclusion grid based on medical imaging studies and protocols (e.g., MRI studiesor protocols, CT studies or protocols, etc.) to enable the correct / suitable combinations of 2D / 3D representations of the patients and medical imaging accessories (e.g., MRI coils, ECG leads, pads, respiratory elbows, patient positioning aids, radiation shields, gating devices, etc.) within the set of imaging scenes 130 that can be collectively rendered or merged, via image processing, by execution component 208 to generate the set of finished images 132.
[0081] In one or more embodiments, scene generation component 204 and execution component 208 can employ defined variables to generate the set of imaging scenes 130 and execute the rendering queue comprising the set of imaging scenes 130, respectively. Table 1 below lists key variables that can define (e.g., for scene generation component 204) the diversity desired within the set of finished images 132 resulting from rendering the set of imaging scenes 130 as well as key variables that can define (e.g., for execution component 208) features, metrics, parameters, etc. for rendering the set of imaging scenes 130 and / or post-processing / photo-editing (e.g., to adjust the viewing direction, resolution, file formats, etc.) the set of finished images 132. In some embodiments, a table such as Table 1 can be input by an end entity (e.g., hardware, software, machine, Al, neural network and / or user) into system 102 via the UI of a computing device (e.g., desktop computer, laptop, tablet, smartphone, etc.), and system 102 can store Table 1 in a memory (e.g., memory 106). In other embodiments, variables such as those listed in Table 1 can be input to system 102 in a different format.
[0082] Table 1 :
[0083] FIG. 3 illustrates example, non-limiting image sets 300 and 310 that show images of different articulating humanoids, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0084] With continued reference to FIGS. 1 and 2, non-limiting image sets 300 and 310 illustrate examples of data synthesis that can be performed by synthetic data generation model 110. For example, as described with reference to FIGS. 1 and 2, in one or more embodiments, data generation component 202 can digitally simulate, based on different scanning protocols, humans assuming different imaging positions. For example, non-limiting image set 300 illustrates images of articulating humanoids representing a woman, and nonlimiting image set 310 illustrates images of articulating humanoids representing a man. The articulating humanoids illustrated in non-limiting image sets 300 and 310 were experimentally generated, based on input data 120, by data generation component 202.
[0085] In one or more embodiments, the articulating humanoids generated by data generation component 202 can be parametric, such that by varying one or more parameters of the articulating humanoids, data generation component 202 can generate new articulating humanoids. For example, in some embodiments, data generation component 202 cangenerate the respective articulating humanoids illustrated in non-limiting image set 300 based directly on input data 120. In other embodiments, data generation component 202 can generate any one articulating humanoid illustrated in non-limiting image set 300 and generate the remaining articulating humanoids illustrated in non-limiting image set 300 by varying parameters and / or surrounding effects of the articulating humanoid. In one or more embodiments, data generation component 202 can similarly generate the articulating humanoids illustrated in non-limiting image set 310. In one or more embodiments, data generation component 202 can generate the articulating humanoids illustrated in non-limiting image set 310 by varying certain parameters of one or more articulating humanoids illustrated in non-limiting image set 300, and vice versa. The articulating humanoids can display multiple imaging poses and reflect changes in ambient lighting, different appearances (e.g., in terms of demographic diversity, clothing, etc.), and so on. Thus, data generation component 202 can generate a variety of articulating humanoids representing humans.
[0086] Non-limiting images set 310 also illustrates 3D CAD representations of medical imaging accessories positioned around articulating humanoids to show how data generation component 202 can employ articulating humanoids to generate 3D CAD models of medical imaging accessories.
[0087] FIG. 4 illustrates an example, non-limiting image set 400 that shows images of different 3D CAD models of medical imaging accessories, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0088] With continued reference to FIGS. 1 and 2, non-limiting image set 400 illustrates additional examples of data synthesis that can be performed by synthetic data generation model 110. For example, as described with reference to FIGS. 1 and 2, data generation component 202 can digitally simulate, based on different scanning protocols, numerous (e.g., hundreds of) different 3D CAD models of medical imaging accessories in different undeformed as well as deformed (e.g., bent, folded, etc.) states, in different orientations and corresponding to different human postures to simulate how the medical imaging accessories are employed in practical scenarios. For example, non-limiting image set 300 illustrates 3D CAD models of inflexible MRI coils. In one or more embodiments, the 3D CAD models illustrated in non-limiting image set 400 were experimentally generated, based on input data 120, by data generation component 202. Data generation component 202 can similarly generate 3D CAD models of the flexible MRI coils illustrated in FIG. 5. The 3D CAD models of the different MRI coils (flexible and inflexible) can correspond tomultiple different poses assumed by humans and reflect changes in ambient lighting, different appearances (e.g., in terms of texture, colors, etc.), and so on. Thus, data generation component 202 can generate a variety of 3D CAD models of medical imaging accessories.
[0089] In one or more embodiments, the 3D CAD models of the medical imaging accessories generated by data generation component 202 can be parametric, such that by varying one or more parameters of the 3D CAD models, data generation component 202 can generate new 3D CAD models. For example, data generation component 202 can modify a 3D CAD model of a flexible MRI coil based on a lean patient and generate a new 3D CAD model of the flexible MRI coil that can correspond to an obese patient. Further, data generation component 202 can add the new 3D CAD model to second library 124.
[0090] FIG. 5 illustrates an example, non-limiting image set 500 that shows images of a flexible MRI coil, in accordance with one or more embodiments described herein.Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0091] With continued reference to FIG. 4, in non-limiting image set 500, image 502 illustrates different flexible MR coils, and images 504 - 518 illustrate different deformed shapes that MRI coils can assume based on the anatomical regions being imaged. For example, image 504 illustrates an MRI coil positioned for imaging the abdomen or the thorax region, image 506 illustrates an MRI coil positioned for imaging the spine region, image 508 illustrates an MRI coil positioned for imaging the shoulder region, image 510 illustrates an MRI coil positioned for imaging the elbow region, image 512 illustrates an MRI coil positioned for imaging the hip joint, image 514 illustrates an MRI coil positioned for imaging the knee and / or portions of the ankle, image 516 illustrates an MRI coil positioned for imaging the chest and / or shoulder regions, and image 518 illustrates an MRI coil positioned for imaging the knee, foot, ankle and / or toe regions.
[0092] FIG. 6 illustrates example, non-limiting image sets 600 and 630 that show images of different MRI coils and coil components, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0093] With continued reference to FIGS. 4 and 5, non-limiting image sets 600 and 630 illustrate images of MRI coils and MRI coil components that can be digitally simulated by data generation component 202. For example, in non-limiting image set 600, image 602 illustrates a head and neck coil, image 604 illustrates a head and neck open face coil adapter, image 606 illustrates a cervical spine coil, image 608 illustrates a tiltable head / neck coil,image 610 illustrates an MRI coil, image 612 illustrates an anterior body array coil, image 614 illustrates flexible surface coils, image 616 illustrates a posterior body array coil, image 618 illustrates a peripheral / vascular coil and image 620 illustrates another MRI coil. Nonlimiting image set 630 illustrates some of the MRI coils and coil components from nonlimiting image set 600 positioned on patients during MRI imaging.
[0094] FIG. 7 illustrates an example non-limiting image set 700 that shows images of MRI scanners located within different imaging environments, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0095] As described with reference to FIGS. 1 and 2, in one or more embodiments, data generation component 202 can generate, based on input data 120, 3D CAD models for different shapes, dimensions and configurations of the different medical imaging equipment and different medical imaging environments such as those illustrated in non-limiting image set 700, wherein input data 120 can comprise information about the physical dimensions of the medical imaging equipment, the brand name of the medical imaging equipment, physical dimensions of the medical imaging environments, ambient lighting, etc. Data generation component 202 can add the different 3D CAD models to third library 126.
[0096] In this regard, non-limiting image set 700 illustrates photographs of MRI setups in different indoor environments (e.g., hospitals, medical imaging centers, etc.). The different images comprised within non-limiting image set 700 illustrate a common MRI scanner and MRI table (also known as patient bed) located in different medical imaging environments. Although only one MRI scanner is illustrated in non-limiting image set 700, it is to be appreciated that different MRI scanners can display different features. For example, different MRI scanners can have different lighting on the gantry, such as edge lighting, ring lighting (i.e., a ring of light around the patient bed that glows), semi-circular lighting, etc. Additionally, one MRI scanner can display different features in different environments. For example, as illustrated in non-limiting image set 700, the gantry lighting on the MRI scanner can be turned on or off depending on the lighting within the medical imaging environment and the time of day. Similarly, different medical imaging environments can have different features. For example, different imaging environments can have different room lighting. For example, some of the images illustrated in non-limiting image set 700 show edge lighting on the ceilings, co-lighting on the ceiling over the patient bed, spot lighting on the ceiling, edge lighting or co-lighting around the periphery of the walls. Additionally, some medical imaging environments can have overhead displays that a patient can view, wooden finishes,colored / painted finished, views of the outdoors such as via windows or simulated outdoor environment visuals to improve patients’ experiences, different fixtures such as ceiling mounted fixtures, backlit / illuminated displays on the ceilings or the walls, etc.
[0097] In one or more embodiments, the different features and details (e.g., ceilings, walls, floor surfaces, textures and other appearances and variables) associated with different medical imaging equipment and environments can be captured by data generation component 202 when generating the 3D CAD models of the different medical imaging devices and the different medical imaging environments. As result, scene generation component 204 can generate a diverse set of imaging scenes (e.g., the set of imaging scenes 130).
[0098] FIG. 8 illustrates diagrams of an example non-limiting medical imaging setup 800 that shows different camera positions and viewing directions of virtual cameras employable to generate synthetic images, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0099] Non-limiting medical imaging setup 800 illustrates different views (view A and view B) of an MRI setup. With continued reference to FIGS. 1 and 2, cameras 802, 804, 806 and 808 can be different virtual cameras defined by scene generation component 204 to generate the set of imaging scenes 130. For example, camera 802 can be a virtual camera at a height of 3 meters (3 m) from the top surface of MRI table 810, and camera 804 can be a virtual camera at a height of 2.3 m from the top surface of MRI table 810. Similarly, camera 806 and cameras 808 can be additional virtual cameras at different heights and viewing angles. The different camera positions and viewing angles can generate a variety of imaging scenes based on different combinations of 3D CAD models of humans, medical imaging accessories, medical imaging equipment, medical imaging environments and 2D graphics of 3D objects and 3D environments.
[0100] FIG. 9 illustrates an example, non-limiting image set 900 that shows an MRI coil positioned on different patients situated on an MRI table, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0101] With continued reference to FIG. 8, non-limiting image set 900 illustrates patient images that can be captured by a real (as opposed to a virtual) camera located at the position of camera 802 in a medical imaging setup. Specifically, image 902, image 904, image 906, image 908 and image 910 illustrate photographs of patients with different demographics positioned on an MRI table according to the imaging poses defined by ascanning protocol for MRI, and an MRI coil is positioned on each patient to scan the abdomen and pelvic area. By defining camera 802 when generating the set of imaging scenes 130, scene generation component 204 can ensure that similar views of articulating humanoids are captured by the set of finished images 132 that can result from rendering the set of imaging scenes 130.
[0102] Further, as evident from images 902 - 910, different patients can have different demographics, genders, physical builds, appearances, hair styles, etc. As stated elsewhere herein, data generation component 202 can generate articulating humanoids having such a variety of features. Doing so, can assist scene generation component 204 in generating realistic imaging scenes and assist execution component 208 in generating naturallooking finished images employable to train deep learning model 134.
[0103] FIG. 10 illustrates a flow diagram of an example, non-limiting workflow 1000 that can simulate humans and medical imaging accessories, equipment and environments by employing 3D CAD models and 2D graphics, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0104] With continued reference to FIGS. 1 and 2, non-limiting workflow 1000 illustrates an automated workflow that can be executed by synthetic data generation model 110 to simulate data and employ the simulated data to train deep learning model 134 for a camera-based MR imaging workflow.
[0105] Non-limiting workflow 1000 can begin at 1002 where synthetic data generation model 110 can access input data 120. As stated elsewhere herein, input data 120 can be input to system 102 by an end entity (e.g., hardware, software, machine, Al, neural network and / or user) via the UI of a computing device (e.g., desktop computer, laptop, tablet, smartphone, etc.).
[0106] At 1004, data generation component 202 can automatically generate a first library (e.g., first library 122) comprising articulating humanoids (i.e., 3D CAD models of humans) to simulate MRI subjects (i.e., patients), their poses and their demographic diversity including their genders, ages, stature (e.g., tall / medium / short), BMI values, facial appearances, clothing, and several other features. In some embodiments, data generation component 202 can source (e.g., access) existing articulating humanoids previously generated by data generation component 202 and comprised within the first library to generate new articulating humanoids by varying parameters of the existing articulating humanoids. The variety of human features captured by the articulating humanoids thus generated can ensurethat deep learning model 134 can be employed in camera-based imaging workflows in different geographies, for patients with different facial appearances, hairstyles, clothing, etc. For example, hospitals in different countries across the world can employ different hospital gowns for patients or some hospitals can allow patients to be imaged in civilian clothing as opposed to hospital gowns. By employing a variety of data to train deep learning model 134, deep learning model 134 can be trained to generate accurate predictions within a variety of camera-based imaging workflows.
[0107] At 1006, data generation component 202 can automatically generate a second library (e.g., second library 124) comprising articulating 3D CAD models of MRI coils and MRI accessories to simulate diverse applications of MRI imaging including brain imaging, head-neck imaging, thorax / abdomen / pelvis imaging, hip / knee / ankle imaging, shoulder / elbow / wrist imaging as well as other types of MRI imaging (e.g., imaging involving breast coils) not listed here. In some embodiments, data generation component 202 can source (e.g., access) existing 3D CAD models of MRI coils previously generated by data generation component 202 and comprised within the second library to generate new 3D CAD models of MRI coils by varying parameters of the existing 3D CAD models. The variety of MRI coils captured by the 3D CAD models can ensure that deep learning model 134 can be employed for different use cases of camera-based MRI workflows. For example, in MRI imaging, a wide range of MRI coils can be employed for brain, head-neck imaging, thorax imaging and pelvis imaging. Additionally, some MRI coils can be multipurpose. For example, a single MRI coil can be employed for shoulder imaging, ankle imaging, etc.Depending on the anatomical region to be imaged as well as the scanning protocol, the same MRI coil can be positioned differently. By employing a variety of data to train deep learning model 134, deep learning model 134 can be trained to generate accurate predictions within a variety of camera-based MRI workflows.
[0108] At 1008, data generation component 202 can automatically generate a third library (e.g., third library 126) comprising 3D CAD models of MRI systems (e.g., MRI scanners / machines) and MRI bay designs to simulate the indoor settings and environments of different MRI imaging setups. In some embodiments, data generation component 202 can source (e.g., access) existing 3D CAD models of MRI systems and MRI bay designs previously generated by data generation component 202 and comprised within the third library to generate new 3D CAD models of MRI systems and MRI bay designs by varying parameters of the existing 3D CAD models. The variety of MRI systems and MRI bay designs captured by the 3D CAD models can further ensure that deep learning model 134 canbe employed for different use cases of camera-based MRI workflows. For example, the MRI gantry and MRI table, gantry facade, in-bore lighting, walls, ceilings, floor surfaces, room lighting, backlit / illuminated displays, etc. can vary across different camera-based MRI workflows. By employing a variety of data to train deep learning model 134, deep learning model 134 can be trained to generate accurate predictions within a variety of camera-based MRI workflows.
[0109] At 1010, data generation component 202 can automatically generate a fourth library (e.g., fourth library 128) comprising 2D graphics to simulate properties of 3D CAD objects and 3D environments. In some embodiments, data generation component 202 can source (e.g., access) existing 2D graphics of 3D CAD models and 3D environments previously generated by data generation component 202 and comprised within the fourth library to generate new 2D graphics of 3D CAD models and 3D environments by varying parameters of the existing 2D graphics. The 2D graphics can comprise color, texture and bump mapping, HDRI images of 3D environments, logos, branding and other elements. The 2D graphics can be applied to 3D CAD objects and 3D environments comprised within imaging scenes (e.g., the set of imaging scenes 130) to render the imaging scenes. By employing a variety of 2D graphics to generate data that can be employed to train deep learning model 134, deep learning model 134 can be trained to generate accurate predictions within a variety of camera-based MRI workflows.
[0110] At 1012, 1014 and 1016, scene generation component 204 can automatically generate a set of imaging scenes (e.g., the set of imaging scenes 130) by selecting data from the first, second, third and fourth libraries.
[0111] For example, at 1012, scene generation component 204 can automatically and selectively compose patient setups. To compose a patient setup, scene generation component 204 can first select MRI studies and an MRI scanning protocol (e.g., from a database of medical imaging studies and scanning protocols stored in a memory such as memory 106). The MRI studies can be employed by scene generation component 204 to replicate medical imaging setups. Based on the MRI studies and the MRI scanning protocol (e.g., a brain routine protocol or abdomen protocol), scene generation component 204 can select a humanoid in an articulated imaging pose (e.g., supine, prone, left / right decubitus, on the left shoulder, knees bent at 90 degrees, arms stretched above the head, etc.) and scan direction (e.g., head-first / feet-first) suitable for the MRI scanning protocol from the first library and a 3D CAD model of an MRI coil in an articulated form suitable for the MRI scanning protocol from the second library. For example, the articulating humanoid selected by scene generationcomponent 204 can show hands, legs, neck, etc. bent to attain a suitable imaging position according to the MRI scanning protocol.
[0112] At 1014, scene generation component 204 can automatically compose the set of imaging scenes to be rendered by execution component 208. To compose an imaging scene, scene generation component 204 can select a 3D CAD model of an MRI system (e.g., MRI gantry and table) and a 3D CAD model of an MRI bay design with a suitable lighting theme from the third library, select 2D graphics for bump mapping surfaces of 3D objects from the fourth library, define camera locations and camera settings (e.g., depth of field and perspective angle) of virtual cameras and add the 3D CAD model of the MRI system, 3D CAD model of the MRI bay design with the suitable lighting theme, 2D graphics, and camera locations and camera settings to the patient setup generated at 1012. In one or more embodiments, the generation of imaging scenes by scene generation component 204 can be automated via inputs such a CSV file, a table or data in another format that can define variables based on which, scene generation component 204 can select data from the first, second, third and fourth libraries and automatically generate the imaging scenes.
[0113] At 1016, scene generation component 204 can similarly compose additional imaging scenes, for example, by repeating the processes at 1012 and 1014 and / or by modifying one or more parameters and / or details associated with the imaging scene generated at 1012 and 1014. For example, scene generation component 204 can change the brand of the MRI system, replace the background, change an HDRI image, replace the MRI gantry, and so on in the imaging scene to generate a new imaging scene. Further, at 1016, rendering component 206 can add the composed imaging scenes to be rendered to a rendering queue. That is, rendering component 206 can queue the rendering tasks. In some embodiments, the queuing of the rendering tasks can be automated, for example, based on certain defined parameters or combinations of imaging scenes, whereas in other embodiments, the queuing of the rendering tasks in the rendering queue can be performed manually by a human.
[0114] Different imaging scenes comprised within the set of imaging scenes can have different combinations of articulating humanoids, MRI accessories, MRI systems, MRI tables, MRI gantries, medical imaging environments, 2D graphics, and so on. Scene generation component 204 can also define different camera locations such as overhead the patient and at a height of 3 m from the MRI table, mounted on to an MRI scanner, oriented from the head side (i.e., the cranial end) of the patient towards the feet (i.e., the caudal end). It should be appreciated that the embodiments of the present disclosure are not limited to MRI imaging. For example, in case of CT imaging, data generation component 202 cangenerate the 3D CAD models (e.g., of patient immobilization systems such as head supports or knee supports, patient immobilization pads, etc.) and 2D graphics suitable for CT imaging setups, and scene generation component 204 can generate the corresponding set of imaging scenes according to different CT studies and CT scanning protocols.
[0115] At 1018, execution component 208 can automatically execute the rendering queue to generate a set of finished images (e.g., the set of finished images 132), and execution component 208 can automatically save the set of finished images 132 in a .png format, a .raw format or other suitable formats in a storage system (e.g., memory 106 or another storage). Execution component 208 can employ a rendering software connected to a render farm to execute the rendering queue.
[0116] The automated workflow can end at 1020. As stated elsewhere herein, training component 210 can employ the finished images thus generated to automatically train deep learning model 134 for generating automatic predictions within camera-based MRI workflows.
[0117] FIG. 11 illustrates a block diagram of an example, non-limiting system 1100 employable for synthetic data generation for a camera-based imaging workflow, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0118] In accordance with an aspect of the present disclosure, as illustrated in FIG. 11, non-limiting system 1100 for synthetic data generation for camera-based imaging workflow assistance is disclosed. Non-limiting system 1100 can comprise system 1108. System 1108 can comprise processor 1101 (e.g., a computer processor) that can build a library articulating 3D models of medical imaging equipment and medical imaging accessories to simulate diverse use cases of interactions of the medical imaging accessories with a variety of human anatomical regions and organs. Processor 1101 can further build a library comprising 3D CAD models that can simulate various medical imaging equipment and indoor settings that can be employed to image patients. Processor 1101 can further build a library comprising 2D graphics that can simulate properties of 3D objects and 3D environment, wherein the 2D graphics can be applied during rendering of imaging scenes to generate the synthetic data. Processor 1101 can also develop a library of patient setups for imaging, by selecting the appropriate imaging protocol (also known as scanning protocol), features of articulating humanoids and medical imaging equipment details. Processor 1101 can further develop an imaging scene by adding 3D CAD models of medical imagingequipment, an imaging environment, 2D graphics representing 3D objects and various camera locations to a patient setup comprised within the library of patient setups. Processor 1101 can add the imaging scene to a rendering queue (i.e., a queue of rendering tasks), repeat the sequence of operations comprising generating a patient setup and adding 3D CAD models of medical imaging equipment, an imaging environment, 2D graphics representing 3D objects and various camera locations to the patient setup to develop additional imaging scenes, and add the additional imaging scenes to the rendering queue. Thereafter, processor 1101 can execute the rendering queue and generate finished images in desired formats. System 1108 can further comprise memory 1102 that can be non-transitory computer readable media employable to store the finished images generated by processor 1101. System 1108 can further comprise display 1103 that can show / display the images generated by processor 1101 to an end entity (e.g., hardware, software, machine, Al, neural network and / or user) at the UI of a computing device (e.g., desktop computer, laptop, tablet, smartphone, etc.). System 1108 can further comprise external display 1104. In traditional image acquisition, a processor may be connected to various medical imaging equipment / devices / scanner systems such as CT scanner 1105 or X-ray machine 1106 through a suitable connecting platform (e.g., connecting platform 1107) for image acquisition. However, embodiments of the present disclosure can eliminate the task of acquiring image data via medical imaging equipment such as CT scanner 1105, X-ray machine 1106, etc. through the connecting platform (1107) since processor 1101 can generate synthetic images equivalent to those produced by a variety of medical imaging equipment. The synthetic images generated by processor 1101 can form synthetic image data that can be employed as ground truth data to develop various Al-based models.
[0119] In one or more embodiments, non-limiting system 1100 can be analogous to non-limiting system 100 such that one or more operations with reference to FIG. 11 can be performed by one or more components of synthetic data generation model 110.
[0120] FIG. 12 illustrates a flow diagram of an example, non-limiting method 1200 for synthetic data generation for a camera-based imaging workflow, in accordance with an aspect of the present disclosure. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0121] With continued reference to FIG. 11, a method for synthetic data generation for camera-based imaging workflow assistance is disclosed. In medical imaging, human subjects vary in several physical aspects such as age, gender, size, length, facial expressions, color, clothes worn by a person, etc. Similarly, the environment in which the medical imagesof a human subject are obtained can vary considerably. For example, the environment can be affected by the temperature, light, and reflection of the light within scan / scanning room, placement of other objects within the scan room, etc. These variations can present themselves as physical limitations when generating synthetic data to train Al models employable in camera-based imaging workflows. To overcome such physical limitations, a method for synthetic data generation is proposed.
[0122] For example, in accordance with an aspect of the disclosure, FIG. 12 illustrates non-limiting method 1200 for synthetic data generation for assistance in a camerabased imaging workflow. At 1202, non-limiting method 1200 can comprise building an image library articulating 3D humanoids to simulate subjects, their poses and demographic diversity. At 1204, non-limiting method 1200 can comprise building a library articulating 3D models of the equipment and accessories to simulate diverse use cases of interactions with a variety of human organs. At 1206, non-limiting method 1200 can comprise building a library to simulate various equipment and indoor settings that may be used for imaging the patient. At 1208, non-limiting method 1200 can comprise building a library of 2D graphics to simulate properties of 3D objects and environments that may be applied during rendering of the images. At 1210, non-limiting method 1200 can comprise developing the patient setup for imaging by selecting the appropriate imaging protocol, features of the humanoid and imaging equipment details. At 1212, non-limiting method 1200 can comprise developing an imaging scene by adding the medical equipment, imaging environment, 2D graphics for 3D objects and various camera locations to the patient setup. At 1214, non-limiting method 1200 can comprise adding the imaging scene to a rendering queue and repeating the sequence of steps (at 1202 to 1212) to generate and add additional scenes to the rendering queue. At 1216, non-limiting method 1200 can comprise executing the rendering queue tasks and generating images in the desired format. The generated images form the synthetic image data that can be employed as a ground truth data to develop various Al-based modules.
[0123] One or more embodiments of the present disclosure will now be described in detail, by way of examples, and with reference to FIG. 12. As illustrated in FIG. 12, nonlimiting method 1200 can comprise building (at 1202) an image library articulating 3D humanoids to simulate subjects, their poses and demographic diversity. As stated elsewhere herein, human beings can vary significantly in their physiological parameters and appearances. For example, the height, width, color, clothing, gender of individuals appearing for medical imaging can vary across and within geographies. In one example, the height and weight of a baby or a child can be significantly different than those of an obese adult.Obtaining image data for such a diverse population of patients and employing the image data for creating Al modules to encompass all the diverse use cases can be significantly challenging for the individuals / teams collecting such data. Other physiological factors that can pose challenges include variations in BMI values of populations from developing countries as compared to populations from developed countries, facial appearances of people from different geographies, and different clothing styles of different populations. To overcome these challenges, a library of images of mannequins (i.e., 3D CAD models) of varying shapes and sizes can be built (e.g., by data generation component 202) by employing a variety of software available in the market. In one example, a CAD software that can generate images of humanoids of varying sizes for medical imaging can be employed. A variety of such images can be generated and stored for future use (e.g., within first library 122). Additionally, the CAD software can be employed to generate images of humanoids in various imaging positions and according to any of the predefined physiological parameters discussed above.
[0124] In accordance with an aspect of the disclosure, illustrated in FIG. 12, nonlimiting method 1200 can comprise building (at 1204) a library articulating 3D models of equipment and accessories to simulate diverse use cases of interactions with a variety of human organs. Some examples of the medical imaging equipment include X-ray machines, CT scanners, MRI machines and ultrasound imaging systems. However, it should be appreciated that the embodiments of the present disclosure are not restricted to any particular type of medical imaging equipment. Different medical imaging equipment can comprise a variety of medical imaging accessories. In one example, an MRI equipment can comprise a variety of MRI coils that are wrapped around the human anatomical regions and organs for imaging. The MRI coils can include body coils, chest coils or head coils that can be wrapped around the body of a patient. Depending on a combination of the size of the patient’s anatomy and the MRI coil, the MRI coils can acquire varying shapes. In some embodiments, a library (e.g., second library 124) can be built (e.g., by data generation component 202), wherein the library can comprise various medical imaging accessories wrapped around different patient bodies in various imaging poses.
[0125] In accordance with an aspect of the disclosure, as illustrated in FIG. 12, nonlimiting method 1200 can further comprise building (at 1206) a library to simulate various equipment and indoor settings that can be employed for imaging the patient. Medical imaging equipment (e.g., MRI machines, CT scanners, X-ray machines, ultrasound imaging systems, etc.) are available in various shapes, dimensions and configurations. Additionally,the environments within which the medical imaging equipment are operated are unique to each equipment. For example, X-ray, MRI or CT rooms are closed environments with controlled temperature and light. The table, gantry size, walls, ceilings and surfaces, in-bore lighting, display lights within the room and other elements can also affect the environment. For example, reflection of light can be different in different imaging environments. In some embodiments, an image library (e.g., third library 126) can be built (e.g., by data generation component 202), wherein the image library can capture various medical imaging equipment, imaging environments and variations in the imaging environments. For example, a rendering software can be utilized to create images of scanning rooms, wherein the images can be equivalent to actual scanning room environments. That is, a library of images can be created to include a variety of imaging equipment and the associated environments. The image library can be created by employing existing CAD software; however, other suitable software (e.g., to be developed in the future) can also be employed to generate the images employable to create the image library.
[0126] In accordance with an aspect of the disclosure, non-limiting method 1200 further comprises building (at 1208) a library of 2D graphics to simulate properties of 3D objects and environments that can be applied during rendering of the images. The 2D image equivalent of a 3D object is also called a bump map of the 3D object, and several bump maps can be created to build a library of 2D equivalents of the 3D objects. In one example, the bump maps can be embossed on other images to generate 3D appearances. Similarly, HDRIs can be employed to create simulated environments to show other objects present within the environment. A library (e.g., fourth library 128) of HDRIs of 3D environments can be created (e.g., by data generation component 202) to provide more robust images of varying dimensions to build Al models.
[0127] In accordance with an aspect of the disclosure, non-limiting method 1200 can further comprise developing (at 1210) a patient set-up for imaging by selecting the appropriate imaging protocol, features of the humanoid and imaging equipment details. Various medical imaging techniques (e.g., MRI, CT, X-ray, ultrasound, etc.) employ standardized imaging protocols that define how a subject (e.g., patient) should undergo scanning. In one example, for MRI scanning, a variety of MRI coils can be wrapped around a subject to image certain organs of the subject, and the subject can be positioned on the MR table in a suitable pose (e.g., supine, prone, etc.). In another example, the subject has to be positioned in various poses on the MR table to obtain images of specific areas of the subject. A library of such imaging protocols, poses and conditions can be created (e.g., by scenegeneration component 204) to employ the library in subsequent steps of non-limiting method 1200, according to one or more embodiments of the present disclosure.
[0128] In accordance with an aspect of the disclosure, as illustrated in FIG. 12, nonlimiting method 1200 can further comprise developing (at 1212) an imaging scene by adding the medical equipment, imaging environment, 2D graphics for 3D objects and various camera locations to the patient setup. In one example, in MR imaging, the MRI system (e.g., the MRI gantry and table), an MRI bay design and lighting theme, 2D graphics for bump mapping surfaces of 3D objects, camera locations and settings (e.g., depth of field and perspective angle) can be added to a patient setup to compose the imaging scenes to be rendered to create the image library.
[0129] In accordance with an aspect of the disclosure, non-limiting method 1200 can comprise adding (at 1214) the imaging scene to a rendering queue and repeating the sequence of steps (at 1202 to 1212) to generate and add additional imaging scenes to the rendering queue. Once the desired images suitable for a particular scanning protocol are created, multiple cameras can be introduced in the environment via a rendering software to generate a rendering queue. In a further aspect, the rendering queue can be executed to generate a finished image set. The execution of the rendering queue can be a computationally heavy task and can consume several hours, depending on the complexity of the imaging scenes. The finished images thus generated represent synthetic images of the humanoids and 3D CAD models of medical imaging accessories, equipment and environments, and the finished images can be employed as ground truth data to develop the Al-based modules.
[0130] FIG. 13 illustrates flow diagrams of example, non-limiting methods 1300 and 1310 that can generate and employ synthetic data to train Al models employable in camerabased imaging workflows, in accordance with one or more embodiments described herein. Repetitive description of like elements and / or processes employed in respective embodiments is omitted for sake of brevity.
[0131] Referring first to non-limiting method 1300, at 1302, non-limiting method 1300 can comprise executing (e.g., by execution component 208), by a system operatively coupled to a processor, a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein the executing generates a set of finished images.
[0132] At 1304, non-limiting method 1300 can comprise training (e.g., by training component 210), by the system, a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camera-based imaging workflow.
[0133] At 1306, non-limiting method 1300 can comprise employing (e.g., by training component 210), by the system, the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0134] With continued reference to non-limiting method 1300, non-limiting method 1310 illustrates steps involved in training the deep learning model.
[0135] At 1312, non-limiting method 1310 can comprise evaluating (e.g., by training component 210), by the system, respective finished images comprised within the set of finished images for accuracy and quality.
[0136] At 1314, non-limiting method 1310 can comprise determining (e.g., by training component 210), by the system, whether the finished images comprised within the set of finished images meet defined accuracy and quality levels. If yes, then at 1316, finished images that meet the defined accuracy and quality levels can be included (e.g., by training component 210) in the ground truth data. If not, then at 1318, finished images that do not meet the defined accuracy and quality levels can be excluded (e.g., by training component 210) from the ground truth data.
[0137] At 1320, non-limiting method 1310 can comprise training (e.g., by training component 210), by the system, the deep learning model on the ground truth data.
[0138] For simplicity of explanation, the computer-implemented and non-computer- implemented methodologies provided herein are depicted and / or described as a series of acts. It is to be understood that the subject innovation is not limited by the acts illustrated and / or by the order of acts, for example acts can occur in one or more orders and / or concurrently, and with other acts not presented and described herein. Furthermore, not all illustrated acts can be utilized to implement the computer-implemented and non-computer-implemented methodologies in accordance with the described subject matter. Additionally, the computer- implemented methodologies described hereinafter and throughout this specification are capable of being stored on an article of manufacture to enable transporting and transferring the computer-implemented methodologies to computers. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from any computer- readable device or storage media.
[0139] The systems and / or devices have been (and / or will be further) described herein with respect to interaction between one or more components. Such systems and / or components can include those components or sub-components specified therein, one or more of the specified components and / or sub-components, and / or additional components. Subcomponents can be implemented as components communicatively coupled to othercomponents rather than included within parent components. One or more components and / or sub-components can be combined into a single component providing aggregate functionality. The components can interact with one or more other components not specifically described herein for the sake of brevity, but known by those of skill in the art.
[0140] In various instances, machine learning algorithms or models can be implemented in any suitable way to facilitate any suitable aspects described herein. To facilitate some of the above-described machine learning aspects of various embodiments, consider the following discussion of artificial intelligence (Al). Various embodiments described herein can employ Al to facilitate automating one or more features or functionalities. The components can employ various Al-based schemes for carrying out various embodiments / examples disclosed herein. In order to provide for or aid in the numerous determinations (e.g., determine, ascertain, infer, calculate, predict, prognose, estimate, derive, forecast, detect, compute) described herein, components described herein can examine the entirety or a subset of the data to which it is granted access and can provide for reasoning about or determine states of the system or environment from a set of observations as captured via events or data. Determinations can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The determinations can be probabilistic; that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Determinations can also refer to techniques employed for composing higher-level events from a set of events or data.
[0141] Such determinations can result in the construction of new events or actions from a set of observed events or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources. Components disclosed herein can employ various classification (explicitly trained (e.g., via training data) as well as implicitly trained (e.g., via observing behavior, preferences, historical information, receiving extrinsic information, and so on)) schemes or systems (e.g., support vector machines, neural networks, expert systems, Bayesian belief networks, fuzzy logic, data fusion engines, and so on) in connection with performing automatic or determined action in connection with the claimed subject matter. Thus, classification schemes or systems can be used to automatically learn and perform a number of functions, actions, or determinations.
[0142] A classifier can map an input attribute vector, z = (zi, Z2, Z3, Z4, z„), to a confidence that the input belongs to a class, as by f(z) = confidence(class). Such classification can employ a probabilistic or statistical-based analysis (e.g., factoring into theanalysis utilities and costs) to determinate an action to be automatically performed. A support vector machine (SVM) can be an example of a classifier that can be employed. The SVM operates by finding a hyper-surface in the space of possible inputs, where the hyper-surface attempts to split the triggering criteria from the non-triggering events. Intuitively, this makes the classification correct for testing data that is near, but not identical to training data. Other directed and undirected model classification approaches include, e.g., naive Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, or probabilistic classification models providing different patterns of independence, any of which can be employed. Classification as used herein also is inclusive of statistical regression that is utilized to develop models of priority.
[0143] In order to provide additional context for various embodiments described herein, FIG. 14 and the following discussion are intended to provide a brief, general description of a suitable computing environment 1400 in which the various embodiments described herein can be implemented. While the embodiments have been described above in the general context of computer-executable instructions that can run on one or more computers, those skilled in the art will recognize that the embodiments can be also implemented in combination with other program modules or as a combination of hardware and software.
[0144] Generally, program modules include routines, programs, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the inventive methods can be practiced with other computer system configurations, including single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things (loT) devices, distributed computing systems, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which can be operatively coupled to one or more associated devices.
[0145] The illustrated embodiments of the embodiments herein can be also practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
[0146] Computing devices typically include a variety of media, which can include computer-readable storage media, machine-readable storage media, or communications media, which two terms are used herein differently from one another as follows. Computer-readable storage media or machine-readable storage media can be any available storage media that can be accessed by the computer and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer- readable storage media or machine-readable storage media can be implemented in connection with any method or technology for storage of information such as computer-readable or machine-readable instructions, program modules, structured data or unstructured data.
[0147] Computer-readable storage media can include, but are not limited to, random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disk read only memory (CD-ROM), digital versatile disk (DVD), Blu-ray disc (BD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, solid state drives or other solid state storage devices, or other tangible or non-transitory media which can be used to store desired information. In this regard, the terms “tangible” or “non-transitory” herein as applied to storage, memory or computer-readable media, are to be understood to exclude only propagating transitory signals per se as modifiers and do not relinquish rights to all standard storage, memory or computer-readable media that are not only propagating transitory signals per se.
[0148] Computer-readable storage media can be accessed by one or more local or remote computing devices, e.g., via access requests, queries or other data retrieval protocols, for a variety of operations with respect to the information stored by the medium.
[0149] Communications media typically embody computer-readable instructions, data structures, program modules or other structured or unstructured data in a data signal such as a modulated data signal, e.g., a carrier wave or other transport mechanism, and includes any information delivery or transport media. The term “modulated data signal” or signals refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in one or more signals. By way of example, and not limitation, communication media include wired media, such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.
[0150] With reference again to FIG. 14, the example environment 1400 for implementing various embodiments of the aspects described herein includes a computer 1402, the computer 1402 including a processing unit 1404, a system memory 1406 and a system bus 1408. The system bus 1408 couples system components including, but not limited to, the system memory 1406 to the processing unit 1404. The processing unit 1404 can be anyof various commercially available processors. Dual microprocessors and other multi-processor architectures can also be employed as the processing unit 1404.
[0151] The system bus 1408 can be any of several types of bus structure that can further interconnect to a memory bus (with or without a memory controller), a peripheral bus, and a local bus using any of a variety of commercially available bus architectures. The system memory 1406 includes ROM 1410 and RAM 1412. A basic input / output system (BIOS) can be stored in a non-volatile memory such as ROM, erasable programmable read only memory (EPROM), EEPROM, which BIOS contains the basic routines that help to transfer information between elements within the computer 1402, such as during startup. The RAM 1412 can also include a high-speed RAM such as static RAM for caching data.
[0152] The computer 1402 further includes an internal hard disk drive (HDD) 1414 (e.g., EIDE, SATA), one or more external storage devices 1416 (e.g., a magnetic floppy disk drive (FDD) 1416, a memory stick or flash drive reader, a memory card reader, etc.) and a drive 1420, e.g., such as a solid state drive, an optical disk drive, which can read or write from a disk 1422, such as a CD-ROM disc, a DVD, a BD, etc. Alternatively, where a solid state drive is involved, disk 1422 would not be included, unless separate. While the internal HDD 1414 is illustrated as located within the computer 1402, the internal HDD 1414 can also be configured for external use in a suitable chassis (not shown). Additionally, while not shown in environment 1400, a solid state drive (SSD) could be used in addition to, or in place of, an HDD 1414. The HDD 1414, external storage device(s) 1416 and drive 1420 can be connected to the system bus 1408 by an HDD interface 1424, an external storage interface 1426 and a drive interface 1428, respectively. The interface 1424 for external drive implementations can include at least one or both of Universal Serial Bus (USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394 interface technologies. Other external drive connection technologies are within contemplation of the embodiments described herein.
[0153] The drives and their associated computer-readable storage media provide nonvolatile storage of data, data structures, computer-executable instructions, and so forth. For the computer 1402, the drives and storage media accommodate the storage of any data in a suitable digital format. Although the description of computer-readable storage media above refers to respective types of storage devices, it should be appreciated by those skilled in the art that other types of storage media which are readable by a computer, whether presently existing or developed in the future, could also be used in the example operating environment, and further, that any such storage media can contain computer-executable instructions for performing the methods described herein.
[0154] A number of program modules can be stored in the drives and RAM 1412, including an operating system 1430, one or more application programs 1432, other program modules 1434 and program data 1436. All or portions of the operating system, applications, modules, or data can also be cached in the RAM 1412. The systems and methods described herein can be implemented utilizing various commercially available operating systems or combinations of operating systems.
[0155] Computer 1402 can optionally comprise emulation technologies. For example, a hypervisor (not shown) or other intermediary can emulate a hardware environment for operating system 1430, and the emulated hardware can optionally be different from the hardware illustrated in FIG. 14. In such an embodiment, operating system 1430 can comprise one virtual machine (VM) of multiple VMs hosted at computer 1402. Furthermore, operating system 1430 can provide runtime environments, such as the Java runtime environment or the .NET framework, for applications 1432. Runtime environments are consistent execution environments that allow applications 1432 to run on any operating system that includes the runtime environment. Similarly, operating system 1430 can support containers, and applications 1432 can be in the form of containers, which are lightweight, standalone, executable packages of software that include, e.g., code, runtime, system tools, system libraries and settings for an application.
[0156] Further, computer 1402 can be enabled with a security module, such as a trusted processing module (TPM). For instance, with a TPM, boot components hash next in time boot components, and wait for a match of results to secured values, before loading a next boot component. This process can take place at any layer in the code execution stack of computer 1402, e.g., applied at the application execution level or at the OS kernel level, thereby enabling security at any level of code execution.
[0157] A user can enter commands and information into the computer 1402 through one or more wired / wireless input devices, e.g., a keyboard 1438, a touch screen 1440, and a pointing device, such as a mouse 1442. Other input devices (not shown) can include a microphone, an infrared (IR) remote control, a radio frequency (RF) remote control, or other remote control, a joystick, a virtual reality controller or virtual reality headset, a game pad, a stylus pen, an image input device, e.g., camera(s), a gesture sensor input device, a vision movement sensor input device, an emotion or facial detection device, a biometric input device, e.g., fingerprint or iris scanner, or the like. These and other input devices are often connected to the processing unit 1404 through an input device interface 1444 that can be coupled to the system bus 1408, but can be connected by other interfaces, such as a parallelport, an IEEE 1394 serial port, a game port, a USB port, an IR interface, a BLUETOOTH® interface, etc.
[0158] A monitor 1446 or other type of display device can be also connected to the system bus 1408 via an interface, such as a video adapter 1448. In addition to the monitor 1446, a computer typically includes other peripheral output devices (not shown), such as speakers, printers, etc.
[0159] The computer 1402 can operate in a networked environment using logical connections via wired or wireless communications to one or more remote computers, such as a remote computer(s) 1450. The remote computer(s) 1450 can be a workstation, a server computer, a router, a personal computer, portable computer, microprocessor-based entertainment appliance, a peer device or other common network node, and typically includes many or all of the elements described relative to the computer 1402, although, for purposes of brevity, only a memory / storage device 1452 is illustrated. The logical connections depicted include wired / wireless connectivity to a local area network (LAN) 1454 or larger networks, e.g., a wide area network (WAN) 1456. Such LAN and WAN networking environments are commonplace in offices and companies, and facilitate enterprise- wide computer networks, such as intranets, all of which can connect to a global communications network, e.g., the Internet.
[0160] When used in a LAN networking environment, the computer 1402 can be connected to the local network 1454 through a wired or wireless communication network interface or adapter 1458. The adapter 1458 can facilitate wired or wireless communication to the LAN 1454, which can also include a wireless access point (AP) disposed thereon for communicating with the adapter 1458 in a wireless mode.
[0161] When used in a WAN networking environment, the computer 1402 can include a modem 1460 or can be connected to a communications server on the WAN 1456 via other means for establishing communications over the WAN 1456, such as by way of the Internet. The modem 1460, which can be internal or external and a wired or wireless device, can be connected to the system bus 1408 via the input device interface 1444. In a networked environment, program modules depicted relative to the computer 1402 or portions thereof, can be stored in the remote memory / storage device 1452. It will be appreciated that the network connections shown are examples and other means of establishing a communications link between the computers can be used.
[0162] When used in either a LAN or WAN networking environment, the computer 1402 can access cloud storage systems or other network-based storage systems in addition to,or in place of, external storage devices 1416 as described above, such as but not limited to a network virtual machine providing one or more aspects of storage or processing of information. Generally, a connection between the computer 1402 and a cloud storage system can be established over a LAN 1454 or WAN 1456 e.g., by the adapter 1458 or modem 1460, respectively. Upon connecting the computer 1402 to an associated cloud storage system, the external storage interface 1426 can, with the aid of the adapter 1458 or modem 1460, manage storage provided by the cloud storage system as it would other types of external storage. For instance, the external storage interface 1426 can be configured to provide access to cloud storage sources as if those sources were physically connected to the computer 1402.
[0163] The computer 1402 can be operable to communicate with any wireless devices or entities operatively disposed in wireless communication, e.g., a printer, scanner, desktop or portable computer, portable data assistant, communications satellite, any piece of equipment or location associated with a wirelessly detectable tag (e.g., a kiosk, news stand, store shelf, etc.), and telephone. This can include Wireless Fidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, the communication can be a predefined structure as with a conventional network or simply an ad hoc communication between at least two devices.
[0164] FIG. 15 is a schematic block diagram of a sample computing environment 1500 with which the disclosed subject matter can interact. The sample computing environment 1500 includes one or more client(s) 1510. The client(s) 1510 can be hardware or software (e.g., threads, processes, computing devices). The sample computing environment 1500 also includes one or more server(s) 1530. The server(s) 1530 can also be hardware or software (e.g., threads, processes, computing devices). The servers 1530 can house threads to perform transformations by employing one or more embodiments as described herein, for example. One possible communication between a client 1510 and a server 1530 can be in the form of a data packet adapted to be transmitted between two or more computer processes. The sample computing environment 1500 includes a communication framework 1550 that can be employed to facilitate communications between the client(s) 1510 and the server(s) 1530. The client(s) 1510 are operably connected to one or more client data store(s) 1520 that can be employed to store information local to the client(s) 1510. Similarly, the server(s) 1530 are operably connected to one or more server data store(s) 1540 that can be employed to store information local to the servers 1530.
[0165] Various embodiments may be a system, a method, an apparatus or a computer program product at any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computerreadable program instructions thereon for causing a processor to carry out aspects of various embodiments. The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium can also include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc readonly memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
[0166] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device. Computer readable program instructions for carrying out operations of various embodiments can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform various aspects.
[0167] Various aspects are described herein with reference to flowchart illustrations or block diagrams of methods, apparatus (systems), and computer program products according to various embodiments. It will be understood that each block of the flowchart illustrations or block diagrams, and combinations of blocks in the flowchart illustrations or block diagrams, can be implemented by computer readable program instructions. These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function / act specified in the flowchart or block diagram block or blocks. The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational acts to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart or block diagram block or blocks.
[0168] The flowcharts and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in theflowchart or block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the Figures. For example, two blocks shown in succession can, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
[0169] While the subject matter has been described above in the general context of computer-executable instructions of a computer program product that runs on a computer or computers, those skilled in the art will recognize that this disclosure also can or can be implemented in combination with other program modules. Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that various aspects can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, mini-computing devices, mainframe computers, as well as computers, hand-held computing devices (e.g., PDA, phone), microprocessor-based or programmable consumer or industrial electronics, and the like. The illustrated aspects can also be practiced in distributed computing environments in which tasks are performed by remote processing devices that are linked through a communications network. However, some, if not all aspects of this disclosure can be practiced on stand-alone computers. In a distributed computing environment, program modules can be located in both local and remote memory storage devices.
[0170] As used herein, the term “non-transitory computer-readable media” is intended to be representative of any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as, computer-readable instructions, data structures, program modules and sub-modules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions embodied in a tangible, non-transitory, computer readable medium, including, without limitation, a storage device and / or a memory device. Such instructions, when executed by a processor, cause the processor to perform at least a portion of the methods described herein. Moreover, as used herein, the term “non-transitory computer-readable media” includes all tangible,computer-readable media, including, without limitation, non-transitory computer storage devices, including, without limitation, volatile and nonvolatile media, and removable and non-removable media such as a firmware, physical and virtual storage, CD-ROMs, DVDs, and any other digital source such as a network or the Internet, as well as yet to be developed digital means, with the sole exception being a transitory, propagating signal.
[0171] As used herein, the terms “software” and “firmware” are interchangeable and include any computer program stored in memory for execution by devices that include, without limitation, mobile devices, clusters, personal computers, workstations, clients, and servers.
[0172] As used herein, the term “computer” and related terms, e.g., “computing device”, “processor”, “controller” are not limited to integrated circuits referred to in the art as a computer, but broadly refers to at least one microcontroller, microcomputer, programmable logic controller (PLC), application specific integrated circuit, and other programmable circuits, and these terms are used interchangeably herein.
[0173] As used in this application, the terms “component,” “system,” “platform,” “interface,” and the like, can refer to and / or can include a computer-related entity or an entity related to an operational machine with one or more specific functionalities. The entities disclosed herein can be either hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and / or thread of execution and a component can be localized on one computer and / or distributed between two or more computers. In another example, respective components can execute from various computer readable media having various data structures stored thereon. The components can communicate via local and / or remote processes such as in accordance with a signal having one or more data packets (e.g., data from one component interacting with another component in a local system, distributed system, and / or across a network such as the Internet with other systems via the signal). As another example, a component can be an apparatus with specific functionality provided by mechanical parts operated by electric or electronic circuitry, which is operated by a software or firmware application executed by a processor. In such a case, the processor can be internal or external to the apparatus and can execute at least a part of the software or firmware application. As yet another example, a component can be an apparatus that provides specific functionality through electroniccomponents without mechanical parts, wherein the electronic components can include a processor or other means to execute software or firmware that confers at least in part the functionality of the electronic components. In an aspect, a component can emulate an electronic component via a virtual machine, e.g., within a cloud computing system.
[0174] In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from context, “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then “X employs A or B” is satisfied under any of the foregoing instances. As used herein, the term “and / or” is intended to have the same meaning as “or.” Moreover, articles “a” and “an” as used in the subject specification and annexed drawings should generally be construed to mean “one or more” unless specified otherwise or clear from context to be directed to a singular form. As used herein, the terms “example” and / or “exemplary” are utilized to mean serving as an example, instance, or illustration and are intended to be nonlimiting. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as an “example” and / or “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art.
[0175] The herein disclosure describes non-limiting examples. For ease of description or explanation, various portions of the herein disclosure utilize the term “each,” “every,” or “all” when discussing various examples. Such usages of the term “each,” “every,” or “all” are non-limiting. In other words, when the herein disclosure provides a description that is applied to “each,” “every,” or “all” of some particular object or component, it should be understood that this is a non-limiting example, and it should be further understood that, in various other examples, it can be the case that such description applies to fewer than “each,” “every,” or “all” of that particular object or component.
[0176] As it is employed in the subject specification, the term “processor” can refer to substantially any computing processing unit or device comprising, but not limited to, singlecore processors; single-processors with software multithread execution capability; multi-core processors; multi-core processors with software multithread execution capability; multi -core processors with hardware multithread technology; parallel platforms; and parallel platforms with distributed shared memory. Additionally, a processor can refer to an integrated circuit, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic controller (PLC), a complexprogrammable logic device (CPLD), a discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Further, processors can exploit nano-scale architectures such as, but not limited to, molecular and quantum-dot based transistors, switches and gates, in order to optimize space usage or enhance performance of user equipment. A processor can also be implemented as a combination of computing processing units. In this disclosure, terms such as “store,” “storage,” “data store,” data storage,” “database,” and substantially any other information storage component relevant to operation and functionality of a component are utilized to refer to “memory components,” entities embodied in a “memory,” or components comprising a memory. It is to be appreciated that memory and / or memory components described herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of illustration, and not limitation, nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), flash memory, or nonvolatile random access memory (RAM) (e.g., ferroelectric RAM (FeRAM). Volatile memory can include RAM, which can act as external cache memory, for example. By way of illustration and not limitation, RAM is available in many forms such as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), direct Rambus RAM (DRRAM), direct Rambus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM). Additionally, the disclosed memory components of systems or computer- implemented methods herein are intended to include, without being limited to including, these and any other suitable types of memory.
[0177] What has been described above include mere examples of systems and computer-implemented methods. It is, of course, not possible to describe every conceivable combination of components or computer-implemented methods for purposes of describing this disclosure, but one of ordinary skill in the art can recognize that many further combinations and permutations of this disclosure are possible. Furthermore, to the extent that the terms “includes,” “has,” “possesses,” and the like are used in the detailed description, claims, appendices and drawings such terms are intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
[0178] The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed.Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
[0179] Various non-limiting aspects of embodiments described herein are provided in the following clauses.
[0180] CLAUSE 1 : A system, comprising: a memory that stores computer-executable components; and a processor that executes the computer-executable components stored in the memory, wherein the computer-executable components comprise: an execution component that executes a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein execution of the rendering queue generates a set of finished images; a training component that trains a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camera-based imaging workflow; and an imaging component that employs the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0181] CLAUSE 2: The system of any preceding clause, further comprising: a data generation component that: generates a first library comprising first three-dimensional (3D) models that represent humans having different demographics and assuming different imaging poses; generates, based on the first library, a second library comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories are represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models; generates a third library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment; and generates a fourth library comprising two-dimensional (2D) graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects are represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments are represented as high dynamic range images (HDRIs).
[0182] CLAUSE 3: The system of any preceding clause, wherein the data generation component automatically generates the first library, the second library, the third library and the fourth library based on input data having different data formats.
[0183] CLAUSE 4: The system of any preceding clause, further comprising: a scene generation component that generates the set of imaging scenes, wherein the set of imaging scenes comprises multi-dimensional representations of the different medical imaging setups, and wherein generating an imaging scene comprised within the set of imaging scenes comprises: selecting, by the scene generation component, a scanning protocol; generating, by the scene generation component, a patient setup by selecting, according to the scanning protocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library; and adding, by the scene generation component, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings employable to render the imaging scene, to the patient setup.
[0184] CLAUSE 5: The system of any preceding clause, wherein the imaging scene represents a first imaging scene employable to generate additional imaging scenes, and wherein the system further comprises: a rendering component that automatically adds the set of imaging scenes to the rendering queue.
[0185] CLAUSE 6: The system of any preceding clause, wherein training the deep learning model comprises: evaluating, by the training component, respective finished images comprised within the set of finished images for accuracy and quality; rejecting, by the training component, based on the evaluating, finished images comprised within the set of finished images that do not meet defined accuracy levels and defined quality levels to generate the ground truth data; and training, by the training component, the deep learning model on the ground truth data.
[0186] CLAUSE 7: The system of any preceding clause, wherein employing the set of imaging scenes to generate the set of finished images eliminates a task of generating the ground truth data via a medical imaging device.
[0187] CLAUSE 8: The system of any preceding clause, wherein the deep learning model is further employable to automatically detect anatomical regions and medical imaging accessories within the camera-based imaging workflow.
[0188] CLAUSE 9: The system of clause 1 above with any set of combinations of clauses 2 - 8 above.
[0189] CLAUSE 10: A computer-implemented method, comprising: executing, by a system operatively coupled to a processor, a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein the executing generates a set of finished images; training, by the system, a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camera-based imaging workflow; and employing, by the system, the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0190] CLAUSE 11 : The computer-implemented method of any preceding clause, further comprising: generating, by the system, a first library comprising first 3D models that represent humans having different demographics and assuming different imaging poses; generating, by the system, based on the first library, a second library comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories are represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models; generating, by the system, a third library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment; and generating, by the system, a fourth library comprising 2D graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects are represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments are represented as HDRIs.
[0191] CLAUSE 12: The computer-implemented method of any preceding clause, wherein the first library, the second library, the third library and the fourth library are automatically generated based on input data having different data formats.
[0192] CLAUSE 13: The computer-implemented method of any preceding clause, further comprising: generating, by the system, the set of imaging scenes, wherein the set of imaging scenes comprises multi-dimensional representations of the different medical imaging setups, and wherein generating an imaging scene comprised within the set of imaging scenes comprises: selecting, by the system, a scanning protocol; generating, by the system, a patient setup by selecting, according to the scanning protocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library; and adding, by the system, based on the scanning protocol, 3D models of medical imagingequipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings employable to render the imaging scene, to the patient setup.
[0193] CLAUSE 14: The computer-implemented method of any preceding clause, wherein the imaging scene represents a first imaging scene employable to generate additional imaging scenes, and wherein the computer-implemented method further comprises: adding, by the system, the set of imaging scenes to the rendering queue.
[0194] CLAUSE 15: The computer-implemented method of any preceding clause, wherein the training comprises: evaluating, by the system, respective finished images comprised within the set of finished images for accuracy and quality; rejecting, by the system, based on the evaluating, finished images comprised within the set of finished images that do not meet defined accuracy levels and defined quality levels to generate the ground truth data; and the training, by the system, the deep learning model on the ground truth data.
[0195] CLAUSE 16: The computer-implemented method of clause 10 above with any combinations of clauses 11 - 15 above.
[0196] CLAUSE 17: A computer program product for synthetic data generation for artificial intelligence (Al) models employable in camera-based imaging workflows, the computer program product comprising a non-transitory computer readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to: execute a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein execution of the rendering queue generates a set of finished images; train a deep learning model by employing the set of finished images as ground truth data for training the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camera-based imaging workflow; and employ the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
[0197] CLAUSE 18: The computer program product of any preceding clause, wherein the program instructions are further executable by the processor to cause the processor to: generate a first library comprising first 3D models that represent humans having different demographics and assuming different imaging poses; generate based on the first library, a second library comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories are represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models; generate athird library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment; and generate a fourth library comprising 2D graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects are represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments are represented as HDRIs.
[0198] CLAUSE 19: The computer program product of any preceding clause, wherein the program instructions are further executable by the processor to cause the processor to: automatically generate, based on input data having different data formats, the first library, the second library, the third library and the fourth library.
[0199] CLAUSE 20: The computer program product of any preceding clause, wherein the program instructions are further executable by the processor to cause the processor to: generate the set of imaging scenes, wherein the set of imaging scenes comprises multi-dimensional representations of the different medical imaging setups, and wherein generating an imaging scene comprised within the set of imaging scenes comprises: selecting, by the processor, a scanning protocol; generating, by the processor, a patient setup by selecting, according to the scanning protocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library; and adding, by the processor, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings employable to render the imaging scene, to the patient setup.
[0200] CLAUSE 21 : The computer program product of any preceding clause, wherein the imaging scene represents a first imaging scene employable to generate additional imaging scenes, and wherein the program instructions are further executable by the processor to cause the processor to: add the set of imaging scenes to the rendering queue.
[0201] CLAUSE 22: The computer program product of any preceding clause, wherein the program instructions are further executable by the processor to cause the processor to: evaluate respective finished images comprised within the set of finished images for accuracy and quality; reject, based on evaluating the respective finished images, finished images comprised within the set of finished images that do not meet defined accuracy levels and defined quality levels to generate the ground truth data; and the train the deep learning model on the ground truth data.
[0202] CLAUSE 23: The computer program product of clause 17 above with any combinations of clauses 18 - 22 above.
Claims
CLAIMSWhat is claimed is:
1. A system, comprising: a memory that stores computer-executable components; and a processor that executes the computer-executable components stored in the memory, wherein the computer-executable components comprise: an execution component that executes a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein execution of the rendering queue generates a set of finished images; a training component that trains a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camerabased imaging workflow; and an imaging component that employs the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
2. The system of claim 1, further comprising: a data generation component that:generates a first library comprising first three-dimensional (3D) models that represent humans having different demographics and assuming different imaging poses; generates, based on the first library, a second library comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories are represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models; generates a third library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment; and generates a fourth library comprising two-dimensional (2D) graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects are represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments are represented as high dynamic range images (HDRIs).
3. The system of claim 2, wherein the data generation component automatically generates the first library, the second library, the third library and the fourth library based on input data having different data formats.
4. The system of claim 2, further comprising: a scene generation component that generates the set of imaging scenes, wherein the set of imaging scenes comprises multi-dimensional representations of the different medical imaging setups, and wherein generating an imaging scene comprised within the set of imaging scenes comprises: selecting, by the scene generation component, a scanning protocol; generating, by the scene generation component, a patient setup by selecting, according to the scanning protocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library; and adding, by the scene generation component, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings employable to render the imaging scene, to the patient setup.
5. The system of claim 4, wherein the imaging scene represents a first imaging scene employable to generate additional imaging scenes, and wherein the system further comprises: a rendering component that automatically adds the set of imaging scenes to the rendering queue.
6. The system of claim 1, wherein training the deep learning model comprises: evaluating, by the training component, respective finished images comprised within the set of finished images for accuracy and quality; rejecting, by the training component, based on the evaluating, finished images comprised within the set of finished images that do not meet defined accuracy levels and defined quality levels to generate the ground truth data; and training, by the training component, the deep learning model on the ground truth data.
7. The system of claim 1, wherein employing the set of imaging scenes to generate the set of finished images eliminates a task of generating the ground truth data via a medical imaging device.
8. The system of claim 1, wherein the deep learning model is further employable to automatically detect anatomical regions and medical imaging accessories within the camerabased imaging workflow.
9. A computer-implemented method, comprising: executing, by a system operatively coupled to a processor, a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein the executing generates a set of finished images; training, by the system, a deep learning model by employing the set of finished images as ground truth data to train the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camera-based imaging workflow; and employing, by the system, the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
10. The computer-implemented method of claim 9, further comprising: generating, by the system, a first library comprising first 3D models that represent humans having different demographics and assuming different imaging poses; generating, by the system, based on the first library, a second library comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories are represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models; generating, by the system, a third library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment; and generating, by the system, a fourth library comprising 2D graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects are represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments are represented as HDRIs.
11. The computer-implemented method of claim 10, wherein the first library, the second library, the third library and the fourth library are automatically generated based on input data having different data formats.
12. The computer-implemented method of claim 10, further comprising: generating, by the system, the set of imaging scenes, wherein the set of imaging scenes comprises multi-dimensional representations of the different medical imaging setups, and wherein generating an imaging scene comprised within the set of imaging scenes comprises: selecting, by the system, a scanning protocol; generating, by the system, a patient setup by selecting, according to the scanning protocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library; and adding, by the system, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings employable to render the imaging scene, to the patient setup.
13. The computer-implemented method of claim 12, wherein the imaging scene represents a first imaging scene employable to generate additional imaging scenes, and wherein the computer-implemented method further comprises: adding, by the system, the set of imaging scenes to the rendering queue.
14. The computer-implemented method of claim 9, wherein the training comprises: evaluating, by the system, respective finished images comprised within the set of finished images for accuracy and quality; rejecting, by the system, based on the evaluating, finished images comprised within the set of finished images that do not meet defined accuracy levels and defined quality levels to generate the ground truth data; and the training, by the system, the deep learning model on the ground truth data.
15. A computer program product for synthetic data generation for artificial intelligence (Al) models employable in camera-based imaging workflows, the computer program product comprising a non-transitory computer readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to: execute a rendering queue comprising a set of imaging scenes representing different medical imaging setups, wherein execution of the rendering queue generates a set of finished images; train a deep learning model by employing the set of finished images as ground truth data for training the deep learning model, wherein the deep learning model is employable to detect patient positioning within a camera-based imaging workflow; and employ the deep learning model within the camera-based imaging workflow to automatically evaluate patient alignment with respect to medical equipment.
16. The computer program product of claim 15, wherein the program instructions are further executable by the processor to cause the processor to: generate a first library comprising first 3D models that represent humans having different demographics and assuming different imaging poses; generate based on the first library, a second library comprising second 3D models that represent different medical imaging accessories associated with different medical imaging equipment, wherein the different medical imaging accessories are represented in their deformed states resulting from positioning the different medical imaging accessories around the first 3D models; generate a third library comprising third 3D models that represent different shapes, dimensions and configurations of the different medical imaging equipment and that represent different imaging environments associated with the different medical imaging equipment; and generate a fourth library comprising 2D graphics that represent 3D objects and 3D environments employable to render the set of finished images, wherein the 3D objects are represented as bump maps or as 2D data based on the bump maps, and wherein the 3D environments are represented as HDRIs.
17. The computer program product of claim 16, wherein the program instructions are further executable by the processor to cause the processor to: automatically generate, based on input data having different data formats, the first library, the second library, the third library and the fourth library.
18. The computer program product of claim 16, wherein the program instructions are further executable by the processor to cause the processor to: generate the set of imaging scenes, wherein the set of imaging scenes comprises multi-dimensional representations of the different medical imaging setups, and wherein generating an imaging scene comprised within the set of imaging scenes comprises: selecting, by the processor, a scanning protocol; generating, by the processor, a patient setup by selecting, according to the scanning protocol, a 3D model of a human from the first library and one or more 3D models of medical imaging accessories from the second library; and adding, by the processor, based on the scanning protocol, 3D models of medical imaging equipment and an imaging environment selected from the third library, 2D graphics selected from the fourth library and camera locations and camera settings employable to render the imaging scene, to the patient setup.
19. The computer program product of claim 18, wherein the imaging scene represents a first imaging scene employable to generate additional imaging scenes, and wherein the program instructions are further executable by the processor to cause the processor to: add the set of imaging scenes to the rendering queue.
20. The computer program product of claim 15, wherein the program instructions are further executable by the processor to cause the processor to: evaluate respective finished images comprised within the set of finished images for accuracy and quality; reject, based on evaluating the respective finished images, finished images comprised within the set of finished images that do not meet defined accuracy levels and defined quality levels to generate the ground truth data; and the train the deep learning model on the ground truth data.
Citation Information
Patent Citations
Generation of 3D models of anatomical structures from 2d radiographs
EP4239581A1
Systems and methods for automated rendering
EP4411653A1
Method for memorable image generation for anonymized three-dimensional medical image workflows
US10722210B2
Multimodality image processing techniques for training image data generation and usage thereof for developing mono-modality image inferencing models
US11727086B2
Algorithm-based methods for predicting and / or detecting a clinical condition related to insertion of a medical instrument toward an internal target
US20230044620A1