Devices, methods and programs that automatically standardize clinical trial images based on artificial intelligence

Abstract

The present disclosure relates to an artificial intelligence-based automatic standardization device for clinical trial images, wherein information regarding at least one attribute is extracted from medical standard data within a clinical trial image, a first output result is obtained using the extracted information and a pre-trained first artificial intelligence model, a second output result is obtained by analyzing the clinical trial image based on images using a pre-trained second artificial intelligence model, and the clinical trial image is standardized based on the first output result and the second output result.

Description

Technology Field

[0001] The present disclosure relates to a device capable of automatically standardizing clinical trial images, and more specifically, to a device capable of automatically classifying and standardizing clinical trial images based on artificial intelligence. Background Technology

[0002] As the use of medical imaging equipped with advanced technology and complex protocols increases in multi-center clinical trials, the importance of standardizing imaging devices and acquisition protocols is growing.

[0003] However, currently, there is a problem in that a lot of time and manpower is consumed as the process of manually verifying the standardization of clinical trial image data, organizing errors and corrections, and then correcting them is repeated.

[0004] Therefore, while there is a need for technology capable of automatically classifying and standardizing clinical trial images, there is currently no publicly available technology that can accurately classify and standardize such images while minimizing manpower consumption. Prior art literature

[0005] Republic of Korea Registered Patent No. 10-2097740, (Registration Date: March 31, 2020) The problem to be solved

[0006] The embodiments disclosed in this disclosure aim to automatically perform standardization by analyzing clinical trial images from various institutions that are not standardized based on an artificial intelligence model.

[0007] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0008] An artificial intelligence-based automatic standardization device for clinical trial images according to one embodiment of the present disclosure for solving the above-described problem comprises: a receiving unit for receiving clinical trial images; a memory storing at least one instruction; and a processor, wherein the processor executes the at least one instruction to extract information regarding at least one attribute from medical standard data within the clinical trial images, obtains a first output result based on the extracted information using a pre-trained first artificial intelligence model, obtains a second output result by analyzing the clinical trial images based on images using a pre-trained second artificial intelligence model, and can standardize the clinical trial images based on the first output result and the second output result.

[0009] In addition, the medical standard data is data regarding the Digital Imaging and Communications in Medicine (DICOM) standard, and the processor can extract information about at least one attribute from the header of the DICOM.

[0010] In addition, the first artificial intelligence model can be trained through rule-based labeling using the extracted information.

[0011] In addition, the first artificial intelligence model can be trained by performing rule-based labeling using information extracted for the first attribute among the at least one attribute.

[0012] In addition, the processor can perform data standardization on information extracted regarding a second attribute other than the first attribute among the at least one attribute and input it into the first artificial intelligence model.

[0013] Additionally, the first attribute may include at least one attribute that may include a keyword associated with at least one sequence type for the clinical trial image.

[0014] In addition, the above rule includes a first rule table and a second rule table, and the first artificial intelligence model can perform labeling based on the first rule table when the extracted information includes one Sequence Type, and perform labeling based on the second rule table when the extracted information includes multiple Sequence Types.

[0015] In addition, the first output result includes a classification result for the Sequence Type of the clinical trial image, and the first artificial intelligence model can generate the first output result based on a random forest algorithm.

[0016] In addition, the processor can obtain the second output result by using the second artificial intelligence model to conduct a majority vote on the classification result for the Sequence Type based on the clinical trial image.

[0017] In addition, if the first output result and the second output result are not the same, the processor checks whether the amount of medical standard data in the clinical trial image satisfies a preset condition, and if it does not satisfy the condition, it can standardize the clinical trial image based on the second output result.

[0018] In addition, the processor can standardize the clinical trial image based on the second output result when the above-mentioned preset condition is satisfied and the Sequence Type included in the first output result includes Perfusion.

[0019] Additionally, an artificial intelligence-based automatic standardization method for clinical trial images according to one embodiment of the present disclosure for solving the above-mentioned problem comprises: a step of extracting information on at least one attribute from medical standard data within a clinical trial image; a step of obtaining a first output result based on the extracted information using a pre-trained first artificial intelligence model; a step of obtaining a second output result by analyzing the clinical trial image based on an image using a pre-trained second artificial intelligence model; and a step of standardizing the clinical trial image based on the first output result and the second output result.

[0020] In addition to this, a computer program stored on a computer-readable recording medium for executing the present disclosure may be further provided.

[0021] In addition, a computer-readable recording medium for recording a computer program for executing a method for implementing the present disclosure may be further provided. Effects of the invention

[0022] According to the aforementioned means for solving the problem of the present disclosure, there is an effect of automatically performing standardization by analyzing clinical trial images from various institutions that are not standardized based on an artificial intelligence model.

[0023] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0024] FIG. 1 is a schematic diagram of an artificial intelligence-based clinical trial image automatic standardization system according to an embodiment of the present disclosure. FIG. 2 is a block diagram of an artificial intelligence-based clinical trial image automatic standardization device according to an embodiment of the present disclosure. FIG. 3 is a diagram illustrating data stored in the memory of an artificial intelligence-based clinical trial image automatic standardization device according to an embodiment of the present disclosure. Figure 4 is a diagram illustrating the standardization process for clinical trial images. Figure 5 is a diagram illustrating the automatic standardization of a standardization process like that of Figure 4 using an artificial intelligence model. FIG. 6 is a flowchart of an artificial intelligence-based method for automatic standardization of clinical trial images according to an embodiment of the present disclosure. Figure 7 is a diagram illustrating the classification of Sequence Types from the DICOM Header of clinical trial images. Figure 8 is a diagram illustrating the extraction of DICOM attributes from the DICOM Header of a clinical trial image. Figure 9 is a diagram illustrating an example in which a specific keyword is included in an attribute extracted from a DICOM Header. Figure 10 is a diagram illustrating a rule table for automatic labeling. Figure 11 is a diagram illustrating the training of the first artificial intelligence model. FIG. 12 is a diagram illustrating the acquisition of a second output result using an image-based second artificial intelligence model. FIG. 13 is a diagram illustrating the accuracy of the output result obtained using the first artificial intelligence model. Figure 14 is a diagram illustrating the accuracy of the output result obtained using the second artificial intelligence model. FIG. 15 is a diagram illustrating a process of ensembling output results obtained using a first artificial intelligence model and a second artificial intelligence model. Specific details for implementing the invention

[0025] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and general content in the art to which this disclosure pertains or content that overlaps between embodiments is omitted. The terms 'part, module, component, block' as used in the specification may be implemented in software or hardware, and depending on the embodiments, a plurality of 'parts, modules, components, blocks' may be implemented as a single component, or a single 'part, module, component, block' may include a plurality of components.

[0026] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are directly connected but also cases where they are indirectly connected, and indirect connections include connections made via a wireless communication network.

[0027] Furthermore, when it is stated that a part "includes" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0028] Throughout the specification, when it is stated that a component is located "on" another component, this includes not only cases where a component is in contact with another component, but also cases where another component exists between the two components.

[0029] Terms such as "first," "second," etc., are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0030] Singular expressions include plural expressions unless there is an obvious exception in the context.

[0031] In each step, identification codes are used for convenience of explanation and do not describe the order of the steps; the steps may be performed differently from the specified order unless a specific order is clearly indicated in the context.

[0032] The operating principles and embodiments of the present disclosure will be described below with reference to the attached drawings.

[0033] In this specification, the 'automatic standardization device for clinical trial images according to the present disclosure' includes all various devices capable of performing computational processing and providing results to a user. For example, the automatic standardization device for clinical trial images according to the present disclosure may include a computer, a server device, and a portable terminal, or may take the form of any one of them.

[0034] Here, the computer may include, for example, a laptop, desktop, notebook, tablet PC, slate PC, etc. equipped with a web browser.

[0035] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0036] The above portable terminal may include, for example, all types of handheld-based wireless communication devices such as PCS, GSM, PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminals, smartphones, etc., as well as wearable devices such as watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted devices (HMDs).

[0037] The artificial intelligence-related functions according to the present disclosure are operated through a processor and a storage unit. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or artificial intelligence models stored in the storage unit. Alternatively, if the one or more processors are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0038] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0039] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weights and performs neural network operations through operations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss value or cost value obtained by the artificial intelligence model is reduced or minimized during the learning process. Artificial neural networks may include deep neural networks (DNNs), such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), or Deep Q-Networks, but are not limited to the examples mentioned above.

[0040] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on artificial neural networks that enables a machine to learn by mimicking human biological neurons. Methodologies of artificial intelligence can be classified according to the learning method into supervised learning, where input data and output data are provided together as training data and the solution (output data) to the problem (input data) is predetermined; unsupervised learning, where only input data is provided without output data and the solution (output data) to the problem (input data) is not predetermined; and reinforcement learning, where a reward is given from the external environment whenever an action is taken in the current state, and learning proceeds in a direction that maximizes such reward. Additionally, methodologies of artificial intelligence may be classified according to the architecture, which is the structure of the learning model. The architectures of widely used deep learning technologies can be classified into convolutional neural networks, recurrent neural networks, transformers, generative adversarial networks, etc.

[0041] The device may include an artificial intelligence model. The artificial intelligence model may be a single model or may be implemented as multiple models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model that possesses problem-solving capabilities by having artificial neurons (nodes) that form a network through synaptic connections and change the strength of synaptic connections through learning. The neurons of a neural network may include combinations of weights or biases. A neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a result to be predicted from an arbitrary input by changing the weights of the neurons through learning.

[0042] The processor can create a neural network, train or learn a neural network, perform operations based on received input data, generate information signals based on the results of the operations, or retrain the neural network. The neural network models may include, but are not limited to, various types of models such as CNN, R-CNN, RPN, RNN, S-DNN, S-SDNN, Deconvolution Network, DBN, RBM, Fully Convolutional Network, LSTM Network, and Classification Network, such as GoogleNet, AlexNet, and VGG Network. The processor may include one or more processors for performing operations according to the neural network models. For example, the neural network may include a deep neural network.

[0043] Neural networks include CNN, RNN, percept, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning) Machine), ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Network) A person skilled in the art will understand that any neural network may be included, but is not limited to, Computer (NTM), Neural Turning Machine (NTM), Capsule Network (CN), Kohonen Network (KN), and Attention Network (AN).

[0044] According to an exemplary embodiment of the present disclosure, the processor comprises a Convolutional Neural Network (CNN) such as GoogleNet, AlexNet, VGG Network, Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA, Text Analysis, Dialog System, GPT-3, GPT-4 for Natural Language Processing, Visual Analytics, Visual Understanding, Video Synthesis for Vision Processing, Anomaly Detection, Prediction, Time-Series Forecasting, Optimization for ResNet Data Intelligence, Various artificial intelligence structures and algorithms, such as recommendation and data creation, may be used, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0045] FIG. 1 is a schematic diagram of an artificial intelligence-based clinical trial image automatic standardization system according to an embodiment of the present disclosure.

[0046] Referring to FIG. 1, an artificial intelligence-based clinical trial image automatic standardization system according to an embodiment of the present disclosure is illustrated.

[0047] The artificial intelligence-based clinical trial image automatic standardization device (100) receives clinical trial images from multiple institutions, and the received clinical trial images are in a state where standardization has not been performed.

[0048] Conventionally, a significant amount of time and manpower was consumed as the process of manually verifying the standardization of clinical trial image data received in this manner, organizing errors and corrections, and then correcting them was repeated.

[0049] However, through the artificial intelligence-based clinical trial image automatic standardization device (100) according to the embodiment of the present disclosure to be described below, standardization of the clinical trial image is performed automatically.

[0050] The artificial intelligence-based clinical trial image automatic standardization device (100) according to an embodiment of the present disclosure is configured to include a server device and can operate as an artificial intelligence-based clinical trial image automatic standardization server.

[0051] Below, with reference to other drawings, an artificial intelligence-based clinical trial image automatic standardization device (100), method, and program according to an embodiment of the present disclosure will be described in detail.

[0052] FIG. 2 is a block diagram of an artificial intelligence-based clinical trial image automatic standardization device (100) according to an embodiment of the present disclosure.

[0053] Referring to FIG. 2, an artificial intelligence-based clinical trial image automatic standardization device (100) according to an embodiment of the present disclosure includes a processor (110), a receiver (120), and a memory (130).

[0054] However, in some embodiments, the device (100) may include fewer or more components than the components shown in FIG. 2.

[0055] FIG. 3 is a diagram illustrating data stored in the memory (130) of an artificial intelligence-based clinical trial image automatic standardization device (100) according to an embodiment of the present disclosure.

[0056] The processor (110) may be implemented as a storage unit that stores data for an algorithm or a program that reproduces the algorithm for controlling the operation of components within the device (100), and at least one processor (110) that performs the aforementioned operation using the data stored in the storage unit. In this case, the storage unit and the processor (110) may each be implemented as separate chips. Alternatively, the storage unit and the processor (110) may be implemented as a single chip.

[0057] In addition, the processor (110) may control one or a combination of the components described above in order to implement various embodiments according to the present disclosure described in the drawings below on the device (100).

[0058] In addition to operations related to the above-mentioned application, the processor (110) can generally control the overall operation of the device (100). The processor (110) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through the components described above, or by running an application stored in the storage unit.

[0059] Additionally, the processor (110) can control at least some of the components of the device (100) to run an application stored in the storage unit. Furthermore, the processor (110) can operate at least two or more of the components included in the device (100) in combination with each other to run the application.

[0060] The receiving unit (120) may include a communication unit or may be the communication unit itself.

[0061] The communication unit may include one or more modules that connect the clinical trial image automatic standardization device (100) to one or more networks.

[0062] The communication unit may include one or more components that enable communication with an external device, and may include, for example, at least one of a broadcast receiving module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0063] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), DVI (Digital Visual Interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).

[0064] In addition to Wi-Fi modules and WiBro (Wireless broadband) modules, the wireless communication module may include wireless communication modules that support various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G.

[0065] The wireless communication module may include a wireless communication interface comprising an antenna and a transmitter that transmit a communication signal. Additionally, the wireless communication module may further include a signal conversion module that modulates a digital control signal output from the processor (110) through the wireless communication interface into an analog wireless signal under the control of the processor (110).

[0066] A short-range communication module is for short-range communication and can support short-range communication by using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0067] The memory (130) can store data that supports various functions of the device (100). The memory (130) can store a number of application programs (or applications) running on the device (100), data for the operation of the device (100), and instructions. At least some of these application programs may exist for the basic functions of the device (100). Meanwhile, the application programs may be stored in the memory (130), installed on the device (100), and driven by the processor (110) to perform operations (or functions).

[0068] The memory (130) can store data supporting various functions of the device (100) and programs for the operation of the processor (110), and can store input / output data (e.g., music files, still images, videos, etc.), and can store a number of application programs (or applications) running on the device (100), data for the operation of the device (100), and instructions. At least some of these application programs can be downloaded from an external server via wireless communication.

[0069] The memory (130) may include at least one type of storage medium among flash memory (130) type, hard disk type, SSD type (Solid State Disk type), SSD type (Silicon Disk Drive type), multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (random access memory; RAM), SRAM (static random access memory), ROM (read-only memory; ROM), EEPROM (electrically erasable programmable read-only memory), PROM (programmable read-only memory), magnetic memory (130), magnetic disk, and optical disk. Additionally, the memory (130) may be a database that is separated from the device (100) but connected via wired or wireless connection.

[0070] Additionally, the memory (130) may have a plurality of processes for the clinical trial image automatic standardization device (100).

[0071] In addition, the clinical trial image automatic standardization device (100) may further include components such as an input unit, an output unit, and an interface unit.

[0072] The input unit is for inputting video information (or signal), audio information (or signal), data, or information input by a user, and may include at least one of at least one camera, at least one microphone, and a user input unit. Voice data or image data collected by the input unit may be analyzed and processed into a user control command.

[0073] The input unit is intended to receive information from a user, and when information is input through the input unit, the processor (110) can control the operation of the device (100) to correspond to the input information. Such an input unit may include a hardware physical key (e.g., a button, dome switch, jog wheel, jog switch, etc. located on at least one of the front, rear, and side of the device (100)) and a software touch key. As an example, the touch key may be composed of a virtual key, soft key, or visual key displayed on a touchscreen-type display unit through software processing, or may be composed of a touch key placed on a part other than the touchscreen. Meanwhile, the virtual key or visual key may have various forms and may be displayed on the touchscreen, for example, as a graphic, text, icon, video, or a combination thereof.

[0074] The output unit is intended to generate output related to sight, hearing, or touch, and may include at least one of a display unit, an audio output unit, a haptic module, and an optical output unit. The display unit may form a layered structure with a touch sensor or be formed integrally to implement a touch screen. Such a touch screen functions as a user input unit that provides an input interface between the device (100) and a user, and at the same time can provide an output interface between the device (100) and a user.

[0075] The display unit displays (outputs) information processed by the device (100). For example, the display unit may display execution screen information of an application program (e.g., an application) running on the device (100), or UI (User Interface) and GUI (Graphic User Interface) information based on such execution screen information.

[0076] The interface section serves as a passage for various types of external devices connected to the device (100). This interface section may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device (100) equipped with an identification module (SIM), an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The device (100) can perform appropriate control related to the external device connected to the interface section.

[0077] Figure 4 is a diagram illustrating the standardization process for clinical trial images.

[0078] Referring to FIG. 4, in an embodiment of the present disclosure, clinical trial images may use DICOM image data, and the standard protocol may use the CDISC standard data format. Additionally, the SDTM dataset may be used as the dataset for submitting clinical data.

[0079] SDTM is a domain-based model defined as a standard for submitting clinical trial data.

[0080] SDTM is classified into several domains according to the characteristics of clinical trial data, and the clinical trial image automatic classification and standardization device (100) can perform mapping using a data library according to the domain.

[0081] The SDTM domain is a dataset name that classifies data on subjects collected during clinical trials into data sharing common characteristics.

[0082] DICOM File Format: DICOM (Digital Imaging and Communications in Medicine) is a standard file format for storing and transmitting medical images. DICOM files contain patient information, image information, imaging equipment information, etc.

[0083] DICOM image is an abbreviation for Digital Imaging and Communications in Medicine and is a standard format for storing and transmitting medical images. A DICOM image can store images obtained through various medical imaging devices (100), such as X-ray, MRI, CT, and PET. Therefore, in the embodiments of the present disclosure, a clinical trial image may refer to a DICOM image.

[0084] The Study Data Tabulation Model (SDTM) is a model designed to standardize, manage, and analyze clinical trial data. SDTM is a standard developed and managed by the Clinical Data Interchange Standards Consortium (CDISC).

[0085] SDTM can classify clinical trial data into the following three areas.

[0086] Interventions: Treatments, drugs, devices, protocols, etc.

[0087] Events: Events occurring in clinical trials, e.g., side effects, adverse reactions, death, etc.

[0088] Findings: Results found in clinical trials, e.g., biological markers, clinical markers, etc.

[0089] SDTM can subdivide each area into the following elements.

[0090] Domains: Categories representing types of data

[0091] Variables: Individual items of data

[0092] Levels: Possible values ​​of a variable

[0093] SDTM provides clear definitions and rules for each element. Through this, SDTM standardizes clinical trial data, enabling easy exchange and analysis across various systems.

[0094] SDTM helps improve the quality of clinical trial data, increase data reusability, and enhance the reliability of clinical trial results.

[0095] The main features of SDTM are as follows.

[0096] Data Standardization: SDTM standardizes clinical trial data, enabling easy exchange and analysis across various systems.

[0097] Data Reusability: SDTM can be reused in various studies by using standardized data.

[0098] Data Quality Improvement: SDTM improves data quality through data standardization and rules.

[0099] Improving the reliability of clinical trial results: SDTM improves the reliability of clinical trial results through data standardization and rules.

[0100] SDTM is an essential standard for managing and analyzing clinical trial data. By using SDTM, the quality and efficiency of clinical trial data can be improved.

[0101] The main uses of SDTM are as follows.

[0102] Collection of clinical trial data: SDTM can define the structure of the database used to collect clinical trial data.

[0103] Analysis of clinical trial data: SDTM can define the data format of analytical tools used to analyze clinical trial data.

[0104] Reporting of clinical trial data: SDTM can define the format of reports used to report clinical trial data.

[0105] SDTM is a standard for end-to-end management of clinical trial data. Using SDTM, clinical trial data can be effectively collected, analyzed, and reported.

[0106] Figure 5 is a diagram illustrating the automatic standardization of a standardization process like that of Figure 4 using an artificial intelligence model.

[0107] Referring to FIG. 5, the clinical trial image automatic standardization device (100) can perform an automatic standardization process for clinical trial images using a first artificial intelligence model based on text analysis and a second artificial intelligence model based on image analysis.

[0108] FIG. 6 is a flowchart of an artificial intelligence-based method for automatic standardization of clinical trial images according to an embodiment of the present disclosure.

[0109] FIGS. 6 to 15 are various example drawings for explaining an artificial intelligence-based clinical trial image automatic standardization device (100), method, and program according to an embodiment of the present disclosure.

[0110] Below, with reference to FIGS. 6 to 15, the process of the artificial intelligence-based clinical trial image automatic standardization device (100), method, and program according to an embodiment of the present disclosure will be described in detail.

[0111] A processor (110) extracts information from medical standard data within a clinical trial image. (S100)

[0112] The processor (110) obtains a first output result using a first artificial intelligence model. (S200)

[0113] Prior to S100, the processor (110) may further include the step of receiving a clinical trial image through a receiver (120).

[0114] The processor (110) can extract information about at least one attribute from medical standard data within a clinical trial image.

[0115] And, the processor (110) performs rule-based labeling using the extracted information and obtains a first output result using a pre-trained first artificial intelligence model.

[0116] In this case, the DICOM (Digital Imaging and Communications in Medicine) standard can be applied to medical standard data.

[0117] In addition, MRI images may be applied to clinical trials, but are not limited to this.

[0118] Figure 7 is a diagram illustrating the classification of Sequence Types from the DICOM Header of clinical trial images.

[0119] Figure 8 is a diagram illustrating the extraction of DICOM attributes from the DICOM Header of a clinical trial image.

[0120] Referring to FIG. 7, the processor (110) extracts information about at least one attribute from the DICOM Header information of the clinical trial image (MRI image).

[0121] At least one attribute is exemplified in FIGS. 7 and FIGS. 8.

[0122] As shown in Fig. 3, the memory (130) stores a pre-set rule.

[0123] The processor (110) performs rule-based labeling using information extracted for at least one attribute from the header of the DICOM.

[0124] The clinical trial image automatic standardization device (100) according to an embodiment of the present disclosure uses a Rule-Based System to solve the existing manual labeling problem. The processor (110) uses the result value of the Rule-Based System as a learning label to train an artificial intelligence model.

[0125] The processor (110) performs rule-based labeling using information extracted for a first attribute among at least one attribute.

[0126] At this time, the processor (110) performs data normalization on the information extracted for an attribute other than the first attribute (second attribute) among at least one attribute and inputs it into the first artificial intelligence model.

[0127] Referring to Fig. 7, an example is shown of performing rule-based virtual labeling on the SeriesDescription and ProtocolName attributes among multiple attributes. Additionally, for the remaining attributes, an example is shown of performing data standardization and executing a random forest-based process using an artificial intelligence model.

[0128] Here, the first attribute includes at least one attribute that may include a keyword associated with at least one Sequence Type for clinical trial images.

[0129] In one embodiment, the first artificial intelligence model is learned through rule-based labeling for the first attribute among at least one attribute of information extracted from standard data in clinical trial images.

[0130] And, when the processor performs automatic standardization of clinical trial images, it obtains output results using the first artificial intelligence model pre-trained as described above.

[0131] Specifically, a first output result is obtained based on information extracted from standard data within clinical trial images using a pre-trained first artificial intelligence model.

[0132] At this time, the first artificial intelligence model generates a first output result based on the data of the second attribute among at least one attribute among the information extracted from standard data within the clinical trial image.

[0133] Figure 9 is a diagram illustrating an example in which a specific keyword is included in an attribute extracted from a DICOM Header.

[0134] There is a problem in that it is difficult to infer the Sequence Type solely from the SeriesDescription attribute, as the SeriesDescription attribute varies depending on the manufacturer or user organization.

[0135] However, it includes identical or similar keywords to indicate a specific Sequence Type.

[0136] Referring to Fig. 9, an example is shown in which the SeriesDescription property and the ProtocolName property contain the keyword TOF.

[0137] Figure 10 is a diagram illustrating a rule table for automatic labeling.

[0138] Referring to FIG. 10, the clinical trial image automatic standardization device (100) may include a plurality of rule tables.

[0139] Rule Table 1 shown in Fig. 10 refers to the first rule table, and Rule Table 2 refers to the second rule table.

[0140] If the information extracted from S100 contains one Sequence Type, the processor (110) performs labeling based on the first rule table.

[0141] And, if the information extracted from S100 contains multiple Sequence Types, the processor (110) performs labeling based on the second rule table.

[0142] The first rule table is based on the SeriesDescription (SD) value among the DICOM header attributes, and if the SD value contains keywords, it is labeled as a related sequence type.

[0143] However, the processor (110) performs labeling based on the second rule table when two or more sequence types are inferred because two or more different keywords are included.

[0144] The second rule table is a rule that determines priority when two or more sequence types are included.

[0145] For example, referring to Fig. 9, the second rule table performs labeling as T1 when sequence types T1 and FLAIR appear together.

[0146] If the SD does not contain a keyword, labeling is not performed.

[0147] I will briefly explain Sequence Types.

[0148] In medical data, Sequence Type is a term indicating the type of image obtained through an MRI scan. MRI is a test that visualizes the internal structure of the human body, and various types of sequence types can be used to obtain diverse information.

[0149] T1, T2, and Flair are the most basic sequence types used in MRI examinations.

[0150] T1 sequencing is a method of obtaining images by measuring the magnetic rotation time of hydrogen atoms. T1 imaging is effective for distinguishing tissues such as nerve tissue, bone, and fat.

[0151] T2 sequencing is a method of obtaining images by measuring the spread of the magnetic rotation of hydrogen atoms. T2 imaging is effective for distinguishing tissues such as white matter, fluid, and inflammation in the brain.

[0152] The Flair sequence is similar to T2 imaging but is designed to better distinguish the differences between the cerebral cortex and ventricles.

[0153] The Suscgre sequence is a method of obtaining images by utilizing the magnetic rotation properties of hydrogen atoms. Suscgre imaging is effective for distinguishing blood vessels in the brain.

[0154] MRA sequencing is a method that uses Suscgre sequences to observe the blood vessels in the brain in greater detail.

[0155] The SCOUT sequence is a sequence used to scan the patient's body and set the scan range before starting the MRI examination.

[0156] Perfusion sequencing is a method of obtaining images by measuring blood flow. Perfusion imaging is used to diagnose diseases such as stroke and cerebral hemorrhage.

[0157] Diffusion sequencing is a method of obtaining images by measuring the diffusion rate of hydrogen atoms. Diffusion imaging is used to diagnose diseases such as stroke and brain tumors.

[0158] Figure 11 is a diagram illustrating the training of the first artificial intelligence model.

[0159] Referring to Fig. 11, the ML algorithm training set is constructed by excluding data that the Rule Based Labeling System cannot infer from the entire dataset.

[0160] At this time, the clinical trial image automatic standardization device (100) can evaluate the model by using it as a test set because it does not use the data that was not inferred for training.

[0161] The clinical trial image automatic standardization device (100) may use some human labeling in the initial stage, and once an artificial intelligence model is trained to achieve a certain level of accuracy, it can proceed entirely with automatic labeling.

[0162] The first output result includes a classification result for the Sequence Type of the clinical trial image.

[0163] The first artificial intelligence model can generate a first output result based on the Random Forest algorithm.

[0164] The processor (110) obtains a second output result using a second artificial intelligence model. (S300)

[0165] FIG. 12 is a diagram illustrating the acquisition of a second output result using an image-based second artificial intelligence model.

[0166] The second artificial intelligence model may be a model trained on image-based analysis methods.

[0167] The processor (110) can go through the process of converting the clinical trial images into a preset image format.

[0168] To this end, the clinical trial image automatic standardization device (100) may further include an image conversion module.

[0169] Referring to FIG. 12, the processor (110) converts a DICOM image into a PNG image.

[0170] Next, the processor (110) performs data classification using an artificial intelligence model and performs a majority vote to obtain a second output result.

[0171] That is, the processor (110) obtains a second output result through the following process.

[0172] The processor (110) receives clinical trial images through the communication unit.

[0173] The processor (110) converts the clinical trial image format using an image conversion module.

[0174] The processor (110) inputs the format-converted image data into the classification model.

[0175] The processor (110) obtains an output result including a Sequence Type from a classification model.

[0176] A majority vote is performed on the output result obtained by the processor (110), and the Type that appears most frequently is obtained as the second output result.

[0177] The processor (110) standardizes the clinical trial image based on the first output result and the second output result. (S400)

[0178] FIG. 13 is a diagram illustrating the accuracy of the output result obtained using the first artificial intelligence model.

[0179] Figure 14 is a diagram illustrating the accuracy of the output result obtained using the second artificial intelligence model.

[0180] FIG. 15 is a diagram illustrating a process of ensembling output results obtained using a first artificial intelligence model and a second artificial intelligence model.

[0181] Text-based first artificial intelligence models, such as Scout and Perfusion, may have slightly lower classification accuracy when there are no attribute values ​​(Features) in the DICOM header.

[0182] Image-based second AI models may experience slightly lower accuracy in sequence types with many variations.

[0183] Referring to FIG. 15, if the first output result and the second output result are the same, the processor (110) obtains the first output result or the second output result as the final output value and standardizes the clinical trial image based thereon.

[0184] At this time, if the first output result and the second output result are not the same, the processor (110) checks whether there is sufficient medical standard data in the clinical trial image.

[0185] In detail, the processor (110) checks whether the amount of medical standard data (DICOM header) satisfies a preset condition.

[0186] And, if the processor (110) determines that there is insufficient standard medical data, it obtains the second output result of the second artificial intelligence model as the final output value and standardizes the clinical trial image based on this.

[0187] The processor (110) obtains a final output value based on the first output result and the second output result, and can select the best value by using the first output result and the second output result complementarily.

[0188] For example, if the processor (110) determines that there is sufficient standard medical data, it can check whether the first output result includes perfusion.

[0189] And, if the first output result includes Perfusion, the processor (110) obtains the second output result of the second artificial intelligence model as the final output value and standardizes the clinical trial image based on this.

[0190] The method according to one embodiment of the present disclosure described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a server, which is hardware.

[0191] The aforementioned program may include code encoded in a computer language such as C, C++, JAVA, or machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.

[0192] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.

[0193] The steps of the method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs.

[0194] Although embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the present disclosure may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0195] 100: Clinical Trial Image Automatic Standardization Device 110: Processor 120: Receiver 130: Memory

Claims

Claim 1 An AI-based automatic standardization device for clinical trial images, comprising: a receiving unit for receiving clinical trial images; a memory storing at least one instruction; and a processor, wherein the processor executes the at least one instruction to extract information regarding at least one attribute from medical standard data within the clinical trial image, obtains a first output result based on the extracted information using a pre-trained first artificial intelligence model, analyzes the clinical trial image based on images using a pre-trained second artificial intelligence model to obtain a second output result, and standardizes the clinical trial image based on the first output result and the second output result, wherein if the first output result and the second output result are not identical, checks whether the amount of medical standard data within the clinical trial image satisfies a preset condition, and if the preset condition is not satisfied, standardizes the clinical trial image based on the second output result, and if the preset condition is satisfied, checks whether Perfusion is included in the Sequence Type included in the first output result, and if Perfusion is included, standardizes the clinical trial image based on the second output result. Claim 2 An artificial intelligence-based clinical trial image automatic standardization device according to claim 1, wherein the medical standard data is data relating to the Digital Imaging and Communications in Medicine (DICOM) standard, and the processor extracts information regarding at least one attribute from the header of the DICOM. Claim 3 An artificial intelligence-based clinical trial image automatic standardization device according to claim 2, wherein the first artificial intelligence model is characterized by being learned through rule-based labeling using the extracted information. Claim 4 An artificial intelligence-based clinical trial image automatic standardization device according to claim 3, wherein the first artificial intelligence model is trained by performing rule-based labeling using information extracted for the first attribute among the at least one attribute. Claim 5 An artificial intelligence-based clinical trial image automatic standardization device according to claim 4, wherein the processor performs data standardization on information extracted for a second attribute other than the first attribute among the at least one attribute and inputs it into the first artificial intelligence model. Claim 6 An artificial intelligence-based clinical trial image automatic standardization device according to claim 4, wherein the first attribute comprises at least one attribute that may include a keyword associated with at least one sequence type for the clinical trial image. Claim 7 An artificial intelligence-based clinical trial image automatic standardization device according to claim 6, wherein the rule includes a first rule table and a second rule table, and the first artificial intelligence model performs labeling based on the first rule table when the extracted information includes one Sequence Type, and performs labeling based on the second rule table when the extracted information includes multiple Sequence Types. Claim 8 An artificial intelligence-based clinical trial image automatic standardization device, wherein, in claim 5, the first output result includes a classification result for a Sequence Type for the clinical trial image, and the first artificial intelligence model generates the first output result based on a random forest algorithm. Claim 9 An artificial intelligence-based clinical trial image automatic standardization device according to claim 1, wherein the processor obtains the second output result by conducting a majority vote on the classification result for the Sequence Type based on the clinical trial image using the second artificial intelligence model. Claim 10 delete Claim 11 delete Claim 12 A method for automatically standardizing an AI-based clinical trial image, comprising: a step of extracting information about at least one attribute from medical standard data within a clinical trial image, performed by a device; a step of obtaining a first output result based on the extracted information using a pre-trained first artificial intelligence model; a step of obtaining a second output result by analyzing the clinical trial image based on an image using a pre-trained second artificial intelligence model; and a step of standardizing the clinical trial image based on the first output result and the second output result, wherein the standardization step comprises: a step of checking whether the amount of medical standard data within the clinical trial image satisfies a pre-set condition when the first output result and the second output result are not identical; a step of standardizing the clinical trial image based on the second output result when the pre-set condition is not satisfied; and a step of checking whether Perfusion is included in the Sequence Type included in the first output result when the pre-set condition is satisfied, and if Perfusion is included, a step of standardizing the clinical trial image based on the second output result. Claim 13 A computer-readable recording medium combined with a computer, which is hardware, and storing a program for executing the method of claim 12.

Images