Multimodal computer-aided diagnosis system and method for prostate cancer

Abstract

Methods and apparatus for computer-aided prostate condition diagnosis are disclosed. An example computer-aided prostate condition diagnosis apparatus includes a memory to store instructions and a processor. The example processor can detect a lesion from an image of a prostate and generate a mapping of the lesion from the image to a sector mapping, the generating the mapping of the lesion including identifying a depth region of the lesion, where the depth region indicates a location of the lesion along a depth axis. The processor can also provide the sector mapping including a representation of the lesion within the prostate mapped from the image to the sector mapping.

Description

[0001] Cross-references to related applications

[0002] This matter is a continuation-in-part of U.S. Patent Application Serial No. 16 / 196,887, filed November 20, 2018, which claims the benefit of U.S. Provisional Patent Application Serial No. 62 / 590,266, filed November 22, 2017. The entire contents of U.S. Patent Application Serial No. 16 / 196,887 and U.S. Provisional Patent Application Serial No. 62 / 590,266 are hereby incorporated herein by reference. Priority is claimed by U.S. Patent Application Serial No. 16 / 196,887 and U.S. Provisional Patent Application Serial No. 62 / 590,266. Technical Field

[0003] This disclosure relates generally to improved medical systems, and more specifically to improved computer-aided diagnostic systems and methods for medical image processing. Background Technology

[0004] A variety of economic, technical, and managerial barriers pose challenges to healthcare institutions (such as hospitals, clinics, and doctors' offices) that provide quality care to patients. Economic motives, under-skilled staff, small workforces, complex equipment, and the recent rise in certification for controlling and standardizing radiation exposure dosages in healthcare enterprises have created difficulties for the effective management and use of imaging and information systems for patient examinations, diagnosis, and treatment.

[0005] The consolidation of healthcare providers has created geographically dispersed hospital networks, where physical contact with the systems is too expensive. Meanwhile, referring physicians want more direct access to supporting data in reports and better collaboration channels. Physicians have more patients, less time, and are overwhelmed by massive amounts of data, and they crave support.

[0006] Healthcare provider tasks, including image processing and analysis, are time-consuming and resource-intensive, and it is impractical, if not impossible, for humans to complete them alone. Summary of the Invention

[0007] In one aspect, a computer-aided diagnostic apparatus for prostate conditions is provided. The exemplary apparatus includes a memory for storing instructions and a processor. The exemplary processor can detect lesions from an image of the prostate and generate a mapping of the lesions from the image to sectors, the generation of the lesion mapping including identifying depth regions of the lesions, wherein the depth regions indicate the location of the lesions along a depth axis. In some examples, the processor may also provide a sector mapping including a representation of the lesions within the prostate from the image mapping to the sector mapping.

[0008] In some examples, the sector mapping can provide a classification of lesions mapped from an image, which provides an assessment of prostate health. In one aspect, the depth region can be identified using apex, middle, and basal regions of the prostate. In some examples, generating the mapping may include calculating one or more polar coordinates relative to the center of the lesion in the image. In some aspects, generating the mapping may include calculating a normalized radius based on the center of the lesion in the image. In some aspects, generating the mapping may include calculating a denormalized radius based on the normalized radius and one or more dimensions of the sector mapping.

[0009] In some examples, generating the mapping may include calculating Cartesian coordinates representing the diameter of the lesion and the sector mapping within it. In some aspects, providing the sector mapping may include transmitting a representation of the sector mapping and the lesion-to-sector mapping to a display device, wherein the representation of the lesion mapping may include the Cartesian coordinates of the lesion and the denormalized radius of the lesion. In some aspects, the display device is electrically connected to the device, or wherein the display device is connected to a remote device that receives the representation of the sector mapping and the lesion-to-sector mapping from the device. In some examples, the image may include a three-dimensional volume. In some aspects, the processor may generate the lesion mapping using a digital twin and generate a score using machine learning techniques and the provided sector mapping including the representation of the mapping.

[0010] In some aspects, a non-transitory machine-readable storage medium may include instructions that, in response to execution by a processor, cause the processor to detect lesions from an image of the prostate and generate a mapping of the lesions from the image to sectors, wherein generating the mapping of the lesions may include identifying depth regions of the lesions, and wherein the depth regions indicate the location of the lesions along a depth axis. The instructions may also cause the processor to: provide a sector mapping including a representation of the lesions within the prostate from the image mapping to the sector mapping; and display the sector mapping having a representation of the lesions within the prostate.

[0011] In some aspects, a method for computer-aided diagnosis of prostate conditions may include detecting lesions from an image of the prostate and generating a mapping of the lesions from the image to sector maps, the generation of the lesion mapping including identifying depth regions of the lesions, wherein the depth regions indicate the location of the lesions along a depth axis. The method may also include providing: providing a sector map including a representation of the lesions within the prostate from the image mapping to the sector map; and displaying the sector map having the representation of the lesions within the prostate. Attached Figure Description

[0012] Figure 1A An exemplary imaging system is shown to which the methods, apparatus and articles of manufacture disclosed herein can be applied.

[0013] Figure 1B An exemplary computer-aided prostate analysis system is shown.

[0014] Figure 2 An exemplary digital twin environment is described.

[0015] Figure 3 This is an example representation of a learning neural network.

[0016] Figure 4 A specific implementation of an exemplary neural network, known as a convolutional neural network, is shown.

[0017] Figure 5 This is a representation of an exemplary implementation of a convolutional neural network for image analysis.

[0018] Figure 6A An exemplary configuration for applying a learning network to process and / or otherwise evaluate an image is shown.

[0019] Figure 6B A combination of multiple learning networks is shown.

[0020] Figure 7 An exemplary training and deployment phase of the learning network is shown.

[0021] Figure 8 An exemplary product is shown that utilizes trained network packets to provide deep learning product offerings.

[0022] Figures 9A to 9C Various deep learning device configurations are shown.

[0023] Figure 10 A flowchart of an exemplary method for computer-driven prostate analysis is shown.

[0024] Figures 11 to 19C An exemplary interface is depicted to facilitate prostate analysis and associated patient diagnosis and treatment.

[0025] Figure 20 It is a block diagram of a processor platform configured to execute exemplary machine-readable instructions to implement the components disclosed and described herein.

[0026] Figure 21 This is a flowchart of an exemplary method for providing a sector mapping with a representation of lesions.

[0027] Figure 22 A schematic diagram illustrating an example of mapping lesions from an image to a sector map, according to the examples in this article, is shown.

[0028] Figure 23A schematic diagram illustrating an example of mapping a lesion to the central or peripheral area of ​​the prostate, according to examples in this article.

[0029] The foregoing summary of the invention and the following detailed description of certain embodiments thereof will be better understood when read in conjunction with the accompanying drawings. Certain embodiments are illustrated in the drawings for illustrative purposes. However, it should be understood that the invention is not limited to the arrangements and tools shown in the drawings. The drawings are not drawn to scale. Throughout all the drawings and the accompanying written description, the same reference numerals will be used to refer to the same or similar parts whenever possible. Detailed Implementation

[0030] In the following detailed description, reference is made to the accompanying drawings, which form a part thereof, and specific examples that can be practiced are illustrated therein. These examples are described in sufficient detail to enable those skilled in the art to practice the subject matter, and it should be understood that other examples may be utilized, and logical, mechanical, electrical, and other changes may be made without departing from the scope of the subject matter of this disclosure. Therefore, the purpose of providing the following detailed description is to describe exemplary embodiments and not to be construed as limiting the scope of the subject matter described herein. Certain features from different aspects of the following description may be combined to form new aspects of the subject matter discussed below.

[0031] When describing elements of various embodiments of this disclosure, the terms “an,” “a,” and “the” are intended to refer to one or more of these elements. The terms “first,” “second,” etc., do not indicate any order, quantity, or importance, but are used to distinguish one element from another. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to the listed elements. As used herein, the terms “connected to,” “linked to,” etc., indicate that an object (e.g., a material, element, structure, component, etc.) may be connected to or linked to another object, regardless of whether the object is directly connected to or linked to the other object, or whether one or more intervening objects exist between the object and the other object.

[0032] As used herein, the terms “system,” “unit,” “module,” “engine,” etc., can include hardware and / or software systems that operate to perform one or more functions. For example, a module, unit, or system can include a computer processor, controller, and / or other logic-based devices that perform operations based on instructions stored on a tangible, non-transitory computer-readable storage medium, such as computer memory. Alternatively, a module, unit, engine, or system can include a hardwired device that performs operations based on device-based hardwired logic. The various modules, units, engines, and / or systems illustrated in the accompanying drawings can represent hardware that operates based on software or hardwired instructions, software that instructs the hardware to perform operations, or a combination thereof.

[0033] Furthermore, it should be understood that references to “one implementation” or “implementation” in this disclosure are not intended to be construed as excluding the existence of additional implementations that also include the features referenced.

[0034] Overview

[0035] Imaging devices (e.g., gamma cameras, positron emission tomography (PET) scanners, computed tomography (CT) scanners, X-ray machines, magnetic resonance (MR) imaging machines, ultrasound scanners, etc.) generate medical images (e.g., raw medical digital imaging and communications (DICOM) images) representing body parts (e.g., organs, tissues, etc.) for the diagnosis and / or treatment of diseases. For example, MR is a medical imaging modality that generates images of the interior of the human body without the use of X-rays or other ionizing radiation. MR uses a master magnet to generate a strong, uniform static magnetic field (e.g., a “master magnetic field”) and gradient coils to generate a spatially varying magnetic field with small amplitudes when an electric current is applied to the gradient coils. When the human body or a part of the human body is placed in the master magnetic field, the nuclear spins associated with hydrogen nuclei in the tissue water become polarized. The magnetic moments associated with these spins preferentially align along the direction of the master magnetic field, resulting in less net tissue magnetization along this axis (conventionally, the “z-axis”), and the gradient coils encode the MR signal.

[0036] The acquisition, processing, analysis, and storage of medical image data play a crucial role in patient diagnosis and treatment within healthcare settings. Medical imaging workflows, and the equipment involved within them, can be configured, monitored, and updated throughout their operation. Machine learning, deep learning, and / or other artificial intelligence technologies can be used, for example, to assist in configuring, monitoring, and updating medical imaging workflows and equipment.

[0037] Some examples provide and / or facilitate improved imaging equipment, thereby increasing diagnostic accuracy and / or coverage. Some examples facilitate improved image reconstruction and further processing, thereby increasing diagnostic accuracy.

[0038] Some examples provide improved management and analysis of medical images, including MR images, where computer-aided diagnostics (CAD) and / or other artificial intelligence can be applied to identify and classify abnormalities / malformations such as prostate cancer.

[0039] Some examples improve MR imaging and image data processing techniques to enable automated, multi-part clinical analysis of oncology scoring and CAD, thereby obtaining a patient disease determination (e.g., prostate cancer) and sending / reporting it to another clinical system, expert, medical record, etc. Other examples provide automated processing techniques to improve image segmentation, oncology scoring, report generation, etc., to reduce, minimize, or eliminate user interaction during the detection / diagnosis process.

[0040] Some examples collect patient medical history and assess patients' prostate-specific antigen (PSA) levels based on blood test data. PSA is a substance produced by the prostate gland, and elevated PSA levels can indicate prostate cancer or non-cancerous conditions such as prostate enlargement. For example, the system can calculate prostate volume and PSA density using image data (e.g., axial, sagittal, etc.), apparent diffusion coefficient (ADC), blood flow mapping information, etc. Then, for example, using computer-aided detection and / or user input, image data, ADC information, density, segmentation, and / or other automated image data analysis can be used to identify lesions in the patient's prostate. Regions of interest (ROIs) can be defined around the identified, probable, and / or approximate lesions to label one or more lesions in one or more images. For example, lesions in the ROI can then be segmented (e.g., along the long axis, etc.) and scored by the system (e.g., to determine the likelihood of lesion validation, malignancy / severity, size, etc.). For example, deep learning, machine learning, and / or other artificial intelligence can be used to automatically segment and calculate prostate volume and / or automatically segment, locate, and score one or more lesions in / on the prostate. It can generate possible prostate cancer diagnoses, patient care plans / treatment triggers, reports for urologists and / or other clinicians, including scores, detailed lesion information, observations, opinions, and conclusions.

[0041] Apparent diffusion coefficient (ADC) images, or ADC mappings, are MR images that show diffusion more specifically than conventional DWI by removing certain (e.g., T2) weightings inherent in conventional diffusion-weighted imaging (DWI). ADC imaging does this by acquiring multiple conventional DWI images with varying amounts of DWI weighting, and the changes in signal are proportional to the diffusion rate.

[0042] For example, scores (such as pirads or pi-rads scores) can indicate the likelihood of cancerous / tumor tissue. PI-RADS is an acronym for Prostate Imaging Reporting and Data Systems, defining quality standards for multi-parameter MR imaging, including image creation and reporting. A PI-RADS score is provided for each variable parameter based on a "yes" or "no" score for the dynamic contrast enhancement (DCE or Dice) parameter; for example, T2-weighted (T2W) and diffusion-weighted imaging (DWI) are scored from 1 to 5. A score is assigned for each detected lesion, where 1 is the most likely benign and 5 is highly suspicious for malignancy. For example, pirads 1 is “very low” (e.g., very unlikely to have clinically significant cancer); pirads 2 is “low” (e.g., unlikely to have clinically significant cancer); pirads 3 is “moderate” (e.g., the presence of clinically significant cancer is uncertain); pirads 4 is “high” (e.g., clinically significant cancer may be present); and pirads 5 is “very high” (e.g., very likely to have clinically significant cancer).

[0043] For example, machine learning techniques (whether deep learning networks or other experiential / observational learning systems) can be used to locate objects in images, understand speech and convert speech to text, and improve the relevance of search engine results. Deep learning is a subset of machine learning that uses a set of algorithms to model high-level abstractions in data using a depth map with multiple processing layers (including linear and non-linear transformations). While many machine learning systems first embed initial features and / or network weights and then modify them through learning and updates by the machine learning network, deep learning networks learn by themselves to identify “good” features for analysis. When using a multi-layered architecture, machines employing deep learning techniques can process raw data better than those using conventional machine learning techniques. Using different layers for evaluation or abstraction facilitates data examination of highly correlated values ​​or distinctive topics.

[0044] Exemplary magnetic resonance imaging system

[0045] Go to Figure 1AThe diagram illustrates the main components of an exemplary magnetic resonance imaging (MRI) system 10. Operation of the system is controlled via an operator console 12, which includes a keyboard or other input device 13, a control panel 14, and a display screen 16. The console 12 communicates with a separate computer system 20 via a link 18, enabling the operator to control the generation and display of images on the display screen 16. The computer system 20 includes multiple modules that communicate with each other via a backplane 20a. These modules include an image processor module 22, a CPU module 24, and a memory module 26 (which may include a frame buffer for storing an array of image data). The computer system 20 is linked to archival media devices, permanent or backup storage, or a network for storing image data and programs, and communicates with a separate system control 32 via a high-speed serial link 34. The input device 13 may include a mouse, joystick, keyboard, trackball, touch-activated screen, light stick, voice control, or any similar or equivalent input device and may be used for interactive geometry requirements.

[0046] System control 32 includes a set of modules connected together via a backplane 32a. These modules include a CPU module 36 and a pulse generator module 38, which is connected to the operator console 12 via a serial link 40. Through link 40, system control 32 receives commands from the operator instructing the scan sequence to be executed. The pulse generator module 38 operates the system components to execute the desired scan sequence and generates data instructing the timing, intensity, and shape of the generated RF pulses, as well as the timing and length of the data acquisition window. The pulse generator module 38 is connected to a set of gradient amplifiers 42 to instruct the timing and shape of gradient pulses generated during the scan. The pulse generator module 38 also receives patient data from a physiological acquisition controller 44, which receives signals from multiple different sensors connected to the patient, such as ECG signals from electrodes attached to the patient. The pulse generator module 38 is connected to a scan chamber interface circuit 46, which receives signals from various sensors associated with the patient and magnet system status. The patient positioning system 48 also receives commands via the scan chamber interface circuit 46 to move the patient to the desired position for scanning.

[0047] The gradient waveform generated by the pulse generator module 38 is applied to a gradient amplifier system 42 having Gx, Gy, and Gz amplifiers. Each gradient amplifier excites a corresponding physical gradient coil in a gradient coil assembly generally labeled 50 to generate a magnetic field gradient for spatial encoding of the acquired signal. The gradient coil assembly 50 forms part of a magnet assembly 52 including a polarizing magnet 54 and a whole-body RF coil 56. In one embodiment of the invention, the RF coil 56 is a multi-channel coil. The transceiver module 58 in system control 32 generates a pulse that is amplified by the RF amplifier 60 and coupled to the RF coil 56 via a transmit / receive switch 62. The resulting signal emitted by the excitation nucleus within the patient can be sensed by the same RF coil 56 and coupled to a preamplifier 64 via the transmit / receive switch 62. The amplified MR signal is demodulated, filtered, and digitized in the receiver section of the transceiver 58. The transmit / receive switch 62 is controlled by a signal from the pulse generator module 38 to electrically connect the RF amplifier 60 to the coil 56 during transmit mode and to connect the preamplifier 64 to the coil 56 during receive mode. The transmit / receive switch 62 may also enable a separate RF coil (e.g., a surface coil) to be used in either transmit or receive mode.

[0048] The MR signal received / detected by the multi-channel RF coil 56 is digitized by the transceiver module 58 and transmitted to the memory module 66 in the system control 32. The scan is complete when an array of raw k-space data is acquired in the memory module 66. For each image to be reconstructed, the raw k-space data is rearranged into separate k-space data arrays, and each of these separate k-space data arrays is input to the array processor 68, which operates to perform a Fourier transform on the data into an array of image data. This image data is transmitted to the computer system 20 via serial link 34, whereby it is stored in memory. In response to a command received from the operator console 12, the image data may be archived in long-term storage or further processed by the image processor 22 and transmitted to the operator console 12, where it is displayed on the monitor 16.

[0049] Exemplary computer-aided prostate analysis system

[0050] Figure 1B An exemplary computer-aided prostate analysis system 100 is shown, which includes an image acquisition module 110, a prostate detector 120, a prostate evaluator 130, a lesion identification and evaluation device 140 (also referred to herein as a lesion evaluator), and a results generator 150.

[0051] Exemplary system 100 implements computer-aided diagnosis and classification of prostate cancer. Some examples analyze prostate information and generate predictions and / or other analyses regarding possible prostate cancer, malignant lesions, and / or other prostate problems. For example, some examples use multimodal, multi-scheme MR data to locate prostate lesions on a prostate sector map and integrate prostate lesion information from a computer-aided diagnosis and classification system for prostate cancer.

[0052] An exemplary image acquisition module 110 acquires image data of a patient, such as ADC images, DWI images, and / or other MR image data. For example, the image data may include the patient's prostate. For example, the image acquisition module 110 may preprocess the image data to prepare it for further analysis. For example, contrast, window level, etc., may be adjusted to emphasize the prostate in the image data.

[0053] An exemplary prostate detector 120 processes image data to identify the prostate in an image. For example, the prostate detector 120 can identify the prostate in image data based on pixel density / intensity values. In other examples, images can be segmented and scored to identify and register the prostate in the image (e.g., MR images, 3D volumes, etc.).

[0054] An exemplary prostate evaluator 130 processes image data in conjunction with patient medical history information and determines the patient's prostate-specific antigen (PSA) level. Elevated PSA levels (indicating that the amount of prostate-specific antigen in the patient's bloodstream is greater than normal) can be an indicator of prostate cancer in an associated patient. For example, the prostate evaluator 130 can segment the prostate in the image and calculate its volume (e.g., using deep learning-based methods, etc.). For example, the prostate evaluator 130 can deposit distances (e.g., three distances, etc.) onto the image (e.g., using a dedicated distance tool, etc.) and can automatically calculate prostate volume and PSA density.

[0055] An exemplary lesion identification and evaluation unit 140 identifies and processes lesions in image data. For example, the lesion identification and evaluation unit 140 can identify and process lesions in images by depositing graphic objects (e.g., indicating regions of interest (e.g., along their long axis, etc.)) on lesions in one or more acquired images. For example, ellipses are deposited on a prostate sector map, where a pattern and one or more sectors below the map are automatically selected (e.g., ellipses are deposited on axial, sagittal, and coronal planes to automatically select corresponding sectors, etc.). The lesions can then be scored by the lesion identification and evaluation unit 140 according to the PIRADS v2 guidelines. Alternatively, one or more lesions can be automatically segmented and then located and scored according to each available MR imaging technique (e.g., using non-rigid registration of the segmented prostate and a 3D model of the prostate sector map, as well as deep learning-based methods, etc.). For example, a global score can be automatically calculated based on lesion scoring from various MR techniques. Alternatively, one or more lesions can be identified using available tools, algorithms, digital twins, etc. In some examples, lesions can be scored based on the representation of lesions mapped from an image to a sector map by the lesion mapper 160 described below.

[0056] Based on lesion information, conclusions, recommendations, and / or other assessments can be made regarding one or more possible prostate problems. Qualitative assessment, hidden layer processing in deep neural networks, and analysis of edges, combinations of one or more edges, object models, etc., enable deep neural networks to correlate MR image data with possible prostate lesions and / or other defects requiring further verification, treatment, etc. For example, convolution, deconvolution, forward inference, and backward learning from image segmentation and pixel intensity data can help drive the correlation between MR image information and prostate cancer identified via CAD.

[0057] Based on lesion analysis, reports and / or triggers for the next action can be generated and output by the exemplary results generator 150. For example, reports can be generated, saved, output, transmitted, etc. For example, patient history (e.g., including identifying trends in PSA levels, etc.), prostate volume, PSA levels, PSA density, lesion details, index lesions, opinions, PI-RADS assessments, conclusions, etc., can be provided (e.g., transmitted to another program, triggered another process, saved, displayed, and / or otherwise output) based on the analysis to drive further actions regarding the patient.

[0058] In some examples, lesion mapper 160 can detect data representing lesions in two-dimensional or three-dimensional space and map the data representing lesions to a sector map. For example, lesion mapper 160 can detect, acquire, or receive two-dimensional images representing slices of the prostate. In some examples, lesion mapper 160 can acquire or compute two-dimensional images by identifying or selecting a subset of data from a three-dimensional image or model representing any suitable two-dimensional slice of the prostate. Lesion mapper 160 can use any suitable technique (such as those described below relative to...) Figures 21 to 23 The described technique maps lesions from an image to a sector map. In some examples, the lesion mapper 160 may also indicate areas in the prostate where the lesion is located (such as the central or peripheral area).

[0059] Digital Twin Examples

[0060] In some examples, digital representations of patients and their anatomical structures / regions (e.g., the prostate, etc.) can be used for computer-aided detection and / or diagnosis of prostate cancer. Digital representations, digital models, digital "twins," or digital "shadows" are digital informatics constructs concerning physical systems, processes, etc. That is, digital information can be realized as a "twin" of a physical device / system / person / process and information associated with and / or embedded within the physical device / system / process. Digital twins are linked to physical systems through the lifecycle of the physical system. In some examples, a digital twin includes a physical object in real space, a digital twin of that physical object existing in virtual space, and information linking the physical object to its digital twin. Digital twins exist in a virtual space corresponding to the real space and include links for data flows from the real space to the virtual space and connections for information flows from the virtual space to the real space and virtual subspaces.

[0061] For example, Figure 2 The diagram illustrates how the patient, prostate, and / or other structural / anatomical regions 210 in the real space 215 provide data 220 to a digital twin 230 in the virtual space 235. The digital twin 230 and / or its virtual space 235 provide information 240 back to the real space 215. The digital twin 230 and / or the virtual space 235 may also provide information to one or more virtual subspaces 250, 252, 254. Figure 2 As shown in the example, virtual space 235 may include one or more virtual subspaces 250, 252, 254 and / or be associated with one or more virtual subspaces, which can be used to model one or more parts of digital twin 230 and / or digital “sub-twins”, thereby modeling subsystems / subparts of the overall digital twin 230.

[0062] Sensors connected to a physical object (e.g., patient 210) can collect data and relay the collected data 220 to a digital twin 230 (e.g., via self-reporting, using clinical or other health information systems such as Image Archiving and Communication Systems (PACS), Radiology Information Systems (RIS), Electronic Medical Record Systems (EMR), Laboratory Information Systems (LIS), Cardiovascular Information Systems (CVIS), Hospital Information Systems (HIS), MR imaging scanners, and / or combinations thereof). For example, interaction between the digital twin 230 and the patient / prostate 210 can help improve the diagnosis, treatment, and health maintenance of the patient 210 (such as the identification of prostate diseases). Benefiting from an accurate digital description 230 of the patient / prostate 210 in real-time or near real-time (e.g., considering data transmission, processing, and / or storage latency) allows the system 200 to predict “failures” that may occur in the form of diseases, functional impairments, and / or other ailments, symptoms, etc.

[0063] In some examples, images overlaid with sensor data, lab results, etc., obtained while a healthcare practitioner is examining, treating, and / or otherwise caring for a patient 210 can be used in augmented reality (AR) applications. For instance, a digital twin 230 uses AR to track a patient's responses to interactions with a healthcare practitioner. Therefore, a patient's prostate can be modeled to identify changes in appearance, lab results, scores, and / or other characteristics to indicate prostate problems such as cancer, assess the problem, model / predict treatment options, etc.

[0064] Therefore, the digital twin 230 is not a general model, but rather a collection of physical, anatomical, and / or biological models reflecting the patient / prostate 210 and his or her associated norms, conditions, etc. In some examples, a three-dimensional (3D) model of the patient / prostate 210 creates a digital twin 230 for the patient / prostate 210. For example, a prostate evaluator 130 can use the digital twin 230 to determine (e.g., model, simulate, infer, etc.) and view the state of the patient / prostate 210 based on dynamically provided input data 220 from sources (e.g., from the patient 210, imaging systems, practitioners, health information systems, sensors, etc.).

[0065] In some examples, the prostate evaluator 130 can use a digital twin 230 of the patient / prostate 210 to monitor, diagnose, and predict the prognosis of the patient / prostate 210. Sensor data can be combined with historical information to identify, predict, and monitor current and / or potential future conditions of the patient / prostate 210 using the digital twin 230. The digital twin 230 can be used to monitor causes, exacerbations, and improvements. The digital twin 230 can be used to simulate and visualize the physical behavior of the patient / prostate 210 for diagnosis, treatment, monitoring, and maintenance.

[0066] Unlike computers, humans do not process information in an orderly, step-by-step manner. Instead, humans attempt to conceptualize problems and understand their context. While humans can view data in reports, tables, etc., they are most effective when they visually examine problems and attempt to discover their solutions. However, information is often lost when humans process information visually, record it in alphanumeric form, and then attempt to visually reconceptualize it, and the problem-solving process becomes extremely inefficient over time.

[0067] However, using a digital twin 230 allows people and / or systems to view and assess visualizations of situations (e.g., patient / prostate 210 and associated patient problems, etc.) without having to translate data back and forth. Utilizing a digital twin 230 with a shared perspective with the actual patient / prostate 210, both physical and virtual information can be viewed dynamically and in real-time (or near real-time, taking into account data processing, transmission, and / or storage latency). Healthcare practitioners do not read reports but instead use the digital twin 230 to view and simulate patient / prostate 210 symptoms, progression, potential treatments, etc. In some examples, features, symptoms, trends, indicators, traits, etc., can be labeled and / or otherwise marked in the digital twin 230 to allow practitioners to quickly and easily view specified parameters, values, trends, alerts, etc.

[0068] The digital twin 230 can also be used for comparisons (e.g., with patient / prostate 210, with “normal,” standard, or reference patients, a set of clinical standards / symptoms, best practices, protocol procedures, etc.). In some examples, the digital twin 230 of patient / prostate 210 can be used to measure and visualize the ideal or “gold standard” value state of that patient / protocol / item, the tolerance or standard deviation around that value (e.g., positive and / or negative deviations relative to the gold standard value), the actual value, the trend of the actual value, etc. The difference between the actual value or the trend of the actual value and the gold standard (e.g., exceeding the tolerance deviation) can be visualized as alphanumeric values, color indicators, patterns, etc.

[0069] Furthermore, the digital twin 230 of patient 210 can facilitate collaboration among patient 210's friends, family, care providers, etc. Using the digital twin 230, a conceptualization of patient 210 and his / her health (e.g., according to a care plan) can be shared among multiple people, including care providers, family, friends, etc. For example, people do not need to be in the same location as patient 210, nor do they need to be in the same location as each other, yet they can still view the same digital twin 230, interact with it, and draw conclusions from it.

[0070] Therefore, a digital twin 230 can be defined as a set of virtual information concepts that describe (e.g., fully describe) patient 210 from a microscopic level (e.g., heart, lungs, feet, prostate, anterior cruciate ligament (ACL), stroke history, etc.) to a macroscopic level (e.g., overall anatomy, holistic view, skeletal system, nervous system, vascular system, etc.). Similarly, a digital twin 230 can represent items and / or protocols at various levels of detail, such as macroscopic, microscopic, etc. In some examples, a digital twin 230 can be a reference digital twin (e.g., a digital twin prototype, etc.) and / or a digital twin instance. A reference digital twin represents a prototype or "gold standard" model of patient / prostate 210 or a specific type / category of patient / prostate 210, while one or more reference digital twins represent one or more specific patient / prostate 210s. Thus, a digital twin 230 of a pediatric patient 210 can be implemented as a pediatric reference digital twin organized according to certain criteria or "typical" pediatric characteristics, with a specific digital twin instance representing a specific pediatric patient 210. In some examples, multiple digital twin instances can be aggregated into a digital twin aggregation (e.g., to represent the accumulation or combination of multiple pediatric patients sharing a common reference digital twin, etc.). For example, digital twin aggregation can be used to identify differences, similarities, trends, etc., between children represented by pediatric digital twin instances.

[0071] In some examples, the virtual space 235 in which the digital twin 230 (and / or multiple digital twin instances, etc.) operates is referred to as a digital twin environment. The digital twin environment 235 provides an integrated multi-domain application space for operating the digital twin 230, including both physical and / or biological applications. For example, the digital twin 230 can be analyzed within the digital twin environment 235 to predict future behavior, symptoms, progression, etc., of a patient / protocol / project 210. The digital twin 230 can also be queried or questioned within the digital twin environment 235 to retrieve and / or analyze current information 240, past medical history, etc.

[0072] In some examples, the digital twin environment 235 can be divided into multiple virtual spaces 250 to 254. Each virtual space 250 to 254 can model different digital twin instances and / or components of the digital twin 230, and / or each virtual space 250 to 254 can be used to perform different analyses, simulations, etc., on the same digital twin 230. Using multiple virtual spaces 250 to 254, the digital twin 230 can be tested inexpensively and efficiently in a variety of ways while keeping the patient 210 safe. For example, a healthcare provider can then understand how the patient / prostate 210 might respond to various treatments in different scenarios. The continuity, triggering, periodicity, and / or other inputs 260 from the real space to the virtual space enable the digital twin 230 to continue evolving.

[0073] Exemplary deep learning and other machine learning

[0074] Deep learning is a class of machine learning techniques that employ representation learning methods, allowing machines to be given raw data and determine the representations needed for data classification. Deep learning uses a backpropagation algorithm to determine the structure of a dataset by altering the machine's internal parameters (e.g., node weights). Deep learning machines can utilize various multi-layer architectures and algorithms. For example, while machine learning involves identifying features to be used to train a network, deep learning processes raw data to identify features of interest without external identification.

[0075] Deep learning in a neural network environment comprises many interconnected nodes called neurons. Input neurons, activated by external sources, activate other neurons based on connections to them controlled by machine parameters. Neural networks function in a certain way based on their own parameters. Learning improves the machine parameters, and more broadly, improves the connections between neurons in the network, causing the neural network to function in the desired manner.

[0076] Deep learning using convolutional neural networks employs convolutional filters to segment data in order to locate and identify learned observable features within it. Each filter or layer in a CNN architecture transforms the input data to increase its selectivity and invariance. This abstraction of the data allows the machine to focus on the features it is attempting to classify and ignore irrelevant background information.

[0077] Deep learning operates on the understanding that many datasets contain high-level features, which in turn contain low-level features. For example, when examining an image, instead of searching for objects, it's more efficient to look for edges; edges form motifs, motifs form parts, and parts form the object being searched for. These hierarchical levels of features are visible in many different forms of data, such as speech and text.

[0078] The learned observable features include the objects the machine learns during supervised learning and quantifiable regularity. Machines with large sets of data that can be effectively classified are better positioned to distinguish and extract features associated with successful classification of new data.

[0079] Deep learning machines that utilize transfer learning can correctly connect data features to certain classifications confirmed by human experts. Conversely, the same machine can update the parameters used for classification when a human expert points out a classification error. For example, settings and / or other configuration information can be guided by the use of learned settings and / or other configuration information, and the number of changes and / or other possibilities in settings and / or other configuration information can be reduced for a given situation as the system is used more frequently (e.g., repeatedly and / or by multiple users).

[0080] For example, an expert classification dataset can be used to train an exemplary deep learning neural network. This dataset constructs the first parameters of the neural network, and this becomes the supervised learning phase. During the supervised learning phase, it is possible to test whether the neural network has achieved the desired behavior.

[0081] Once the desired neural network behavior has been achieved (e.g., the machine is trained to operate according to a specified threshold), the machine can be deployed for use (e.g., testing the machine with "real" data). During operation, the neural network classification can be affirmed or rejected (e.g., by an expert user, expert system, reference database, etc.) to continue improving the neural network behavior. The exemplary neural network is then in a transfer learning state because the classification parameters that determine the neural network behavior are updated based on the ongoing interactions. In some examples, the neural network may provide direct feedback to another process. In some examples, the data output by the neural network is buffered (e.g., via the cloud) and validated before being provided to another process.

[0082] Deep learning machines using convolutional neural networks (CNNs) can be used for image analysis. CNN analysis stages can be used for face recognition in natural images, lesion identification in image data, computer-aided diagnosis (CAD), and more.

[0083] High-quality medical image data can be acquired using one or more imaging modalities such as X-ray, computed tomography (CT), molecular imaging and computed tomography (MICT), and magnetic resonance imaging (MRI). The quality of medical images is generally not affected by the machine that produces the image, but rather by the patient. For example, patient movement during MRI can create blurred or distorted images, which can hinder accurate diagnosis.

[0084] Interpreting medical images without regard to quality is a relatively recent development. Medical images are largely interpreted by physicians, but these interpretations can be subjective, influenced by the physician's experience and / or fatigue in the field. Image analysis via machine learning can support the workflow of healthcare practitioners.

[0085] For example, deep learning machines can provide computer-aided detection support to improve image analysis in terms of image quality and classification. However, deep learning machines applied in the medical field often face problems that lead to many misclassifications. For instance, deep learning machines must overcome small training datasets and require iterative tuning.

[0086] For example, deep learning machines can be used to determine the quality of medical images with minimal training. Semi-supervised and unsupervised deep learning machines can be used to quantitatively measure aspects of image quality. For example, deep learning machines can be used after images have been acquired to determine whether the image quality is sufficient for diagnosis. Supervised deep learning machines can also be used for computer-aided diagnosis. For example, a lesion identification and evaluation unit 140 can use a deep learning network model to analyze lesion data identified in an image. For example, a prostate evaluator 130 can use a deep learning network model to assess prostate health based on prostate tissue identified in an image and associated patient health information. For example, supervised learning can help reduce susceptibility to misclassification.

[0087] Deep learning machines can leverage transfer learning to offset the small datasets available during supervised training when interacting with physicians. These deep learning machines can improve their computer-aided diagnosis over time through training and transfer learning. In some examples, a digital twin 230 (e.g., as a whole and / or one of its sub-parts 250 to 254) can utilize deep learning network models to model the behavior of its components such as the prostate, lesions, other organs, etc.

[0088] Exemplary learning network system

[0089] Figure 3 This is a representation of an exemplary learning neural network 300. The exemplary neural network 300 includes layers 320, 340, 360, and 380. Layers 320 and 340 are connected using neural connections 330. Layers 340 and 360 are connected using neural connections 350. Layers 360 and 380 are connected using neural connections 370. Data flows from the input layer 320 to the output layer 380 and reaches the output 390 via inputs 312, 314, and 316.

[0090] Layer 320 is the input layer, which is in Figure 3The example includes multiple nodes 322, 324, and 326. Layers 340 and 360 are hidden layers, and... Figure 3 The example includes nodes 342, 344, 346, 348, 362, 364, 366, and 368. The neural network 300 may include more or fewer hidden layers 340 and 360 than shown. Layer 380 is the output layer, and... Figure 3 The example includes node 382 with output 390. Each input 312 to 316 corresponds to nodes 322 to 326 of input layer 320, and each node 322 to 326 of input layer 320 has a connection 330 to each node 342 to 348 of hidden layer 340. Each node 342 to 348 of hidden layer 340 has a connection 350 to each node 362 to 368 of hidden layer 360. Each node 362 to 368 of hidden layer 360 has a connection 370 to output layer 380. Output layer 380 has an output 390 to provide output from exemplary neural network 300.

[0091] In connections 330, 350, and 370, some exemplary connections 332, 352, and 372 may be assigned increased weights, while other exemplary connections 334, 354, and 374 may be assigned smaller weights in the neural network 300. For example, input nodes 322 to 326 are activated by receiving input data via inputs 312 to 316. Nodes 342 to 348 and 362 to 368 of hidden layers 340 and 360 are activated by data flowing forward through network 300 via connections 330 and 350, respectively. After data processed in hidden layers 340 and 360 is sent via connection 370, node 382 of output layer 380 is activated. When output node 382 of output layer 380 is activated, node 382 outputs an appropriate value based on the processing performed in hidden layers 340 and 360 of neural network 300.

[0092] Figure 4 A specific implementation of an exemplary neural network 300, which is a convolutional neural network 400, is shown. For example... Figure 4 As shown in the example, input 310 is provided to a first layer 320, which processes input 310 and propagates it to a second layer 340. Input 310 is further processed in the second layer 340 and propagated to a third layer 360. The third layer 360 classifies the data to be provided to the output layer e80. More specifically, as... Figure 4As shown in the example, a convolution 404 (e.g., a 5×5 convolution, etc.) is applied to a portion or window (also called a “receptive field”) 402 of the input 310 (e.g., a 32×32 data input, etc.) in the first layer 320 to provide a feature map 406 (e.g., a (6×)28×28 feature map, etc.). The convolution 404 maps elements from the input 310 to the feature map 406. The first layer 320 also provides subsampling (e.g., 2×2 subsampling, etc.) to generate a reduced feature map 410 (e.g., a (6×)14×14 feature map, etc.). The feature map 410 undergoes a convolution 412 and propagates from the first layer 320 to the second layer 340, where the feature map 410 becomes an expanded feature map 414 (e.g., a (16×)10×10 feature map, etc.). After subsampling 416 in the second layer 340, the feature map 414 becomes a reduced feature map 418 (e.g., (16×)4×5 feature map, etc.). The feature map 418 undergoes convolution 420 and propagates to the third layer 360, where the feature map 418 becomes a classification layer 422, thereby forming an output layer 424 with N categories, for example, having connections 426 to the convolutional layer 422.

[0093] Figure 5 This is a representation of an exemplary implementation of an image analysis convolutional neural network 500. The convolutional neural network 500 receives an input image 502 and abstracts the image in a convolutional layer 504 to identify learned features 510 to 522. In a second convolutional layer 530, the image is transformed into multiple images 530 to 538, where each of the learned features 510 to 522 is enhanced in its respective sub-image 530 to 538. Images 530 to 538 are further processed to focus on the features of interest 510 to 522 in images 540 to 548. The resulting images 540 to 548 are then processed by a pooling layer, which reduces the size of images 540 to 548 to separate portions 550 to 554 of images 540 to 548, including the features of interest 510 to 522. The outputs 550 to 554 of the convolutional neural network 500 receive values ​​from the last non-output layer and classify the image based on the data received from the last non-output layer. In some examples, a convolutional neural network 500 can contain many different variations of convolutional layers, pooling layers, learned features, and outputs.

[0094] Figure 6A An exemplary configuration 600 is shown, in which a learning network (e.g., machine learning, deep learning, etc.) is applied to process and / or otherwise evaluate an image. Machine learning can be applied to a variety of processes, including image acquisition, image reconstruction, image analysis / diagnosis, etc. Figure 6AAs shown in the exemplary configuration 600, raw data 610 (e.g., raw data 610 obtained from imaging scanners such as X-ray, computed tomography, ultrasound, MRI, etc., raw spectrogram data, etc.) is fed into a learning network 620. The learning network 620 processes the data 610 to associate and / or otherwise incorporate the raw image data 620 into the resulting image 630 (e.g., a “good quality” image and / or other images of sufficient quality for diagnosis, etc.). The learning network 620 includes nodes and connections (e.g., paths) to associate the raw data 610 with the completed image 630. For example, the learning network 620 may be a training network that learns these connections and processes feedback to establish connections and recognize patterns. For example, the learning network 620 may be a deployed network generated by the training network and utilizes the connections and patterns established in the training network to acquire the input raw data 610 and generate the resulting image 630.

[0095] Once the learning network 620 has been trained and produces good images 630 from the original image data 610, the network 620 can continue the "self-learning" process and improve its performance during operation. For example, there is "redundancy" in the input data (original data) 610, and there is redundancy in the network 620, which can be utilized.

[0096] If we examine the weights assigned to the nodes in the learning network 620, we may find many connections and nodes with extremely low weights. Low weights indicate that these connections and nodes contribute little to the overall performance of the learning network 620. Therefore, these connections and nodes are redundant. Such redundancy can be evaluated to reduce redundancy in the input (raw data) 610. For example, reducing input 610 redundancy can save scanner hardware, reduce component requirements, and also reduce patient exposure doses.

[0097] In deployment, configuration 600 forms package 600, which includes input definition 610, trained network 620, and output definition 630. Package 600 can be deployed and installed relative to another system such as an imaging system, analysis engine, etc.

[0098] like Figure 6B As shown in the example, learning network 620 can be linked with and / or otherwise combined with multiple learning networks 621 to 623 to form a larger learning network. For example, the combination of networks 620 to 623 can be used to further improve the response to input and / or to assign networks 620 to 623 to various aspects of the system.

[0099] In some examples, during operation, "weak" connections and nodes can initially be set to zero. The learning network 620 then processes its nodes during the hold-through process. In some examples, changing the nodes and connections set to zero is not allowed during retraining. Considering the redundancy within network 620, it is highly likely that equally good images will be generated. Figure 6B As shown, after retraining, learning network 620 becomes DLN 621. Learning network 621 is also examined to identify weak connections and nodes, and these are set to zero. The network that is further retrained is learning network 622. Exemplary learning network 622 includes the "zeros" from learning network 621, plus a new set of nodes and connections. Learning network 622 continues to repeat this process until good image quality is achieved at learning network 623 (which is called the "Minimum Viable Network (MVN)"). Learning network 623 is an MVN because if additional connections or nodes are attempted to be set to zero in learning network 623, the image quality will deteriorate.

[0100] Once the MVN has been obtained using the learning network 623, "zero" regions (e.g., irregular dark areas in the figure) are mapped to input 610. Each dark area may map to one or a set of parameters in the input space. For example, one of the zero regions may be associated with the number of views and channels in the original data. Since redundancy in the network 623 corresponding to these parameters can be reduced, it is highly likely that the input data can be reduced and that the input data can produce equally good output. To reduce the input data, a new original dataset corresponding to the reduced parameters is obtained and this new original dataset is run through the learning network 621. Networks 620 to 623 may or may not be simplified, but one or more of the learning networks 620 to 623 are processed until the "Minimum Viable Input (MVI)" of the original data input 610 is reached. At the MVI, further reduction of the input original data 610 may result in a decrease in the quality of image 630. For example, the MVI may reduce the complexity of data acquisition, require fewer system components, reduce patient stress (e.g., less breath-holding or contrast agent), and / or reduce the dose to the patient.

[0101] By forcing some connections and nodes in learning networks 620 to 623 to zero, networks 620 to 623 construct "side branches" to compensate. In this process, insights into the topological structure of learning networks 620 to 623 are gained. It should be noted that networks 621 and 622, for example, have different topological structures due to the forced zeroing of some nodes and / or connections. This process of effectively removing connections and nodes from a network goes beyond "deep learning" and can be termed, for example, "deep-deep learning."

[0102] In some examples, input data processing and the deep learning stage can be implemented as separate systems. However, as separate systems, neither module may be aware of the larger input feature evaluation loop used to select the input parameters of interest / importance. Since input data processing selection is critical to producing high-quality output, feedback from the deep learning system can be used to perform input parameter selection optimization or improvement via the model. Instead of forming the raw data by scanning the entire set of input parameters (e.g., this is brute-force and expensive), a variant of active learning can be implemented. Using this variant of active learning, an initial parameter space can be determined to produce the desired or “optimal” results in the model. The parameter values ​​can then be randomly reduced to generate the raw input, which reduces the quality of the results while still maintaining an acceptable range or threshold of quality, and reduces runtime by processing inputs that have little impact on model quality.

[0103] Figure 7 Exemplary training and deployment phases of a learning network, such as a deep learning or other machine learning network, are shown. Figure 7 As shown in the example, during the training phase, a set of inputs 702 is provided to network 704 for processing. In this example, the set of inputs 702 may include facial features of the image to be recognized. Network 704 processes the inputs 702 along the forward path 706 to associate data elements and recognize patterns. Network 704 determines that input 702 represents a dog 708. During training, the network result 708 is compared 710 with a known result 712. In this example, the known result 712 is a human face (e.g., the input dataset 702 represents a human face, not a dog face). Since the determination 708 of network 704 does not match the known result 712 710, an error 714 is generated. Error 714 triggers a backpropagation analysis of the known result 712 and the associated data 702 along the backward path 716 through network 704. Therefore, training network 704 learns data 702, 712 from the forward path 706 and the backward path 716 through network 704.

[0104] Once the network output 708 is compared with the known output 712 and matches according to a specific criterion or threshold 710 (e.g., matching n times, matching greater than x%, etc.), the trained network 704 can be used to generate a network for deployment with an external system. Once deployed, a single input 720 is provided to the deployed learning network 722 to generate an output 724. In this case, based on the trained network 704, the deployed network 722 determines that the input 720 is an image of a face 724.

[0105] Figure 8 An exemplary product is shown that utilizes trained network packets to provide deep learning and / or other machine learning products. For example... Figure 8As shown in the example, input 810 (e.g., raw data) is provided for preprocessing 820. For example, the raw input data 810 is preprocessed 820 to check for format, integrity, etc. Once the data 810 has been preprocessed 820, patches of data 830 are created. For example, patches, portions, or “blocks” of data with a specific size and format are created 830 for processing. The patches are then fed into a training network 840 for processing. Based on learned patterns, nodes, and connections, the training network 840 determines the output based on the patches of input. The output is assembled 850 (e.g., combined and / or otherwise grouped together to generate a usable output, etc.). The output is then displayed 860 and / or otherwise output to a user (e.g., human users, clinical systems, imaging modalities, data storage (e.g., cloud storage, local storage, edge devices, etc.) etc.).

[0106] As discussed above, learning networks can be packaged as devices for training, deployment, and application in a variety of systems. Figures 9A to 9C Various learning device configurations are shown. For example, Figure 9A A general learning device 900 is illustrated. The exemplary device 900 includes an input definition 910, a learning network model 920, and an output definition 930. The input definition 910 may include one or more inputs that are transformed into one or more outputs 930 via the network 920.

[0107] Figure 9B An exemplary training device 901 is shown. That is, training device 901 is an example of device 900 configured to train a learning network device. Figure 9B In the example, multiple training inputs 911 are provided to network 921 to develop connections in network 921 and to provide an output evaluated by output evaluator 931. Output evaluator 931 then provides feedback to network 921 to further develop (e.g., train) network 921. Additional inputs 911 may be provided to network 921 until output evaluator 931 determines that network 921 has been trained (e.g., the output has satisfied a known correlation between input and output according to a specific threshold, error magnitude, etc.).

[0108] Figure 9C An exemplary deployment device 903 is shown. Once the training device 901 has learned to the necessary level, it can be deployed for use. For example, while the training device 901 learns by processing multiple inputs, the deployed device 903 determines the output by processing a single input. Figure 9CAs shown in the example, the deployed device 903 includes an input definition 913, a trained network 923, and an output definition 933. For example, once the network 921 has been sufficiently trained, the trained network 923 can be generated from the network 921. The deployed device 903 receives system input 913 and processes the input 913 via the network 923 to generate an output 933, which can then be used, for example, by a system already associated with the deployed device 903.

[0109] Exemplary Image Analysis and Prostate Assessment System and Method

[0110] Some examples provide systems and methods for computer-aided diagnosis and classification of prostate cancer. For instance, some examples use multimodal, multi-scheme MR data to locate prostate lesions on a prostate sector map and integrate prostate lesion information from computer-aided diagnosis and classification systems for prostate cancer.

[0111] For example, in the first workflow, a graphical object (e.g., ROI / major axis) is deposited on a lesion in one or more acquired images. Additionally, ellipses are deposited on a prostate sector map, where a pattern and one or more sectors are automatically selected below the map. The lesions can then be scored according to the PIRADS v2 guidelines. Based on the lesion mapping and scoring, a report and / or the next action trigger can be generated and output.

[0112] In another workflow, for example, MR image acquisition is performed, and one or more resulting images are loaded and displayed. The patient's medical history is obtained (e.g., from clinicians, patients, electronic medical records, etc.), and the patient's PSA level is determined. The prostate is automatically segmented, and its volume is calculated (e.g., using deep learning-based methods, etc.). Graphical objects (e.g., ROI / long axis) are deposited on the MR data, and one or more corresponding sectors are automatically selected (e.g., using non-rigid registration of the segmented prostate and a 3D model of the prostate sector mapping, etc.). One or more lesions can then be scored according to the PIRADS v2 guidelines. Based on region analysis and lesion scoring, reports and / or next action triggers can be generated and output.

[0113] In another workflow, for example, MR image acquisition is performed, and one or more resulting images are loaded and displayed. The patient's medical history is obtained (e.g., from clinicians, patients, electronic medical records, etc.), and the patient's PSA level is determined. The prostate is automatically segmented and its volume is calculated (e.g., using deep learning-based methods, etc.). One or more lesions are automatically segmented and then located and scored according to each available MR imaging technique (e.g., using a 3D model of the segmented prostate with non-rigid registration and sector mapping, and deep learning-based methods, etc.). Based on lesion segmentation, analysis, and scoring, reports and / or next action triggers can be generated and output.

[0114] In some examples, a deep learning network model can process image data to generate a binary mask output to identify lesions on the prostate in one or more images. The model can obtain one or more image slices, 3D volumes, etc. (e.g., preprocessed to normalize intensity and / or resolution, etc.) and segment the image data via the network to provide a binary mask for identifying lesions in the image data. For example, lesions can be located on a prostate sector map using multimodal, multiprotocol MR data via the network model.

[0115] Therefore, some examples provide for the processing, viewing, analysis, and communication of 3D reconstructed images and their relationship to images initially acquired from MR scanning equipment. For instance, a combination of image acquisition, reconstructed images, annotation, and measurements performed by clinicians and / or automatically using deep learning and / or other artificial intelligence provides reference physicians with clinically relevant information that can aid in diagnosis and treatment planning.

[0116] Figure 10 Exemplary methods and associated infrastructure for analyzing prostate information and generating predictions and / or other analyses regarding potential prostate cancer, malignant lesions, and / or other prostate problems are illustrated. At box 1002, patient history and PSA levels are determined (see, for example...). Figure 11 (Example interface). At box 1004, the prostate volume is calculated. For example, distances (e.g., 3 distances, etc.) are deposited on the image (e.g., using a dedicated distance tool, etc.), and the prostate volume and PSA density are automatically calculated (see example). Figure 12(Exemplary interface). For example, prostate volume can be automatically calculated using three distances drawn on the prostate in an image via the user interface to label the length (d1), width (d2), and height (d3) of the prostate in the image. The prostate volume can then be calculated as length × width × height × 0.52 = prostate volume, where 0.52 is an example of a scaling factor that describes the difference between the actual size and the representation in the image data. The PSA density can then be calculated, for example, based on the prostate volume and other factors.

[0117] At box 1006, one or more lesions are identified and evaluated. In various specific implementations, one or more lesions may be identified and analyzed. For example, new lesions may be added (e.g., marked) on one or more MR images (see example in...). Figure 13 (e.g., at point 1302 in the exemplary interface). For example, the major axis distance and the ADC region of interest (ROI) can be deposited on one or more images via the interface. Alternatively, one or more lesions can be identified using available tools, algorithms, digital twins, etc. (see, for example...) Figure 14 (Example interface 1402). Another example of lesion location determination 1502 is in Figure 15 An exemplary graphical user interface is shown. Figure 16 As shown in the exemplary interface, ellipses are deposited in the axial, sagittal, and coronal planes of interface 1602 to automatically select corresponding sectors. For example, once the ellipse is positioned in the prostate sector mapping mode, one or more sectors below the ellipse are automatically selected. Figure 17 For example, lesions are scored according to each available MR technique, and a global score is automatically calculated based on the lesion scores from the MR techniques. For example, lesion scores may be based on their size (e.g., length, width, volume, etc.), location, etc., and the scores may include T1-weighted pulse sequence scores, T2-weighted pulse sequence scores, diffusion-weighted imaging (DWI) scores, dynamic contrast-enhanced (DCE) MRI scores, overall scores, etc.

[0118] At box 1008, reports can be generated, saved, output, transmitted, etc. (see example). Figure 18 (Example interface). Figures 19A to 19C Exemplary reports displaying prostate assessment, scores, PI-RADS assessments, ADC information, etc., are shown. For example, patient history (e.g., including identifying trends in PSA levels), prostate volume, PSA levels, PSA density, lesion details, index lesions, opinions, PI-RADS assessments, conclusions, etc., can be provided based on analysis (e.g., transferred to another program, triggered another process, saved, displayed, and / or otherwise output) to drive further actions regarding the patient.

[0119] Therefore, axial and sagittal MR image views can be used in both training and evaluation sets for developing and testing deep learning networks (such as Networks 300, 400, and 500) to analyze MR prostate image data and identify and classify one or more lesions in the images. Based on lesion information, conclusions, recommendations, and / or other assessments can be made regarding one or more possible prostate problems. Qualitative assessments, hidden layer processing in deep neural networks, and analysis of edges, combinations of one or more edges, object models, etc., enable deep neural networks to correlate MR image data with possible prostate lesions and / or other defects requiring further verification, treatment, etc. For example, convolution, deconvolution, forward inference, and backward learning from image segmentation and pixel intensity data can help drive the correlation between MR image information and prostate cancer identified via CAD.

[0120] Although combined with Figure 1 to Figure 19C Exemplary implementations are shown, but in conjunction with Figures 1 to 12, the specific implementations are as follows: Figure 19C The elements, processes, and / or devices illustrated may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other manner. Furthermore, the components disclosed and described herein may be implemented by hardware, machine-readable instructions, software, firmware, and / or any combination of hardware, machine-readable instructions, software, and / or firmware. Thus, for example, the components disclosed and described herein may be implemented by analog and / or digital circuits, logic circuits, programmable processors, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs). When reading any claim of this patent covering a device or system implemented purely by software and / or firmware, at least one of these components is hereby expressly defined as a tangible computer-readable storage device or disk including storage software and / or firmware, such as a memory, a digital versatile disk (DVD), a compact disc (CD), a Blu-ray disc, etc.

[0121] A flowchart representing exemplary machine-readable instructions for implementing the components disclosed and described herein, combined with at least Figure 10 As shown. In the example, machine-readable instructions include those generated by the processor (such as those combined below). Figure 20 The program executed by the processor 2012 shown in the exemplary processor platform 2000 discussed herein. The program may be embodied in machine-readable instructions stored on a tangible computer-readable storage medium (such as a CD-ROM, floppy disk, hard disk drive, digital multi-disc (DVD), Blu-ray disc, or memory associated with the processor 2012), but the entire program and / or portions thereof may alternatively be executed by a device other than the processor 2012 and / or embodied in firmware or dedicated hardware. Further, although references are combined with at least Figure 10 The flowchart shown illustrates an exemplary procedure, but many other methods of implementing the components disclosed and described herein may be used alternatively. For example, the execution order of the blocks may be changed, and / or some of the blocks may be modified, eliminated, or combined. Although at least Figure 10 The flowcharts depict exemplary operations in the order shown, but these operations are not exhaustive and are not limited to the order shown. Furthermore, various changes and modifications can be made by those skilled in the art within the spirit and scope of this disclosure. For example, the blocks shown in the flowcharts may be executed in an alternative order or in parallel.

[0122] As mentioned above, at least Figure 10 The exemplary process can be implemented using coded instructions (e.g., computer and / or machine-readable instructions) stored on a tangible computer-readable storage medium, such as a hard disk drive, flash memory, read-only memory (ROM), optical disc (CD), digital versatile optical disc (DVD), cache, random access memory (RAM), and / or any other storage device or disk, wherein information is stored for any duration (e.g., extended time period, permanent, transient, for temporary buffering, and / or for caching information). As used herein, the term tangible computer-readable storage medium is explicitly defined to include any type of computer-readable storage device and / or disk, excluding propagation signals and transmission media. As used herein, "tangible computer-readable storage medium" and "tangible machine-readable storage medium" are used interchangeably. In addition or alternatively, at least Figure 10 and Figure 21 The exemplary process can be implemented using coded instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium, such as a hard disk drive, flash memory, read-only memory, compact disk, digital universal disk, cache, random access memory, and / or any other storage device or disk, wherein information is stored for any duration (e.g., extended time period, permanent, transient, for temporary buffering, and / or for caching information). As used herein, the term non-transitory computer-readable medium is explicitly defined to include any type of computer-readable storage device and / or disk, excluding propagated signals and transmission media. As used herein, when the phrase “at least” is used as a transitional term in the preamble of a claim, it is also open-ended, just as the term “comprising” is open-ended. Additionally, just as the term “comprising” is open-ended, the term “including” is also open-ended.

[0123] Figure 20This is a block diagram of an exemplary processor platform 2000, which is configured to perform at least Figure 10 The instructions are provided to implement the exemplary components disclosed and described herein. The processor platform 2000 can be, for example, a server, a personal computer, a mobile device (e.g., a mobile phone, smartphone, tablet computer such as an iPad). TM Personal digital assistants (FDA), internet applications, or any other type of computing device.

[0124] The processor platform 2000 shown in the example includes a processor 2012. The processor 2012 shown in the example is hardware. For example, the processor 2012 may be implemented by an integrated circuit, logic circuit, microprocessor, or controller from any desired product family or manufacturer.

[0125] The processor 2012 shown in the example includes local memory 2013 (e.g., cache). Figure 20 The exemplary processor 2012 performs at least Figure 10 The instructions are to implement Figure 1 to Figure 19C and Figures 21 to 23 The system and infrastructure, as well as related methods such as image acquisition modules, prostate detectors, prostate evaluators, lesion identification and evaluation devices, result generators, lesion mappers, etc., are described. The processor 2012 of the example shown communicates via bus 2018 with main memory, including volatile memory 2014 and non-volatile memory 2016. Volatile memory 2014 may be implemented using synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS dynamic random access memory (RDRAM), and / or any other type of random access memory device. Non-volatile memory 2016 may be implemented using flash memory and / or any other desired type of memory device. Access to main memory 2014 and 2016 is controlled by a clock controller.

[0126] The processor platform 2000 shown in the example also includes interface circuitry 2020. Interface circuitry 2020 can be implemented using any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB) interface, and / or a PCI express interface.

[0127] In the example shown, one or more input devices 2022 are connected to interface circuitry 2020. Input devices 2022 allow users to input data and commands into processor 2012. One or more input devices may be implemented as, for example, sensors, microphones, cameras (still camera or video camera), keyboards, buttons, mice, touchscreens, touchpads, trackballs, isopoint devices, and / or voice recognition systems.

[0128] One or more output devices 2024 are also connected to the interface circuitry 2020 of the illustrated example. The output devices 2024 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays, cathode ray tube displays (CRTs), touchscreens, haptic output devices, and / or speakers). Therefore, the interface circuitry 2020 of the illustrated example typically includes a graphics driver card, a graphics driver chip, or a graphics driver processor.

[0129] The interface circuit 2020 of the example shown also includes communication devices, such as transmitters, receivers, transceivers, modems and / or network interface cards, to facilitate the exchange of data with external machines (e.g., any kind of computing device) via a network 2026 (e.g., Ethernet connection, digital subscriber line (DSL), telephone line, coaxial cable, cellular telephone system, etc.).

[0130] The processor platform 2000 shown in the example also includes one or more mass storage devices 2028 for storing software and / or data. Examples of such mass storage devices 2028 include floppy disk drives, hard disk drives, optical disk drives, Blu-ray disc drives, RAID systems, and digital universal disc (DVD) drives.

[0131] Figure 20 The encoded instructions 2032 may be stored in a mass storage device 2028, in volatile memory 2014, in non-volatile memory 2016, and / or on a removable tangible computer-readable storage medium (such as a CD or DVD).

[0132] Exemplary techniques for providing lesions from prostate images using sector mapping

[0133] In some examples, users (such as clinicians, etc.) may instruct or otherwise transpose lesions from prostate images onto sector maps. Additionally, in some examples, lesions identified from prostate images may be automatically transposed or mapped to sector maps, as described below relative to... Figure 21 to Figure 23This will be discussed in more detail. The technical advantages of mapping lesion representations from images to sector maps can include generating sector maps with lesion representations without additional user input, such as ellipses indicating lesion boundaries. In some examples, this technical advantage can also include reducing the latency of generating sector maps by producing and providing sector maps with representations of lesions in the prostate without waiting for user input. The generated sector maps can also include a more accurate representation of the lesions by mapping them directly from the image or 3D model to the sector map without requiring approximations input as user input. Furthermore, the generated sector maps may be more reproducible than manual techniques where the user analyzes an image of the prostate and then annotates the sector map with the location and dimensions (such as ellipses) corresponding to the lesions identified in the image.

[0134] Figure 21 This is a process flowchart of an exemplary method for providing a sector mapping with a representation of lesions. In some examples, method 2100 can be used with any suitable device (such as...). Figure 20 This is achieved through processor platforms such as the 2000 series processor.

[0135] At box 2102, method 2100 may include detecting lesions within the prostate in an image. In some examples, lesions are detected within a two-dimensional or three-dimensional image, volume, or model of the prostate. Lesions may be detected and mapped from images of the patient, digital twins representing imaging data from the patient, etc. In some examples, axial slices of the lesion may be identified based on user input including the coordinates of the lesion's center. In some examples, the diameter of the lesion may also be detected, obtained, or otherwise acquired from user input.

[0136] In some examples, a prostate mask is applied to a three-dimensional model of the prostate or a two-dimensional image obtained from a three-dimensional model. The three-dimensional model of the prostate can be a three-dimensional model including the prostate and any other organs, or it can be generated as a product of applying a prostate mask to three-dimensional data from a patient, where the mask excludes any data representing tissue from outside the prostate. The prostate mask can be a two-dimensional set or three-dimensional volume of binary values ​​indicating whether each pixel of the two-dimensional image or three-dimensional model is located within the prostate. In some examples, the prostate mask can indicate any number of regions of the prostate, such as a central region or a peripheral region, etc. For example, a prostate mask can separate pixels in an image corresponding to a specific region of the prostate by applying a value of one to pixels included in a specific region or a value of zero to pixels outside the boundaries of the specific region. In some examples, any suitable mask (such as a non-binary mask, etc.) can be applied to an image of the prostate to identify lesions or regions of interest.

[0137] In some examples, the above relative to Figure 10 The technique described in box 1006 detects lesions originating from the prostate in an image. For example, machine learning techniques, algorithms, digital twins, etc., can be used to identify the long axis distance and the region of interest (ADC).

[0138] At box 2104, method 2100 may include generating a lesion-to-sector mapping from an image. In some examples, the lesion-to-sector mapping can automatically generate a sector mapping that includes a representation of the lesion, which may be scored by a user, machine learning techniques, etc. For example, if the lesion-to-sector mapping is automatically generated from an image or model, detection of user input for one or more elliptical, circular, or any other suitable geometry of the lesion may not be performed.

[0139] In some examples, generating a mapping of lesions may include identifying depth regions of the lesions at box 2106. For example, a lesion may be located in any number of depth regions of the prostate or correspond to any number of depth regions. In some examples, depth regions may include apex regions, middle regions, and basal regions, etc. A lesion may be identified as being located within one or more of these depth regions. For example, a lesion may be identified based on the depth region in which its center is located. Identifying depth regions of the prostate for a lesion may include determining the value or location of the lesion along a depth axis (such as the z-axis) or any other suitable coordinate value, and determining whether that value is within the first, middle, or last third of the representation of the prostate. For example, a three-dimensional model of the prostate may be divided into three regions along the z-axis or depth axis. In some examples, the prostate may be divided into any number of depth regions.

[0140] In some examples, generating the lesion mapping may include using any suitable technique at box 2108 to convert the coordinates of the lesion representation (such as the centroid) from a first coordinate system to a second coordinate system. For example, the first coordinate system may include a Cartesian coordinate system, and the second coordinate system may be a polar coordinate system, etc. In some examples, the x and y coordinates of the lesion's center may be converted to polar coordinates R and θ. When a two-dimensional slice of the lesion (represented by x and y values) is mapped to a depth region of the sector mapping (such as a vertex region, intermediate region, or basal region, etc.), the z-value may not be converted to polar coordinates. In some examples, any number of slices of the lesion along the z-axis may be mapped individually to the sector mapping.

[0141] Converting the x and y coordinates of the lesion center to polar coordinates may include calculating the r and θ values ​​as follows:

[0142]

[0143] θ = atan2(y, x) Formula (2)

[0144] In some examples, at box 2110, the polar coordinates R and diameter D of the lesion can be normalized based on the dimensions of the lesion image. These dimensions may include the distance from the center of the image to the outer edge of the image, etc. In some examples, the polar coordinates R can be normalized to a value between zero and one based on the maximum radius of the image representing the prostate. In some examples, the diameter D of the lesion representation can also be normalized to a value between zero and one based on the maximum radius of the prostate. In some examples, the polar coordinates R can be normalized based on R divided by Rmax, and the diameter value of the lesion D can be normalized based on D divided by Rmax, as shown below with respect to Equations 3 and 4. The diameter value D can be used to calculate the size of the lesion representation on the sector map, where the representation can be circular or elliptical, etc.

[0145] In some examples, at box 2112, the following formulas 3 and 4 can be used to denormalize the normalized polar coordinates R and D of the lesion based on the normalized radius of the sector mapping and one or more dimensions.

[0146]

[0147]

[0148] In Equations 3 and 4, Rmax represents the maximum radius of the prostate gland from a 2D slice or image, and rmax represents the maximum radius of the sector mapping. The R value represents the polar coordinate R derived from the x and y values ​​in Equation 1, and D represents the diameter of the lesion in the 2D image. In the following text, relative to... Figure 22 Let's discuss the Rmax and rmax values ​​in more detail.

[0149] In some examples, for each θ, different rmax values ​​can be calculated to represent the lesion representation at different angles to the image.

[0150] In some examples, at box 2114, the polar coordinates r, d, and θ can be converted to Cartesian coordinates or any other suitable coordinate system. In some examples, any suitable technique can be used to convert polar coordinates to Cartesian coordinates. For example, the following formulas 5 and 6 can be used to calculate the x and y coordinates representing the center of a lesion on a sector map:

[0151] x=rcosθ formula (5)

[0152] y=rsinθ formula (6)

[0153] The x and y values ​​calculated using Formulas 5 and 6 can be used to visualize lesions on a sector map by mapping the x-center and y-center of the lesion, as well as the diameter of the lesion, to represent the center and edge of the lesion on the sector map.

[0154] At box 2116, method 2100 may include providing a sector map that includes a representation of a mapping of lesions within the prostate. In some examples, the representation of the mapping of lesions on the sector map (such as a circle with the center of the lesion, or other shapes) may be transmitted to any suitable display device or external device for viewing and approval by a user (such as a clinician). The representation of the lesion mapping may include the Cartesian coordinates of the center of the lesion in the sector map and the denormalized radius of the lesion based on the sector map. In some examples, the display device is electrically connected to a device that identifies the mapping of lesions from an image to a sector map, or the display device may be connected to a remote device that receives or obtains the sector map and the representation of the lesion-to-sector mapping.

[0155] Laryngeal lesions can be scored based on the output of a representation of the lesion on a sector map. In some examples, the user interface may include a lesion map to a sector and measurements based on the techniques described herein. For example, the sector map may provide a classification of lesions mapped from an image, where the classification provides an assessment of prostate health. In some examples, lesion scores may be automatically generated based on the techniques described above with respect to Figures 1 through 19C. For example, image data, ADC information, density, segmentation, and / or other automated image data analysis may be used to identify any number of lesions in a patient's prostate. Regions of interest (ROIs) may be defined around the identified, probable, and / or approximate lesions to label one or more lesions in one or more images. Lesions in the ROI may then be segmented by the system (e.g., along the long axis, etc.), mapped to a sector map, and scored (e.g., to determine the likelihood of lesion validation, malignancy / severity, size, etc.). In some examples, deep learning, machine learning, and / or other artificial intelligence may be used to score lesions in / on the prostate based on a sector map with a representation of the lesion, as well as other information. Scoring may include pi-rads scoring, etc. It can generate possible prostate cancer diagnoses, patient care plans / treatment triggers, reports for urologists and / or other clinicians, including scores, detailed lesion information, observations, opinions, and conclusions.

[0156] Figure 21 The process flowchart for method 2100 is not intended to indicate that all operations of blocks 2102-2116 of method 2100 will be included in every example. Additionally, Figure 21The process flowchart of Method 2100 describes the possible order of operations. However, it should be understood that the operations of Method 2100 can be implemented in various orders or sequences. Furthermore, in some examples, Method 2100 may include fewer or additional operations. For example, lesions in multiple acquisitions may be normalized so that one or more lesions of the prostate can be automatically mapped to a sector map without user annotation. In some examples, multiple region masks may be used to identify regions containing lesions. For example, as described below relative to... Figure 23 To discuss in more detail, these zones may include the central zone, the peripheral zone, or any other number of zones.

[0157] In some examples, the method may include using machine learning techniques and a provided sector map that includes a representation of the mapping to generate a score. For example, the machine learning techniques described above can be used to automatically generate a pi-rads score based on a representation of the lesion mapped to the sector map as input.

[0158] Figure 22 Example diagrams illustrating the mapping of lesions from images to sector maps, according to examples in this paper, are shown. In some examples, mapping lesions from images to sector maps can be performed by devices (such as...) Figure 20 Processor platform 2000) and methods (such as Figure 21 This can be achieved using method 2100.

[0159] In some examples, a two-dimensional image 2200 of the prostate or a two-dimensional dataset acquired or obtained from a three-dimensional model of the prostate, etc., can be mapped to a sector map 2202. The mapping of the lesion representation from the two-dimensional image 2200 to the sector map 2202 can be performed using Equations 3 and 4 above, as well as any additional formulas (such as Equations 1, 2, 5, and 6 for converting x and y coordinates to polar coordinates).

[0160] In some examples, Rmax2204 represents the maximum radius of the prostate starting from the center of the 2D image 2200, and rmax2206 represents the maximum radius based on the center of the sector map 2202. The R value 2208 and the θ value 2210 represent polar coordinates converted from the x and y values ​​in Equation 1. In some examples, r2212 represents the denormalized radius calculated by Equation 3 above, expressed in polar coordinates mapped to sector map 2202.

[0161] In some examples, sector mapping 2202 may include any suitable number of sectors or areas. For example, sector mapping 2202 may include fewer sectors or additional sectors.

[0162] Figure 23An exemplary schematic diagram is shown, according to examples herein, for mapping lesions to the central or peripheral areas of the prostate. In some examples, the area for mapping lesions from an image to a sector map can be determined by a device (such as...) Figure 20 Processor platform 2000) and methods (such as Figure 21 This can be achieved using method 2100.

[0163] Figure 23 The schematic diagram includes an image 2300 of a two-dimensional slice of the prostate and a sector map 2302. In some examples, image 2300 may be a two-dimensional dataset acquired from a three-dimensional model of the prostate, etc.

[0164] In some examples, the lesion can be mapped to any number of areas of the prostate. For example, the lesion can be mapped to the central or central area 2304, the peripheral area 2306, etc. If a peripheral area mask is detected, provided, or obtained, the lesion center of the two-dimensional image 2300 of the lesion can be calculated based on the following formulas 7, 8, 9, and 10.

[0165]

[0166]

[0167]

[0168]

[0169] If the lesion is located in the central region 2304 of the prostate, formulas 7-8 and 9-10 can be used to determine whether the lesion is located in the peripheral region 2306 of the prostate, or a combination thereof. In formulas 7-10, the variable Rp 2308 represents the maximum radius of the central region 2304 for a corresponding angle, and the R value 2316 represents the distance from the center of the lesion representation in image 2300 to the center of the prostate. In some examples, the variable Rpz 2310 represents the maximum radius of the peripheral region 2306 for a given angle or θ value for a lesion originating from the center of the prostate. In some examples, r 2320 represents the location of the lesion in sector mapping 2302, and 2312 represents the maximum radius of the central region 2304 of sector mapping 2302 for a given angle or θ value. In some examples, the variable rpz 2314 represents the radius of the peripheral region 2306 of sector mapping 2302 for a given angle.

[0170] In some examples, the R value 2316 and the θ value 2318 represent the polar coordinates derived from the x and y values ​​of the center of the lesion representation in image 2300 using Equation 1. The value of r 2320 can represent the denormalized radius of the lesion mapped to sector mapping 2302.

[0171] In some examples, when scoring lesions, the location of the lesion within the central region 2304, the peripheral region 2306, or a combination thereof can be used. For example, machine learning techniques, users, etc., can be used to determine whether a lesion is located within the central region 2304 or the peripheral region 2306 to determine a lesion score (such as a pi-rads score). In some examples, this score can be used as a visual aid to discuss biopsy and treatment options with the patient. Dividing the prostate and associated structures into multiple sectors standardizes reporting and facilitates precise localization for MR-targeted biopsies and treatments, pathological relevance, and studies. In some examples, sector mapping 2302 can provide a route through surgical anatomy during radical prostatectomy or any other suitable surgical procedure.

[0172] It should be understood that the two-dimensional slice image 2300 and sector map 2302 of the prostate are examples, and any number of different sectors or regions can be identified or shown in the image 2300 or sector map 2302.

[0173] Based on the foregoing, it should be understood that the methods, apparatus, and articles disclosed above are used to monitor, process, and improve the operation of imaging and / or associated / including processor / computing devices and other healthcare systems by combining various deep learning and / or other machine learning techniques with patient imaging data, and to arrive at computer-aided prostate diagnoses. Some examples provide automated and / or guided workflows and associated systems that utilize artificial intelligence networks and / or other systems to determine patient history, prostate volume, lesion identification and assessment, and recommendations / reports. Some examples correlate lesions with sector mappings of the prostate and automatically segment the prostate and lesions. Artificial intelligence enables PIRADS and / or other scoring systems to establish computer-aided diagnoses and / or subsequent actions in further diagnosis, treatment, reporting, triggering, etc. While MR readout times can be long and difficult, some examples automate MR image analysis and / or assist users in evaluating relevant information highlighted in one or more images. Additionally, automated analysis can help reduce the amount of unnecessary prostate biopsies while improving the early detection, treatment, and monitoring of prostate problems.

[0174] While certain exemplary methods, apparatuses, and articles of manufacture have been described herein, the scope of this patent is not limited thereto. Rather, this patent covers all methods, apparatuses, and articles of manufacture that reasonably fall within the scope of the claims of this patent.

Claims

1. A device for providing diagnosis of prostate diseases, the device comprising: The memory is used to store instructions; and Processor, the processor being configured to execute the instructions to: Detecting lesions from prostate images; Depending on whether the lesion is located in the central or peripheral region of the prostate, a mapping of the lesion from the image to a sector is generated in different ways. Generating the mapping of the lesion includes identifying a depth region of the lesion, wherein the depth region indicates the position of the lesion along a depth axis; and The sector mapping provides a representation of the lesion within the prostate, including a mapping from the image to the sector mapping. The mapping from the image to the sector, generated in different ways depending on whether the lesion is located in the central or peripheral area of ​​the prostate, includes: If the lesion is located in the central region, the lesion center of the two-dimensional image of the lesion is calculated in the following way: If the lesion is located in the peripheral area, the lesion center of the two-dimensional image of the lesion is calculated in the following way: in, R p This represents the maximum radius of the central region for the corresponding angle. R The distance from the center of the lesion in the image to the center of the prostate. R pz For a given angle or θ value, the maximum radius of the peripheral region of a lesion originating from the center of the prostate. D The diameter of the lesion. r This indicates the location of the lesion in the sector mapping. r p For a given angle or θ value, the maximum radius of the central region of the sector mapping is given. r pz Let be the radius of the surrounding area mapped to the sector for a given angle, and d This is the mapping value of the lesion diameter in the sector mapping.

2. The apparatus of claim 1, wherein the sector mapping provides a classification of the lesions mapped from the image, the classification providing an assessment of prostate health.

3. The apparatus of claim 1, wherein the depth region is identified using the apex region, middle region, and base region of the prostate.

4. The apparatus of claim 3, wherein generating the mapping comprises calculating one or more polar coordinates relative to the center of the lesion in the image.

5. The apparatus of claim 4, wherein generating the mapping comprises calculating a normalized radius based on the center of the lesion in the image.

6. The apparatus of claim 5, wherein generating the mapping comprises calculating the denormalized radius based on the normalized radius and one or more dimensions of the sector mapping.

7. The apparatus of claim 6, wherein generating the mapping comprises calculating Cartesian coordinates, the Cartesian coordinates representing the diameter of the lesion and the sector within the mapping.

8. The apparatus of claim 7, wherein providing the sector mapping comprises transmitting the sector mapping and a representation of the mapping from the lesion to the sector mapping to a display device, wherein the representation of the mapping of the lesion includes the Cartesian coordinates of the lesion and the denormalized radius of the lesion.

9. The apparatus of claim 8, wherein the display device is electrically connected to the apparatus, or wherein the display device is connected to a remote device, the remote device receiving from the apparatus the representation of the sector mapping and the mapping of the lesion to the sector mapping.

10. The apparatus of claim 1, wherein the image comprises a three-dimensional volume.

11. The apparatus of claim 1, wherein the processor is configured to generate the mapping of the lesion by using a digital twin, and wherein the processor is configured to generate a score using machine learning techniques and a provided sector mapping including the representation of the mapping.

12. A method for computer-aided diagnosis of prostate diseases, the method comprising: Detecting lesions from prostate images; The mapping of the lesion from the image to the sector is generated in different ways depending on whether the lesion is located in the central or peripheral area of ​​the prostate. The generation of the mapping of the lesion includes identifying the depth region of the lesion, wherein the depth region indicates the position of the lesion along the depth axis. Provides the sector mapping, which includes a representation of the lesion within the prostate gland mapped from the image to the sector mapping; and The sector mapping showing the representation of the lesion within the prostate gland. The mapping from the image to the sector, generated in different ways depending on whether the lesion is located in the central or peripheral area of ​​the prostate, includes: If the lesion is located in the central region, the lesion center of the two-dimensional image of the lesion is calculated in the following way: If the lesion is located in the peripheral area, the lesion center of the two-dimensional image of the lesion is calculated in the following way: in, R p This represents the maximum radius of the central region for the corresponding angle. R The distance from the center of the lesion in the image to the center of the prostate. R pz For a given angle or θ value, the maximum radius of the peripheral region of a lesion originating from the center of the prostate. D The diameter of the lesion. r This indicates the location of the lesion in the sector mapping. r p For a given angle or θ value, the maximum radius of the central region of the sector mapping is given. r pz Let be the radius of the surrounding area mapped to the sector for a given angle, and d This is the mapping value of the lesion diameter in the sector mapping.

Images