Medical image processing apparatus and medical image processing method

Abstract

To compress a data volume while allowing an influence on a rendering image to be reduced.SOLUTION: A medical image processing apparatus relating to the present embodiment comprises processing circuitry. The processing circuitry acquires a data volume to be compressed. The processing circuitry acquires a function relating at least one of a color value and an opacity value to each data value of the data volume. The processing circuitry determines a compression rate per region of the data volume based on at least one of a plurality of color values and a plurality of opacity values in each region.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] SUMMARY OF THE DISCLOSURE The embodiments described herein generally relate to medical image processing apparatus and methods for processing image data, for example, compressing image data volumes using multiple compression ratios that are dependent on gradient visibility. [Background technology]

[0002] For example, volumetric medical imaging techniques capable of generating three-dimensional medical imaging data using any of a variety of imaging modalities, such as CT, PET, MRI, ultrasound, and X-ray, are widely used for imaging or diagnostic purposes.

[0003] Volumetric medical image data (volume data) may comprise a three-dimensional array of voxels, each voxel representing a particular location in three-dimensional space and each voxel having one or more data values. For example, in the case of CT data, each voxel may have an associated intensity value representing the attenuation of applied X-ray radiation imparted to the location that the voxel represents. The intensity values ​​may be referred to as image values, gray values, gray levels, voxel values, or CT values. The intensity values ​​may be measured in Hounsfield units (HU).

[0004] In some data processing methods, a Graphics Processing Unit (GPU) is used to perform volume rendering of volumetric data, such as volumetric imaging data acquired from a medical imaging scanner.

[0005] A typical GPU has dedicated graphics memory that may be located on a graphics card. For data to be manipulated by a typical GPU, the data must be accessible in the dedicated graphics memory. Input data is uploaded to the graphics card for manipulation.

[0006] A typical GPU has a parallel architecture that allows it to perform the same calculation multiple times simultaneously on different inputs. For example, a GPU might calculate all the pixels of an image simultaneously by applying the same algorithm at each pixel location.

[0007] GPU volume rendering may entail utilizing the GPU to facilitate standard volume rendering techniques such as Multi-Planar Reformatting (MPR), Shaded Volume Rendering (SVR), Intensity Projection (IP), or slab rendering.

[0008] Volume rendering (SVR) involves the rendering of a three-dimensional voxel data set using a transfer function to specify color and opacity values ​​for any given data value. The transfer function maps each possible set of voxel values ​​to a distinct opacity and color value (usually represented by a combination of red, green, and blue color values). The transfer function is typically chosen to simulate a particular material or feature. For example, muscle might be rendered in red and bone in white.

[0009] Typically, most of a volume may be completely transparent, or the volume may have large regions that are similar in color and / or transparency.

[0010] It is becoming increasingly common to use large imaging volumes. For example, in the case of medical imaging, increased medical scanner resolutions may result in very large imaging volumes. Some volumes may be too large to fit entirely into the dedicated graphics memory of a GPU.

[0011] Larger image matrices are used in medical imaging. For example, a slice may comprise 512x512, 1024x1024, or 2048x2048 pixels. More slices may be acquired. As an example, a 2048x2048x6000 volume requires 47 GB of storage. Matrix sizes larger than 512x512 are sometimes referred to as Large Scale.

[0012] It is desirable to immediately render images from a volume, and sometimes it is desirable to render images in real time.

[0013] The processing costs of moving data around may be high. The large data volumes may be larger than the data capacity of a typical graphics processing unit (GPU). Even high-performance GPUs may struggle to accommodate the largest volumes. Mobile and virtual reality (VR) devices are typically limited in memory to the extent that the volume of a 512 matrix extends memory and capacity.

[0014] Some existing compression schemes are unacceptably lossy. For example, they may result in visible blocking artefacts. Some existing compression schemes are not amenable to use for real-time rendering. Some existing compression schemes are too costly to decode on the fly. Some existing compression schemes are more interested in the data values ​​themselves rather than the image that results from rendering the data values.

[0015] For example, atlas-based methods, such as multi-map atlas-based methods, may be used to render large volumes. An example of an atlas-based method is described in U.S. Patent 10,719,907, which is incorporated herein by reference.

[0016] In an atlas-based method, instead of uploading all the volume data at once, sub-regions of the volume data are sent to the device memory as needed for rendering. In an atlas-based system, current compression systems are not always sufficient to obtain good performance. The upload process is relatively slow and may create a bottleneck in the rendering process. One criterion for a system with good performance may be that the system does not need to upload all the volume data to the rendering device in order to render each frame.

[0017] When memory capacity is heavily over-subscribed, data transfer overhead may dominate. Memory capacity may be said to be oversubscribed when the data required to render each frame is too large. Memory capacity may be considered heavily over-subscribed when the data to be rendered is twice the amount that can be stored by the rendering device. If the data required to render an image is more than twice the available memory of the rendering device, the entire volume may be uploaded for every frame.

[0018] It may be desirable to compress the data to be rendered so that it occupies less than twice the memory capacity. For large-scale data, the data volume size may be large enough that standard lossless compression techniques do not sufficiently reduce the data volume size.

[0019] In some situations, the currently implemented compression scheme may not be able to compress the volume to a size smaller than twice the memory capacity, and the system may need to transmit at least 50% of the total data volume for each frame that is rendered.

[0020] For example, if the size of the data volume to be rendered is 12 GB, a standard lossless compression method may only reduce the size of the data volume to 10 GB, if the size of the memory of the rendering device used to render the compressed data is 4 GB, the rendering process will be very inefficient due to the overhead of uploading data.

[0021] A best-case GPU transfer rate, for example, may be around 8 GB / s. For large-scale volumes, even the best-case GPU transfer rate may result in a delay of a second or more. [Prior art documents] [Patent documents]

[0022] [Patent Document 1] China Patent Application Publication No. 102096939 [Patent Document 2] US Patent Application Publication No. 2013 / 314417 Summary of the Invention [Problem to be solved by the invention]

[0023] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to compress volume data so as to reduce the impact on a rendering image. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0024] The medical image processing apparatus according to the present embodiment includes a processing circuit. The processing circuit acquires volume data to be compressed. The processing circuit acquires a function that associates at least one of a color value and an opacity value with each data value of the volume data. The processing circuit determines a compression rate for each region of the volume data based on at least one of a plurality of color values ​​and a plurality of opacity values ​​of each region. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a schematic diagram of an apparatus according to an embodiment. [Diagram 2] FIG. 2 is a flow chart outlining a method according to an embodiment. [Diagram 3] FIG. 3 is a plot of a transfer function according to an embodiment. [Figure 4] FIG. 4 is a plot of gradient visibility curves according to an embodiment. [Diagram 5] FIG. 5 is an example of a simplified summed gradient visibility table. [Figure 6] FIG. 6 is a representation of a simplified sparse octree structure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] A medical image processing apparatus 10 according to an embodiment is shown diagrammatically in Fig. 1. The apparatus 10 is configured to perform volume rendering of image data acquired by any one or more of a plurality of medical imaging scanners (not shown). The medical imaging scanners may comprise at least one of a CT (Computed Tomography) scanner, an MRI (Magnetic Resonance Imaging) scanner, an X-ray scanner, a PET (Positron Emission Tomography) scanner, a SPECT (Single Photon Emission Computed Tomography) scanner, an ultrasound scanner, or any suitable scanner. In other embodiments, the apparatus 10 may be configured to process any suitable two-dimensional or three-dimensional image data.

[0027] Apparatus 10 comprises a computing device 12, which in this example is a personal computer (PC) or a workstation. In other embodiments, computing device 12 may be any suitable computing device, such as, for example, a server, a desktop computer, a laptop computer, a mobile device, etc. In further embodiments, the functionality of computing device 12 may be provided by two or more computing devices.

[0028] Computing device 12 is connected to a display screen 16, or other display device, and one or more input devices 18, such as a computer keyboard or mouse. In an alternative embodiment, display screen 16 is a touch screen that also functions as input device 18.

[0029] In this embodiment, computing device 12 is configured to receive medical image data to be processed from a data store 20. Data store 20 stores data acquired by a number of medical imaging scanners.

[0030] In alternative embodiments, computing device 12 receives data from one or more further data stores (not shown) instead of or in addition to data store 20. For example, computing device 12 may receive data from one or more remote data stores (not shown) that may form part of a Picture Archiving and Communication System (PACS) or other information system, such as, for example, a laboratory archive, an Electronic Medical Record (EMR) system, an Admission Discharge and Transfer (ADT) system, etc. In further embodiments, computing device 12 may receive data directly from one or more scanners.

[0031] The computing device 12 comprises a central processing unit (CPU) 22 and a graphics processing unit (GPU) 30. The CPU 22 comprises a memory 24. The CPU 22 further comprises processing circuitry comprising a downsizing circuit 26 configured to create a downsized version of the image dataset, a visibility circuit 27 that determines a summed gradient visibility table from a transfer function and uses the summed gradient visibility table to determine per-block visibility of the downsized dataset, and a rendering circuit 28 configured to render an image from the mixed resolution version of the downsized dataset. The GPU 30 comprises a graphics memory 32. The GPU 30 is configured to perform high speed parallel processing of data stored in the graphics memory 32.

[0032] Together, the CPU 22 and GPU 30 provide processing resources for automatically or semi-automatically processing medical imaging data.

[0033] In this embodiment, the circuits 26, 27, and 28 are each implemented in the computing device 12 by a computer program having computer readable instructions executable to perform the method of the embodiment, however, in other embodiments, the various circuits may be implemented as one or more Application Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs).

[0034] Computing device 12 has a hard drive and other components of a PC, including RAM, ROM, a data bus, an operating system including various device drivers, and hardware devices including a graphics card, such components being not shown in FIG.

[0035] The apparatus of FIG. 1 is configured to carry out the method of processing medical image data shown in FIG.

[0036] At stage 40, the downsizing circuitry 26 receives an image data set. In this embodiment, the image data set comprises volumetric image data from a CT scan. A volumetric image data set may also be referred to as a data volume, volume data, or image data volume. That is, the processing circuitry obtains the volume data to be compressed.

[0037] The image dataset comprises individual intensity values ​​of a plurality of voxels of the image dataset.In further embodiments, the image dataset may comprise any suitable medical or non-medical image data.

[0038] In some embodiments, the data set received at stage 40 may have undergone pre-processing, for example pre-filtering to reduce speckle noise.

[0039] In stage 41, the downsizing circuit 26 creates a set of downsized volumes. Each of the downsized volumes is a downsized version of the image data set received in stage 40. The level of downsizing in each downsized volume is half the resolution of the previous level. That is, the processing circuit generates multiple downsized volumes from the volume data using different resolutions. For example, if the original image data set has a size of 2048×2048×3000, the first downsized volume has a size of 1024×1024×1500, the second downsized volume has a size of 512×512×7500, and so on. In this embodiment, four downsized volumes are created in addition to the original volume. In other embodiments, any suitable number of downsized volumes may be created, for example, 5, 6, 7, 8, 9, or 10.

[0040] Any suitable downsizing method may be used to create the downsized volume, for example, an averaging downsizing scheme may be used.

[0041] The downsizing circuit 26 provides the image data set and the downsized volume to a visibility circuit 27 .

[0042] In some embodiments, the downsized volume has already been created by other circuitry for use in interactive rendering modes, and in such embodiments, visibility circuitry 27 may receive the image data set and the downsized volume at stage 40 and omit stage 41. Downsizing circuitry 26 may be omitted in some embodiments.

[0043] In another embodiment, the downsized volume produced by downsizing circuitry 26 is also provided to rendering circuitry 28 for use in an interactive rendering mode.

[0044] At stage 42, the visibility circuit 27 receives a transfer function tf. For example, the processing circuitry obtains a function that associates at least one of a color value and an opacity value with each data value of the volume data. The function further associates an opacity value with each data value of the volume data. In particular, the transfer function tf corresponding to the function maps the data values ​​to corresponding opacity and color values. In this embodiment, the data values ​​are CT values ​​in Hounsfield units. The transfer function maps each CT value, which is an integer, to a corresponding opacity and color value. In other embodiments, any suitable type of data value may be used.

[0045] The transfer function tf may be a predefined transfer function that is stored, for example, in the data store 20 or the memory 24. In other embodiments, the transfer function tf may be specified by a user and obtained via user input.

[0046] Figure 3 shows an example of a transfer function tf. The transfer function tf comprises opacity curves 50A to 50F and a color gradient 52, each plotted against the same horizontal axis 54 of data values, which in this embodiment are voxel CT values. In the example shown in Figure 3, the scale of the data values ​​is from 0 to 1600. The vertical axis 56 is the opacity of the material. The scale of the opacity values ​​is from 0 to 1.

[0047] The opacity curve includes six sections 50A, 50B, 50C, 50D, 50E, and 50F, each section having a distinct opacity value slope.

[0048] Section 50A consists of data values ​​ranging from 0 to 40. In section 50A, the opacity values ​​are low and fairly constant. In section 50B, the data values ​​range from 40 to 260 and the opacity rises slowly. In section 50C, the data values ​​range from 260 to 550 and the opacity rises more quickly than in section 50B. In section 50D, the data values ​​range from 550 to 1020 and the opacity rises more rapidly. In section 50E, the data values ​​range from 1020 to 1470 and the rate of increase in opacity decreases. In section 50F, the data values ​​range from 1470 to 1600 and the opacity is fairly constant.

[0049] In the example shown in Figure 3, a common opacity value is used for all red, green, and blue. opacity,r =tf opacity,g =tf opacity,b In other embodiments in the present invention, color-dependent opacity values ​​may be used.

[0050] The color gradient 52 is a color mapping per CT value. The color values ​​of the color gradient 52 are represented in grayscale in FIG. 3 for ease of reproduction. In a full-color version of the color gradient 52, data values ​​below 500 would be represented as red gradually changing to orange. Data values ​​around 600 would be colored yellow. Above 750, the color values ​​become gray and then eventually white.

[0051] At stage 43, the visibility circuit 27 uses the transfer function tf to create a summed gradient visibility table that is designed to be used to analyze a block of pixels or voxels to determine whether a visible color gradient exists in a given block of pixels or voxels.

[0052] To construct the sum gradient visibility table, the visibility circuit 27 first determines the minimum data value DataMin and the maximum data value DataMax of the sum gradient visibility table. In this embodiment, the sum gradient visibility table has a length equal to the transfer function tf of interest. The minimum data value DataMin is the lowest data value in the transfer function, which is zero for the transfer function of Figure 3. The maximum data value DataMax is the highest data value in the transfer function, which is 1600 for the transfer function of Figure 3. The visibility circuit 27 considers the set of integer data values ​​i from DataMin to DataMax.

[0053] The sum gradient visibility value sgv(i) for each data value i in the transfer function tf is calculated as follows and stored in a sum gradient visibility table. The sum gradient visibility value sgv(i) may be described as an index value. The processing circuitry determines the index value for each data value using the above function. The index value depends, for example, on at least one of the color value and the opacity value. The index value comprises the sum gradient visibility value.

[0054] The visibility circuit 27 computes for each integer data value i a color value tf color (i) and opacity value tf opacity (i) is multiplied to obtain the original color tf premul To calculate (i), a transfer function tf is used. The unmultiplied color is a combination of the color value and the opacity value for a given data value.

[0055] In this embodiment, colors are described using the RGB (Red, Green, Blue) color space. color (i) is the three color components tf color (i) r , tf color (i) g , tf color (i) b These are collectively called tf color (i) rgb It is sometimes written as tf opacity(i) is the three color components tf opacity (i) r , tf opacity (i) g , tf opacity (i) b These are collectively called tf opacity (i) rgb It is sometimes written as:

[0056]

number

[0057] Since the effect of chrominance depends on opacity, which is included in the premultiplied color, even large chrominance in areas of high transparency can still have some visual impact.

[0058] In other embodiments, any suitable color space may be used to represent color values. In some embodiments, the CIELAB color scale is used instead of RGB. The CIELAB color scale is a measure of lightness L from black to white. * , green to red a * , blue to yellow b * The CIELAB color space is expressed as a three-component color space, L * , a * , or b * The color space is designed so that a given numerical change in the value of causes an amount of color change that a viewer will perceive as consistent, no matter where in the color space the numerical change occurs.

[0059] Each entry in the total gradient visibility table is calculated using the following formula:

[0060]

number

[0061] where gt is the threshold for acceptable gradients. In the embodiment of Figure 2, the threshold is gt = 0.00031.

[0062] The threshold gt is stored in memory 24. In other embodiments, the threshold gt may be stored in any suitable data store. In further embodiments, the threshold gt may be input by a user.

[0063] In summary, Equation 2 compares pre-multiplication values ​​obtained by multiplying the color and opacity as shown in Equation 1. If the difference between the pre-multiplication color of a given data value and the pre-multiplication color of the immediately preceding data value is less than a threshold gt, the interval gradient visibility value of the given data value is set to 0. If the difference between the pre-multiplication values ​​of the given data value and the immediately preceding data value is greater than or equal to a threshold gt, the interval gradient visibility value is set to 1. The total gradient visibility sgv(i) of a data value i is the sum of all the interval gradient visibility values ​​corresponding to the data values ​​from DataMin to i. That is, in determining the index value of a given data value, the processing circuit obtains the difference between the color and opacity combination of the given data value and the color and opacity combination of the immediately preceding data value, and uses the difference to determine the interval gradient visibility of the given data value. Specifically, the processing circuit compares the difference with a threshold to determine the interval gradient visibility. The processing circuitry then sums the interval gradient visibility values ​​from the minimum data value to the given data value to obtain a total gradient visibility value for the given data value, where the processing circuitry may calculate the index values ​​in the CIELAB color space.

[0064] The result is a flat function at all locations where there is no significant gradient in the pre-multiplication transfer function.

[0065] Figure 4 shows a curve of the total gradient visibility value obtained with the transfer function of Figure 3. The horizontal axis 54 shows the data values ​​on the same scale as in Figure 3. The vertical axis 58 shows the total gradient visibility sgv(i). The vertical axis 58 of the total gradient visibility function is the sum, which is the count of the number of visible color transitions in the transfer function between the minimum CT value of the transfer function and the current CT value shown on the x-axis.

[0066] The total gradient visibility curves 60A, 60B, 60C are shown for the transfer function tf of FIG. 3 and the threshold gt=0.00031.

[0067] The first section 60A of the total gradient visibility curve is from data values ​​0 to 120. There is no significant slope in the pre-multiplication transfer function from 0 to 120, so the total gradient visibility curve is flat.

[0068] The second section 60B of the total gradient visibility curve is from data values ​​120 to 1520. In this region, the total gradient visibility increases due to the slope of the transfer function. In section 60B, an increase of 1 is seen for each data value because all color transitions in this region are calculated to be visible. The value of the sgv(x) function increases by 1 for each increment of the x-axis.

[0069] The third section 60C of the total gradient visibility curve is from data values ​​1520 to 1600. From 1520 to 1600 there is no significant slope in the pre-multiplication transfer function, so the total gradient visibility curve is flat.

[0070] The visibility circuit 27 stores the sum gradient visibility table in the memory 24. That is, the memory 24 stores the data values ​​and the index values ​​in the volume data. In other embodiments, the sum gradient visibility table may be stored in any suitable location. In further embodiments, any suitable method may be used to store the sum gradient visibility. For example, the sum gradient visibility may be stored as a list, a lookup table, or a function.

[0071] Figure 5 shows a simplified version of a total gradient visibility table 70. The total gradient visibility table of Figure 5 has only eleven data values ​​ranging from 0 to 10. The data values ​​of Figure 5 are independent of the transfer function shown in Figure 3. The total gradient visibility table of Figure 5 is shown for a simple example of how to calculate the total gradient visibility values.

[0072] A first row 71 of the total gradient visibility table 70 includes data values ​​from DataMin, which in this example is 0, to DataMax, which in this example is 10. A second row 72 of the total gradient visibility table 70 includes color values ​​for each data value in the first row 70 obtained from an appropriate transfer function (not shown). The color values ​​include RGB values ​​represented in FIG. 5 by letters, e.g., (x,x,x). In the notation used in the exemplary total visibility table 70, primes are used to indicate very similar values, e.g., (x,x,x) and (x,x,x'). Different letters are used to indicate more distinct colors, e.g., (x,x,x) and (x,x,y).

[0073] In reality, the second row stores a set of three numbers which are the red, green and blue color values.

[0074] The third row 73 of the total gradient visibility table 70 contains an opacity value for each data value in the first row 70 obtained from a suitable transfer function (not shown). In the example shown in Figure 5, the data values ​​0, 1, 9 and 10 have an opacity value of zero. The other data values ​​have an opacity value of 0.1.

[0075] The fourth row 74 of the sum gradient visibility table 70 contains a value of 0 or 1 for each data value. These are the gradient visibility values ​​before summation. premul (i) and tf premul If the difference with (i-1) is less than the threshold gt, then the corresponding interval gradient visibility in the fourth row 74 is 0, and tf premul (i) and tf premul If the difference from (i-1) is greater than or equal to the threshold gt, then the corresponding interval gradient visibility in the fourth row 74 is one.

[0076] The fifth row 75 of the total gradient visibility table 70 contains the value of the total gradient visibility value, which is obtained by adding the interval gradient visibility values ​​from DataMin to i.

[0077] Returning to the flow chart of FIG. 2, at stage 44 the visibility circuit 27 calculates the gradient visibility for each block of each downsized volume obtained at stage 41 .

[0078] Consider an example of a downsized volume. The visibility circuit 27 considers a number of blocks within the downsized volume. For example, the processing circuit divides the volume data into a number of blocks, each having a number of pixels or voxels. Specifically, the processing circuit divides each of the downsized volumes into a number of separate blocks, each having a block size of k×k×k pixels or voxels. k×k×k is k 3In this embodiment, the block size is 8x8x8 voxels. In other embodiments, any suitable block size may be used. Smaller block sizes may result in greater compression, but require more overhead to calculate the visibility of the blocks. In some embodiments, a small block size of 1x1x1 may be used.

[0079] For each level of the downsized volume, the visibility circuitry 27 uses the sum gradient visibility table to find regions where the source data will display a visible gradient. For each voxel in the block, the visibility circuitry 27 checks the minimum and maximum interpolated data values ​​against the sum gradient visibility table. For example, the processing circuitry determines a range of data values ​​obtainable by interpolation of data values ​​of pixels or voxels in the block.

[0080] Consider each individual voxel in the block. Each voxel has an individual set of coordinate values ​​(x,y,z). The visibility circuit 27 establishes a region of interest around the voxel, also called a voxel neighborhood. For example, the region of interest may include the voxel of interest and its nearest neighbors. The region of interest may include voxels that may be combined into the selected voxel during interpolation. In this embodiment, the size of the region of interest is the same as the block size. In other embodiments, any suitable region of interest may be used. For example, the region of interest may be larger than the block size.

[0081] The visibility circuit 27 reads the data values ​​of the voxels in the region of interest and determines the minimum data value of the voxels in the region of interest. The visibility circuit 27 calculates a sum gradient visibility value gv of the minimum data values ​​in the region of interest surrounding the voxel at (x,y,z). min To get (x,y,z), look up the stored sum gradient visibility table.

[0082]

number

[0083] According to Equation 3, the block size is k 3 find the minimum data value in the volume of the 3D region from x to x+k, y to y+k, and z to z+k, and determine the total gradient visibility of the minimum data value.

[0084] The visibility circuit 27 reads the data values ​​of the voxels in the region of interest and determines the maximum data value of the voxel in the region of interest. The visibility circuit 27 calculates a sum gradient visibility value gv of the maximum data values ​​in the region of interest surrounding the voxel at (x,y,z). max To get (x,y,z), look up the stored sum gradient visibility table.

[0085]

number

[0086] According to Equation 4, the block size is k 3 find the maximum data value in the volume of the 3D region from x to x+k, y to y+k, and z to z+k, and determine the total gradient visibility of the maximum data value.

[0087] The minimum and maximum data values ​​may be considered to define a range of possible interpolated data values. In other embodiments, any suitable method may be used to find the maximum and minimum data values.

[0088] The sum of the minimum data values ​​gradient visibility gv min (x,y,z) is the sum of the maximum data values ​​gradient visibility gv max If it is equal to (x,y,z), the visibility circuit 27 designates the voxel as invisible.

[0089] The sum of the minimum data values ​​gradient visibility gv min (x,y,z) is the sum of the maximum data values ​​gradient visibility gv max If it is not equal to (x,y,z), then the visibility circuit 27 designates the voxel as visible.

[0090]

number

[0091] If the generated values ​​are different, then the voxel is determined to contain a potentially visible gradient and VoxelVisible(x,y,z) is set to true. If the generated values ​​are equal, then the voxel is determined to not contain a potentially visible gradient and VoxelVisible(x,y,z) is set to false.

[0092] If the total gradient visibility of the minimum data value is equal to the total gradient visibility of the maximum data value, the voxel's neighborhood may be considered to have no color gradients above the threshold. The combination of color and opacity values ​​will be similar for all possible interpolated data values ​​in the voxel's neighborhood. The voxel's neighborhood may be considered to be homogenous.

[0093] The intention is to retain in the final rendering those blocks whose gradients are potentially visible: the visibility circuit 27 designates a block as visible if any of the voxels within the block are designated as visible.

[0094]

number

[0095] A visibility circuit 27 determines the block-by-block gradient visibility for each block of the downsized volume, with volumes with a higher level of downsizing being formed from fewer blocks than volumes with a lower level of downsizing.

[0096] In the embodiment of FIG. 2, the total gradient visibility value is used as an index value in determining visibility for each block. That is, the processing circuitry designates each block as having or not having a visible gradient using index values ​​corresponding to data values ​​of pixels or voxels in the block. For example, the processing circuitry designates a block as not having a visible gradient if there is no difference in the index values ​​within the range of data values, and designates a block as having a visible gradient if there is a difference in the index values ​​within the range of data values. More specifically, the processing circuitry designates each block of each of the downsized volumes as having or not having a visible gradient. In other embodiments, any suitable index value may be used. Visibility may be determined based on interval gradient visibility values ​​for each data value. Visibility may be determined from any suitable combination of parameters or functions based on opacity and color values, or opacity and color values. The index values ​​may be stored in any suitable format, such as any suitable table, list, or function, and the processing circuitry may recalculate the index values ​​using further reduced thresholds.

[0097] At stage 45, the visibility circuitry 27 computes a sparse octree structure. The sparse octree structure has as many levels as there are downsized volumes. In this embodiment, the sparse octree structure has four downsized levels and a level representing the original, non-downsized volume. The sparse octree structure is computed using the per-block visibility values ​​computed at stage 44. That is, for each region of the volume data, the processing circuitry computes a sparse octree representing the most downsized blocks in that region that have been designated as not containing visible gradients.

[0098] An octree is a node structure in which each node branches into eight child nodes. Octree structures are often used in 3D rendering.

[0099] In this embodiment, the first level nodes of the octree structure represent the blocks of the most downsized volume.

[0100] If a block of the most downsized volume does not have a gradient greater than the threshold and is designated as invisible in stage 44, then no child nodes are generated from the node representing that block in the first level.

[0101] If a block of the most downsized volume has a gradient greater than the threshold and is designated as visible in stage 44, then from the node representing that block in the first level, eight child nodes in the second level are generated, each of which is the next highest level of downsizing and is one-eighth the size of the block in the most downsized level.

[0102] If a child node does not have a gradient greater than the threshold, no grandchild nodes are generated from that child node. If the child node has a gradient greater than the threshold, eight grandchild nodes at the third level are generated from that child node.

[0103] The depth of the octree at each point is equal to the most downsized block in the region that does not display a gradient greater than the threshold. Where there is a gradient greater than the threshold, the octree proceeds down, generating further nodes until there are no more gradients greater than the threshold or until the original undownsized volume is reached.

[0104] 6 shows a simplified example of a portion of a sparse octree structure 80. Nodes 82 at a first level of the sparse octree structure 80 correspond to blocks of the most downsized volume. The first level of the sparse octree structure also includes other nodes that correspond to other blocks of the most downsized volume, not shown in FIG.

[0105] In the example of Figure 6, it was determined at stage 44 that the block represented by node 82 contains at least one visible gradient. Therefore, eight child nodes 84A to 84H are generated from node 82. The eight child nodes represent blocks that together occupy the same volume as the block represented by node 82. The eight child nodes are a second level of a sparse octree structure 80. Each block in the second level is one-eighth the size of a block in the first level.

[0106] The visibility circuit 27 determines the visibility of each of the eight child nodes 84A to 84H at stage 44. In the example of FIG. 5, the blocks of child nodes 84A and 84G are each determined to contain a visible gradient. The blocks of child nodes 84B, 84C, 84D, 84E, 84F, and 84H are determined to not contain a visible gradient. Thus, grandchild nodes are generated from child nodes 84A and 84G, but grandchild nodes are not generated from child nodes 84B, 84C, 84D, 84E, 84F, and 84H. The octree structure 80 is sparse because only a portion of the nodes at a given level of the octree have child nodes.

[0107] Eight grandchild nodes 86A to 86H are generated from child node 84A, and eight grandchild nodes 88A to 88H are generated from child node 84G.

[0108] In practice, the sparse octree structure 80 is likely to have more than three levels. The visibility determination of stage 44 is used to determine which nodes in the sparse octree structure 80 have child nodes.

[0109] The sparse octree structure 80 is deeper in regions of the image volume where there is greater color variation, which may suggest greater anatomical detail in these regions. The sparse octree structure 80 is shallower in regions of the image volume where the color is substantially homogenous.

[0110] The sparse octree structure 80 may provide a fast way to perform sparse spatial lookups of an image volume without having to index all the data to perform the spatial lookup.

[0111] At stage 46 , the rendering circuitry 28 renders an image from the volumetric image dataset using the sparse octree structure 80 determined at stage 45 .

[0112] The rendering circuitry 28 uses a sparse octree structure 80 to determine the mixed resolution data set. The mixed resolution data set may also be referred to as a mixed resolution data volume or mixed resolution volume data. For each region of the image volume, the mixed resolution data set contains the most downsized blocks of the region that do not contain visible gradients. Thus, different parts of the mixed resolution data set come from different downsized volumes. Some regions of the mixed resolution data set compress better than other regions.

[0113] For example, the processing circuitry calculates the mixed resolution volume data using a sparse octree such that the mixed resolution volume data includes multiple blocks from different ones of the downsized volumes. That is, the processing circuitry determines a compression ratio for each region of the volume data based on at least one of the multiple color values ​​and multiple opacity values ​​of each region. In other words, the processing circuitry determines the compression ratio depending on at least one of the multiple color values ​​and multiple opacity values ​​of each block. For example, the processing circuitry determines a compression ratio for each region of the volume data based on the multiple color values ​​and multiple opacity values ​​of each region. Specifically, the processing circuitry determines the compression ratio depending on multiple index values ​​corresponding to multiple data values ​​in each region. Note that if the multiple index values ​​are recalculated using a further reduced threshold, the processing circuitry updates the mixed resolution data volume based on the further reduced threshold.

[0114] The sparse octree structure 80 determines the best downsizing level to use for each used block of the volumetric imaging data set. The rendering circuitry 28 queries the sparse octree structure 80 to find the most downsized block that was designated invisible in stage 44.

[0115] By referencing the downsizing levels at the time of sampling, rendering may be performed using the most downsized level that does not include a visible gradient. For example, the processing circuitry may render the volumetric data using a compression ratio based on at least one of the color values ​​and the opacity values ​​of each region such that different regions are rendered using different compression ratios. A more downsized version of the volumetric imaging dataset may be used to render regions that include little color variation, and a less downsized or non-downsized version may be used to render regions that include more color variation.

[0116] The rendering circuitry 28 passes the mixed resolution data set to the GPU memory 32. The GPU 30 samples the mixed resolution data set using conventional sampling methods. In this embodiment, an atlas-based system is used to handle the management and uploading of the various volume data blocks. An example of an atlas-based system is described in U.S. Patent 10,719,907, which is incorporated herein by reference. An area of ​​memory on the GPU 30 is pre-allocated for storage of the volume data and an index giving the memory location of the specified location. Based on the allowable compression calculated as above, a downsized area of ​​the data is sent to the GPU, along with an updated index to point to the downsized data.

[0117] In this embodiment, the rendering method used is Shaded Volume Rendering (SVR) including ray tracing. When rendering a block, the rendering circuitry 28 looks up the sparse octree structure 80 to find the downsizing level to use for that block. The ray step size used in ray tracing is modified based on the downsizing level of the block currently in use. If the block through which the ray passes is not downsized, a first ray step size between ray sampling points is used. If the block through which the ray passes is downsized, the ray step size between ray sampling points may be increased. For example, when sampling from a 2× downsized volume, the ray step size used may be twice the first ray step size. Using the increased ray step size may accelerate rendering. For example, the processing circuitry selects a ray step size for ray traversal based on the sparse octree to render the mixed resolution volume data.

[0118] In some circumstances, lookups of the sparse octree structure 80 may cause a slowdown in rendering speed. Reducing the ray step size in downsized blocks may compensate for such slowdown.

[0119] At stage 47 , rendering circuitry 28 displays the rendered image on display screen 16 .

[0120] The method of Figure 2 may enable rendering of volumes much larger than the available device memory. Large-scale memory requirements may be significantly reduced. The amount of data uploaded to the GPU may be significantly reduced.

[0121] The method of FIG. 2 may provide a transfer function aware block-based compression scheme for volume rendering that can achieve compression on the order of 10:1 in practical cases. That is, when the technical idea according to the present embodiment is realized in a medical image processing method according to the procedure shown in FIG. 2, the medical image processing method obtains volume data to be compressed, obtains a function that associates at least one of a color value and an opacity value with each data value of the volume data, and determines a compression ratio for each region of the volume data based on at least one of a plurality of color values ​​and a plurality of opacity values ​​of each region. For example, the compression range may be from 3:1 to 30:1 depending on the data used and the transfer function applied. A calibration value (threshold gt) may be used to change the degree of compression.

[0122] The method of Figure 2 has been shown to scale well for larger volumes.

[0123] The sparse octree structure 80 may provide a quick way to perform sparse spatial lookups without indexing all of the data. Similar areas of the data set may be used for higher level nodes of the sparse octree structure 80, i.e., higher levels of compression, without further partitioning.

[0124] An exemplary large matrix volume was compressed using the method described above with reference to FIG. 2. At full resolution 1024×1024×221, the size of the volumetric image data set was 442 MB. At mixed resolution, where the resolution of each region of the image volume is determined using the method of FIG. 2, the size of the volumetric image data set was 45 MB. Images rendered using full resolution and mixed resolution are very similar. Thus, a significant level of compression was achieved using the method of FIG. 2. As described above, according to the present embodiment, volume data can be compressed with reduced impact on the rendered image.

[0125] In experiments testing the method of Figure 2, it was found that a compression ratio of 9.8:1 was achieved using the transfer function of Figure 3. A compression ratio of 30:1 was achieved when using a box bone transfer function where there is no flash and only bone is visible. A compression ratio of 3:1 was achieved when using a foggy transfer function where all the volume is visible but some are only dimly visible.

[0126] By modifying the gradient visibility threshold gt, compression may be traded off against visual fidelity. A higher threshold gt results in higher compression, but the image may be somewhat different than that produced at full resolution. In some circumstances, such changes in the image may be acceptable to the user. In some embodiments, the user can directly select the gradient visibility threshold gt or can select an acceptable level of image degradation.

[0127] In one example, multiple images are rendered from the same volumetric image dataset using different values ​​of the threshold gt. The largest volume has a size of 422MB.

[0128] The first image is rendered with gt=0.001. The size of the resulting compressed dataset is 14.3MB, with a compression ratio of 30.8:1.

[0129] The second image is rendered with gt=0.0003. The size of the resulting compressed dataset is 20.2MB, with a compression ratio of 21.9:1.

[0130] The third image is rendered with gt=0.00005. The size of the resulting compressed dataset is 33.4MB, with a compression ratio of 13.2:1.

[0131] The 4 images are rendered with gt=0.000025. The size of the resulting compressed dataset is 82.2MB, with a compression ratio of 5.4:1.

[0132] For each of the first through fourth images, a separate difference image is obtained that represents the difference between that image and an image rendered with the largest uncompressed data set. As the threshold gt is decreased, the separate difference images show smaller differences. At the lowest threshold, it may be very difficult for a user to perceive a difference in the images.

[0133] The fifth image was rendered using a downsized volume using 2x downsizing. The downsized volume has a fixed resolution rather than a mixed resolution. The downsized volume was not obtained using the method of Figure 2. The size of the downsized volume is 55MB. The compression ratio is 8:1.

[0134] Take the difference between the image rendered with the full size volume and a fifth image rendered with a 2x downsized volume.

[0135] The sixth image is rendered using the method of FIG. 2 with gt=0.0000275. The threshold is chosen to result in a dataset of similar size to the 2× downsized dataset used to render the fifth image. The dataset for the mixed resolution dataset used when rendering the sixth image is 54MB. The compression ratio is 8.2:1.

[0136] Obtain the difference between the image rendered using the maximum size volume and a sixth image rendered from the mixed resolution dataset obtained using the method of FIG.

[0137] The difference between the sixth image (rendered using the method of Figure 2) and an image rendered from a full-size volume is much smaller than the difference between the fifth image (rendered using a 2x downsize of a fixed resolution) and an image rendered from a full-size volume. The sixth image rendered using the method of Figure 2 retains much more detail than the fifth image obtained using pure downsizing.

[0138] By using the method of FIG. 2, improved image quality may be obtained from comparable data size by using a compression ratio that varies across regions of the data set rather than a fixed compression ratio.

[0139] The application of the method of FIG. 2 may not be limited to large-scale volume data. The degree of compression for smaller matrix volumes may be reduced because smaller matrix volumes typically contain larger gradients. However, compression for smaller matrix volumes may still be achieved in a manner similar to compression for larger matrix volumes.

[0140] The method of Fig. 2 may also be used for applications on hardware with more limited available memory. Mobile devices, augmented reality (AR) or virtual reality (VR) may benefit from compression of the rendered data even at conventional data resolutions. For example, a true 6:1 compression of the rendered data may be very useful in enabling rendering of large conventional volumes by mobile devices.

[0141] In some embodiments, the method of Figure 2 is used to facilitate data transfer between devices. For example, a mixed resolution data set may be acquired by a first device using the method of Figure 2. The mixed resolution data set may be transferred to a second device for rendering.

[0142] Sending a compressed data set rather than the full data volume may, for example, allow a traditional data volume to be rendered very quickly in a browser. For example, sending a 400MB traditional volume (512x512x800) over a network may be expensive. Sending a 40MB compressed volume over the network may be much faster.

[0143] In some embodiments, by varying the threshold gt, the volume quality is gradually refined. For example, in one embodiment, a first device creates a first data set by compressing an original data volume using a first threshold gt. For example, the first data set may have a compression of 30:1. The first device transfers the first data set to a second device. The second device renders an image from the first data set and displays the rendered image.

[0144] While the second device renders an image from the first data set, the first device creates and transmits a second data set that is a less compressed representation of the original data volume. The second data set is compressed using a second, lower threshold gt. The second data set is larger than the first data set. The second device renders the second data set to refine the displayed data.

[0145] Any number of data sets may be transmitted with different degrees of compression: as the threshold gt is lowered, the level of detail in the rendered image increases.

[0146] In some embodiments, even when all rendering is performed on a single device, a threshold drop may be used in the rendering: by using a high degree of compression initially, the image may be displayed in a time that the user considers acceptable, and then the image may be refined over time.

[0147] In the method of Figure 2, rendering is performed using SVR. In another embodiment, Global Illumination rendering is performed. In systems using Global Illumination, memory is shared between volumetric image data and photon maps. Often, 50% of memory is allocated to volumetric image data and 50% of memory is allocated to photon maps. Significant compression of the volumetric image data may result in significant savings in the overall memory requirements of the system.

[0148] In some embodiments, the total gradient visibility logic is applied to the transmittance transfer function used during photon mapping. The transmittance transfer function describes the color of light that can pass through a material. The transmittance transfer function is used to perform the photon rendering method, which aims to model the transmittance to achieve more realism by obtaining realistic shadows. Photon rendering includes a photon mapping stage before the primary rendering. In the photon mapping stage, the path of each individual photon in the scene is traced. The energy loss of the photon as it passes through the material is modeled. The transmittance transfer function is used to compress the photon volume using the total gradient visibility method.

[0149] In some embodiments, an intensity projection (IP) view is rendered. The rendering may include a multi-planar reconstruction. A total gradient visibility table is constructed using a window width and a window level. In some embodiments, the IP view is rendered using a mapping from data values ​​to grayscale shades. In other embodiments, a full color transfer function is applied to the data values. An interval visibility value is determined by taking the difference in color values ​​(which may include grayscale) for each interval and comparing the difference in color values ​​to a threshold. The interval visibility values ​​are then summed from a minimum value to a given data value to obtain a total gradient visibility value for that data value. The total gradient visibility value is used to perform compression as described above.

[0150] In some embodiments, a min-max tree may be used to speed up the calculation of the downsampling required for each block. The min-max tree contains the minimum and maximum data values ​​for each block.

[0151] In some embodiments, the use of the sum gradient visibility is combined with the use of an atlas-based system. One atlas-based system is described in U.S. Patent 10,719,907, which is incorporated herein by reference. The atlas-based system employs a bricked structure, which may also be described as a building block. The offset compression system used in the atlas-based system may be used in conjunction with the sum gradient visibility compression using the method of FIG. 2. To transform the data values ​​by weighting or offsetting, a region of the data set may be stored in a region of a multi-channel atlas texture using a transformation vector. The data values ​​are then restored using the transformation vector.

[0152] Further performance improvements may be achieved by using atlas-based methods in conjunction with mixed-resolution datasets, which may reduce the upload burden.

[0153] Although the above embodiments have described medical imaging data, in other embodiments, the method may be used to compress and / or render any data. For example, the data may include oil and gas data. The data may include three-dimensional microscopy data. The data may include, for example, climate data, geological surveys, demographics, or gaming data. The method may be used in any application where high speed parallel sampling occurs, for example, high speed parallel sampling of different data types. The method may be used in any system based on a voxelized transfer function.

[0154] Although particular circuits are described herein, in alternative embodiments, the functionality of one or more of those circuits may be provided by a single processing resource or other component, or the functionality provided by a single circuit may be provided by a combination of two or more processing resources or other components. Reference to a single circuit encompasses multiple components that provide the functionality of that circuit, whether or not such components are separate from one another. Reference to multiple circuits encompasses single components that provide the functionality of those circuits.

[0155] While certain embodiments have been described, these embodiments are presented for illustrative purposes only and are not intended to limit the scope of the invention. In fact, the novel methods and systems described herein may be embodied in a variety of other forms. Furthermore, various omissions, substitutions, and changes in the form of the methods and systems described herein may be made without departing from the spirit of the invention. The appended claims and their equivalents are intended to cover such forms and modifications as fall within the scope of the invention.

[0156] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention.

[0157] Regarding the above-described embodiments, the following supplementary notes are disclosed as one aspect and optional features of the invention. (Appendix 1) In a first aspect, a medical image processing device is provided that has a processing circuit configured to acquire a data volume to be compressed, acquire a function that associates a color value with each data value of the data volume, and change the compression rate for each region of the data volume based on the color value of each region. (Appendix 2) The function may further associate an opacity value with each data value of the data volume.The variation of the compressibility for each of the regions of the data volume may be based on a colour value and an opacity value of each region. (Appendix 3) The function may be a transfer function. The function may be a reflectance transfer function. The function may be a transmittance transfer function. (Appendix 4) The processing circuitry may be further configured to render an image from the data volume using a compression ratio based on a color value of each region, such that different regions are rendered using different compression ratios. The processing circuitry may be further configured to render an image from the data volume using a compression ratio based on a color value and an opacity value of each region, such that different regions are rendered using different compression ratios. (Appendix 5) The processing circuitry may be further configured to divide the data volume into a number of blocks, each block having a number of pixels or voxels. The variation of the compression ratio may depend on a color value of each block. The variation of the compression ratio may depend on a color value and an opacity value of each block. (Appendix 6) The processing circuitry may be further configured to determine an index value for each data value using the function. The index value may be dependent on a color value. The index value may be dependent on a color value and an opacity value. (Appendix 7) The apparatus may further comprise a memory configured to store the data values ​​and index values. The variation of the compression ratio may be dependent on the index values ​​corresponding to the data values ​​in each region. (Appendix 8) The determining of the index value for a given data value may include obtaining a difference between a color and opacity combination of the given data value and a color and opacity combination of an immediately preceding data value and using the difference to determine an interval gradient visibility for the given data value. Using the difference to determine interval gradient visibility may include comparing the difference to a threshold value. (Appendix 9) The index value may comprise a summed gradient visibility value, which for a given data value may be obtained by adding interval gradient visibility values ​​from a minimum data value to the given data value. (Appendix 10) The index value may be calculated in the CIELAB color space. The color value may include a grayscale value. (Appendix 11) The processing circuitry may be further configured to designate each block as containing a visible gradient or not containing a visible gradient, the designation of each block being made using the index values ​​corresponding to the data values ​​of pixels or voxels within the block. (Appendix 12) The designation of each block as having or not having a visible gradient may include determining a range of data values ​​obtainable by interpolation of data values ​​of pixels or voxels within the block, and designating the block as not having a visible gradient if there is no difference in index values ​​within the range of data values, and designating the block as having a visible gradient if there is a difference in index values ​​within the range of data values. (Appendix 13) The processing circuitry may be configured to generate a plurality of downsized volumes from the data volume, each of the downsized volumes may be generated using a different compression ratio. (Appendix 14) The processing circuitry may be further configured to divide each of the downsized volumes into a plurality of separate blocks and designate each block of each of the downsized volumes as including or not including a visible gradient. (Appendix 15) The processing circuitry may be further configured to compute, for each region of the data volume, a sparse octree indicating the most downsized blocks in that region that are designated as not containing visible gradients. (Appendix 16) The processing circuitry may be further configured to compute a mixed resolution data volume using the sparse octree, the mixed resolution data volume may include blocks from different ones of the downsized volumes. (Appendix 17) The processing circuitry may be configured to select a ray step size for ray traversal based on the sparse octree. (Appendix 18) The processing circuitry may be configured to send the mixed resolution data volume over a network connection to a further device for rendering. (Appendix 19) The processing circuitry may be configured to recalculate the index value using a further reduced threshold value. (Appendix 20) The processing circuitry may be configured to render a further image from the data volume based on index values ​​calculated using the further reduced threshold value for compressibility. (Appendix 21) The processing circuitry may be configured to update the mixed resolution data volume based on the further reduced threshold and the processing circuitry may be configured to send the updated mixed resolution data volume to the further device for rendering over the network connection. (Appendix 22) In a further aspect, a method for processing medical images is provided, comprising obtaining a data volume to be compressed, obtaining a function relating a color value to each data value of the data volume, and varying a compression ratio for each region of the data volume based on the color value of each region. (Appendix 23) The function may further associate an opacity value with each data value of the data volume.The variation of the compressibility for each of the regions of the data volume may be based on a colour value and an opacity value of each region. (Appendix 24) In a further aspect, there is provided a non-transitory computer readable medium encoded with computer readable instructions that, when executed on a computer, cause the computer to perform a method comprising obtaining a data volume to be compressed, obtaining a function that associates a color value with each data value of the data volume, and varying a compression ratio for each region of the data volume based on the color value of each region. (Appendix 25) In a further aspect that can be provided independently, a medical image processing device is provided that includes a processing circuit configured to acquire data to be compressed and change the compression ratio based on assignment information of color values ​​and opacity values ​​corresponding to each data value of the data. (Appendix 26) The processing circuitry may be further configured to divide the data into blocks consisting of a plurality of voxels, and to change the compression ratio based on assignment information of color values ​​and opacity values ​​corresponding to each data value of the plurality of voxels included in the block. (Appendix 27) The processing circuitry may be further configured to determine a need for compression of each block based on differences in index values ​​corresponding to the data values ​​of voxels within the block. (Appendix 28) The index value may be determined by an sgv value that is determined based on a difference between index values ​​of transfer functions corresponding to the data values. (Appendix 29) The index value may be determined by adding the sgv value, which is proportional to the data value. (Appendix 30) In a further aspect, which may be provided independently, a method is provided for visualizing volume data using a compressed block-based data format in which the required level of downsizing for each block is calculated through the use of a sum gradient visibility table generated from a rendering transfer function, and the resulting mixed resolution volume is rendered into an image for display. (Appendix 31) The total gradient visibility table may be generated using color gradients calculated in the CIELAB visually uniform color space. (Appendix 32) The ray step size may be modified to match the scaling of the resident downsampled block. (Appendix 33) A sparse octree may be computed that describes the most downsized blocks in a given region that pass the gradient threshold test and is then used during ray traversal to dynamically vary the step size. (Appendix 34) The rendering subsystem may reside on a second linked machine. The mixed resolution data volume may be sent over a network connection for rendering at the client site. (Appendix 35) The threshold used to calculate the sum gradient visibility table may be gradually reduced over time so that the mixed resolution data volume slowly gains detail as additional high resolution regions are streamed over the network. (Appendix 36) In a further aspect that may be provided independently, there is provided a method of rendering a view of a volumetric imaging dataset representing a volume, in which a compressed block-based data format is used and the volumetric imaging dataset is downsized into blocks of a lower resolution than the dataset, the method comprising: determining, for blocks of different sizes, whether potentially visible image features of interest, e.g. gradients, are present in an image resulting from the rendering applied to the blocks; selecting a block size dependent on the determination of whether potentially visible image features of interest, e.g. gradients, are present; and rendering using the selected block size to create an image. (Appendix 37) The rendering may comprise using a transfer function to determine color and / or opacity and / or other image parameters based on values ​​of the data set. The method may comprise creating a gradient visibility table based on the transfer function and using the table to determine the block size. (Appendix 38) For blocks of different sizes, said determining whether potentially visible image features of interest, such as gradients, are present may include determining whether gradients are present that have gradient values ​​greater than a threshold. (Appendix 39) Different downsizing levels may be used for different regions within the volume, depending on which downsizing levels for the different regions result in gradients greater than a threshold in an image rendered from the downsized data. (Appendix 40) The gradient visibility table may be generated using color gradients calculated in a perceptually uniform color space. (Appendix 41) The rendering may include using a ray casting process, and a step size of the rays may be modified to match the scaling of the downsampled block or blocks. (Appendix 42) A sparse octree may be computed that describes the most downsized blocks in a given region that pass the gradient threshold test and is then used during ray traversal to dynamically vary the step size. (Appendix 43) The method may comprise performing the downsizing using a first processing resource, transmitting the downsized data over a network to a second processing resource, and performing the rendering at the second processing resource to generate an image. (Appendix 44) As the high resolution downsized data is transmitted over the network and rendered, the or some gradient threshold may decrease over time so that the rendered image gains detail over time. (Appendix 45) Features in one embodiment may be provided as features in any other embodiment as appropriate. For example, method features may be provided as apparatus features and vice versa. Any one or more features of an embodiment may be provided in combination with any suitable one or more features in any other embodiment. [Explanation of symbols]

[0158] 10 equipment 12 Computing Equipment 16 Display Screens 18 Input Devices 20 Data storage unit 22 CPU 24 Memory 26 Downsizing Circuit 27 Visibility Circuit 28 Rendering Circuit 30 GPU 32 Graphics Memory

Claims

1. Obtain volume data to be compressed; obtaining a function relating at least one of a color value and an opacity value to each data value of the volumetric data; determining a compression ratio for each region of the volume data based on at least one of a plurality of color values ​​and a plurality of opacity values ​​of each region; Dividing the volume data into a number of blocks, each block having a number of pixels or voxels; determining the compression ratio in dependence on at least one of a plurality of color values ​​and a plurality of opacity values ​​of each block; determining an index value for each data value using said function; A processing circuit is provided, The index value depends on the color value, a memory for storing a plurality of data values ​​and a plurality of index values ​​in the volume data; the processing circuit determines the compression ratio in dependence on the index values ​​corresponding to the data values ​​within each region. Medical imaging equipment.

2. the function further associates an opacity value with each data value of the volume data; the processing circuit determines a compression ratio for each of the regions of the volume data based on a plurality of color values ​​and a plurality of opacity values ​​of each region; The medical image processing device according to claim 1 .

3. the processing circuitry renders the volume data using a compression ratio based on a plurality of color values ​​of each region such that different regions are rendered using different compression ratios. The medical image processing device according to claim 1 .

4. the processing circuitry, in determining the index value for a given data value, obtains a difference between a color and opacity combination of the given data value and a color and opacity combination of an immediately preceding data value and uses the difference to determine an interval gradient visibility for the given data value. The medical image processing device according to claim 1 .

5. the processing circuitry compares the difference to a threshold to determine the interval gradient visibility. The medical image processing device according to claim 4 .

6. The index value includes a total gradient visibility value; the processing circuitry sums a plurality of interval gradient visibility values ​​from a minimum data value to the given data value to obtain the total gradient visibility value for the given data value. The medical image processing device according to claim 5 .

7. The plurality of index values ​​are calculated in the CIELAB color space. The medical image processing apparatus according to claim 1 .

8. the processing circuitry designates each block as having or not having a visible gradient using the index values ​​corresponding to the data values ​​of the pixels or voxels in the block. The medical image processing apparatus according to claim 1 .

9. The processing circuitry includes: determining a range of data values ​​obtainable by interpolation of data values ​​of pixels or voxels within the block; designating the block as not containing a visible gradient if there is no difference in the index values ​​within the range of data values; designating the block as including a visible gradient if there is a difference in index values ​​within the range of data values; The medical image processing apparatus according to claim 8 ,

10. the processing circuitry generates a plurality of downsized volumes from the volumetric data using different resolutions. The medical image processing device according to claim 5 .

11. The processing circuitry includes: Dividing each of the downsized volumes into individual blocks; designating each block of each of the downsized volumes as containing a visible gradient or not containing a visible gradient; The medical image processing device according to claim 10.

12. the processing circuitry computes, for each region of the volumetric data, a sparse octree indicating the most downsized blocks in that region designated as not containing visible gradients; The medical image processing device according to claim 11.

13. the processing circuitry computes the mixed resolution volume data using the sparse octree, such that the mixed resolution volume data includes a plurality of blocks from different ones of the downsized volumes. The medical image processing device according to claim 12.

14. the processing circuitry selects a ray step size for ray traversal based on the sparse octree to render the mixed resolution volume data. The medical image processing device according to claim 13.

15. the processing circuitry recalculates the plurality of index values ​​using a further reduced threshold value. The medical image processing device according to claim 13.

16. the processing circuitry updates the mixed resolution volume data based on the further reduced threshold. The medical image processing device according to claim 15.

17. Obtain volume data to be compressed; obtaining a function relating at least one of a color value and an opacity value to each data value of the volume data; determining a compression ratio for each region of the volume data based on at least one of a plurality of color values ​​and a plurality of opacity values ​​of each region; Dividing the volume data into a number of blocks, each block having a number of pixels or voxels; determining the compression ratio in dependence on at least one of a plurality of color values ​​and a plurality of opacity values ​​of each block; Using said function, determining an index value for each data value that is dependent on the color value; storing a plurality of data values ​​and a plurality of index values ​​in the volume data; determining said compression ratio in dependence on said index values ​​corresponding to said data values ​​within each region; A medical image processing method comprising:

Images