System and method for image transformation based on API calls

Abstract

To provide a method and a system for processing an image and transform the same into a high resolution and high-definition image using a computationally efficient image transformation procedure.SOLUTION: In a computing system 102 for image processing, transformation of an image includes: receiving an intensity image 101; generating an application programming interface (API) call for transforming the received intensity image; transmitting the generated API call to an image processing server 402 for transforming the intensity image into a layered distance field (DF) image; and receiving a response from the image processing server, the response comprising one or more functions for obtaining the layered DF image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates generally to image processing, and more particularly to methods and systems for rendering images using a distance field procedure (DFP). [Background technology]

[0002] The fields of image processing and computer graphics have become all-pervasive with billions of uses in daily life. From navigation to shopping, from trains to automobiles, from entertainment / recreation systems to medical equipment, image processing has a wide range of uses and applications. Particularly in the field of navigation technology, there is a current need for high-quality and high-definition map images to show higher resolution, clarity, and finer details in map images, even after scaling, for efficient and safe navigation. Despite such numerous uses, there are still some deficiencies in image representation. While traditional approaches have attempted to address this issue, a more suitable approach is needed.

[0003] Another problem with processing images is that while images offer rich texture and detail, they also require significant storage space. Also, when images are transformed, such as scaled and rotated, they exhibit various visual artifacts, such as pixelation, excessive blurring, moiré patterns, and jagged edges. Furthermore, images do not provide an adequate level of abstraction for conveying various forms of information or when attempting to edit specific elements.

[0004] To overcome this problem, one solution for reducing image storage requirements is to represent images in the Scalable Vector Graphics (SVG) format. While the SVG format is compact and can be scaled to any size without loss of quality, it lacks the texture and richness of images. Therefore, there is a need for an image representation solution that can provide the pixel richness of full-intensity images and the scalability and size of the SVG format. Thus, even after the SVG format becomes available, there is a need for a superior image representation and processing solution that can provide the pixel richness and size of the SVG format with artifact-free transformations (i.e., resolution independence).

[0005] Another requirement in image processing is to provide high-quality images that provide as much image detail as possible without consuming excessive processing power and storage space. Some existing graphics processing applications are based on calculating distance fields for two-dimensional (2D) images / shapes. The distance field of an object represents the distance from any point in space to the boundary of the object. In many applications, the distance chosen is often the shortest distance from the point to the boundary of the object. The 2D distance field of a shape represents the shortest signed distance from any point P in space to the boundary (i.e., contour) of the shape. The signed distance can be used to distinguish between the interior and exterior of an object (e.g., a negative distance indicates the point is outside the object, and a positive distance indicates the point is inside the object). The criterion used to measure distance can take many forms, but Euclidean distance is often preferred due to its usefulness in several applications, such as collision detection and rendering. 2D distance fields are used to represent many details of a 2D shape, such as the exterior of the shape, the interior of the shape, the contour of the shape, and some offset surfaces.

[0006] Such distance field-based representations of 2D shapes and objects have several advantages over more traditional geometric methods for representing objects and have been used effectively in many fields, including computer-aided design, medical imaging, surgical simulation, font rendering, games, filmmaking, deformation modeling, fluid simulation, and robotics.

[0007] The distance field is smooth (C 0 Because distance fields are continuous (continuous) and vary across space, they also offer efficiency and quality benefits. In terms of computational efficiency, distance fields are computed with simple and fast Boolean operations, including but not limited to sum, difference, and intersection. In addition, distance fields offer fast and simple offset operations, easier blending computations, smooth reconstruction of edge shapes, easier collision detection, and fast geometric queries. However, distance fields are represented using analytical representations, and their interpretation can sometimes be complex for complex shapes.

[0008] Another representation related to 2D distance fields is the distance map, which is a regularly sampled map of the distance field for various points in a plane representing the 2D shape. Distance maps are obtained by sampling the shape at a very high rate to capture all shape details such as corners and Voronoi region boundaries.

[0009] According to yet another expression, an adaptively sampled distance field (ADF) can be used to perform detail-oriented sampling of the distance field at a shape and then reconstruct the distance field from these adaptively sampled points. Such ADFs may include bilinear cell ADF representations, centroid cell ADF representations, quartic cell ADF representations, etc. Along with rendering an image using ADFs, anti-aliasing is used to estimate the intensity at every pixel of the shape. However, until now, the performance of ADFs has been limited to rendering primitives, such as rendering fonts and glyphs. Procedural ADFs are primarily based on distance field techniques, which are used to convert a set of curves representing a shape, such as the characters of a font, into an equivalent set of distance fields, which helps provide improved performance and quality in processing tasks related to such sets of curves and / or fonts, including collision detection, selection, animation, blending, tuning, and editing. Using ADF in this way allows shapes and fonts to be rendered properly, but it is not known to use them to render high definition images.

[0010] In some approaches, distance-based anti-aliasing is used to render shapes or fonts. Distance-based anti-aliasing provides better performance than traditional anti-aliasing because the distance field varies smoothly in this case. Also, because the distance field of moving shapes such as fonts or glyphs does not change significantly from frame to frame, distance-field-based anti-aliasing provides superior quality in frame-to-frame rendering, even for moving fonts or glyphs. Another feature of distance-based anti-aliasing is that it provides continuous stroke modulation (CSM) by offering a continuous range of stroke width and edge sharpness settings. However, this approach also has a flaw: implementing CSM becomes excessively complex for complex topological changes in the underlying shapes.

[0011] Therefore, there is a need for an efficient system for rendering high intensity and detailed images with higher processing efficiency and lower storage requirements that can overcome the shortcomings of the above approaches. Summary of the Invention [Problem to be solved by the invention]

[0012] Numerous approaches are known for processing 2D shapes and fonts such as those described above. However, processing high-intensity, high-resolution images is often computationally expensive and memory-intensive. One of these approaches is distance fields, a mature graphics representation technique. Distance field techniques use metrics such as Euclidean distance to measure distance because of their usefulness in many applications, such as collision detection and rendering. Distance fields have several advantages over more traditional geometric methods for representing objects and have been used effectively in many fields, including computer-aided design, medical imaging, surgical simulation, font rendering, games, filmmaking, deformation modeling, fluid simulation, and robotics. Distance fields are an example of an implicit function with a long history of use and research.

[0013] Various embodiments disclosed herein provide various forms of distance field representations, including detail-oriented distance fields, periodically sampled distance fields, procedural distance fields, analytical distance fields, memory-stored distances, etc. Thus, distance fields can also be derived from geometric primitives such as strokes, filled regions, textured regions, etc. [Means for solving the problem]

[0014] Some embodiments are based on the recognition that objects can be visualized using a distance field rendering pipeline. A distance field rendering pipeline can be defined as: given a known geometry of an object, calculating a distance field representing that geometry and then mapping the calculated distance field to pixels in an image. Indeed, since different pixels in an image have different distances to the object's surface, their intensity can be estimated as a function of the distance value. In other words, distance values ​​can be mapped to intensity values. One simple example of such a mapping is to brighten pixels at locations with negative distance values ​​and darken pixels at locations with positive distance values. As mentioned earlier, negative distances correspond to pixels inside the object and positive distances correspond to pixels outside the object; thus, using such a mapping, a bright representation of the object's shape can be visualized on a dark background.

[0015] Some embodiments are based on the recognition that a distance field rendering pipeline can be used to visualize not only complete geometric shapes, but also geometric primitives that form various types of shapes. For example, geometric primitives such as curves are first converted to a distance field and then mapped to pixels and their corresponding intensity values ​​for rendering and viewing on a display. In addition to estimating the intensity, the mapping can be used to perform various functions such as anti-aliasing, colorization, etc. Additionally or alternatively, the geometric primitives converted to a distance field can be queried to perform various functions such as collision detection, selection, path planning, etc. According to some embodiments, the results of these operations can also be rendered on a display.

[0016] Some embodiments are based on the recognition that the distance field rendering pipeline can be inverted, and the distance field can be calculated from an intensity image, i.e., inverse to the current rendering pipeline. For example, some embodiments disclose methods and systems for representing the intensity of a richly textured image as a set of distance fields. To do so, such inverse rendering requires finding a virtual shape with a distance field such that visualization of the distance field values ​​produces the original intensity image. It is therefore an objective of some embodiments to provide methods and systems for implementing such an inverse rendering pipeline that uses distance fields to take advantage of the distance fields and use them in processing high-intensity images.

[0017] Additionally, some embodiments disclose methods and systems that can provide richness of intensity pixels at the size of SVG format with artifact-free conversion (i.e., resolution independence), which can be used for a variety of applications such as surgical planning, data compression, texture mapping, and surveillance.

[0018] Various embodiments disclosed herein provide an inverse rendering pipeline based on intensity image displacement using a distance field (DF)-based solution that solves many technical challenges, including, but not limited to, eliminating visual artifacts present in images during viewing transformation (e.g., pixelation, moiré patterns, excessive blur, jaggies), reducing storage requirements, reducing image-specific memory and processing requirements by being adaptable to content complexity, and providing fast, high-quality, progressive browsing over slow networks. Thus, unlike the prior art, various embodiments disclosed herein provide superior compression and the ability to directly manipulate (e.g., directly render, directly query) without performing a decompression step.

[0019] Therefore, an objective of some embodiments is to provide a system and method for converting image intensities into a distance field representation. Additionally or alternatively, an objective of some embodiments is to provide a structure for a distance field representation of image intensities that, when reconstructed, renders a distance field that represents the original image with a desired accuracy. In computer science and its numerous applications, changing the representation of an object from one form to another often enables significant improvements in many technical challenges related to the object. This technique is referred to herein as the "transform-and-conquer" technique. While this transform-and-conquer technique has been applied to geometry and distance fields in various fields, such as font rendering, it has not yet been applied to intensity images and distance fields. One reason for this deficiency is the complexity of such a transformation.

[0020] Some embodiments are based on the recognition that it may be advantageous to provide an optimization procedure that searches for the best DF representation of the image intensities while optimizing several optimization parameters so that the overall complexity of the transformation can be reduced.

[0021] Thus, an example of an optimization parameter may be minimization of the visualization error, e.g., the distance between the original image and the image reconstructed from the distance field. However, given the complexity of the intensity-to-distance field transformation, such optimization is challenging and difficult to converge. To address this issue, some embodiments are based on the way the actual painting process is performed. For example, when viewing any intensity image, the final intensity and / or color each pixel has is independent of how the artist arrived at this intensity. However, during the painting process, rather than immediately painting a point at its final intensity, the artist may add layers upon layers of paint with different intensities / colors, so that the combination of the ordered paint layers results in the final intensity / color at each location in the paint. As a result, each rich intensity image can be viewed not as a two-dimensional (2D) image of intensity, but as a 3D image of layers of intensity, i.e., layers of 2D images, which combine to form the desired image.

[0022] Therefore, various embodiments provide methods and systems for converting an intensity image of arbitrary complexity into a layered distance field image by applying a transform-and-conquer technique to the intensity image. Each layer of the layered distance field image contains a distance field procedure that defines distance field values ​​at every location in the intensity image and rules for mapping these distance field values ​​to intensity values. In addition, the layered distance field image contains information about the order in which to combine different layers so that an intensity image reconstructed from the layers of the layered distance field image approximates the original intensity image.

[0023] Some embodiments provide methods, systems, and devices for image processing. The methods, systems, and devices are based on converting intensity images, such as highly detailed, high-definition, and richly textured images, into layered DF images. The layered DF images are obtained by performing a layer-by-layer DF transformation of the intensity image to simplify processing and reduce overall system memory and computational requirements while achieving a desired level of performance. The layers are obtained in an ordered sequence, with each layer in the sequence associated with a DF procedure for specifying DF values ​​at multiple locations in the intensity image and a set of rules for mapping the DF values ​​to the intensity values ​​of the respective layer. The layer-by-layer transformation performed in this manner results in a layered DF image, which can then be rendered to obtain a superior-quality intensity image. The layer-by-layer transformation achieves the goal of reversing the rendering pipeline and providing a high-quality image with rich details and textures as output.

[0024] Some embodiments are based on the recognition that each of a plurality of locations in the intensity image can be represented by a respective candidate region of a plurality of candidate regions in the intensity image.

[0025] Some embodiments provide an intensity reconstruction function for combining the mapped intensities of each layer according to their order in the sequence of layers to reconstruct an intensity image.

[0026] Some embodiments provide for determining an error value associated with the difference between the intensity of the original or received intensity image and the intensity image reconstructed from the layered DF image. The error value is then compared to an error threshold. This comparison is then used to continue the layered transformation or to stop the transformation process and update the reconstructed image accordingly.

[0027] Various embodiments disclose a Backus-Naur Form (BNF) grammar that describes operations for each layer in the layered DF transform.

[0028] Some embodiments are based on the recognition that the layered DF transform can be used to recreate the asymmetric strokes of a painting process by defining an asymmetric stroke procedure. By doing so, we ensure optimality of the image transformation process, thereby making the overall image transformation process computationally efficient and convergent. Various embodiments provide resolution-independent intensity image reconstruction.

[0029] Various embodiments provide distance field operations including one or more of a distance map operation, an adaptive distance field computation operation, an analytical distance field computation operation, a procedural distance field computation operation, an in-memory distance-distance field computation operation, a stroke distance field computation operation, an in-domain distance field computation operation, and a unary operator distance field computation operation.

[0030] Some embodiments are based on the recognition that a DF procedure may include an asymmetric stroke procedure associated with a spline curve, the spline curve being associated with a corresponding distance field, and the asymmetric stroke defining rules for mapping the distance field of the spline curve to different gradients of intensity change on different sides of the spline curve's central axis, such that the intensity of the spline curve varies in a direction perpendicular to its central axis.

[0031] Various embodiments provide a DF procedure that includes a DF visualized with masked graded intensities, where the masked graded intensities include null intensity values ​​at specific locations in the layered DF image.

[0032] Some embodiments are further based on the recognition that different layers of the layered DF image represent elements corresponding to different resolutions of the received intensity image.

[0033] Some embodiments provide an image processing system including an input interface configured to receive an intensity image. The image processing system further includes at least one processor configured to execute computer-executable instructions for generating application programming interface (API) calls to transform the received intensity image. The API calls are sent to an image processing server to transform the intensity image into a layered distance field (DF) image, the layered DF image including an ordered sequence of layers. Each layer in the ordered sequence includes a DF procedure for determining DF values ​​at multiple locations of the received intensity image and a set of rules for mapping the DF values ​​to intensity values ​​for each layer. The image processing system further includes instructions configured to receive a response from the image processing server, the response including one or more functions for obtaining the layered DF image, such that the layered DF image is obtained by transforming the intensity image. The layered DF image is then rendered on an output interface.

[0034] Some embodiments provide one or more functions that include reconstructing the transformed received intensity image by combining the mapped intensities of each layer according to their order in the sequence of layers.

[0035] Various embodiments further provide for determining an error value associated with the difference between the intensities of the received intensity image and an intensity image reconstructed from the layered DF image, and the error value is compared to an error threshold, and the reconstructed intensity image is updated based on the comparison.

[0036] Some embodiments provide an API call that includes a transformation instruction, where the transformation instruction is for transforming the layered DF image based on the transformation instruction.

[0037] Some embodiments provide API calls that include pen type data for specifying asymmetric stroke procedures, including at least a pinstroke pen type option, a pressure-sensitive pen type option, a customizable pen type option, a scalable pen type option, and a textured pen type option.

[0038] Some embodiments provide a method for image processing based on API calls. The method includes receiving an intensity image and generating an API call to transform the received intensity image. The API call is then sent to an image processing server to transform the intensity image into a layered distance field (DF) image, the layered DF image including an ordered sequence of layers, each layer in the ordered sequence including a DF procedure for determining DF values ​​at multiple locations of the received intensity image and a set of rules for mapping the DF values ​​to intensity values ​​for each layer. The method further includes receiving a response from the image processing server, the response including the layered DF image such that the layered DF image is obtained by transforming the intensity image, and the method further includes rendering the layered DF image.

[0039] Thus, various embodiments disclosed herein provide efficient, resolution-independent, and adaptive techniques for processing and transforming images to provide richly textured, high-definition, highly detailed, and low-memory-intensive image rendering based on DF technology. Various embodiments also provide a more intuitive, paint-like experience for rendering high-quality images, capable of resolution-based rendering with infinite zoom. [Brief explanation of the drawings]

[0040] [Figure 1A] FIG. 1 shows a block diagram illustrating an image processing system according to some embodiments of the present disclosure. [Figure 1B]FIG. 1B shows a schematic diagram illustrating the image processing system of FIG. 1A configured for layered DF image transformation, according to some embodiments of the present disclosure. [Figure 1C] 1A-1C show schematic diagrams illustrating the change in distance field and concentration profile of a shape, according to some embodiments of the present disclosure. [Figure 1D-1] FIG. 1 shows a schematic diagram illustrating various operations performed on distance fields to combine them, according to some embodiments of the present disclosure. [Figure 1D-2] FIG. 1 shows a schematic diagram illustrating various operations performed on distance fields to combine them, according to some embodiments of the present disclosure. [Figure 1E] FIG. 1 shows a flow diagram of a method for image processing based on a layered DF transform of intensity images, according to some embodiments of the present disclosure. [Figure 1F] FIG. 10 shows another flow diagram of a method for image processing based on refining the layered DF transform of an intensity image, according to some embodiments of the present disclosure. [Figure 2A] 1 shows a block diagram illustrating a possible implementation of a layered DF transform of an intensity image based on the concept of inverting the rendering pipeline, according to some known solutions. [Figure 2B] 1 shows a block diagram illustrating a possible implementation of a layered DF transform of an intensity image based on the concept of inverting the rendering pipeline, according to some known solutions. [Figure 2C] 1 illustrates a flow diagram of a method for image processing based on iterative optimization of error values, according to some embodiments of the present disclosure. [Figure 2D] 10 shows another flow diagram of a method for level of detail based image processing according to some embodiments of the present disclosure. [Figure 2E] 2C and 2D, according to some embodiments of the present disclosure. [Figure 3A] 1 shows a high-level block diagram and example images illustrating converting an intensity image into a distance field-based image, according to some embodiments of the present disclosure. [Figure 3B] FIG. 1 illustrates a high-level flow diagram of a method for layered distance field-based image transformation based on transformation instructions, according to some embodiments of the present disclosure. [Figure 3C] FIG. 1 shows a flow diagram of a detailed method for layered distance field-based image transformation based on transformation instructions, according to some embodiments of the present disclosure. [Figure 4] FIG. 1 illustrates a block diagram of an architecture of a computing system for image processing according to some embodiments of the present disclosure. [Figure 5] 1 illustrates a flow diagram of a method for determining a resolution-independent transform of an image according to some embodiments of the present disclosure. [Figure 6] FIG. 1 illustrates a flow diagram of a method for transforming an image using a brushstroke-based procedure according to some embodiments of the present disclosure. [Figure 7] 1 illustrates a flow diagram of a method for transforming an image using a set of primitives according to some embodiments of the present disclosure. [Figure 8] FIG. 1 illustrates a block diagram of a computing system that may be used to implement various embodiments for converting an image into a layered DF image as disclosed herein, according to some embodiments of the present disclosure. [Figure 9] 1 illustrates a method for curve fitting using an image processing system according to some embodiments of the present disclosure. [Figure 10] 1 illustrates a computer-implemented method for performing successive refinement operations on a set of data points for curve fitting, according to some embodiments of the present disclosure. [Figure 11] 1 illustrates a method for incrementally fitting a set of curves to a series of data points according to some embodiments of the present disclosure. [Figure 12] 1 illustrates a method for rendering an ordered set of primitives using a depth buffer according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0041] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown in block diagram form solely to avoid obscuring the present disclosure.

[0042] As used in this specification and claims, the terms "for example," "for instance," and "such as," as well as "comprising," "having," "including," and other forms of these verbs, when used in conjunction with a list of one or more components or other items, should be construed as open-ended, meaning that the list should not be considered to exclude further components or items. The term "based on" means based at least in part on. Furthermore, it should be understood that the style and terminology used herein are for purposes of description and should not be considered to be limiting. Any headings used herein are for convenience only and have no legal or limiting effect.

[0043] Various embodiments disclosed herein provide a novel approach to digital drawing that enables the creation of detailed textured artwork with artifact-free scalability and a small memory footprint.

[0044] Some embodiments disclosed herein provide methods, systems, and computer program products for image processing. The image processing disclosed in various embodiments provides several advantages over existing solutions known in the art. These various advantages include the artifact-free scalability of the SVG image format and the ability to create graphic elements that exhibit pixel richness along with compact size. The image processing methods and systems disclosed herein provide users, such as artists, developers, media editors, engineers, animators, and the like, with the realization that they can work on an infinite canvas in both spatial (x and y) extent and scale (z), providing seamless infinite zoom and infinite detail capabilities that are unavailable from any other system known in the existing art.

[0045] Additionally, various embodiments provide an efficient and effective solution by reducing memory storage requirements for image processing for memory-constrained and bandwidth-limited environments. Another advantage of the methods and systems disclosed herein is that they enable high image adjustability by providing distance-based anti-aliased rendering capabilities during image processing. Furthermore, the capabilities disclosed in various embodiments provided herein also include support for rich primitives such as variable-width textured strokes with complex sub-pixel characteristics, and support real-time, fast rendering of images by enabling immediate feedback during drawing and interactive canvas transformations.

[0046] Some embodiments also provide methods and systems for enabling stylization of different looks and levels of abstraction in images obtained by distance field-based reconstruction and rendering, which can also be easily integrated with distance field-based font rendering. Thus, various embodiments can provide intuitive content creation by unfolding images layer by layer, just like a real-world painting experience, providing users, such as content creators, with a superior drawing-like experience.

[0047] Some embodiments are based on the recognition that distance field-based rendering can provide an unprecedented capability to represent arbitrary dimensional geometry, consisting of both smooth and sharp features, in a purely procedural, resolution-independent manner, through the use of simple and highly efficient Boolean, blending, offsetting, and arithmetic operations inherent to distance fields, and do so with less memory, higher accuracy, higher quality, and less computational load than other techniques known in the art. Thus, the modeled geometry is realistic and can be derived or simulated.

[0048] The methods and systems disclosed herein are based on distance field technology and its application in a layered manner and can be used in multiple applications including, but not limited to, medical applications, geographic information systems, gaming systems, entertainment systems, maps and navigation, compression technologies, UI design, video encoding, animation creation, video editing, etc.

[0049] 1A shows a schematic diagram 100 illustrating an image processing system 102, according to some embodiments of the present disclosure. The image processing system 102 is configured to receive an intensity image 101 and convert the received intensity image 101 into a reconstructed intensity image 103.

[0050] The intensity image 101 can be any image with highly detailed and precise information about various channels, raw images, geometric objects, fonts, etc. For example, the intensity image 101 can be a representation including a set of spatially coherent pixels, a distribution of color amplitudes like a JPEG or PNG image, a set of colors, and corresponding locations, etc. The pixel representation can be procedural in nature, discrete in nature, or continuous in nature, with a spatially related or temporally related set of values ​​associated with the image. Additionally, the intensity image 101 is associated with a sampling mechanism or process capable of generating the associated set of values, which can be 2D or 3D.

[0051] Thus, the intensity image 101 may be an image used in any of the applications such as video editing, UI design, animation, maps used in navigation, entertainment related applications, digital TV, images used in transportation systems such as trains, images used in building services such as elevators, or any other application having an associated display screen for viewing the intensity image 101 or the reconstructed intensity image 103.

[0052] For the transformation of the intensity image 101, the image processing system 102 uses a layered DF transformation to convert the received intensity image 101 layer by layer into a layered DF image, which is then used to reconstruct an image for rendering. This image for rendering is a reconstructed intensity image 103, which may then be rendered on an output interface.

[0053] The reconstructed intensity image 103 has more desirable definition and visual quality than the received intensity image 101, and is also compressed in size, resulting in an overall improved quality and more desirable storage characteristics of the received intensity image 101. This is achieved by the performance improvements provided by the layered DF transform performed by the image processing system 102. Details of the image processing system 102 are shown in FIG. 1B.

[0054] 1B shows a detailed schematic diagram illustrating the image processing system 102 of FIG. 1A configured for a layered DF transform, according to some embodiments of the present disclosure. The image processing system 102 may be implemented as a computing system. The computing system includes an input interface 107 configured to receive an intensity image 101. The input interface 107 may be configured to receive input from any input device and / or technique known in the art, including, but not limited to, keyboard input, touch-based input, mouse-based input, input submission by uploading an intensity image from a second computing system, uploading the input image 101 from the same computing system, etc.

[0055] The received intensity image 101 is then sent to at least one processor, such as the processor 109 shown in FIG. 1B. The processor 109 is configured to execute computer-executable instructions, which may be stored in a memory or a computer-readable storage medium. The computer-executable instructions, in turn, may be configured to perform various operations to transform the intensity image 101 into a reconstructed intensity image 103 based on layered DF calculations performed by a combination of a DF procedure (DFP) module 104 and a rule module 105. As shown in FIG. 1B, the operations of the DFP module 104 and the rule module 105 can transform the intensity image 101 into a layered DF image 106. Each layer in the layered DF image 106 is an ordered layer within an ordered sequence of multiple layers. The ordered sequence of multiple layers allows for incremental refinement of the intensity image 101 by dividing the overall computational task of image transformation into smaller, manageable portions of image processing, resulting in a stage-by-stage or layer-by-layer compression of the layered DF image 106. This makes the entire image processing task computationally feasible, less complex, and more memory efficient, while at the same time rendering the image at a higher level of detail by successively increasing the level of detail at each layered transformation step.

[0056] Each layer in the ordered sequence has associated with it a DFP 104a provided by the DFP module 104. The DFP 104a allows the image to be represented by defining DF values ​​at multiple locations in the received intensity image 101. A set of rules provided by the rules module 105 then maps the DF values ​​to corresponding intensity values ​​for each layer.

[0057] Thus, each DFP 104a and its associated rules 105 are configured to directly model and approximate the luminance and chrominance channels of an image, such as a layered image in a sequence of layers. First converting the original channels of an image, such as RGB and CMYK, to other formats and color models, such as LUV and LAB, allows for faster, simpler, and better convergence when matching a set of DFPs to an image. In one embodiment of the present invention, the image is 2D, and the image is viewed as a raw 2D distance field, determining the inverse of a traditional distance field rendering pipeline (i.e., given a known 1D geometry, computing a 2D distance field, and mapping the 2D distance field to pixels) (i.e., given a 2D distance field, discovering a generation of an unknown 1D geometry, which is combined to match the 2D distance field). This provides a significant advantage over other methods in terms of computational complexity by reducing the dimensionality of the problem from two to one when the image is two-dimensional. One-dimensional problems can be solved with one-dimensional methods, which are faster, simpler, and have better convergence properties. Such convergence is achieved layer-by-layer for each layered DF image obtained at a layer.

[0058] The layer-by-layer transformation in this manner is performed until a stopping criterion for the transformation is met, which may be determined by several factors, including but not limited to, level of detail, error value minimization, target compression value, etc.

[0059] Once the layered transformation is completed in this manner, the layered DF image 106 resulting from the final stage of the transformation is sent to an output interface 108. The output interface 108 may be configured to render the layered DF image 108. This rendered DF image 106 may be viewed as a reconstructed intensity image 103 using display or viewing technology associated with the output interface 108. For example, the output interface 108 may provide a display of the reconstructed intensity image 103 in a map-based application, an entertainment application, a transportation system display, a factory automation environment, etc. In many applications, it is advantageous, in part for the reasons described above, to use distance field calculations to convert the received intensity image 101 into a layered DF image 106, which is then rendered as a reconstructed intensity image 103. Accordingly, this disclosure makes extensive use of DF calculations for image processing. Considerations and discussion related to DF calculations will now be discussed in connection with FIG. 1C .

[0060] FIG. 1C shows a schematic diagram illustrating DF calculations and intensity profile changes for a shape, according to some embodiments of the present disclosure. FIG. 1C shows an object 110, such as a shape or font, and its corresponding DF representation 111. For illustrative purposes, FIG. 1C shows a 2D object. However, the concepts described herein are applicable to 3D objects without departing from the scope of the present disclosure. FIG. 1C also shows intensity profile changes for the object 110 and its DF representation 111. Intensity profile 112 represents the intensity profile of object 110 along an axis defined along line AA′, and intensity profile 113 represents the intensity profile of the DF representation of object 110 along an axis defined by line BB′.

[0061] As is known, the distance field of an object describes, for any point in space, the distance from this point to the boundary of this object (such as object 110). In many applications, the distance chosen is often the shortest distance from the point to the boundary of the object. Signed distances can be used to distinguish between the interior and exterior of the object (e.g., a negative distance indicates that the point is outside the object, and a positive distance indicates that the point is inside the object).

[0062] Alternatively or additionally, the distance from a point to the boundary of the object is replaced by a vector that originates at this point and is traced to the nearest location on the boundary of the object. The criterion used to measure distance can take many forms, such as the Euclidean distance form.

[0063] As such, various methods can be used to represent an object 110 and its DF 111, including detail-oriented distance fields, periodically sampled distance fields, procedural distance fields, analytical distance fields, distances stored in memory, distances derived from geometric primitives such as strokes, field regions, textured regions, etc. A procedural distance field based DF representation 111 uses a DFP (such as DFP 104) that includes a set of distance fields and a set of operations that operate on the set of distance fields.

[0064] Thus, each DFP 104 implemented by the DFP module distance field procedure directly models and approximates the luminance and chrominance channels of an image, to name just one example. First converting the original channels of an image, such as RGB and CMYK, to other formats and color models, such as LUV and LAB, allows for faster, easier, and better convergence when matching a set of distance field procedures to an image. In one embodiment of the present invention, the image is 2D, and the image is viewed as a raw 2D distance field, determining the inverse of a traditional distance field rendering pipeline (i.e., given a known 1D geometry, computing a 2D distance field, and mapping the 2D distance field to pixels) (i.e., given a 2D distance field, discovering a generation of an unknown 1D geometry, which is combined to match the 2D distance field). This provides a significant advantage over other methods by reducing the dimensionality of the problem from two to one when the image is two-dimensional. One-dimensional problems can be solved with one-dimensional methods that are faster, simpler, and have better convergence properties.

[0065] 1D-1 and 1D-2 show schematic diagrams illustrating various operations performed on distance fields for distance field combination, according to some embodiments of the present disclosure.

[0066] 1D-1 and 1D-2, various operations in the set of operations involve Boolean combinations of distance fields. For example, one operation may be a distance field blend 114, another may be a distance field sum operation 115, another may be a difference operation 116, and yet another may be an intersection operation 117. All of these operations may be very simple calculations using distance fields.

[0067] For example, distance field blend 114 shows the blending of two different fonts, the "a" in Palatine and the "W" in Times New Roman. This would be difficult to calculate using other image processing techniques, but is easy with distance fields. Similarly, other operations are very easy. For example, the sum operation is: Sum operation: A∪B => distance (A∪B)=max(distance (A), distance (B)) 115, where the difference operation is Difference operation: AB => distance(AB) = min(distance(A), - distance(B)) 116, where the multiplication operation is Multiplication: A∩B => Distance(A∩B) = min(Distance(A), Distance(B)) It may be 117 given as:

[0068] Similarly, many other operations can be defined, including, but not limited to, distance field offset operations, distance field arithmetic combinations, distance field unary operations, distance field conditional operations, distance field logical operations, distance field masking operations, etc. A distance field masking operation defines a region of an image that prevents changes to the region when performing operations on the image within the region, the region being defined by a set of distance field procedures, which are resolution independent. Masks can be binary or continuous.

[0069] The operations in the set of operations are not limited to distance field operations but also include image processing operations, computer graphics operations, and signal processing operations. Distance fields offer distinct computational advantages when performing various operations such as blending, offsetting, and rendering to derive composite shapes from Boolean operations. Further distance field procedures, such as DFP 104, can take both positive and negative values ​​when evaluated, enabling a wide range of applications, such as modeling brushes, pens, and erasers in digital drawing systems. Additionally, Boolean combinations of DFs can be used to capture sharp, discontinuous features in an image, arithmetic (e.g., linear) combinations of DFs can be used to capture smooth regions in an image, implicit blending can be used to smoothly combine DFs, and offsets can enable easy feature fitting and sizing of candidate DFs. Furthermore, DF primitives can be adjusted using a Correspondences by Sensitivity to Movement (CSM) algorithm. Furthermore, procedural masking may allow for more efficient representation of images, and several procedural DF-based templates can be provided for texture modeling. Another advantage of using DF procedures is that they allow for massive parallelization of operations.

[0070] Additionally, some embodiments provide various operations for DF computation defined by a Backus-Naur Form (BNF) grammar 118. As such, each layered DF image is associated with a BNF grammar 118, and the BNF grammar 118 for each structure of the layered DF image includes a number of operations including at least one of a distance field operation, a unary operator operation, a composite distance field computation operation, a join operation, a DF image reconstruction operation, a DF image generation operation, a blend operation, a Porter-Duff compositing blend operation, etc.

[0071] The distance field operations include at least one of a distance map operation, an adaptive distance field computation operation, an analytical distance field computation operation, a procedural distance field computation operation, an in-memory distance-distance field computation operation, a stroke distance field computation operation, an in-domain distance field computation operation, and a unary operator distance field computation operation.

[0072] The unary operation includes at least one of a CSM operation, an offset operation, an inset operation, and a probability map calculation operation.

[0073] The composite distance field computation operation includes at least one of a distance field computation operation and a combination operation, where the combination operation includes at least one of a Boolean operation, an implicit blending operation, a linear combination operation, and an arithmetic combination operation.

[0074] The DF image reconstruction operation includes at least one of a DF map-intensity operation and a masked DF map-intensity operation, and the DF image generation operation includes at least one of a reconstruction DF operation and a blending operation.

[0075] The blending operation includes at least one of an addition operation, a subtraction operation, a substitution operation, a darkening operation, a brightening operation, and a Porter-Duff blending operation. The Porter-Duff blending operation includes at least one of a src operation, an over operation, and a dest operation. By using an appropriate combination of the various operations, a layered DF image of the corresponding layer can be realized. Then, by using successive refinements of the layered DF image until a stopping condition is met, a higher quality, finer definition, and better resolution intensity image is reconstructed.

[0076] FIG. 1E shows a flow diagram of a method 119 for image processing based on a layered DF transform of an intensity image 101 according to some embodiments of the present disclosure.

[0077] The method 119 includes receiving an intensity image 101 in step 120. The intensity image 101 may be received at an interface 107 of an image processing system 102 from an image processing related application.

[0078] Next, in step 121, the received intensity image is converted into a layered DF image. The layered DF image includes an ordered sequence of layers, each layer having a better level of refinement than the previous layer. Furthermore, as previously disclosed, each layer in the ordered sequence is associated with a DF procedure, such as DFP 104a, for determining DF values ​​at multiple locations in the received intensity image 101 and a set of rules, such as the rules provided in rule module 105, for mapping the DF values ​​to intensity values ​​for each layer. Thus, the ordered sequence of multiple layers in the layered DF image may be associated with iterative DFPs. Each layer may be associated with a different DFP, which may be selected from at least one of a parameterized DF procedure, an analytical DF procedure, a sampled DF procedure, and an asymmetric stroke procedure associated with a spline curve.

[0079] The asymmetric stroke procedure includes asymmetric strokes that define rules for mapping the distance field of a spline curve to different gradients of intensity change on different sides of the spline curve's central axis, such that the spline curve's intensity varies in a direction perpendicular to its central axis. These intensities of the spline curve can vary gradually and are different on different sides of the spline curve's central axis. In some embodiments, the spline curve may be shifted relative to the central axis. The use of asymmetric stroke procedures in layered transformations enables more intuitive and computationally convergent solutions to image processing and transformation problems, similar to real-world painting processes.

[0080] Another type of computationally efficient DFP may include a DF visualized with masked intensity gradients, which include null intensity values ​​at specific locations in the layered DF image. The null intensity values ​​result in a layer of the layered DF image not modifying the intensity of the previous layer at the corresponding location, thus resulting in fewer computations being performed for the entire DFP run.

[0081] Some embodiments are based on the recognition that different layers of the layered DF image 106 represent elements corresponding to different resolutions of the received intensity image. Thus, the layered DF transform implemented by method 119 (and by image processing system 102) provides a resolution-independent method for performing image transformation, since each subset of DFPs selected from the beginning of the ordered sequence of DFPs reconstructs the received intensity image 101 at a different resolution, but with the same bounds on the reconstruction error, which will be discussed below in connection with Figures 2C and 2D.

[0082] Additionally, for each layered DF image, a multi-scale normalization may be performed that iteratively estimates each layer of the layered DF image by varying the optimization parameters in different iterations.

[0083] Such successive iterations for transforming DF images between different layers are performed until a stopping condition is met, which will be explained below in connection with Figures 2C and 2D.

[0084] Finally, in step 122, the last layered DF image of the layer where the continuous transformation stops is used as the output layered DF image for rendering on the output interface. This image is then rendered as the reconstructed intensity image 103. As such, the layered DF image includes an intensity reconstruction function for combining the mapped intensities of each layer according to their order in the sequence of layers to reconstruct the received intensity image.

[0085] Another variation of method 119 is shown in FIG. 1F. 1F illustrates another flow diagram of a method 123 for image processing based on a layered DF transform of an intensity image 101, according to some embodiments of the present disclosure. Hereinafter, for the sake of explanation, the intensity image 101 may be referred to as IMG 101 in some flow diagrams, but these are synonymous.

[0086] The method 123 includes, in step 126, assigning DFPs 104a to an initial configuration of the image IMG 101. The initial configuration may be an empty set, and the assignment uses the image IMG 101 to determine an initial configuration of a set of DFPs 104a, the initial configuration being a set of DFPs 104a determined by various means for establishing a starting point for refinement, including interpolation, regression, optimization, human editing, an AI system, search methods (random search, grid search, cell search, divide and conquer search, trial and error search, etc.), and combinations thereof. The starting point includes auxiliary data, such as a compressed version of the image, a low-resolution version of the image, an approximation of the image, an interpolation of the image, etc., stored with the set of DFPs 104a to enable reconstruction of the image IMG 101.

[0087] In some embodiments, the initial configuration may be determined by storing a small, low-resolution version of image IMG101 along with a set of DFPs 104a, such that reconstruction of the image begins with rendering IMG101 followed by rendering a set of DFPs 104a on top of IMG101.

[0088] In the next step 127, the DFP determined in step 125 is refined until a stopping criterion is met, at which point the method 123 is stopped and the available DFP 104a in step 127 is stored as the DFP 104a for the particular layer.

[0089] For DFP refinement, the image is decomposed into tiles, and these tiles are processed sequentially and in parallel. Furthermore, refinement works at increasing levels of detail (stepwise, continuous, or discrete steps, such as low-medium-high) for efficient computation and to obtain a progressive (coarse to fine) representation of the image. Therefore, refinement performs one-time preprocessing to generate preprocessed image data, which may include edge maps, gradient maps, Laplacian maps, zero-crossing maps, filtered versions of the image, frequency-domain versions of the image, wavelet decompositions of the image, statistics, etc. Edge, gradient, Laplacian, and zero-crossing maps may be obtained at sub-pixel resolution.

[0090] Some embodiments provide for performing a filtering operation that is applied once to the image before refinement, where the filtering operation performs sharpening, noise removal, etc. to compensate for or complement the rendering characteristics of a set of DFPs. Thus, refinement uses smaller subsampled versions of the image to improve refinement performance, and the image and the subsampled versions of the image are periodically alternated during refinement. Refinement continues in the above manner until a stopping criterion is met.

[0091] The stopping criteria may include multiple factors, including but not limited to exceeding a time limit, exceeding a maximum iteration count, reaching an error tolerance, experiencing a lack of convergence over a series of refinement steps, etc. The stopping criteria may be image independent, image dependent, may be dynamically updated (changing during a refinement step), may be static, may be controlled by visual inspection, human direction, AI, may be determined by a procedure, or may be determined by a table.

[0092] If the stopping criteria are not met, method 123 proceeds to step 128. In step 128, candidate regions (CRs) 124 are selected from IMG 101 or from the layered DF image, depending on whether this is the first or subsequent iteration of method 123. The CRs 124 are regions corresponding to respective locations in IMG 101 or any layered DF image. Thus, each of the multiple locations in IMG 101 or any layered DF image corresponds to a respective one of the multiple candidate regions in the received intensity image IMG 101 or any layered DF image. Therefore, in step 129, a new set of distance field procedures (NDFPs) 125 is identified. The NDFPs 125 are identified based on the entire set of DFPs 104a stored in the DF module 104. Finally, in step 130, the DFPs 104a and the NDFPs 125 are combined and stored as an updated set of DFPs 104a associated with the corresponding candidate regions CRs 124. The process of refining the DFP may then be repeated until a stopping condition or criterion is met. After refinement is complete, the final layered DF image is used to reconstruct an intensity image.

[0093] Some embodiments further include determining an error value associated with the difference between the intensity of the received intensity image IMG101 and the intensity of an intensity image reconstructed from the layered DF image. This error value is then compared to an error threshold, and the reconstructed image is updated based on this comparison. This is further described in connection with FIGS. 2A-2B.

[0094] 2A-2B show block diagrams illustrating a possible implementation of a layered DF transform of an intensity image based on the concept of inverting the rendering pipeline, according to some known solutions.

[0095] FIG. 2A illustrates a rendering pipeline 200. Rendering pipeline 200 represents the key elements required for rendering any intensity image, such as intensity image 203. Rendering pipeline 200 may begin with a basic unit of viewing or rendering, such as a geometric shape 201, which may correspond to any shape, object, part, location, region, or primitive within the overall image. First, a distance field 202 corresponding to various points or locations in geometric shape 201 is determined. Distance field 202 is then used to render intensity image 203 using an image reconstruction function specifically targeted at rendering an image from distance field data for various points in intensity image 203. For example, in the case of a 2D geometric shape, it has been previously disclosed that distance field 202 includes a set of signed shortest distances for various points in the 2D geometric shape space from the boundary of the corresponding 2D geometric shape. The sign and magnitude of the distances are then appropriately converted to pixel intensity values. Thus, some geometric shapes may be represented by their distance map, which comprises a map of distance values ​​sampled periodically for a shape (such as geometric shape 201).

[0096] Another representation of the distance field can be derived using an adaptively sampled distance field (ADF), which can use detail-oriented sampling of the distance field for the geometry 201. Thus, where there is significant variation in the distance field and therefore excessive detail in the geometry 201, a higher sampling rate is used. Similarly, where the distance field varies slowly or where precision is not required, such as in low-detail portions of the geometry 201, a lower sampling rate is used. The sampled distances resulting from this detail-oriented sampling are then stored in an appropriate data structure, for example, by using a hierarchical representation for data storage or by using on-demand sampling of the distance field. An intensity image 203 can then be reconstructed using a reconstruction function to reconstruct the distance field from the adaptively sampled points in the space of the geometry 201.

[0097] This rendering pipeline requires computationally expensive analysis filters for sampled distance calculations, and the large storage requirements of the continuous hierarchical data structure make the overall image processing based on rendering pipeline 200 computationally and storage-wise expensive.

[0098] Some embodiments are based on the recognition that an alternative method based on an inverse of the rendering pipeline 200 can be computationally superior in terms of storage and performance.

[0099] 2B shows a block diagram of such a method, based on an inverse rendering pipeline 204. The inverse rendering pipeline 204 starts with an original intensity image 206, and distance field data 207 for each point or location in the original intensity image 206 is determined from the original intensity image. A set of visualization rules 208 is then applied, possibly iteratively, to this distance field data 207 to obtain a reconstructed intensity image 209. The application of the visualization rules is performed iteratively based on a visualization error parameter 210, which determines a stopping condition for stopping the application of the visualization rules and rendering the reconstructed intensity image 209.

[0100] Furthermore, some embodiments are based on the recognition that the computation of the inverse rendering pipeline 204 may be more efficient if the processing tasks required for the computation of the distance field data 207 and the intermediate steps of the visualization rules 208 can be divided into smaller portions of processing tasks.

[0101] Therefore, FIG. 2B illustrates an architecture of the inverse rendering pipeline 204 for more efficient operation, in which the original intensity image 206 is transformed into a reconstructed intensity image 209 by using a layered distance field image 211, which involves a smaller portion of the processing tasks, including layer-by-layer distance field and visualization rule calculations, governed by visualization error minimization for each layer of processing. Furthermore, such layer-by-layer processing does not require a large hierarchical data structure for storage because it operates directly on the various color or intensity channels of the original intensity image 206. To that end, such layer-by-layer processing is implemented by the image processing system 102 shown in FIGS. 1A-1F, as previously described. As previously described, the image processing system 102 is configured to perform methods 119 and 123 to transform the intensity image into a layered DF image including an ordered sequence of multiple layers. Thus, different layers are associated with a set of DF procedures equivalent to DFP 104a shown in Figure 1B and a set of rules, such as rules 105 shown in Figure 1B, equivalent to visualization rules 208 shown in Figure 2A. Such layered DF image transformations are further used to implement the inverse rendering pipeline 204 shown in Figure 2A, which is subject to visualization error 210 as a stopping condition, as further explained by method 212 shown in Figure 2C.

[0102] 2C illustrates a method 212 for converting an intensity image into a layered DF image, according to one embodiment. The method 212 may be performed by the image processing system 102 shown in FIG. 1B.

[0103] The method 212 includes receiving the intensity image at step 213. As previously mentioned, the image processing system includes an input interface 107 configured to receive the intensity image 101.

[0104] The method 212 further includes converting the received intensity image 101 into a layered DF image at step 214. To that end, the received intensity image 101 is then sent to at least one processor, such as the processor 109 shown in FIG. 1B. The processor 109 is configured to execute computer-executable instructions, which may be stored in a memory or a computer-readable storage medium. The computer-executable instructions may, in turn, be configured to perform various operations to convert the received intensity image 101 into a layered distance field (DF) image including an ordered sequence of layers. Each layer in the ordered sequence, including a DF procedure, such as one of the DFPs 104a shown in FIG. 1B, defines DF values ​​at every location in the received intensity image 101 and rules, such as the visualization rule 208 shown in FIG. 2A, for mapping these DF values ​​to intensity values ​​for that layer. To that end, the processor is configured to iteratively convert the intensity image 101 into the layered DF image 106 until the error value is less than an error threshold or until the error value reaches a value less than the error threshold. The error value is a layer-by-layer measure of the error or difference between the intensity image 101 and the intensity image reconstructed from the layered DF image 106 by combining the intensity values ​​of each level in their corresponding order. When such an error value condition is reached at a layer of the transformation, error value minimization is achieved. Then, at step 215, the layered DF image is output for rendering by using the intensity values ​​at this layer of the transformation.

[0105] As discussed above in connection with FIG. 1B, the layered DF image may be rendered by the output interface 108.

[0106] Therefore, the error value can be considered as the reconstruction error or visualization error 210, and reaching a visualization error below the error threshold is a stopping condition for stopping the iterative conversion of the intensity image 101 to the layered DF image 106. Therefore, the visualization error 210 value can be calculated by calculating the sum of the distances between the intensities of corresponding pairs of pixels at corresponding locations in the received intensity image 101 and the reconstructed intensity image for that particular layer. The intensity image is reconstructed from the layered DF image 106 of this layer using the image reconstruction function described above. To do this, the maximum distance between corresponding pairs of pixels in the received intensity image and the reconstructed intensity image is identified from the sum of the distances calculated for an iteration. Then, to minimize the visualization error 210 to a value less than the error threshold, a current DFP for the current layer of the iteration is selected that reduces the maximum distance between corresponding pairs of pixels in the received intensity image and the reconstructed intensity image. This is called a greedy optimization method for searching for a DFP for the current layer in the current iteration of method 212. Such greedy optimization is performed to select the current DFP that reduces the maximum distance between pairs of corresponding pixels and reduces the visualization error 210 of the entire reconstructed image to the minimum possible value defined by the error threshold.

[0107] Some embodiments are based on the recognition that the received intensity image 101 can be divided into a set of candidate regions. The candidate regions correspond to various locations on the intensity image 101. For example, the candidate regions may be identified as curves, tiles, or grids, each centered on a corresponding location in the intensity image 101. Next, for each candidate region, a local current DFP is identified that subtracts the maximum distance between corresponding pairs of pixels of the candidate region in the received intensity image and the reconstructed intensity image to generate a set of local current DFPs. This local current DFP is then combined with the set of local current DFPs to generate a current DF procedure.

[0108] As such, the set of candidate regions may be a union of non-overlapping tiles covering the entirety of the received intensity image 101. Furthermore, the current DFP comprises a Boolean combination of a set of local current DFPs. In some embodiments, dividing the intensity image 101 into the set of candidate regions is performed based on a greedy optimization methodology. The greedy optimization methodology includes setting the dimensions of the divided candidate regions for a current iteration of the greedy optimization to be larger than the dimensions of the divided candidate regions for a previous iteration of the greedy optimization.

[0109] Some embodiments are based on the recognition that the segmentation of the intensity image 101 into candidate regions and the minimization of the visualization error value 210 are performed using greedy optimization to achieve iterative optimization at each layer level for the transformation of the intensity image 101 at each level into a layered DF image 106. This layer-by-layer iterative optimization is thus performed to achieve successive refinement of the image across different layers. The successive refinement may be based on a level of detail (LOD) associated with the visualization error value 210. At each layer, the image obtained by rendering the current set of DFPs for this layer may hereinafter be referred to as the working canvas WC.

[0110] In some embodiments, the LOD includes the current resolution of the layered DF image (such as layered DF image 106 or layered DF image 211). Thus, method 212 includes converting an intensity image (such as intensity image 101 or intensity image 206) into a layered DF image using iterative optimization, where the iterative optimization includes generating a current DF procedure for the current layer that reduces an error between the current resolution of the received intensity image and a current intensity image reconstructed from a current sequence of layers including the current layer and a previous layer (such as reconstructed intensity image 209) determined by a previous iteration in the iterative optimization. Thus, different iterations in the iterative optimization use different resolutions of the received intensity image, varying from lower to higher resolutions in proportion to the iteration index. For example, layer 1 may have resolution R1, layer 2 may have resolution R2, and so on.

[0111] Thus, in some embodiments, method 212 may be performed iteratively to determine a sequence of layers of layered DF image 211 and to initialize and iteratively update the sequence of DFPs based on the LODs until a termination condition (equivalent to the stopping condition disclosed above) is met, where each iteration may include selecting a candidate region of the received intensity image, determining a new DFP for the selected candidate region, and combining the new DFP with the sequence of DFPs identified in the previous iteration, which may be further illustrated in the flow diagram shown in FIG. 2D.

[0112] 2D shows a flow diagram of a method 212a for converting an intensity image into a layered DF image based on LOD to search for a set of DFPs until a termination condition is met. For purposes of discussion of method 212a, the intensity image is represented by IMG 206, the current set of DFPs is represented by DFPs 216, the rendered image of each layer is a working canvas WC 217, the level of detail is LOD 218, the candidate regions at one or more locations are CR 219, and the new set of DFPs obtained by updating the current set is NDFPs 220.

[0113] Method 212a includes, in step 221, assigning DFPs to an initial configuration of IMG 206. The initial configuration may be an empty set of DFPs. Alternatively, the initial configuration may be a set of DFPs determined by various means for establishing a starting point for refining IMG 206. The various means include interpolation, regression, optimization, human editing, AI systems, search methods (random search, grid search, cell search, divide and conquer search, trial and error search, etc.), and combinations thereof. After assigning the DFPs to the initial configuration, method 212a includes, in step 222, initializing a WC. The WC may be initialized by rendering the initial configuration of the set of DFPs. Thus, the rendering may include rendering a blank image, a void image, or a constant color image (e.g., equal to the average color of the image, a common color of the image, etc.).

[0114] The starting point for initializing the WC can be determined by one or more of interpolation, regression, search, filtering, etc., and the starting point includes auxiliary data, which is stored with the set of DFPs to enable reconstruction of the image. The auxiliary data may be a compressed version of the image, a lower-resolution version of the image, an approximation of the image, an interpolation of the image, etc. As such, the assignment of the set of DFPs 216 to the initial configuration and initialization of the WC 217 is frequently adjusted (i.e., designed en masse) to determine an appropriate starting point for the refinement step. For example, one particularly effective adjustment design is to assign the initial configuration to an empty set and initialize the WC to a small, lower-resolution version of image S, which is stored with the set of DFPs 216, such that image reconstruction begins with rendering S, followed by rendering the set of DFPs 216 on top of S.

[0115] Additionally, in step 223, the LOD 218 is also initialized to the first level. Thereafter, in step 224, the set of DFPs 216 is iteratively refined until a stopping condition is met. As previously mentioned, the stopping condition involves determining an error value between the current layered DF image and the original intensity image and checking whether the error value falls below an error threshold. This is further described in connection with steps 225-229, which define the operations performed at each iteration for refining the DFPs.

[0116] Candidate regions are selected in step 225. The selection of candidate regions is guided by an error criterion, which may be determined using, for example, a greedy method, an optimization method, a trial-and-error method, a grid search to ensure coverage, a cell search to ensure coverage, a random search, or a divide-and-conquer search. Thus, candidate regions are determined by the maximum error due to the substantial difference in locations over a set of locations, using either a local or global error criterion. The global error criterion may be increased by allowing branching steps or by other means, including trial-and-error, grid search, cell search, random search, divide-and-conquer search, or a combination thereof.

[0117] The error criterion or error value may correspond to a measure of closeness between the rendering of the set of distance field procedures (i.e., WC 217) and the image (i.e., IMG 206) using various criteria, including L1, L2, perceptual, AI-based, etc. In some embodiments, each criterion may further include a penalty term to encourage simpler and smaller expressions (e.g., fewer distance field procedures) that emerge as optimized choices or criteria. After selecting a candidate region in this manner, in step 226, a new set of DFPs for the candidate region is determined using the previously stored set of DFPs 216, WC 217, and the original image IMG 206. Thereafter, in step 227, the new DFPs, NDFPs 220, are combined with the previous DFPs 216 to form an updated set of DFPs 216. Next, in step 228, WC 217 is updated based on the updated DFPs, and in step 229, the LOD 218 is incremented to the next level. These steps are then repeated by checking a stopping condition in step 224. If the stopping condition is met, the method 212a is stopped and the final stage WC 217 is rendered to provide the output image.

[0118] As such, LOD 218 is incremented at each level in a coarse-to-fine manner for each level to generate different degrees of coarse-to-fine (or dense-to-sparse) progression between successive refinement layers in transformation methods 212 and 212a. In some embodiments, the increment of LOD 218 is governed by one or more processes, including, but not limited to, being constant between the first and next level (i.e., zero increment), allowing a single level of detail (flat progression, zero progression, etc.), determined by a procedure, determined by a table, controlled by visual inspection, human-guided, by an AI module, image-independent, image-dependent, manually and / or procedurally set to control and / or achieve specific criteria for the nature of the image progression (e.g., smoothness) and magnitude and quality of presentation, dynamic updating, statistical updating, etc.

[0119] In some embodiments, the initial configuration determined in step 221 may be an empty set, and WC 217 may be initialized by rendering the initial configuration of a set of DFPs 216 onto the working canvas. Then, LOD 218 may be initialized to a coarse level. Furthermore, a stopping criterion may be specified such that densification stops when the difference between the image and the working canvas is less than an error threshold. For densification, a one-time processing of image IMG 206 is performed to generate preprocessed image data including an edge map determined at sub-pixel resolution. The error value for determining the stopping condition may be selected as an L2 error criterion for measuring the difference between image IMG 206 and working canvas WC 217.

[0120] Further, candidate regions are selected by finding a small set L of locations in the image with the maximum L2 error (i.e., by using a greedy method to select locations with the lowest match between the image and the working canvas). Next, a new set S of DFPs, i.e., NDFPs 220, is derived near each location P in L such that when S is combined with the set of DFPs, it improves the local L2 error criterion within the region surrounding P. Each distance field procedure in S is derived from strokes along edges determined in the preprocessed image data that reduce the difference between the working canvas and the image within the region surrounding P. The strokes are determined by curve-fitting location and color attributes. The precision of the strokes is determined by the level of detail: coarse levels of detail use large, soft brushes, while finer levels of detail use smaller (finer), harder, and more complex brushes, resulting in a progressive reconstruction of the image. The distance field procedure in S represents strokes at the appropriate level of detail by using a detail-oriented, resolution-independent distance field.

[0121] Additionally, the working canvas is updated from a set of distance field procedures DFPs 216, where the working canvas is updated by rendering the set of distance field procedures onto the working canvas. Additionally, the LOD 218 is incremented to the next level, which may be the final level.

[0122] In this manner, method 212a can be implemented in examples. To that end, in some embodiments, an arithmetic procedure is used to combine NDFPs 220 with DFPs 216. Also, as noted in the examples above, candidate regions are selected based on the error between the intensities of the selected candidate regions in the received intensity image and the corresponding regions in the reconstructed intensity image as obtained from WC 217.

[0123] In some embodiments, dividing the received intensity image into candidate regions includes dividing it into a grid of rectangular regions. For example, a grid search method may be used to divide the received intensity image into a square 16x16 grid. Then, as described in method 212a and the examples, points within each region of the grid whose reconstruction error is greater than an error threshold are selected. For example, a random point P within each square whose L2 error is greater than a specified threshold is identified. The neighborhood around each P constitutes a collective candidate region. These selected points are then used to solve the optimization problem for each selected point, using the region to generate a DF procedure for each selected point, such that an algebraic combination of the DF procedures for the selected points reduces the reconstruction error within the region. Therefore, in different iterations of the iterative optimization, the grid size may be increased with each subsequent iteration. The grid size may be increased with each refinement step (e.g., 32x32, 64x64, 128x128, etc.) to ensure convergence to a solution. Therefore, for the combining step 227, we solve an optimization problem for each square to find the union of a new set S (NDFPs 220) of detail-oriented, resolution-dependent distance field procedures located near P that minimizes a local L2 error criterion in the region surrounding P when S is arithmetically combined with the set of distance field procedures (DFPs 216). If the underlying distance field of S captures details comparable in size to the squares it belongs to, then larger squares will capture broader and lower-frequency image components, and smaller squares will capture finer and higher-frequency image components, resulting in a smooth, visually appealing progressive image reconstruction that progresses from high-level structure to fine, intricate details.

[0124] In some embodiments, the received intensity image may be preprocessed to extract preprocessed image data at sub-pixel resolution, and the preprocessed image data may be used to convert the received intensity image into a layered DF image, where the preprocessed image data includes one or a combination of an edge map, a gradient map, a Laplacian map, a zero-crossing map, a filtered version of the received intensity image, a frequency-domain version of the received intensity image, a wavelet decomposition of the received intensity image, and intensity statistics of the received intensity image.

[0125] In some embodiments, after the stopping condition is met, the intensity operations on the intensity image reconstructed from the layered DF images are estimated and stored to further reduce the error with the received intensity image.

[0126] Therefore, methods 212 and 212a are configured to achieve the goal of successive refinement of an image using layer-by-layer transformation, which is governed by a convergent solution to an optimization problem that optimizes computation, complexity, size, quality, and storage in the image transformation process. A convergent solution is possible by using a greedy optimization method to minimize an error value and select candidate regions, while reduced computational complexity is achieved by dividing the overall computation into processing parts for each layer to select candidate regions of appropriate size and achieve successive refinement. Such successive refinement is illustrated by the example shown in FIG. 2E.

[0127] Figure 2E shows, by way of example, a schematic diagram of the layer-by-layer DF transformation 230 realized by the method shown in Figures 2C and 2D. The layer-by-layer transformation operates successively on layers 230a, 230b, ..., 230n. The output of each layer is used as input for the refinement of the successive layer, and this successive refinement continues until a stopping condition is met. For example, the output image rendered in layer 230a is used as the input image to layer 230b, and so on.

[0128] At each layer, the transformation is performed according to methods 119 and 123 shown in FIGS. 1E and 1F, respectively, and methods 212 and 212a shown in FIGS. 2C and 2D, respectively. The layer-by-layer process begins with a series of refinements, proceeding from top to bottom, i.e., from layer 230a to layer 230n, and from left to right, i.e., from stage or frame 230a to 230an. Each frame contains a new set of distance field procedures (NDFPs) 220 for the refinement step, and the working canvas (WC) after the refinement step shows the effect of the determined NDFPs. At each step, the convergence speed of the refinement and the smooth, progressive, and detail-oriented multi-resolution performance gradually increase.

[0129] In this manner, successive refinements may be configured to select a set of operations (e.g., Boolean, blending, arithmetic, etc.) that operate on a set of distance fields located at a set of locations determined from the candidate region. The set of locations may be associated with the candidate region either by being near the candidate region or by being centered on the candidate region. The distance fields may then take a variety of forms, including strokes, filled regions, textured regions, gradients, edges, parameterized templates (all types for matching regions of images including noise (varying in amplitude, orientation, and frequency), textures, etc.), detail-directed distance fields, regularly sampled distance fields, procedural distance fields, analytical distance fields, distances stored in memory, etc.

[0130] In some embodiments, the solution of various operations on the distance field may be guided by an error criterion using either a greedy optimization method, or a greedy stepwise optimization method that avoids solving high-dimensional problems and instead solves one dimension at a time (at each step at each layer) and combines steps. Further, successive refinements proceed by either using curve fitting methods, using the location of the maximum error, and using the location of the substantial difference to identify the distance field at a set of locations, using a local error criterion, using a global error criterion, allowing for variance steps, etc.

[0131] In some embodiments, curve fitting may be performed to resemble the intuitive brush strokes of a painting process. Brush strokes may be fixed width, variable width, fixed density, variable density, solid, textured, procedural, statistical, or dynamic, producing positive and negative values ​​that allow for modeling both drawing and erasing on a surface, and strokes include a series of splines, polynomials, procedures, functions, points, curves, etc., along with associated attributes (location, pressure, width, color, luminance values, chrominance values, opacity, contour, texture data, time data, etc.).

[0132] Successive refinement of an image in the manner described in all of the previous embodiments provides a smooth, natural, progressive viewing of the reconstructed image without unexpected jumps and visual artifacts. This is achieved by allowing the expressiveness of the new set of DFPs to increase with each refinement step. Expressiveness is determined by the number of operations, the number of distance fields, the number of locations, the complexity of the operations, etc. In practice, this is done to mimic the action used by artists, who often begin their paintings with a rough sketch using a large, soft brush, followed by painting with a series of smaller (i.e., finer), harder brushes to add detail. As a result, fine brushstrokes are used only when necessary to refine the painting, while the rest remains coarse. At the same time, the progressive refinement and search of DFPs provides a graceful (i.e., smooth, expected, predictable) gradual visual refinement of the image without visual artifacts and unexpected changes as the progression unfolds, and can ensure that specified error tolerances are smoothly met when essential (e.g., in medical applications). Therefore, the above method results in extremely high-quality images.

[0133] Some embodiments are based on the recognition that converting an intensity image to a layered DF image also provides flexibility to convert the image to achieve desired performance and optimization feature levels, including, but not limited to, compression, texture, resolution, and detail. As such, image processing system 102 may be configured to provide the ability to specify desired performance feature levels.

[0134] 3A shows an image processing system 102 configured to receive transformation instructions 301 for transforming an input image to a desired performance and optimized feature level. The image processing system 102 has already been described in the previous embodiment.

[0135] As shown in FIG. 3A, the image processing system 102 includes an input interface 107, which may further include a first input interface 107a and a second input interface 107b. The first input interface 107a is configured to receive an input image, which may be a layered DF image 106. The input interface 107a may include an interface configured to receive external input from a user, an external source, a remote server, a database, etc. However, the input interface 107a may also be an internal interface, such as a system bus, that transfers input within the image processing system 102. The layered DF image 106 is received via either an internal or external source. The layered DF image 106 includes an ordered sequence of multiple layers, such as that shown in FIG. 2E, and each layer of the layered DF image 106 includes a DF procedure that defines DF values ​​at every location in the received intensity image and rules for mapping these DF values ​​to intensity values ​​for that layer.

[0136] The input interface 107 also includes a second input interface 107b configured to receive transformation instructions 301. The transformation instructions 301 may be received by a user from an external source, an internal source, etc. Thus, the second input interface 107b may also be an internal input interface or an external input interface. The second input interface 107b may receive the transformation instructions 301, transform the layered DF images 106 based on the transformation instructions, and output the reconstructed intensity images 103 obtained from the transformed layered DF images through the output interface 108.

[0137] As such, the transformation instructions 301 may specify one or a combination of: (1) compression parameters that cause the transformed layered DF image to include a compression of the layered DF image; (2) texture mapping parameters that cause the transformed layered DF image to include a layered DF image with a modified texture; (3) scaling instructions that cause the transformed layered DF image to include a scaled version of the layered DF image; and (4) algebraic instructions that cause the transformed layered DF image to include the results of an algebraic operation on the layered DF image.

[0138] The compression parameters may define a compression level for the layered DF image 106. The layered DF image may be repeatedly transformed to compress the layered DF image until the compression level specified in the transformation instructions is met, generating a transformed layered DF image. The desired compression level is met when the compression level of the transformed layered DF image is equal to or less than the desired compression level. Therefore, to achieve the desired compression level, a set of DFPs for the layered DF image at a layer may define parameters for lossless compression to generate a compressed layered DF image. Furthermore, in situations where the compression level is not met, compressed layered DF images at higher levels are removed from successive layers to generate a transformed layered DF image until the compression level is met.

[0139] In an alternative embodiment, to achieve a desired compression level, a set of DFPs for a layered DF image may define parameters for lossy compression to generate a compressed layered DF image. Also, in these situations, if the compression level is not satisfied, compressed layered DF images of higher levels are removed until the compression level is satisfied to generate a transformed layered DF image. Therefore, the compression of the DF procedure may be set to resolution-independent. A resolution-independent DF procedure is added to the top or final layer of the layered DF image 106 to generate the transformed layered DF image.

[0140] Compressing the layered DF image 106 in this manner is essential for many applications, such as electronic devices like modern microscopes, which produce massive amounts of data—gigabytes per second, terabytes of data per day—all in the form of images. Storing and processing this data is a serious obstacle. In these applications, the compressed representation described in the previous paragraph is advantageous because it not only compresses the data but also achieves two other goals: 1) the representation can be directly operated on (e.g., directly rendered, directly queried) without performing a decompression step, and 2) the representation adapts to the complexity of the data's content. Such adaptive representations are also detail-oriented, concentrating more expressive power in high-frequency regions of the data and using less expressive power for low-frequency regions of the data, thereby overcoming the storage and processing obstacles inherent in previously known solutions.

[0141] The transformation instructions 301 may also specify texture mapping parameters. Correspondingly, the texture mapping parameters are stored in the memory of the image processing system 102 in the form of a set of texture DF procedures that define different textures for at least a portion of the layered DF image. Furthermore, the transformation instructions 301 indicate a desired texture, which is used to select a texture DF procedure indicated by the desired texture from the set of texture DF procedures and add the selected texture DF procedure to the top layer of the layered DF image. Alternatively, the DF procedure in the top layer of the layered DF image 106 may be replaced with the selected DF procedure. Therefore, the image processing system 102 may execute the method 302 shown in FIG. 3B to transform the layered DF image 106 according to the desired transformation characteristics specified in the transformation instructions 301.

[0142] The method 302 includes, at step 303, a layered distance field (DF) image including an ordered sequence of layers. The layered DF image may be received at a first input interface 107a. As previously mentioned, each layer of the layered DF image includes a DF procedure that defines DF values ​​at every location in the received intensity image and rules for mapping these DF values ​​to intensity values ​​for that layer. In addition, transformation instructions 301 are also received at a second input interface 107b. The transformation instructions 301 specify at least one of compression parameters, texture mapping parameters, scaling instructions, algebraic instructions, or a combination thereof.

[0143] Further, in step 304, the layered DF image is transformed based on the transformation instructions 301. The transformation is performed by selecting a set of DFPs together with transformation parameters and adding or replacing the topmost DFP of the layered DF image with the transformed DFP. Based on this transformation, in step 305, the transformed layered DF image may be output at the output interface 108.

[0144] 3C illustrates another flow diagram of a method 306 for transforming a layered DF image into a transformed image based on the transformation instructions. For purposes of discussion of method 306, the intensity image is represented by IMG 307, the current set of DFPs is represented by DFPs 308, the rendered image of each layer is the working canvas WC 309, the level of detail is LOD 310, the candidate regions at one or more locations are CR 312, and the new set of DFPs obtained by updating the current set is NDFPs 313. These are as previously described for method 212a. However, method 306 also includes data associated with schedule 311 that specifies the transformation instructions for selecting a transformation schedule 323 for transforming the set of DFPs based on the transformation instructions. The resulting image is seen to be a transformed IMG 324.

[0145] The method 306 includes assigning the DFPs to an initial configuration in step 314. Setting the initial configuration has been previously described with reference to FIG. 2D . Next, in step 315, the WC 309 is initialized. For example, if the initial configuration is an empty set, the WC 309 may also be a blank or black image. Then, the LOD 310 and the schedule 311 are also set to initial values. In some examples, the LOD 310 may vary from coarse to fine. The LOD 310 may also be initialized according to the schedule 311.

[0146] The schedule 311 may be configured to control how the set of DFPs 308 is refined step-by-step in a data-efficient, distance-field-procedure-independent manner, thereby avoiding additional data storage per step and / or per DFP. Thus, the schedule 311 allows for step-specific data and DFP-specific data processing. Furthermore, the schedule 311 specifies instructions for processing the layered DF images at each layer, progressing from coarse to fine (or from fine to coarse) in various increments to achieve different degrees of progressive compression in the image. Thus, in some embodiments, the processing of the layered DF images progresses at a constant level of detail, resulting in a single LOD 310 and non-progressive (flat progressive, zero progressive) compression. The LOD 310 may be determined, for example, by any of a number of procedures, a table, visual inspection, human guidance, or AI component control; it may be image-independent or image-dependent; and it may be dynamically or statistically updated. Once the DFP 308 is initialized, the WC 309, LOD 310, and schedule 311 are thus performed, and in step 317, the refinement process of the DFPs begins, with the refinement being incremental until a stopping criterion is met. The stopping criterion, in one example, may determine the size and quality level of the compressed image. The stopping criterion is determined based on a trade-off between the size and quality level of the compressed image. In some embodiments, the stopping criterion is set manually and / or procedurally to control and / or achieve specific criteria for the size and quality level of the compressed image.

[0147] Refinement of the IMG 307 and DFPs 308 continues through steps 318 through 322 until a stopping criterion is met. At each refinement, the DFPs 308 are updated according to a transformation schedule 323 and a transformed IMG 324, which are used to determine the quality (or size or similar constraint) of the transformed IMG 324. This quality is then compared to the quality (or size or similar constraint) specified by the stopping criterion. If the desired quality set by the stopping criterion is reached, refinement of the DFPs is stopped; otherwise, steps 318 through 322 continue.

[0148] In step 318, a CR 312 is selected from the WC 309, and a set of NDFPs 313 is selected from the set of DFPs 308 for the WC 309. Next, in 320, the DFPs 308 and the NDFPs 313 are combined (e.g., by substitution or addition), and based on this update, the WC 309 is updated in step 321. Further, in step 322, the LOD 310 is incremented according to the transformation schedule 311. For example, if the transformation schedule specifies transformation instructions that specify compression parameters, the LOD 310 for the next layer may be set to the next compression level. Steps 310-322 are then repeated until a stopping criterion that defines the desired level of compression is reached. The compression can be either lossy, lossless, etc.

[0149] Therefore, image transformation based on the updated set of DFPs is more accurate and provides more inherent redundancy, which helps to self-fill any missing data in the image compression because the DFPs are missing distance values ​​for locations surrounding a distance value or distance values ​​surrounding a location.

[0150] Similarly, if the transformation instructions specify texture mapping parameters, the stopping criteria may determine the desired level of texture in the final output image. For any image, the detail supported by an image depends on its resolution. This detail provides information about the image's texture. Obtaining sufficient detail everywhere can require extremely large images, especially compared to the limited space in GPU memory. Some well-known image reconstruction techniques, such as Structured Vector Graphics (SVG), provide edge detail at any scale, but are overly simplistic, lack rich detail, are less general, and are more complex to encode, evaluate, and filter.

[0151] However, using schedule 311 to define texture mapping information for a set of DFPs for each layer of an image provides a resolution-independent distance field procedure that significantly improves image synthesis quality, memory utilization, and performance characteristics. Additionally, due to the single instruction, multiple data (SIMD) nature of the determined resolution-independent distance field procedure, method 306 provides efficient rendering on the display and various processing tasks imposed on the texture maps.

[0152] In some embodiments, the transformation instructions specify scaling instructions for generating a layered DF image based on a scaled version of the layered DF image associated with a scaling level, the scaling level including an upper bound of infinity.

[0153] In some embodiments, the transformation may be applied to any type of image, such as a font image, an image associated with a map of a geographic region, or the like.

[0154] Thus, using methods 302 and 306, a desired level of transformation that satisfies any desired output characteristics, such as compression or texture, can be obtained for the image with efficient processing and reduced storage requirements. As such, image processing system 102 may be implemented as a remote computing module accessible from any client device, potentially a light client, that accesses the features of image processing system 102 without excessive processing load.

[0155] 4 illustrates such an architecture for image processing system 102. Image processing system 102 is configured to perform all of the operations described so far in the previous embodiments. Image processing system 102 may further communicate with computing devices 400 in a user layer, and image processing system 102 itself resides in an application or service layer, further interacting with an image processing server 402 in a server layer of interaction. Layers such as user layer, application layer, and server layer are used to represent the separation between different computing components in the image processing ecosystem, which can be facilitated by image processing system 102.

[0156] The image processing system 102 may be accessed from a computing device 400, which may be any dedicated or general-purpose computing device. The computing device 400 may provide images, such as the original intensity image 101, that need to be converted into a high-quality DF layered image by the image processing system 102. For example, the computing device 400 may be a navigation device, and the intensity image 101 may be an image of a map to be displayed for navigation services.

[0157] The image processing system 102 includes an input interface 107 that receives the intensity image 101 from the computing device 400 and converts it into a high-quality layered DF image by accessing an image processing application programming interface (API) 401. To that end, the processor 102a may be configured to generate API calls to the image processing API 401. The image processing API further encapsulates various functions useful for converting the intensity image 101 into a high-quality layered DF image. Thus, the API calls may be directed to an image processing server 402 that stores various libraries corresponding to different types of transformations that can be performed on the intensity image 101 to obtain a layered DF image including an ordered sequence of multiple layers, as previously described. Each layer includes a DF procedure for defining DF values ​​at multiple locations of the received intensity image 101 and a set of rules for mapping the DF values ​​to intensity values ​​for the respective layer. The DF procedure and the set of rules may be stored in a DFP library 403 on the image processing server. Each DF procedure is associated with a layer in the sequence of layers of the layered DF image, and each DF procedure contains instructions for initializing and repeatedly updating the DF procedure until a termination condition is met. The API calls are configured to send image features in the form of standard API messages to the image processing server 402 and to obtain the functions required for image transformation from the DFP library 403.

[0158] The processor 102a may then be configured to implement the layer-by-layer transformation subject to the LOD and stopping function by executing the instructions or processes specified in the received functions, as described in the previous embodiment. To that end, the processor 102a may be configured to receive a response from the image processing server 402, the response including one or more functions required to obtain a layered DF image. The layered DF image may then be rendered on the output interface 108.

[0159] The one or more functions may include an intensity reconstruction function for combining the mapped intensities of each layer according to their order in the layer sequence to reconstruct a received intensity image after transformation. The one or more functions may also include a function for transforming the received intensity image by determining an error value associated with the difference between the received intensity image and an intensity image reconstructed from the layered DF image. Furthermore, the error value is compared with an error threshold, and the reconstructed intensity image is updated based on the comparison. The transformation may be performed iteratively until the error between the received intensity image and an intensity image reconstructed from the layered DF image by combining the intensity values ​​of each level in their corresponding order is less than the error threshold. The error may be a visualization error including the sum of the distances between the intensities of corresponding pixel pairs at corresponding locations in the received intensity image and the reconstructed intensity image.

[0160] In some embodiments, the API call further includes transformation instructions for transforming the layered DF image based on the transformation instructions. The transformation instructions include one or a combination of: (1) compression parameters that cause the transformed layered DF image to include a compression of the layered DF image; (2) texture mapping parameters that cause the transformed layered DF image to include the layered DF image with a modified texture; (3) scaling instructions that cause the transformed layered DF image to include a scaled version of the layered DF image; and (4) algebraic instructions that cause the transformed layered DF image to include the result of an algebraic operation on the layered DF image.

[0161] The image processing server 402 stores various function libraries, including, but not limited to, an operations library 404 containing a collection of functions for performing algebraic and other manipulation operations on distance fields in layered DF images, a brushstroke library 405 containing functions for implementing DF procedures as a series of brushstrokes, a curve fitting library 406 containing curve fitting functions, a primitive library 407 for implementing a set of primitives, and a database 408 storing any data related to images, mathematical constants and formulas, training data, etc.

[0162] The DFP library 403 may include functions for implementing various types of procedures, functions for selecting candidate regions, determining new DF procedures for selected candidate regions, iteratively updating DF procedures, combining DF procedures, etc. The DFP library 403 may also include an adaptive DF procedure based on an adaptive sampling rate associated with the intensity image 101.

[0163] The brush stroke library 405 may include functions for implementing asymmetric stroke procedures associated with spline curves. For example, an API call includes pen type data for specifying the asymmetric stroke procedure, where the pen type data includes at least a pinstroke pen type option, a pressure-sensitive pen type option, a customizable pen type option, a scalable pen type option, and a textured pen type option, so that corresponding related functions are retrieved from the brush stroke library 405. The spline curve associated with the corresponding distance field and the asymmetric stroke procedure define rules for mapping the distance field of the spline curve to different gradients of intensity change on different sides of the spline curve's central axis, where the intensity of the spline curve varies in a direction parallel to its central axis. Such procedures are further described in FIG. 6.

[0164] The asymmetric stroke procedure may further be associated with a real-time curve fitting operation performed on a spline curve, which may be defined by a function stored in the curve fitting library 406 .

[0165] Therefore, the image processing server 402 provides the necessary functions from different libraries in response to an API call to implement a method for converting the intensity image 101 into a layered DF image (such as the layered DF image 106 shown in FIG. 1B ). The method includes receiving the intensity image 101 and generating an API call to convert the received intensity image 101. The API call is sent to the image processing server 402 to convert the intensity image 101 into a layered distance field (DF) image. The layered DF image includes an ordered sequence of layers. Each layer in the ordered sequence includes a DF procedure for determining DF values ​​at multiple locations in the received intensity image and a set of rules for mapping the DF values ​​to intensity values ​​of the respective layer. The image processing server 402 responds to the API call with a response message, which includes one or more functions obtained from different libraries to obtain the layered DF image, which is obtained by converting the intensity image. Once the conversion is complete, the layered DF image may be rendered (in the form of a working canvas WC or, if a stopping condition is reached, as a reconstructed intensity image).

[0166] Therefore, the image processing server 402 can provide various libraries for realizing high-quality, highly efficient, resolution-independent image conversion and distributed computing by storing large amounts of data separately on the image processing server 402 and accessing this data from the image processing system 102 via API calls as needed.

[0167] The image processing system 102 is also configured to provide a resolution-independent channel-level image transform, as shown in the method of Figure 5. Figure 5 shows a flow diagram of a method 500 for determining a resolution-independent transform for an image 501, according to some embodiments. The method 500 includes, in step 503, decomposing the image 501 into a set of channels 502.

[0168] Next, in step 504, the channels 502 are used to configure a composition engine 506 for the set of channels 502, and parameters P1, P2, P3, P4, ..., Pn-1, Pn of the composition engine 506 determine a set of resolution-independent DFPs such that rendering of the set of resolution-independent DFPs directly reconstructs the set of channels 502. The composition engine may be a module in processor 102a of image processing system 102 specifically configured to execute instructions for implementing the resolution-independent channels and their transformations as described in this method 500.

[0169] Method 500 also includes training synthesis engine 506 to configure various parameters in step 505. To that end, this configuration may further include identifying resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, Pn in step 508. Next, in step 509, determining DFPs corresponding to the resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, Pn. Next, in step 510, rendering the determined DFPs and, in step 511, reconstructing channel 502 from the rendered DFPs. This reconstruction process is further provided in training block 505, and an image may be reconstructed by trained synthesis engine 507 based on trained synthesis engine 506, as illustrated in steps 512-515. Steps 512 to 515 are the same as steps 508 to 511, except that in step 511, resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, Pn are identified. Next, in step 513, DFPs corresponding to the resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, Pn are determined. Next, in step 514, the determined DFPs are rendered, and in step 515, channel 502 is reconstructed from the rendered DFPs.

[0170] Therefore, training 505 of the synthesis engine 506 on the set of channels 502 is performed to adjust parameters of the synthesis engine 506 to determine a resolution-independent representation of the image 501 by a set of resolution-independent DFPs 509 and 513 for the set of channels 502.

[0171] Thus, configuring 504 a composition engine 506 for a set of channels 502 is performed to determine a set of resolution-independent DFPs 509 such that rendering of the set of resolution-independent DFPs reconstructs the set of channels 503. This reconstruction may be performed directly from the channels, for example, through a linear or non-linear mapping from distance to intensity.

[0172] Thus, the decomposition 503 determines a luminance channel of the image 501, a chrominance channel of the image 501, multiple chrominance channels, luminance and multiple chrominance channels, performs a color space transformation from the image color space to a set of RGB (HSV, HSB, YUV, LAB, LUV, CMYK, etc.) channels, and uses the unchanged image channels as the set of channels, the decomposition transforming the image 501 into a frequency domain of the channels, a wavelet domain of the channels, etc. Additionally, the decomposition 503 may perform one-time preprocessing on the set of channels 502 to generate preprocessed channel data, the preprocessed channel data including an edge map, a gradient map, a Laplacian map, a zero-crossing map, a filtered version of the set of channels, statistics, etc., where the edge, gradient, Laplacian, and zero-crossing maps are determined at sub-pixel resolution, etc. The decomposition 503 may further include applying a filtering operation to the image 501 once before decomposing the image 501 into a set of channels 502, the filtering operation performing sharpening, noise removal, etc. (to compensate for (or complement) the rendering characteristics of the set of resolution-independent DFPs 509).

[0173] The composition engine 506 may be implemented as one of a procedure, a neural network, or a composition of procedures. The composition may be static during training or dynamic during training, with each procedure in the composition of procedures determining an element (or a subset of elements) of a set of resolution-independent DFPs, and each procedure in the composition of procedures determining a portion (or a subset of elements) of an element (or a subset of elements) of the set of resolution-independent DFPs, with the portion being a set of operations (e.g., Boolean, blending, arithmetic, etc.) operating on a set of distance fields or a set of locations relative to the set of distance fields.

[0174] In some embodiments, configuring 504 and training 505 are performed separately for each channel in the set of channels, with configuring (training) being performed once for the set of channels 502 and using pre-processed channel data to perform its function.

[0175] Thus, training 505 uses renderings of a set of resolution-independent DFPs 509, or renderings of images 501, or both, at a set of viewpoints to adjust parameters of a synthesis engine 506. Viewpoint-dependent versions of the set of resolution-independent DFPs (and images) are used to adjust the parameters of the synthesis engine 506, guided by an error criterion, such as methods 123 or 212a. The viewpoint determines scale, translation, rotation, etc., and a single line of sight is used during training in a viewpoint-independent manner to adjust the parameters of the synthesis engine 506. Various means are used to adjust the parameters of the synthesis engine 506, including trial-and-error, range enumeration, grid search, cell search, random search, divide-and-conquer search, or combinations thereof. Thus, to minimize the size of the set of resolution-independent DFPs, training may be progressive in nature (coarse-to-fine), multi-resolution in nature, SIMD in nature, etc.

[0176] Thus, the goal of this training is to select, for each resolution-independent DFP in the set of resolution-independent DFPs 509, a set of operations (e.g., Boolean, blending, arithmetic, etc.), a subset of parameters of the composition engine, that operate on the set of distance fields such that the distance fields take various forms including strokes, filled regions, textured regions, gradients, edges, parameterized templates (all kinds for matching regions of images containing noise (different amplitudes, orientations, and frequencies), textures, etc.). The distance fields may be detail-oriented, periodically sampled, procedural, analytical, memory-stored distances, etc.

[0177] In one example, an image 501 is decomposed into a set of channels in the LAB color space. Separate synthesis engines 506 for the L and AB channels are then trained 504 to construct a synthesis engine 507 for the set of channels 502. The first layer of the synthesis engine 506 for L includes a set of fixed-size procedures L1, each of which includes a set of parameters defining detail-oriented, resolution-independent DFPs 509 capable of capturing low-frequency components of the image 501. The second layer of the synthesis engine 506 for L includes a set of fixed-size procedures L2, each of which includes a set of parameters defining detail-oriented, resolution-independent distance field procedures 509 capable of capturing slightly higher-frequency components of the image 501 than those modeled by L1. L2 receives the output of L1, and the procedures of L1 are combined in a distance-field-specific manner (e.g., Boolean combination, blend combination) determined during training 505 to generate an output for L2 to consume. The synthesis of layers continues in this manner until the final layer, Ln, is configured to capture the high-frequency components of the image 501, and the output of Ln is a reconstruction of the L channel of the image 501. A similar setup is repeated for the AB channels. The number of procedures for each layer, L1, L2, ..., Ln, can be adjusted to suit the needs of its intended application. More procedures per layer enable a better overall reconstruction of the image 501. The number of procedures per layer can also be determined dynamically during training rather than remaining static. While training the synthesis engine for a set of channels, the parameters of the synthesis engine for the L and AB channels are adjusted separately. Correspondingly, an optimization method is used to adjust the L2 error criterion between the image 501 and the rendering of a set of detail-oriented, resolution-independent distance field procedures at multiple viewpoints.

[0178] In this manner, the image processing system 102 may implement a method 500 for transforming an image based solely on that channel.

[0179] Another possible image transformation is to mimic the painting process in the form of brush strokes, as shown in method 600 of FIG. 6 shows a flow diagram of a method 600 for transforming an image using a brushstroke-based procedure, according to some embodiments. Functions for implementing the method 600 may be provided in the brushstroke 405 library shown in FIG.

[0180] A brush stroke is a series of points with attributes, a path with attributes, a centerline with attributes, a sweep with attributes, a set of curves with attributes, or a set of distance field procedures with attributes. Attributes may include width, contour, distance field procedure, stamp, pressure, color, luminance value, chrominance value, opacity, noise level, texture data, time data, arc length data, etc., and paths include movement to commands, lines, and curves. A brush stroke's centerline is defined by a series of points, a path, or a set of curves, and brush strokes generate positive and negative values ​​that allow for modeling both drawing and erasing on a working canvas.

[0181] Brushstroke-centric primitives are more easily converted into an equivalent set of brushstrokes (e.g., edges and gradients). These primitives resemble brushstrokes, and when combined, produce primitives of the same type (in mathematical terms, a closed set of combination operations, such as distance fields, and Boolean operations that combine them). These primitives can take a variety of forms, including edges, gradients, solid regions, textured regions, strokes, parameterized templates (all kinds for matching regions of images, including noise (varying in amplitude, orientation, and frequency), textures, etc.), detail-directed distance fields, regularly sampled distance fields, procedural distance fields, analytical distance fields, memory-stored distances, etc.

[0182] The method 600 includes rendering an image 601 through a set of brush strokes BSTROKES 616 such that rendering the set of brush strokes 616 progressively reconstructs the image. The method 600 includes assigning a set of DFPs 602 to an initial configuration in step 606. Next, a working canvas WC is initialized in step 607. Further, a level of detail LOD 604 is set to a first level in step 608.

[0183] Further, in step 609, the set of DFPs 602 is iteratively refined based on the image 601 and the working canvas WC 603 until a stopping criterion is met. The iterative refinement is performed based on steps 610-614. In step 610, candidate regions CR 605 of the image 601 are selected at a set of locations where the image differs from the working canvas WC 603. Further, in step 611, a new set of distance field procedures NDFPs 617 is determined from the candidate regions and the set of locations at the level of detail defined by LOD 604. The NDFPs 617 define brushstroke-centered primitives provided by BSTROKES 616. Next, in step 612, the NDFPs 617 are combined (e.g., by addition or replacement) with the DFPs 602.

[0184] Then, in step 613, WC 603 is updated by rendering a set of distance field procedures DFPs 602, and in step 614, LOD 604 is incremented to the next level.

[0185] The iteration then returns to step 609 to check for a stopping condition. If a stopping condition is reached, the brushstroke-centered primitives BSTROKES 616 of the set of DFPs 602 are processed to determine a set of brushstrokes, the rendering of which progressively reconstructs the image 601.

[0186] In step 615, to convert image 601 into a set of brush strokes, image 601 is first converted into a constrained set S of distance field procedures, and then the set of brush strokes is determined from S. This two-step indirection has access to more information (i.e., is richer), and therefore is more efficient and simpler to convert than prior art methods that infer a set of brush strokes directly from image pixels.

[0187] Brush strokes have several properties that make them ideal candidates for representing images: 1) because brush strokes are inherently 1D, they are rendered faster than more common, more complex 2D forms such as triangles; 2) the points that define a brush stroke are more cohesive and therefore compress better; 3) they can be stylized from one form (e.g., a pencil) to another (e.g., a paintbrush) to achieve different looks and effects for various applications such as filmmaking and games; 4) brush strokes can be easily converted into distance fields, which are inherently SIMD, and then efficiently processed (e.g., rendered, selected, transformed, stylized) on the GPU by leveraging the structure of the fragment SIMD pipeline on the GPU; and 5) as determined by the present invention, they can be physically painted onto a canvas for artistic creation.

[0188] These brush strokes may then undergo various forms of processing, including but not limited to edge following, smoothing, curve fitting, attribute fitting, region filling, template fitting, distance field fitting, image sampling, optimization, greedy search, greedy stepwise optimization, trial-and-error search, random search, divide-and-conquer search, etc.

[0189] Such processing of brush strokes and their use for rendering images by the image processing system 102 provides a more intuitive method of image transformation that closely resembles the real-world painting process.

[0190] Another use of the image processing system 102 is to convert an image into a set of primitives, as shown in FIG.

[0191] 7 shows a method 700 for converting an image 701 into a set of primitive SOPs 703, where each primitive in the set includes a set of distance field procedures, and rendering the set of primitives reconstructs the image 701. The primitives may include brush strokes, solid areas, textured areas, edges, gradients, etc.

[0192] The method 700 includes setting the working canvas WC702 to an initial state in step 706. The initial state may be a blank (i.e., empty) state, a constant color (e.g., a common color of the image, equal to the average color of the image), etc. Setting may include various means, such as interpolation, regression, optimization, search, filtering, etc., to determine a starting point for the working canvas, which starting point includes auxiliary data. The auxiliary data is stored with a set of primitive SOPs 703 to enable reconstruction of the image 701. The auxiliary data may be a compressed version of the image 701, a low-resolution version of the image 701, an approximation of the image 701, an interpolation of the image 701, etc.

[0193] Next, in step 707, initial SOPs 703 are determined. In some examples, once the working canvas is set to blank, specific primitives are selected to approximate each pixel in image 701, and the specific primitives are added to SOPs 703. Image 701 is decomposed into a set of regions that cover image 701, and to determine initial SOPs 703, a specific set of primitives is selected to approximate each region in the set of regions, and the specific set of primitives is further added to SOPs 703. Add to.

[0194] In another example, when working canvas WC702 is set to a non-blank state (e.g., set to a low-resolution version of the image as a starting point), a difference image is generated as the difference between image 701 and WC702. Next, to determine SOPs 703, specific primitives are selected to approximate each pixel in the difference image, the specific primitives are added to a set of primitives, and the difference image is decomposed into a set of regions that cover the difference image. Next, a specific set of primitives is selected to approximate each region in the set of regions, and the specific set of primitives are added to set of primitive SOPs 703.

[0195] Next, in step 708, the SOPs 703 are rendered onto the working canvas. Further, from step 709, the SOPs 703 are refined until a stopping condition is met. The refinement includes, in step 710, selecting candidate regions CR705 in a set of locations where the image 701 resembles the working canvas WC702. Next, in step 711, a set of primitives SSOP704 is found from the CR705 and the set of locations.

[0196] Then, in step 712, subsets of the set of primitives are merged to reduce the size of the set of primitive SOPs 703. Merging combines sets of distance field procedures corresponding to the primitives in the subsets of the set of primitives SSOPs 704. Merging takes advantage of the computational advantages of distance fields to perform various operations that are complex, difficult, slow, and sometimes impossible with other representations, including blends, Boolean offsets, sweeps, and morphs. Merging blends sets of distance field procedures using, for example, any of Boolean operations, blending operations, arithmetic operations, conditional operations, etc. Merging further uses a search method to combine sets of distance field procedures, which is guided by an error criterion such that the error criterion is not allowed to exceed a specified error tolerance during merging. The search method may be a greedy optimization method, a greedy stepwise optimization method, a curve fitting method, a trial-and-error method, a range enumeration method, a grid-based method, a cell-based method, a random method, a divide-and-conquer method, or a combination of these methods, which minimizes the size of the set of primitives while maintaining a specified error tolerance during merging.

[0197] Additionally, in step 713 , working canvas WC 702 is updated from SOPs 703 , the rendering of which reconstructs image 701 .

[0198] In this manner, the above iterations are repeated, and method 700 stops when a stopping condition is met. Method 700 starts with a large set of primitives S that are close matches to image 701, and refines S into a smaller set while maintaining (i.e., not exceeding) a predetermined error tolerance between S and image 701 (the stopping condition) until a stopping criterion is met.

[0199] The successive refinements provided by method 700 are guided by an error criterion using a variety of methods, such as greedy optimization, trial-and-error, grid search to ensure coverage, cell search to ensure coverage, random search, and divide-and-conquer search. The error criterion may be a local error criterion, a global error criterion that allows branching steps to increase the global error criterion using simulated annealing temperature cooling, and so on. The CR 705 in the iterative refinement is determined by a set of locations, which may be locations with minimal error, locations with small differences, locations with high similarity, or a combination thereof. Successive refinements map a set of points to a new set of points by applying refinement rules. Refinement rules typically increase the number of points to produce a smoother sequence. Refinement rules can be adaptive, changing their behavior based, among other things, on the characteristics of the points and their neighborhoods. For example, a refinement rule may insert a new point between consecutive points in a set of data points only if the distance between the consecutive points exceeds a threshold value.

[0200] The method 700 may be implemented by the image processing system 102 to provide a highly efficient method for image transformation based on a set of primitives derived from an image.

[0201] The various embodiments described above may also be implemented using the computing system shown in FIG.

[0202] FIG. 8 shows a block diagram of a computing system 800 used to implement various embodiments disclosed herein for converting an image into a layered DF image. FIG. 8 is described in conjunction with FIGS. 1A-7. The computing system 800 may correspond to the image processing system 102 or the computing device 400. The computing system 800 may have several interfaces connecting the computing system 800 to one or more image rendering devices 813. For example, a network interface controller (NIC) 808 may be adapted to connect the computing system 800 to a network 812 via a bus 807. Via the network 812, either wirelessly or wired, the computing system 800 can accept an input intensity image 815. Additionally, additional information associated with the input intensity image 815 can be received via an input interface 816. The input interface 816 can connect the computing system 800 to a keyboard and / or a pointing device. By way of example, the pointing device may include a mouse, a trackball, a touchpad, a joystick, a pointing stick, a stylus, or a touchscreen, among others. The input interface 816 may be connected to one or more sensors 801 that capture input images 815. To this end, the one or more sensors may include a camera.

[0203] The computing system 800 includes a processor 802 configured to execute stored instructions and a memory 810 that stores instructions executable by the processor 802. The processor 802 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 810 may include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system. Additionally, the computing system 800 may include a storage device 803 adapted to store different modules that store executable instructions for the processor 802, such as the image processing system 102. The storage device 803 may be implemented using a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof.

[0204] The storage module 803 is configured to store a set of DFPs 804 and their corresponding rules. In addition, the storage module 803 can store working canvases WC at various stages of processing. The storage module 803 can also store layered DF images IMG 806 at all layers of processing. In addition, the storage module 803 can store data regarding levels of detail, a set of API-related data regarding API calls made to the image processing API, error value-related data, etc. Thus, the storage module 803 may be configured to cause the processor 802 to receive an input image 815 and perform all the functions of the image processing system 102 disclosed in the previous embodiment. As a result, the input image 815 can be converted into a layered DF image, which can then be reconstructed and rendered via the output interface 809. The output interface 809 may be configured to connect the computing system 800 to an image rendering device 813. By way of example, the image rendering device 813 includes, among others, a computer monitor, a television, a projector, or a mobile device.

[0205] Computing system 800 may also be configured to implement further features, which are described below in conjunction with FIGS. 9, 10, 11, and 12.

[0206] FIG. 9 illustrates a method 900 of curve fitting using the image processing system 102 according to an embodiment of the present disclosure.

[0207] In some embodiments, method 900 may be a computer-implemented method for fitting a set of curves 945 to a set of data points 910. Method 900 includes, at step 950, filtering the set of data points DPs 910 to obtain a filtered set of data points FILTERED DPs 915. Further, at step 955, method 900 includes finding a first set of steady points STAT.PTs:1 920 from the filtered set of data points FILTERED DPs 915. Further, method 900 includes, at step 960, fitting a first set of curves CURVES:1 925 between successive elements of the first set of steady points STAT.PTs:1 920 and, at step 965, tessellating the first set of curves CURVES:1 925 to obtain a tessellated set of data points TESS.DPs 930. Next, in step 970, the tessellated set of data points TESS.DPs 930 is filtered to find a second set of stationary points of the filtered tessellated set of data points, FILTERED TESS.DPs 935. Further, in step 975, a second set of stationary points STAT.PTs:2 940 of FILTERED TESS.DPs 935 is determined. Further, in step 980, these curves are fitted between successive elements of the second set of stationary points STAT.PTs:2 940 to obtain a second set of curves CURVEs:2 945. Thus, the second set of curves CURVEs:2 945 determines a set of curves that fit the second set of data points DPs 910.

[0208] The data points include a set of attributes, where the attributes in the set of attributes may be location, number, value, measurement, order, sequence number, width, contour, distance field procedure, stamp, pressure, color, luminance value, chrominance value, opacity, noise level, texture data, time data, arc length data, etc., and may be of any dimension (e.g., 2D, 3D, etc.). The set of attributes includes independent attributes and dependent attributes, and the set of curves determines the relationship between the independent and dependent attributes of the set of data points.

[0209] Data points in a set that are required to remain unchanged (i.e., their attributes remain fixed during the fitting process) are called stationary points. Any point in a set of data points can be labeled as stationary before the fitting method begins (examples include the start and end data points in a set of data points that specify an independent sequence, such as an independent curve in a digital drawing).

[0210] The data points in the set of data points are It can be dynamically classified as stationary by a procedure that detects features (performs feature detection), where the features include corners, edges, points of maximum curvature, points of inflection, common templates of features, and combinations thereof.

[0211] The curves may include Bezier curves of various degrees (e.g., quadratic, cubic, quartic, etc.), splines of various degrees (e.g., quadratic, cubic, quartic, etc.), polynomials of various degrees, mathematical functions, procedures with parameters that produce a set of output values ​​given a set of input values, a set of distance field procedures, combinations of these forms with and without constraints to achieve particular goals such as C1 and C2 continuity, vexels, piecewise variations of these forms to allow long sequences of data points to be properly fitted, etc. A vexel is a cubic Bézier curve C whose first and second off-curve points are constrained such that the perpendicular projection from the first off-curve point of C onto a line L between a first endpoint p0 of C and a second endpoint p1 of C intersects L at a distance one-third of the length of L from p0, and the perpendicular projection from the second off-curve point of C onto a line L intersects L at a distance two-thirds the length of L from p0.

[0212] Tessellation approximates each curve in a set of curves by another primitive, typically of lower degree or less complexity. For example, in the tessellation of a cubic Bézier curve (of degree 3) into a sequence of line segments (of degree 1) that closely approximate the cubic Bézier curve, primitives include curves, lines, points, procedures, functions, etc.

[0213] Most geometric smoothing methods suffer from several problems. Perhaps the most serious is the shrinkage problem. When a geometric smoothing method is repeatedly applied to a shape, the shape eventually collapses to a point. Perhaps the most popular geometric smoothing technique is Gaussian smoothing, which is performed by convolving a set of data points P that defines a curve C with a Gaussian filter G. Gaussian methods are known to cause shrinkage. In one embodiment of the present invention, a two-pass geometric smoothing method is used to reduce shrinkage of a curve C defined by a set of data points P: pass 1 calculates the smoothed curve G(C) and the residual CG(C), where G(C) is the Gaussian-filtered curve C, and pass 2 calculates G(C)+G(CG(C)) to obtain a smoothed set of data points with reduced shrinkage.

[0214] A corner can be defined as the intersection of two edges. A corner can also be defined as a point for which two different principal edge directions exist in a local neighborhood of this point. One approach for corner detection is as follows:

[0215] The dot product formula is used: dot(A,B) = ||A|| ||B|| cos(angle). A and B are normalized vectors around each candidate point p, eliminating points near the start and end of each independent sequence in the set of data points as candidates. Furthermore, a normalized vector is found around each candidate point p. To do this, the dot product formula is used to find cos(angle) around the candidate point p. This angle is compared with a specified tolerance to find a result (as either true or false). Some other methods of corner detection include Harris corner detection, Shi-Tomasi corner detection, robust corner detection, and previously trained neural networks.

[0216] In some embodiments, a set of data points is filtered, and a non-shrinking smoothing operation is performed to obtain a filtered set of data points, a smoothing operation is performed, a convolution operation is performed, a noise reduction operation is performed, a moving average operation is performed, a corner-preserving smoothing operation is performed, a successive refinement operation is performed, or a combination of these operations is performed.

[0217] In some embodiments, a successive refinement operation maps a set of data points to a new set of data points by applying refinement rules, where the new set of data points replaces the set of data points, and the refinement rules resize (e.g., increase) the set of data points to generate a smoother sequence of connecting points within the set of data points. Refinement rules may be static or adaptive. Adaptive refinement rules are based on features of a point and its neighborhood. The features may be distance, differences between specific attributes of a set of attributes of a set of data points, etc.

[0218] FIG. 10 illustrates a computer-implemented method 1000 for performing successive refinement operations on a set of data points for curve fitting, according to some embodiments of the present disclosure.

[0219] The method 1000 includes, in step 1050, filtering the set of data points DPs 1010 to obtain a filtered set of data points FILTERED DPs 1015. The method 1000 further includes, in step 1055, performing a successive refinement operation on the filtered set of data points FILTERED DPs 1020 to obtain a set of successively filtered data points R.FILTERED.DPs 1020. Next, the method 1000 includes, in step 1060, filtering the set of successively filtered data points R.FILTERED.DPs 1020 to obtain a set of filtered and successively refined filtered data points FRFILTERED.DPs 1025. Further, in step 1065, a set of steady points STAT.PTs 1030 are found from the filtered and successively refined filtered data points FRFILTERED.DPs 1025. Finally, in step 1070, a set of curves CURVEs 1035 are fitted between successive elements of the set of steady points STAT.PTs 1030.

[0220] In some embodiments, discovering the first (or second) set of stationary points from the filtered set of data points (filtered tessellated set of data points) includes performing a procedure to find the first (or second) set of stationary points, where the procedure detects features such as corners, edges, points of maximum curvature, points of inflection, common templates of features, and combinations thereof, to identify data points in the filtered set of data points (filtered tessellated set of data points) that are labeled as stationary to find the first (second) set of stationary points.

[0221] In some embodiments, tessellating the first set of curves includes performing an order reduction (complexity reduction) of the first set of curves to obtain a tessellated set of data points, decomposing the first set of curves into a sequence of approximated line segments (approximate lower-order curves, approximate data points) to obtain the tessellated set of data points, and performing recursive refinement on the first set of curves to obtain the tessellated set of data points. The order reduction (decomposition, recursive refinement) includes an error tolerance (error measured as the difference between the tessellation and the first set of curves), and the tessellation continues until the error tolerance is met (typically until the tessellation closely approximates the first set of curves).

[0222] In some embodiments, fitting the first (second) set of curves between consecutive elements of the first (second) set of steady-state points involves first identifying a subset of the filtered set of data points (filtered, tessellated set of data points) between each pair of consecutive elements of the first (second) set of steady-state points, and then performing a fitting operation on the subset of the filtered set of data points (filtered, tessellated set of data points) to determine a fitted curve to add to the first (second) set of curves.

[0223] The fitting operation may include linear regression, nonlinear regression, least squares, optimization, robust regression, or pre-trained neural network regression, and includes first determining a distance field from a subset of the filtered set of data points (the filtered, tessellated set of data points), second using the distance field to deform a set of candidate curves to fit the subset of the filtered set of data points (the filtered, tessellated set of data points), and third assigning the deformed set of candidate curves to a fitting curve, and deforming the set of candidate curves to determine an error criterion by sampling the distance field. The error criterion may be used to move and reshape the set of candidate curves, and determining a distance field from the filtered set of data points (filtered tessellated set of data points) involves first constructing an approximation to a subset of the filtered set of data points (filtered tessellated set of data points), and then determining a distance field from the approximation, where the approximation is a polyline connecting the subset of the filtered set of data points (filtered tessellated set of data points), where the approximation is a polyline joining successive refinements of the subset of the filtered set of data points (filtered tessellated set of data points), and where the approximation is a set of curves connecting the subsets of the filtered set of data points (filtered tessellated set of data points).

[0224] In some embodiments, filtering the tessellated set of data points performs an operation to determine the filtered tessellated set of data points, which may include a smoothing operation, a convolution operation, a non-erosive smoothing operation, a noise reduction operation, a moving average operation, a corner-preserving smoothing operation, a successive refinement operation, or a combination of these operations.

[0225] In some embodiments, viewpoint-dependent fitting defines a subset of these steps (e.g., filtering and tessellation) that are viewpoint-dependent. Viewpoint-dependent fitting can provide distinct advantages (e.g., faster performance and higher quality) for some applications. The viewpoint determines the viewing characteristics (e.g., scale, rotation, translation, resolution, etc.) of how a set of data points is viewed. An example of viewpoint dependency is as follows: When tessellating, the scale of the viewpoint (i.e., magnification level) can be used to moderate the fineness with which the tessellation is performed. As the viewpoint zooms in on the set of data points, the tessellation becomes finer, and as the viewpoint zooms out on the set of data points, the tessellation becomes coarser. Similarly, when filtering, the scale of the viewpoint can be used to moderate how the filtering is performed; as the viewpoint zooms in on the set of data points, the filtering uses fewer points from the set of data points, and as the viewpoint zooms out on the set of data points, the filtering uses more points from the set of data points.

[0226] For example, in one embodiment, the curve type may be a vexel, with each data point including a triplet of x, y, and pen pressure for digital drawing applications. Filtering the set of data points may include a non-erosive smoothing filter. For further filtering, a first set of stationary points from the filtered set of data points is identified using a dot product formula to identify corners as stationary points. Next, a first set of curves is fitted between consecutive elements of the first set of stationary points. For fitting, vexels can be directly fitted to data points between consecutive elements in a single step by using their definition (no iterative regression method is required for vexels). Further, tessellation is performed to fit the first set of curves to the tessellated set of data points by recursively subdividing the vexels into line segments. The tessellated set of data points is filtered using a Gaussian smoothing filter. Further, a second set of stationary points from the filtered tessellated set of data points is identified. This is done by identifying corners as stationary points using the dot product formula. A second set of curves is then fitted between successive elements of the second set of stationary points. To this end, vexels can be directly fitted to data points between successive elements in a single step using their definition (no iterative regression is required for vexels).

[0227] In some embodiments, a computer-implemented method is provided for incrementally fitting a set of curves to a series of data points.

[0228] 11 shows such a method 1100 for incrementally fitting a set of curves CURVEs 1105 to a set of data points DPs 1110. The method 1100 may include, in step 1160, setting a current steady state point CUR.STAT.PT 1115 to an initial value. Next, in step 1162, the set of data points DPs 1110 are processed a batch at a time, where the batch includes a batch subset BATCH.SUBSET 1120 of the set of data points. The batch processing further includes, in step 1164, finding a set of steady points STAT.PTs 1125 from a batch subset BATCH SUBSET 1120 of the series of data points DPs 1110 and the current steady point CUR.STAT.PT 1115, in step 1166 updating the current steady point CUR.STAT.PT 1115 from the set of steady points STAT.PTs 1125, and in step 1168 fitting a new set of curves NEW CURVEs 1130 to each subset SUBSET.DPs 1135 of the series of data points identified by consecutive pairs of elements of the set of steady points STAT.PTs 1125.Fitting the new set of curves further includes filtering the subset of data points SUBSET.DPs 1135 in step 1170, fitting a first set of curves CURVEs:1 1140 to the filtered subset of data points FILTERED.SUBSET.DPs 1145 in step 1172, tessellating the first set of curves CURVEs:1 1140 to a tessellated set of data points TESS.DPs 1150 in step 1174, filtering the tessellated set of data points TESS.DPs 1150 in step 1176 to obtain filtered tessellated data points FILTERED.TESS.DPs 1155, and fitting the new set of curves NEW CURVEs 1130 to the filtered tessellated set of data points FILTERED. and adding, in step 1180, a new set of curves, NEW CURVEs 1130, to the set of curves, CURVEs 1105, which are fitted to the set of data points processed so far.

[0229] Some embodiments provide an interactive, incremental curve-fitting component necessary for digital drawing systems. In such systems, data points are generated with pen pressure as an artist draws with a tablet. The data points are typically processed to fit a curve in small batches (e.g., five points) at a time, providing immediate visual feedback to the artist as they draw. Subsequent processing and curve-fitting of all data points can be computationally excessive for some systems, necessitating batching of smaller subsets of data points. Each batch of data points is curve-fitted and integrated with a previously calculated curve derived from the previous batch. Curve fitting begins upon a pen-down event and ends during a pen-up event. In such a process, the first data point in a series is set to null. The series of data points terminates when a stopping condition is met, such as some event, such as the occurrence of a pen-up event, exceeding a time limit, exceeding a maximum number of iterations, meeting an error tolerance, or lack of convergence over a series of processing steps, and may be, for example, data-independent, data-dependent, dynamically updated, static, visual inspection, human directed, AI-controlled, procedure-determined, or table-determined.

[0230] In some embodiments, a set of steady-state points from a batch subset of a series of data points is discovered, and updating the current steady-state point from the set of steady-state points is done using a state machine. Generally, a state machine is any device that stores the status of something at a given point in time and can act on inputs to change the state and / or cause an action or output for any given change. More formally, a state machine can be described as an initial state or record of something stored somewhere, a set of possible input events, a set of new states that can result from the input events, and a set of possible actions or output events that result from the new states.

[0231] In some embodiments, updating the current steady point from the set of steady points assigns the current steady point to the last steady point in the set of steady points.

[0232] Some embodiments provide a method for rendering an ordered set of primitives using a depth buffer, where each primitive in the ordered set defines a complex shape determined by a set of Boolean operations performed on a set of distance fields.

[0233] 12 illustrates a method 1200 for rendering an ordered set of primitives using a depth buffer 1205. The method 1200 may be implemented by a processor configured to perform the operations of the method 1200, including setting the depth buffer 1205 to an initial state at step 1250. The operations further include dividing the value range of the depth buffer 1205 into an ordered set of sub-ranges 1215 at step 1255. The operations of the method 1200 further include assigning the ordered set of sub-ranges 1215 to each of the ordered set of primitives in a one-to-one sorted manner at step 1260. The operations further include rendering each primitive of the ordered set of primitives 1210 in order at step 1265, where a primitive has a sub-range 1220, a set of Boolean operations 1225, and a set of distance fields 1230. The method 1200 further includes, at step 1270, using the depth buffer 1205 to determine a composite shape 1235 of the primitive from the subrange 1220, the set of Boolean operations 1225, and the set of distance fields 1230, and rendering the composite shape 1235.

[0234] The depth buffer includes a memory buffer and a processor that operates on the memory buffer. The processor may be a CPU, a GPU, etc. The depth buffer includes a GPU z-buffer that may be dynamic, e.g., has a numeric range, includes addressable elements (e.g., fragments, pixels), performs Boolean operations on the addressable elements, performs arithmetic operations on the addressable elements, and operates on the addressable elements in parallel. The Boolean operations include sum, multiplication, and difference, and the sum, multiplication, and difference operations are decomposed into a set of min-max operations, and the depth buffer operates in a SIMD manner.

[0235] The embodiments disclosed herein avoid clearing the depth buffer for each primitive in an ordered set of primitives by dividing the depth buffer's value range into a set of sub-ranges, typically one sub-range for each primitive, and reformulate the Boolean operations and set of distance field ranges that define primitives to operate strictly within their sub-ranges in such a way that all previous values ​​assigned to the depth buffer are ignored when rendering previous primitives in the ordered set in the requested order. This significantly improves computational efficiency by removing the penalty imposed for clearing the depth buffer, as is done in existing technology solutions.

[0236] In some embodiments, the depth buffer's numeric range ranges from a first value to a second value, the first value being less than the second value, and the setting assigns the first value to 0 and the second value to 1. The setting can assign the depth buffer to the first value, and a subrange of the ordered series of subranges is a subset of the numeric range, the subrange having a first subrange value and a second subrange value, and the first subrange value being less than the second subrange value.

[0237] In some embodiments, the dividing separates the depth buffer's numeric range into an ordered series of sub-ranges by dividing the depth buffer's numeric range into N sub-ranges, where N is equal to the total number of primitives in the ordered series of primitives. The depth buffer's numeric range may be divided into N numerically increasing sub-ranges or N numerically decreasing sub-ranges.

[0238] In some embodiments, determining the composite shape uses a depth buffer to perform a set of Boolean operations on a set of distance fields within a sub-range, reformulating the Boolean operations and the range of the set of distance fields that define primitives to operate strictly within that sub-range in such a way that when rendering previous primitives in an ordered series in the requested order, all previous values ​​assigned to the depth buffer are ignored.

[0239] In some embodiments, the rendering process uses the depth buffer obtained by determining the composite shape to render the composite shape. For example, an ordered set of four primitives, A, B, C, and D, may require rendering. Each of these consists of a Boolean union operator, from which the composite shape can be determined. According to embodiments disclosed herein, the depth buffer is initially initialized to the numeric range 0 to 1, which holds the maximum value for future operations applied to the depth buffer fragments. Primitive A may be assigned to the subrange starting at 0.00 and ending at 0.25, primitive B may be assigned to the subrange starting at 0.25 and ending at 0.50, primitive C may be assigned to the subrange starting at 0.50 and ending at 0.75, and primitive D may be assigned to the subrange starting at 0.75 and ending at 1.00.

[0240] Further, for this example, rendering proceeds as follows: The depth buffer is cleared once. Next, a set of distance fields for primitive A is programmed to work within its assigned sub-range [0.00, 0.25] and to calculate its composite shape by using max arithmetic with the depth buffer set to hold the maximum value. Primitive A is then rendered using the composite shape in the depth buffer. Furthermore, a set of distance fields for primitive B is programmed to work within its sub-range [0.25, 0.50] and to calculate its composite shape by using max arithmetic with the depth buffer set to hold the maximum value. Note that all distance values ​​determined by primitive B are greater than any distance values ​​currently being processed for primitive A. As a result, the previously rendered distance values ​​for primitive A do not affect the calculation of the composite shape for primitive B and therefore do not need to be cleared before processing primitive B. Primitive B is then rendered using the composite shape in the depth buffer. The same steps for primitive C and primitive D achieve progressive rendering of the primitives.

[0241] The above description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the above description of exemplary embodiments provides those skilled in the art with an enabling description for implementing one or more exemplary embodiments. What is intended is various changes that may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter, as set forth in the appended claims.

[0242] Specific details are provided in the above description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. Furthermore, like reference numbers and names in the various drawings indicate like elements.

[0243] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Moreover, the order of operations may be rearranged. A process may be terminated when its operations are completed, but may have other steps not discussed or included in the diagram. Moreover, not all operations in any process that are specifically described may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to a return of the function to the calling function or the main function.

[0244] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automated implementation may be performed or at least assisted through the use of machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor may perform the necessary tasks.

[0245] The various methods or processes outlined herein may be coded as software executable on one or more processors employing any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0246] Embodiments of the present disclosure may be implemented as a method, an example of which is provided. The order of operations performed as part of this method may be determined in any suitable manner. Thus, embodiments may be configured to perform operations in an order different from that illustrated, which may include performing some operations simultaneously even though they are shown as a series of operations in the illustrated embodiments. While the present disclosure has been described with reference to certain preferred embodiments, it should be understood that various other adaptations and modifications may be made within the spirit and scope of the present disclosure. Therefore, it is the aspect of the appended claims to cover all such variations and modifications that fall within the true spirit and scope of the present disclosure.

Claims

1. An image processing system, comprising: an input interface configured to receive an intensity image; at least one processor, the at least one processor executing computer-executable instructions to: generating application programming interface (API) calls to transform the intensity image; sending the application programming interface (API) call to an image processing server to convert the intensity image into a layered distance field (DF) image, the layered distance field (DF) image comprising an ordered sequence of a plurality of layers; Each layer in the ordered sequence is a DF procedure for determining DF values ​​at a plurality of locations of the intensity image; a set of rules for mapping the DF values ​​to intensity values ​​for each layer; the at least one processor, by executing the computer-executable instructions, is further configured to receive a response from the image processing server, the response including one or more functions for obtaining the laminar distance field (DF) image, such that the DF image is obtained by transforming an intensity image; the image processing system further comprising an output interface configured to render the laminar distance field (DF) image; the one or more functions include an intensity reconstruction function for reconstructing the transformed intensity image by combining the mapped intensities of each layer according to their order in the sequence of layers.

2. Transforming the intensity image by the image processing server includes: determining an error value associated with a difference between the intensities of the intensity image and an intensity image reconstructed from the layered distance field (DF) image; comparing the error value to an error threshold; The image processing system of claim 1 , further comprising: updating the reconstructed intensity image based on the comparison.

3. 3. The image processing system of claim 2, wherein the intensity image is repeatedly transformed until an error between the intensity image obtained by combining intensity values ​​at each level in their corresponding order and the intensity image reconstructed from the layered distance field (DF) image is less than the error threshold.

4. The image processing system of claim 3 , wherein the error is a visualization error that includes a sum of distances between intensities of corresponding pairs of pixels at corresponding locations in the intensity image and the reconstructed intensity image.

5. The image processing system of claim 1 , wherein the application programming interface (API) calls include transformation instructions for transforming the layered distance field (DF) image based on the transformation instructions.

6. The conversion instruction is (1) a compression parameter such that the transformed layered distance field (DF) image includes a compression of the layered distance field (DF) image; (2) texture mapping parameters such that the transformed laminar distance field (DF) image includes the laminar distance field (DF) image with a modified texture; (3) scaling instructions such that the transformed laminar distance field (DF) image comprises a scaled version of the laminar distance field (DF) image; and (4) algebraic instructions that cause the transformed layered distance field (DF) image to contain the results of algebraic operations on the layered distance field (DF) image. The image processing system of claim 5 , comprising at least one or a combination of:

7. 2. The image processing system of claim 1, wherein each DF procedure is associated with a layer in a sequence of layers of the layered distance field (DF) image, and each DF procedure includes instructions for initializing and repeatedly updating the DF procedure until a termination condition is met.

8. Iteratively updating the DF procedure comprises: selecting candidate regions of the intensity image; determining a new DF procedure for the selected candidate region; and combining the new DF procedure with a sequence of previous DF procedures.

9. 9. The image processing system of claim 8, wherein at least one DF procedure includes an asymmetric stroke procedure associated with a spline curve, the spline curve being associated with a corresponding distance field, the asymmetric stroke procedure defining rules for mapping the distance field of the spline curve to different gradients of intensity change on different sides of a central axis of the spline curve, such that the intensity of the spline curve varies in a manner perpendicular to the central axis of the spline curve.

10. The image processing system of claim 9 , wherein the asymmetric stroke procedure includes a real-time curve-fitting operation performed on the spline curve.

11. 11. The image processing system of claim 10, wherein the application programming interface (API) calls include pen type data for defining the asymmetric stroke procedure, and the pen type data includes at least a pinstroke pen type option, a pressure-sensitive pen type option, a customizable pen type option, a scalable pen type option, and a textured pen type option.

12. The image processing system of claim 8 , wherein the DF procedure includes an adaptive DF procedure based on an adaptive sampling rate associated with the intensity image.

13. 1. A method for image processing, said method comprising: receiving an intensity image; generating an application programming interface (API) call to transform the intensity image; sending the application programming interface (API) call to an image processing server to convert the intensity image into a layered distance field (DF) image, the layered distance field (DF) image comprising an ordered sequence of a plurality of layers; Each layer in the ordered sequence is a DF procedure for determining DF values ​​at a plurality of locations of the intensity image; a set of rules for mapping the DF values ​​to intensity values ​​for each layer; The method further includes receiving a response from the image processing server, the response including one or more functions for obtaining the laminar distance field (DF) image, such that the DF image is obtained by a transformation of an intensity image; The method further comprises rendering the layered distance field (DF) image; The method, wherein the one or more functions include an intensity reconstruction function for reconstructing the transformed intensity image by combining the mapped intensities of each layer according to their order in the sequence of layers.

Images