System and method for image transformation

Abstract

To provide a method and a system for processing an image and transform the same into a high resolution and high-definition image using a computationally efficient image transformation procedure.SOLUTION: Transformation of an image in an image processing system comprises first transforming the image, also referred to as an intensity image, into a layered distance field (DF) image comprising an ordered sequence of multiple layers. Each layer in the ordered sequence is associated with a DF procedure (DFP) and one set of rules for mapping the DF values to intensity values of the respective layers. A result of applying the DFP to each location in the intensity image is a transformed intensity image of high definition and high resolution. The application of the DFP is governed by a stopping criteria based on error values between the intensity image and a reconstructed intensity image.SELECTED DRAWING: Figure 1B

Description

Technical Field

[0001] The present disclosure generally relates to image processing, and more specifically to methods and systems for rendering images using a distance field procedure (DFP).

Background Art

[0002] The fields of image processing and computer graphics have become widespread and are used in billions of applications in daily life. From navigation to shopping, trains to cars, entertainment news / entertainment systems to medical devices, image processing is widely used and applied in various fields. Particularly in the field of navigation technology, for efficient and safe navigation, high-quality and high-definition map images are currently required to show high resolution, sharpness, and more detailed details in map images even after zooming. Despite such a large number of uses, there are still some deficiencies in image representation. Conventional approaches have tried to address this problem, but more qualified approaches are needed.

[0003] Another problem with image processing is that images provide rich textures and details but require a significant amount of storage space. Also, images exhibit various visual artifacts such as mosaic processing, excessive blurring, moiré patterns, jagging, etc. when transformations such as zooming and rotation are performed. Furthermore, images cannot provide an appropriate level of abstraction when transmitting various forms of information or when trying to edit specific elements.

[0004] To overcome this problem, one solution to reduce image storage requirements is to represent images in the Scalable Vector Graphics (SVG) format. While the SVG format is compact and can be scaled to any size without loss of quality, it lacks the texture and richness of images. Therefore, a solution for image representation is needed that can provide the richness of pixels in full-intensity images, along with the scalability and size of the SVG format. Thus, even after the availability of the SVG format is achieved, there is a need for a superior solution for image representation and processing that can provide the richness of pixels with artifact-free transformation (i.e., resolution-independent) and the size of the SVG format.

[0005] Another requirement in image processing is to provide high image quality, offering as much detail as possible without consuming excessive processing power and storage space. Some existing graphics processing applications are based on the calculation of distance fields for two-dimensional (2D) images / shapes. The distance field of an object represents the distance from any point in space to the boundary of that object. The distance chosen in many applications is usually the shortest distance from the point to the object's boundary. The 2D distance field of a shape represents the signed shortest distance from any point P in space to the boundary (i.e., contour) of the shape. Signed distance can be used to distinguish between the inside and outside of an object (for example, a negative distance indicates that the point is outside the object, and a positive distance indicates that the point is inside the object). Numerous forms of criteria can be adopted for measuring distance, but Euclidean distance is often preferred because of its usefulness in several applications such as collision detection and rendering. 2D distance fields are used to represent numerous details of a 2D shape, such as the outside of the shape, the inside of the shape, the contour of the shape, and some offset planes.

[0006] Such distance-field-based representations of 2D shapes and objects offer several advantages over more conventional geometric methods for representing objects and have been effectively used in many fields, including computer-aided design, medical imaging, surgical simulation, font rendering, games, filmmaking, deformation modeling, fluid simulation, and robotics.

[0007] The distance field is smooth (C 0 Because it is continuous and changes throughout space, it also offers benefits in terms of efficiency and quality. From a computational efficiency standpoint, distance fields are computed using simple and fast Boolean operations, including but not limited to addition, subtraction, and multiplication. In addition, distance fields also offer fast and simple offset processing, easier blend calculations, smoother reconstruction of edge shapes, simpler collision detection, and fast geometric queries. However, distance fields are represented using analytical representations, and their decisions can sometimes be complex for complex shapes.

[0008] Another representation associated with a 2D distance field is a distance map, which is a periodically sampled map of the distance field for various points on a plane representing a 2D shape. Distance maps are obtained by sampling the shape at a very high rate to capture all the details of the shape, such as corners and Voronoi boundary lines.

[0009] According to another expression, an Adaptively Sampled Distance Field (ADF) can be used to perform detail-oriented sampling of the distance field in a given shape and then reconstruct the distance field from these adaptively sampled points. Such ADFs may include ADF representations of bilinear cells, centroid cells, quaternary cells, etc. Along with rendering images using ADFs, anti-aliasing is used to estimate the intensity at all pixels of the shape. However, until now, the performance of ADFs has been limited to the rendering of primitives such as fonts and glyphs. Procedural ADFs are primarily based on distance field techniques, which are used to convert a set of curves representing a certain shape, such as the characters of a font, into an equivalent set of distance fields, which helps to provide improved performance and quality in processing tasks related to such sets of curves and / or fonts, in this case including collision detection, selection, animation, blending, tuning, and editing. While ADFs can be used to render shapes and fonts appropriately, they are not known to be used for rendering high-resolution images.

[0010] In some approaches, distance-based anti-aliasing is used for rendering shapes or fonts. Distance-based anti-aliasing offers superior performance compared to conventional anti-aliasing because the distance field changes smoothly in this case. Also, since the distance field of moving shapes such as fonts or glyphs does not change very much from frame to frame, distance-based anti-aliasing provides superior rendering quality per frame, even for moving fonts or glyphs. Another feature of distance-based anti-aliasing is that it provides continuous stroke modulation (CSM) by providing a continuous range of stroke width and edge sharpness settings. However, this approach also has a drawback: implementing CSM becomes overly complex for complex topological changes in the underlying shape.

[0011] Therefore, there is a need for an efficient system for rendering high-intensity, detailed images that can overcome the shortcomings of the above approach, offering higher processing efficiency and lower storage requirements. [Overview of the Initiative] [Problems that the invention aims to solve]

[0012] Numerous approaches are known for processing 2D shapes and fonts as described above. However, processing high-intensity and high-resolution images is often computationally expensive and requires a lot of memory. One of these approaches is the distance field, which is a mature graphics representation technique. Distance field techniques use a standard such as Euclidean distance to measure distance, for this reason due to its usefulness in numerous applications such as collision detection and rendering. Distance fields have several advantages over more conventional geometric methods for representing objects and have been effectively used in many fields, including computer-aided design, medical imaging, surgical simulation, font rendering, games, filmmaking, deformation modeling, fluid simulation, and robotics. Distance fields are a concrete example of implicit functions with a long history of use and research.

[0013] The various embodiments disclosed herein provide a variety of distance field representations, including detail-oriented distance fields, periodically sampled distance fields, procedural distance fields, analytical distance fields, and distances stored in memory. Thus, distance fields can also be derived from geometric primitives such as strokes, filled areas, and textured areas. [Means for solving the problem]

[0014] Some embodiments are based on the understanding that objects can be visualized by using a distance-field rendering pipeline. A distance-field rendering pipeline can be defined as being given a known geometric shape of an object, calculating a distance field representing that geometric shape, and then mapping the calculated distance field to pixels in an image. In fact, different pixels in an image have different distances from the object's surface, so their intensity can be estimated as a function of the distance value. In other words, distance values ​​can be mapped to intensity values. A simple example of such mapping is to brighten pixels where there are negative distance values ​​and darken pixels where there are positive distance values. As mentioned earlier, negative distances correspond to pixels inside the object, and positive distances correspond to pixels outside the object, and thus, by using such mapping, a bright representation of an object's shape can be visualized on a dark background.

[0015] Some embodiments are based on the understanding that by using a distance-field rendering pipeline, not only complete geometric shapes but also geometric primitives that form various types of shapes can be visualized. For example, geometric primitives such as curves are first converted to a distance field and then mapped to pixels and their corresponding intensity values ​​for rendering on a display. In addition to intensity estimation, the mapping allows for the execution of various functions such as anti-aliasing and colorization. Alternatively, various functions such as collision detection, selection, and path planning can be performed by querying the geometric primitives converted to a distance field. According to some embodiments, the results of these operations can also be rendered on a display.

[0016] Some embodiments are based on the understanding that a distance-field rendering pipeline can be reversed, and that the distance field can be calculated from an intensity image, i.e., inversely to the current rendering pipeline. For example, some embodiments disclose a method and system for representing the intensity of a richly textured image as a set of distance fields. To this end, such inverse rendering requires discovering a virtual shape having a distance field such that the original intensity image is generated by visualizing the distance field values. Therefore, an objective of some embodiments is to provide a method and system for realizing such an inverse rendering pipeline that uses distance fields to provide the advantages of the distance fields and to use them for processing high-intensity images.

[0017] Furthermore, several embodiments disclose methods and systems that can provide richness of intensity pixels in artifact-free (i.e., resolution-independent) and SVG format size. These methods and systems can be used for a variety of applications such as surgical planning, data compression, texture mapping, and monitoring.

[0018] Various embodiments disclosed herein provide inverse rendering pipelines based on intensity image replacement using distance field (DF)-based solutions that solve many technical challenges, including, but not limited to, eliminating visual artifacts present in images during field transformation (e.g., mosaic effects, moiré patterns, excessive blurring, jagged edges), reducing storage requirements, reducing image-specific memory and processing requirements by adapting to content complexity, and providing fast, high-quality, incremental browsing over slow networks. Thus, unlike the prior art, the various embodiments disclosed herein offer superior compression and the ability to be directly manipulated (e.g., direct rendering, direct querying) without performing a decompression step.

[0019] Therefore, the objective of some embodiments is to provide systems and methods for converting image intensity to a DF representation. In addition to or instead of this, the objective of some embodiments is to provide a structure for a DF representation of image intensity that, when reconstructed, renders a distance field to represent the original image with desired accuracy. In computer science and its many applications, changing the representation of an object from one form to another often allows for significant improvement in many of the technical challenges associated with that object. This technique is referred to herein as the "transformation and conversion" technique. This transformation and conversion technique has been applied to geometric shapes and distance fields in various fields such as font rendering, but has not yet been applied to intensity images and distance fields. One reason for such a deficiency is the complexity of such transformations.

[0020] Some embodiments are based on the recognition that it may be advantageous to provide an optimization procedure that seeks the best DF representation of the image intensity while optimizing several optimization parameters so as to reduce the overall complexity of the above transformation.

[0021] Therefore, one example of an optimization parameter might be minimizing the visualization error, for example, the distance between the original image and the image reconstructed from the distance field. However, given the complexity of the transformation from intensity to distance field, such optimization is difficult and hard to converge. To address this problem, some embodiments are based on how the actual painting process is carried out. For example, when viewing an arbitrary intensity image, the final intensity and / or color that each pixel has is independent of how the artist arrived at that intensity. However, during the painting process, the artist may not immediately paint a point with its final intensity, but rather add layers on top of layers of paint with different intensities / colors so that the ordered combination of layers of paint results in the final intensity / color at each location of the paint. As a result, each rich intensity image can be viewed not as a two-dimensional (2D) image of intensity, but as a 3D image of layers of intensity, i.e., layers of 2D images, which combine to form the desired image.

[0022] Therefore, various embodiments provide methods and systems for converting intensity images of arbitrary complexity into layered distance-field images by applying conversion and processing techniques to intensity images. Each layer of the layered distance-field image includes a distance-field procedure that defines distance-field values ​​at all locations in the intensity image, and rules for mapping these distance-field values ​​to intensity values. In addition, the layered distance-field image includes information about the order in which different layers are combined so that the intensity image reconstructed from the layers of the layered distance-field image approximates the original intensity image.

[0023] Some embodiments provide a method, system, and apparatus for image processing. The method, system, and apparatus are based on converting an intensity image, such as an image with extremely detailed, high-definition, and rich texture, into a layered DF image. The layered DF image is obtained by performing the DF conversion of the intensity image layer by layer to simplify processing and reduce the memory and computational requirements of the entire system while achieving a desired level of performance at the same time. Those layers are obtained in an ordered sequence, and each layer in this sequence is associated with a DF procedure for defining DF values at multiple locations in the intensity image and a set of rules for mapping the DF values to the intensity values of each layer. The layer-by-layer conversion thus performed results in a layered DF image, which is then rendered to obtain an intensity image of excellent quality. The layer-by-layer conversion achieves the purpose of reversing the rendering pipeline and providing a high-quality image with rich details and textures as an output.

[0024] Some embodiments are based on the recognition that each of a plurality of locations in the intensity image can be represented by a respective one of a plurality of candidate regions in the intensity image.

[0025] Some embodiments provide an intensity reconstruction function for reconstructing an intensity image by combining the mapped intensities of each layer according to their order in the sequence of layers.

[0026] Some embodiments provide for determining an error value associated with the difference between the intensity of the original or received intensity image and the intensity image reconstructed from the layered DF image. The error value is then compared to an error threshold. This comparison is then used to continue the layered conversion or stop the conversion process and update the reconstructed image accordingly.

[0027] Various embodiments disclose a Backus-Naur Form (BNF) grammar that describes multiple operations for each layer in a layered DF transformation. Some embodiments are based on the recognition that, by using a layered DF transformation, the asymmetric strokes of a painting process can be reproduced by defining an asymmetric stroke procedure, thereby ensuring the optimality of the image conversion process and thus making the overall image conversion process calculation efficient and convergent.

[0028] Various embodiments provide resolution-independent intensity image reconstruction. Various embodiments provide a distance field operation that includes one or more of a distance map operation, an adaptive distance field calculation operation, an analytical distance field calculation operation, a procedural distance field calculation operation, an in-memory distance-distance field calculation operation, a stroke distance field calculation operation, an intra-region distance field calculation operation, and a unary operator distance field calculation operation.

[0029] Some embodiments are based on the recognition that a DF procedure can include an asymmetric stroke procedure associated with a spline curve, the spline curve is associated with a corresponding distance field, and the asymmetric stroke defines rules for mapping the distance field of the spline curve to different gradients of intensity change on different sides of the central axis of the spline curve such that the intensity of the spline curve changes in a direction perpendicular to its central axis.

[0030] Various embodiments provide a DF procedure that includes a DF visualized with masked stepwise intensity, and the masked stepwise intensity includes a null intensity value at a specific location in the layered DF image.

[0031] Some embodiments are further based on the recognition that different layers of a layered DF image represent elements corresponding to different resolutions of the received intensity image.

[0032] Some embodiments provide a method for image processing. This method includes the steps of receiving an intensity image and converting the received intensity image into a layered DF image, the layered DF image comprising an ordered sequence of multiple layers. Each layer in the ordered sequence includes a DF procedure for determining DF values ​​at multiple locations in the received intensity image and a set of rules for mapping the DF values ​​to the intensity values ​​of each layer. This method further includes the step of outputting the layered DF image for rendering on an output interface.

[0033] Some embodiments provide a non-temporary computer-readable storage medium on which a processor-executable program is implemented to perform a method for image processing.

[0034] Accordingly, the various embodiments disclosed herein provide efficient, resolution-independent, and adaptive techniques for processing and transforming images to provide richly textured, high-definition, extremely detailed, and low-memory rendering of images based on DF technology. Furthermore, the various embodiments provide a more intuitive, paint-like experience for rendering high-quality images, enabling resolution-based rendering with infinite zoom. [Brief explanation of the drawing]

[0035] [Figure 1A] A block diagram illustrating an image processing system according to some embodiments of this disclosure is shown. [Figure 1B] Figure 1A shows a schematic diagram illustrating an image processing system configured for layered DF image conversion according to some embodiments of this disclosure. [Figure 1C] A schematic diagram illustrating the changes in a distance field and concentration profile of a certain shape, according to some embodiments of this disclosure, is shown. [Figure 1D-1] A schematic diagram illustrating various operations performed on a distance field to combine distance fields, according to some embodiments of this disclosure, is shown. [Figure 1D-2]A schematic diagram illustrating various operations performed on a distance field to combine distance fields, according to some embodiments of this disclosure, is shown. [Figure 1E] A flowchart of a method for image processing based on layered DF transformation of intensity images, according to some embodiments of this disclosure, is shown. [Figure 1F] Another flowchart shows a method for image processing based on the refinement of layered DF transformation of intensity images, according to some embodiments of the present disclosure. [Figure 2A] A block diagram is shown illustrating possible implementations of layered DF transformation of intensity images based on the concept of reversing the rendering pipeline, following several well-known solutions. [Figure 2B] A block diagram is shown illustrating possible implementations of layered DF transformation of intensity images based on the concept of reversing the rendering pipeline, following several well-known solutions. [Figure 2C] A flowchart of a method for image processing based on iterative optimization of error values, according to some embodiments of this disclosure, is shown. [Figure 2D] Another flowchart of a method for image processing based on the level of detail, according to some embodiments of the present disclosure, is shown. [Figure 2E] Figures 2C and 2D show schematic diagrams of layer-by-layer refinement achieved by the methods shown in some embodiments of the present disclosure. [Figure 3A] High-level block diagrams and image examples illustrating the conversion of intensity images to distance-field-based images according to some embodiments of this disclosure are shown. [Figure 3B] A high-level flowchart of a method for transforming a layered distance field-based image based on a transformation command, according to some embodiments of this disclosure, is shown. [Figure 3C] A flowchart of a detailed method for transforming a layered distance field-based image based on a transformation command, according to some embodiments of this disclosure, is shown. [Figure 4]A block diagram of the architecture of a computing system for image processing according to some embodiments of this disclosure is shown. [Figure 5] A flowchart of a method for determining resolution-independent image transformation according to some embodiments of this disclosure is shown. [Figure 6] A flowchart of a method for transforming an image using a brushstroke-based procedure, according to some embodiments of this disclosure, is shown. [Figure 7] A flowchart of a method for transforming an image using a set of primitives, according to some embodiments of this disclosure, is shown. [Figure 8] A block diagram of a computing system used to implement various embodiments for converting images disclosed herein into layered DF images, according to some embodiments of this disclosure, is shown. [Figure 9] This disclosure describes a method for curve fitting using an image processing system according to some embodiments of this disclosure. [Figure 10] This disclosure describes a computer-implemented method for performing continuous refinement operations on a set of data points for curve fitting, according to some embodiments of this disclosure. [Figure 11] The present disclosure describes a method for incrementally fitting a set of curves to a series of data points, according to some embodiments of this disclosure. [Figure 12] This disclosure describes a method for rendering an ordered set of primitives using a depth buffer, according to some embodiments of this disclosure. [Modes for carrying out the invention]

[0036] The following description includes numerous specific details for illustrative purposes to ensure that the disclosure is fully understood. However, it will be apparent to those skilled in the art that the disclosure can be implemented without these specific details. In other cases, the apparatus and methods are shown in block diagram form solely to avoid obscuring the disclosure.

[0037] As used herein and in the claims, the terms “for example,” “for instance,” and “such as,” as well as “comprising,” “having,” “including,” and other forms of these verbs, when used with an enumeration of one or more components or other items, should be interpreted as open-ended, meaning that the enumeration should not be considered to exclude any further components or items. The term “based on” means based at least partially. Furthermore, it should be understood that the style and terminology used herein are for illustrative purposes only and should not be considered restrictive. Any headings used herein are for convenience only and have no legal or restrictive effect.

[0038] The various embodiments disclosed herein provide a novel approach to digital drawing that enables the creation of detailed, textured artwork with artifact-free scalability and a small memory footprint.

[0039] Several embodiments disclosed herein provide methods, systems, and computer program products for image processing. The image processing disclosed in various embodiments offers several advantages over existing solutions known in the art. These various advantages include the ability to create graphic elements that exhibit pixel richness along with artifact-free scalability and small size in the SVG image format. The image processing methods and systems disclosed herein provide users such as artists, developers, media editors, engineers, and animators with the understanding that it is possible to provide seamless infinite zoom and infinite detail capabilities by working on an infinite canvas in terms of both spatial (x and y) range and scale (z), which cannot be obtained from any other system known in the art.

[0040] Furthermore, various embodiments provide efficient and effective solutions by reducing the memory storage requirements for image processing in memory-constrained and bandwidth-constrained environments. Another advantage of the methods and systems disclosed herein is that they enable high image adjustment capabilities by providing distance-based anti-aliasing rendering capabilities during image processing. In addition, the capabilities disclosed in the various embodiments provided herein also include support for rich primitives such as variable-width textured strokes with complex subpixel features, supporting real-time high-speed rendering of images by enabling immediate feedback during drawing and interactive canvas transformations.

[0041] Furthermore, some embodiments provide methods and systems for stylizing different appearances and levels of abstraction in images obtained by distance-field-based reconstruction and rendering, which can also be easily integrated with distance-field-based font rendering. Thus, various embodiments can provide intuitive content creation by unfolding images layer by layer, just like a real-world painting experience, and provide users such as content creators with an excellent drawing-like experience.

[0042] Some embodiments of distance-field-based rendering are based on the understanding that, in a purely procedural, resolution-independent manner, distance-field-based rendering can provide unprecedented capabilities to represent geometric shapes of arbitrary dimensions, consisting of both smooth and sharp features, by using simple and highly efficient Boolean, blending, offsetting, and arithmetic operations inherent to distance fields, achieving this with less memory, higher precision, higher quality, and lower computational load than other techniques known in the art. Thus, the modeled geometric shapes are realistic and can be derived or simulated.

[0043] The methods and systems disclosed herein are based on the application of distance field technology and layered methods and can be used in a number of applications including, but not limited to, medical applications, geographic information systems, game systems, entertainment systems, maps and navigation, compression technology, UI design, video encoding, animation creation, and video editing.

[0044] Figure 1A shows a schematic diagram 100 illustrating an image processing system 102 according to some embodiments of the present disclosure. The image processing system 102 is configured to receive an intensity image 101 and convert the received intensity image 101 into a reconstructed intensity image 103.

[0045] The intensity image 101 can be any image that has very detailed and high-resolution information about various channels, source images, geometric objects, fonts, etc. For example, the intensity image 101 may be a representation that includes a set of spatially coherent pixels, a distribution amplitude of colors such as in a JPEG or PNG image, a set of colors, and corresponding locations. The pixel representation may be intrinsically procedural, intrinsically discrete, or intrinsically continuous and have a set of spatially or temporally related values ​​associated with the image. The intensity image 101 may also be associated with a sampling mechanism or process that can generate a set of related values ​​that may be 2D or 3D.

[0046] Therefore, the intensity image 101 can be an image used in any of the following applications: video editing, UI design, animation, maps used in navigation, entertainment-related applications, digital TV, images used in transportation systems such as trains, images used in building equipment such as elevators, or any other application that has a related display screen for viewing the intensity image 101 or a reconstructed intensity image 103.

[0047] For the conversion of the intensity image 101, the image processing system 102 uses a layered DF conversion to convert the received intensity image 101 into a layered DF image layer by layer, and the layered DF image is then used to reconstruct the image for rendering. This image for rendering is the reconstructed intensity image 103, which may then be rendered on the output interface.

[0048] The reconstructed intensity image 103 has a more desirable level of detail and visual quality compared to the received intensity image 101, and its size is also compressed, resulting in an overall improvement in the quality of the received intensity image 101 and more desirable storage characteristics. This is achieved through the performance improvements provided by the layered DF transformation performed by the image processing system 102. Details of the image processing system 102 are shown in Figure 1B.

[0049] Figure 1B shows a detailed schematic diagram of the image processing system 102 of Figure 1A configured for layered DF conversion according to some embodiments of the present disclosure. The image processing system 102 may be implemented as a computing system. The computing system includes an input interface 107 configured to receive an intensity image 101. The input interface 107 may be configured to receive any input from input devices and / or technologies known in the art, including, but not limited to, keyboard input, touch-based input, mouse-based input, input submission by uploading an intensity image from a second computing system, and uploading an input image 101 from the same computing system.

[0050] Next, the received intensity image 101 is sent to at least one processor, such as the processor 109 shown in Figure 1B. The processor 109 is configured to execute computer executable instructions that can be stored in memory or a computer-readable storage medium. The computer executable instructions may be configured to perform various operations to convert the intensity image 101 into a reconstructed intensity image 103 based on layered DF calculations performed by a combination of a DF procedure (DFP) module 104 and a rule module 105. As shown in Figure 1B, the operations of the DFP module 104 and the rule module 105 can convert the intensity image 101 into a layered DF image 106. Each layer in the layered DF image 106 is an ordered layer in an ordered sequence of multiple layers. The ordered sequence of multiple layers divides the entire computational task of image conversion into smaller, manageable parts of the image processing, and as a result allows for stepwise refinement of the intensity image 101 by compressing the layered DF image 106 step by step or layer by layer. This makes the entire image processing task computationally feasible, reducing complexity, improving memory efficiency, and simultaneously rendering images at a higher level of detail by continuously increasing the level of detail with each layering step.

[0051] Each layer in the ordered sequence has a DFP 104a associated with it, provided by the DFP module 104. The DFP 104a allows the image to be represented by defining DF values ​​at multiple locations in the received intensity image 101. Then, a set of rules provided by the rule module 105 maps the DF values ​​to the corresponding intensity values ​​for each layer.

[0052] Therefore, each DFP 104a and its associated rule 105 are configured to directly model and approximate the luminance and chrominance channels of an image, such as a layered image in a sequence of layers. By first converting the original channels of the image, such as RGB and CMYK, to other forms and other color models such as LUV and LAB, faster, simpler, and better convergence is possible when matching a set of DFPs to the image. In one embodiment of the present invention, the image is 2D, and the image is viewed as a raw 2D distance field, and the inverse of the conventional distance field rendering pipeline (i.e., given a known 1D geometric shape, compute a 2D distance field, and map the 2D distance field to pixels) is determined (i.e., given a 2D distance field, discover the generation of an unknown 1D geometric shape, which combines to match the 2D distance field). This provides a significant advantage over other methods in terms of computational complexity by reducing the dimension of the problem from 2 to 1 when the image is 2D. One-dimensional problems can be solved by one-dimensional methods which have faster, simpler, and better convergence characteristics. Such convergence is achieved layer by layer for each layered DF image obtained in a given layer.

[0053] Layer-by-layer transformations in this manner are performed until the transformation termination criteria are met. The termination criteria may be defined by multiple factors, including but not limited to the level of detail, error value minimization, and target compression value.

[0054] Once the layered transformation in this manner is complete, the layered DF image 106 obtained in the final stage of the transformation is sent to the output interface 108. The output interface 108 may be configured to render the layered DF image 108. This rendered DF image 106 may be viewed as a reconstructed intensity image 103 using display or viewing technology associated with the output interface 108. For example, the output interface 108 can provide a display of the reconstructed intensity image 103 in map-based applications, entertainment applications, transportation system displays, factory automation environments, etc. In many applications, it is convenient, for some of the reasons mentioned above, to use distance field calculations to transform the received intensity image 101 into a layered DF image 106, which is then rendered as a reconstructed intensity image 103. Therefore, this disclosure makes extensive use of DF calculations for image processing. Next, in relation to Figure 1C, considerations and explanations related to DF calculations are described.

[0055] Figure 1C shows a schematic diagram illustrating DF calculation and density profile changes for a certain shape according to some embodiments of the present disclosure. Figure 1C shows an object 110, such as a shape or font, and its corresponding DF representation 111. For illustrative purposes, Figure 1C shows a 2D object. However, the concepts shown herein are also applicable to 3D objects without departing from the scope of the present disclosure. Figure 1C also shows the density profile changes for object 110 and its DF representation 111. Density profile 112 represents the density profile of object 110 along an axis defined along the AA' line, and density profile 113 represents the density profile of the DF representation of object 110 along an axis defined by the BB' line.

[0056] As is already known, the distance field of an object represents the distance from any point in space to the boundary of the object (object 110, etc.). In many applications, the distance chosen is usually the shortest distance from the point to the object's boundary. Signed distances can be used to distinguish between inside and outside an object (for example, a negative distance indicates that the point is outside the object, and a positive distance indicates that the point is inside the object).

[0057] Alternatively, or in addition to this, the distance from a point to the object's boundary can be replaced with a vector that, starting from this point and tracing back, reaches the nearest point on the object's boundary. The criterion used to measure distance can take many forms, such as the Euclidean distance form.

[0058] Therefore, various methods can be used to represent object 110 and its DF111, including detail-oriented distance fields, periodically sampled distance fields, procedural distance fields, analytical distance fields, distances stored in memory, and distances derived from geometric primitives such as strokes, field regions, and textured regions. A DF representation 111 based on a procedural distance field uses a DFP (DFP104, etc.) which includes a set of distance fields and a set of operations that act on the set of distance fields.

[0059] Therefore, each DFP104 executed by the DFP module distance field procedure directly models and approximates the luminance and chrominance channels of an image, to take just one example. By first converting the original channels of the image, such as RGB and CMYK, to other forms and other color models such as LUV and LAB, it enables faster, simpler, and better convergence when matching a set of distance field procedures with an image. In one embodiment of the present invention, the image is 2D, and the image is viewed as a raw 2D distance field, and the inverse of the conventional distance field rendering pipeline (i.e., given a known 1D geometric shape, compute a 2D distance field, and map the 2D distance field to pixels) is determined (i.e., given a 2D distance field, discover the generation of an unknown 1D geometric shape, which combines to match the 2D distance field). This offers a significant advantage over other methods by reducing the dimension of the problem from 2 to 1 when the image is 2D. One-dimensional problems can be solved by one-dimensional methods that are faster, simpler, and have better convergence characteristics.

[0060] Figures 1D-1 and 1D-2 show schematic diagrams illustrating various operations performed on a distance field for the joining of distance fields, according to some embodiments of the present disclosure.

[0061] As shown in Figures 1D-1 and 1D-2, various operations within a set of operations involve Boolean combinations of metric fields. For example, one operation may be a blend of metric fields 114, another a sum of metric fields 115, yet another a difference operation 116, and yet another a multiplication operation 117. All of these operations can be very simple calculations using metric fields.

[0062] For example, the distance field blend 114 shows a blend of Palatine's "a" and Times New Roman's "W" as two different fonts. This is difficult to compute using other image processing techniques, but is easy using a distance field. Similarly, other operations are also very easy. For example, the union operation is, Sum operation: A∪B => distance (A∪B)=max(distance (A), distance (B)) It may also be given as 115, and the difference operation is, Difference calculation: AB => Distance(AB) = min(Distance(A), -Distance(B)) It may also be given as 116, and the product operation is, Multiplication operation: A∩B => Distance(A∩B)=min(Distance(A), Distance(B)) It may also be given as 117.

[0063] Similarly, many other operations can be defined, including, but not limited to, distance field offsetting, distance field arithmetic joins, distance field unary operations, distance field conditional operations, distance field logical operations, and distance field masking operations. Distance field masking operations define an area of ​​an image that prevents changes to the area when performing operations on the image within that area. This area is defined by a set of distance field procedures, which are resolution-independent. The mask can be binary or continuous.

[0064] The operations within a set of operations are not limited to distance field operations, but also include image processing operations, computer graphics operations, and signal processing operations. Distance fields offer clear computational advantages when performing various operations such as blending, offsetting, and rendering, and when deriving complex shapes from Boolean operations. Other distance field procedures, such as DFP104, can take both positive and negative values ​​when evaluated, enabling a wide range of applications in digital drawing systems, such as modeling brushes, pens, and erasers. In addition, Boolean combinations of DFs can capture sharp, discontinuous features in an image, arithmetic (e.g., linear) combinations of DFs can capture smooth regions in an image, implicit blending can be used to smoothly join DFs, and offsets allow for easy feature fitting and sizing of candidate DFs. Furthermore, DF primitives can be adjusted using the Correspondences by Sensitivity to Movement (CSM) algorithm. Furthermore, procedural masking can enable more efficient representation of images, and several procedural DF-based templates can be provided for texture modeling. Another advantage of using DF procedures is that they enable large-scale parallelization of operations.

[0065] Furthermore, some embodiments provide various operations for DF calculations defined by a Backus-Naur notation (BNF) grammar 118. Thus, each layered DF image is associated with a BNF grammar 118, and the BNF grammar 118 for each structure of the layered DF image includes multiple operations, including at least one of the following: distance field operations, unary operator operations, compound distance field calculation operations, join operations, DF image reconstruction operations, DF image generation operations, blend operations, Porter-Duff composite blend operations, etc.

[0066] Distance field calculations include at least one of the following: distance map calculations, adaptive distance field calculations, analytical distance field calculations, procedural distance field calculations, in-memory distance-distance field calculations, stroke distance field calculations, in-domain distance field calculations, and unary operator distance field calculations.

[0067] A unary operation includes at least one of the following: CSM operation, offset operation, inset operation, and probability map calculation operation.

[0068] A composite distance field calculation operation includes at least one of the following: a distance field calculation operation and a combination operation.

[0069] A join operation includes at least one of the following: Boolean operations, implicit blend operations, linear join operations, and arithmetic join operations.

[0070] The DF image reconstruction operation includes at least one of the following: DF-map-intensity calculation and DF-map-intensity calculation using a mask.

[0071] The DF image generation operation includes at least one of the following: a reconstruction DF operation and a blending operation.

[0072] A blend operation includes at least one of the following: addition, subtraction, substitution, darkening, brightening, and Porter-Duff blend operation.

[0073] The Porter-Duff blend operation includes at least one of the following operations: src, over, and dest.

[0074] By using an appropriate combination of various calculations, a layered DF image of the corresponding layer can be realized. Next, by using continuous refinement of the layered DF image until the stopping condition is met, an intensity image with better quality, higher definition, and better resolution is reconstructed.

[0075] Figure 1E shows a flowchart of a method 119 for image processing based on layered DF transformation of an intensity image 101, according to some embodiments of the present disclosure.

[0076] This method 119 includes receiving an intensity image 101 in step 120. The intensity image 101 can be received from an image processing-related application at the interface 107 of the image processing system 102.

[0077] Next, in step 121, the received intensity image is converted into a layered DF image. The layered DF image includes an ordered sequence of multiple layers, each layer having a better level of refinement than the previous layer. Furthermore, as previously disclosed, each layer in the ordered sequence is associated with a DF procedure, such as DFP 104a, for determining DF values ​​at multiple locations in the received intensity image 101, and a set of rules, such as the rules given in the rule module 105, for mapping the DF values ​​to the intensity values ​​of each layer. Thus, the ordered sequence of multiple layers in the layered DF image may be associated with iterations of the DFP. Each layer may be associated with a different DFP, which can be selected from at least one of the following: a parameterized DF procedure, an analytical DF procedure, a sampled DF procedure, and an asymmetric stroke procedure associated with a spline curve.

[0078] The asymmetric stroke procedure includes an asymmetric stroke that defines rules for mapping the distance field of a spline curve to different gradients of intensity changes on different sides of the spline curve's central axis, such that the intensity of the spline curve changes in a direction perpendicular to its central axis. These intensities of the spline curve can change gradually and are different on different sides of the spline curve's central axis. In some embodiments, the spline curve may be shifted relative to the central axis. Using the asymmetric stroke procedure in layered transformations enables more intuitive and computationally converging solutions to image processing and transformation problems, which are similar to real-world painting processes.

[0079] Another type of computationally efficient DFP may include DF visualized with masked, stepped intensities, where the masked, stepped intensities include null intensity values ​​at specific locations in the layered DF image. As a result of the null intensity values, in a given layer of the layered DF image, the intensity of the previous layer at the corresponding location is not modified, thus reducing the computations performed for the entire DFP.

[0080] Some embodiments are based on the understanding that different layers of the layered DF image 106 represent elements corresponding to different resolutions of the received intensity image. Therefore, the layered DF transformation implemented by method 119 (and by the image processing system 102) provides a resolution-independent method for performing image transformation. This is because each subset of DFPs selected from the beginning of the ordered sequence of DFPs reconstructs the received intensity image 101 at different resolutions, but the range of reconstruction errors is the same. Reconstruction errors will be discussed later in relation to Figures 2C and 2D.

[0081] In addition, for each layered DF image, multiscale normalization may be performed to repeatedly estimate each layer of the layered DF image by changing the optimization parameters in different iterations.

[0082] Such continuous iterations for transforming the DF image between different layers are carried out until a stopping condition is met. The stopping condition will be explained later in relation to Figures 2C and 2D.

[0083] Finally, in step 122, the last layered DF image of the layer where the continuous transformation is stopped is used as the output layered DF image for rendering onto the output interface. This image is then rendered as the reconstructed intensity image 103. Thus, the layered DF image includes an intensity reconstruction function for combining the mapped intensities of each layer according to their order in the sequence of layers in order to reconstruct the received intensity image.

[0084] Another variation of Method 119 is shown in Figure 1F.

[0085] Figure 1F shows another flowchart of a method 123 for image processing based on layered DF transformation of intensity image 101, according to some embodiments of this disclosure. In the following flowcharts, intensity image 101 may be represented as IMG101 for explanatory purposes, but these are synonymous.

[0086] Method 123 includes, in step 126, assigning DFP104a to an initial configuration of IMG101. The initial configuration may be an empty set, and the assignment uses image IMG101 to determine an initial configuration of a set of DFP104a, which is a set of DFP104a determined by various means for establishing a starting point for refinement, which includes interpolation, regression, optimization, human editing, AI systems, search methods (random search, grid search, cell search, divide and conquer search, trial and error search, etc.) and combinations thereof. The starting point includes auxiliary data, which is stored with the set of DFP104a to enable reconstruction of image IMG101, and the auxiliary data includes compressed versions of the image, low-resolution versions of the image, image approximations, image interpolations, etc.

[0087] In some embodiments, the initial configuration may be determined by storing a small, low-resolution version of the image IMG101 together with a set of DFP104a, such that the image reconstruction begins with rendering IMG101 followed by rendering a set of DFP104a on top of IMG101.

[0088] In the next step 127, the DFP determined in step 125 is refined until the stopping criteria are met. Once the stopping criteria are met, method 123 is stopped, and the DFP 104a available in step 127 is stored as DFP 104a for a specific layer.

[0089] For DFP refinement, the image is decomposed into tiles, which are processed sequentially and in parallel. Furthermore, refinement functions by increasing the level of detail (stepwise, continuous, discrete steps, such as low-medium-high) for efficient computation and to obtain a progressive (coarse to fine) representation of the image. Therefore, refinement performs a one-time preprocessing to generate preprocessed image data, which includes edge maps, gradient maps, Laplacian maps, zero-crossing maps, filtered versions of the image, frequency-domain versions of the image, wavelet decomposition of the image, statistics, etc. Edge, gradient, Laplacian, and zero-crossing maps may be obtained at sub-pixel resolution.

[0090] Some embodiments provide the performance of a filtering operation applied once to the image before refinement, where the filtering operation performs sharpening, denoising, etc., to compensate for or complement the rendering features of a set of DFPs. Thus, refinement uses a smaller subsampled version of the image to improve refinement performance, and the image and the subsampled version of the image are periodically alternated during refinement. Refinement continues in the manner described above until a stopping criterion is met.

[0091] The stopping criteria may include, but are not limited to, exceeding the time limit, exceeding the maximum iteration count, reaching the tolerance level, or the occurrence of a loss of convergence across a series of refinement steps. The stopping criteria may be image-independent or image-dependent, dynamically updated (changes during refinement steps), static, controlled by visual inspection, human instruction, AI, procedure, or table.

[0092] If the stopping criteria are not met, method 123 proceeds to step 128.

[0093] In step 128, candidate regions (CRs) 124 are selected from IMG 101 or from the layered DF image, depending on whether it is the first or subsequent iteration of method 123. CR 124 is a region corresponding to each location in IMG 101 or either of the layered DF images. Thus, each of the multiple locations in IMG 101 or either of the layered DF images corresponds to each of the multiple candidate regions in the received intensity image IMG 101 or any layered DF image. Therefore, in step 129, a new set of distance field procedures (NDFPs) 125 are identified. NDFPs 125 are identified based on the entire set of DFPs 104a stored in the DF module 104. Finally, in step 130, the DFPs 104a and NDFPs 125 are combined and stored as an updated set of DFPs 104a associated with the corresponding candidate region CR 124. The DFP refinement process may then be repeated until the stopping conditions or criteria are met. After refinement is complete, the intensity image is reconstructed using the final layered DF image.

[0094] Some embodiments further include determining an error value associated with the difference between the intensity of the received intensity image IMG101 and the intensity of the intensity image reconstructed from the layered DF image. This error value is then compared with an error threshold, and the reconstructed image is updated based on this comparison. This will be further explained in relation to Figures 2A and 2B.

[0095] Figures 2A and 2B show block diagrams illustrating possible implementations of layered DF transformation of intensity images based on the concept of reversing the rendering pipeline, following several well-known solutions.

[0096] Figure 2A shows the rendering pipeline 200. The rendering pipeline 200 represents the essential elements required for rendering any intensity image, such as an intensity image 203. The rendering pipeline 200 may begin with a basic unit of viewing or rendering, such as a geometric shape 201, corresponding to any shape, object, part, location, region, or primitive in the entire image. First, a distance field 202 is obtained, corresponding to various points or locations in the geometric shape 201. Next, the distance field 202 is used to render the intensity image 203 using an image reconstruction function that specifically aims to render the image from data relating to the distance field of various points in the intensity image 203. For example, in the case of a 2D geometric shape, as previously disclosed, the distance field 202 includes a set of signed shortest distances from the corresponding boundary of the 2D geometric shape for various points in the space of the 2D geometric shape. Next, the sign and magnitude of the distances are appropriately converted to the intensity values ​​of the pixels. Therefore, some geometric shapes may be represented by their distance maps, which contain a map of distance values ​​periodically sampled for a given shape (such as geometric shape 201).

[0097] Another representation of the distance field can be derived using an adaptively sampled distance field (ADF), which allows for detail-oriented sampling of the distance field for the geometric shape 201. Therefore, if there are significant changes in the distance field resulting in excessive detail in the geometric shape 201, sampling is performed at a higher rate. Similarly, if the distance field changes gradually, such as in the low-detail areas of the geometric shape 201, or if high precision is not required, sampling is performed at a lower rate. The sampled distances obtained as a result of this detail-oriented sampling are then stored in an appropriate data structure, for example, by using a hierarchical representation of the data storage or by using on-demand sampling of the distance field. The intensity image 203 can then be reconstructed using a reconstruction function to reconstruct the distance field from the adaptively sampled points in the space of the geometric shape 201.

[0098] This rendering pipeline requires computationally expensive analytical filters for calculating sampled distances. Furthermore, the large storage requirements of the continuous hierarchical data structure result in high computational and storage costs for the entire image processing based on rendering pipeline 200.

[0099] Some embodiments are based on the recognition that an alternative method based on an inverted rendering pipeline 200 could be computationally superior in terms of storage and performance.

[0100] Figure 2B shows a block diagram of such a method based on an inverse rendering pipeline 204. The inverse rendering pipeline 204 starts with the original intensity image 206, from which distance field data 207 is obtained for each point or location in the original intensity image 206. Then, a set of visualization rules 208 is applied to this distance field data 207 (this application may be repeated) to obtain a reconstructed intensity image 209. The application of the visualization rules is performed repeatedly based on a visualization error parameter 210, which determines the stop condition for stopping the application of the visualization rules and rendering the reconstructed intensity image 209.

[0101] Furthermore, some embodiments are based on the understanding that if the processing tasks required for the intermediate steps of calculating the distance field data 207 and the visualization rules 208 can be divided into smaller parts of the processing tasks, the computation of the inverse rendering pipeline 204 can be made more efficient.

[0102] Therefore, Figure 2B shows the architecture of a reverse rendering pipeline 204 for more efficient work, in which the original intensity image 206 is transformed into a reconstructed intensity image 209 by using a layered distance field image 211, which includes a smaller portion of the processing task, including layer-wise distance field and visualization rule calculations, depending on minimizing the visualization error for each layer of processing. Furthermore, such layer-wise processing does not require a large hierarchical data structure for storage, as it acts directly on the various color or intensity channels of the original intensity image 206. For this purpose, such layer-wise processing is implemented by the image processing system 102 shown in Figures 1A to 1F, as previously described. As previously described, the image processing system 102 is configured to perform methods 119 and 123 to transform the intensity image into a layered DF image containing an ordered sequence of multiple layers. Therefore, different layers are associated with a set of DF procedures equivalent to DFP104a shown in Figure 1B and a set of rules such as rule 105 shown in Figure 1B, which is equivalent to visualization rule 208 shown in Figure 2A. Such layered DF image transformations are further used to implement the inverse rendering pipeline 204 shown in Figure 2A, which is subject to a visualization error 210 as a stopping condition, as further explained by method 212 shown in Figure 2C.

[0103] Figure 2C shows a method 212 for converting an intensity image to a layered DF image according to one embodiment. Method 212 may be performed by the image processing system 102 shown in Figure 1B.

[0104] Method 212 includes receiving an intensity image in step 213. As previously stated, the image processing system includes an input interface 107 configured to receive the intensity image 101.

[0105] Method 212 further includes converting the received intensity image into a layered DF image in step 214. Therefore, the received intensity image 101 is then sent to at least one processor, such as the processor 109 shown in Figure 1B. The processor 109 is configured to execute computer executable instructions that can be stored in memory or a computer-readable storage medium. The computer executable instructions may be configured to perform various operations to convert the received intensity image 101 into a layered distance field (DF) image containing an ordered sequence of multiple layers. Each layer in the ordered sequence containing a DF procedure, such as one of the DFP 104a shown in Figure 1B, defines DF values ​​at all locations in the received intensity image 101 and rules, such as the visualization rule 208 shown in Figure 2A, for mapping these DF values ​​to the intensity values ​​of this layer. To this end, the processor is configured to repeatedly convert the intensity image 101 into a layered DF image 106 until the error value is less than an error threshold, or until the error value reaches a value less than an error threshold. The error value is a layer-by-layer measure of the error or difference between the intensity image 101 and the intensity image reconstructed from the layered DF image 106 by combining the intensity values ​​of each level in their corresponding order. Error value minimization is achieved when such an error value condition is reached in a layer of transformation. Next, in step 215, the layered DF image is output for rendering by using the intensity values ​​in this layer of transformation.

[0106] As previously mentioned in relation to Figure 1B, the layered DF image may be rendered by the output interface 108.

[0107] Therefore, the error value can be considered as the reconstruction error or visualization error 210, and reaching a visualization error below the error threshold is a stopping condition for stopping the repeated transformation of the intensity image 101 to the layered DF image 106. Therefore, the visualization error 210 value can be calculated for that particular layer by calculating the sum of the distances between the intensities of corresponding pixel pairs at corresponding locations in the received intensity image 101 and the reconstructed intensity image. The intensity image is reconstructed from the layered DF image 106 of this layer using the image reconstruction function described above. For this purpose, the maximum distance between corresponding pixel pairs in the received intensity image and the reconstructed intensity image is identified from the sum of distances calculated for a given iteration. Next, in order to minimize the visualization error 210 to a value smaller than the error threshold, the current DFP for the current layer of the iteration is selected by subtracting the maximum distance between corresponding pixel pairs in the received intensity image and the reconstructed intensity image. This is called the greedy optimization method for searching the DFP for the current layer in the current iteration of method 212. Such greedy optimization is performed to select the current DFP, which reduces the maximum distance between corresponding pixel pairs and the visualization error of the entire reconstructed image 210 to the minimum possible value determined by the error threshold.

[0108] Some embodiments are based on the understanding that the received intensity image 101 can be divided into a set of candidate regions. The candidate regions correspond to various locations on the intensity image 101. For example, the candidate regions may be identified as curves, tiles, or grids, each of which centers on a corresponding location in the intensity image 101. Next, for each candidate region, a local current DFP is identified by subtracting the maximum distance between corresponding pixel pairs in the candidate region in the received intensity image and the reconstructed intensity image, in order to generate a set of local current DFPs. Then, this local current DFP is combined with a set of local current DFPs to generate the current DF procedure.

[0109] Therefore, a pair of candidate regions may be a union of non-overlapping tiles that cover the entire received intensity image 101. Furthermore, the current DFP includes a Boolean join of a pair of local current DFPs. In some embodiments, the division of the intensity image 101 into a pair of candidate regions is performed based on a greedy optimization methodology. The greedy optimization methodology includes setting the dimension of the candidate region divided for the current iteration of the greedy optimization to be greater than the dimension of the candidate region divided for the previous iteration of the greedy optimization.

[0110] Some embodiments are based on the understanding that the division of the intensity image 101 into candidate regions and the minimization of the visualization error value 210 are performed using greedy optimization to achieve iterative optimization at each layer level for the conversion of the intensity image 101 at each level to a layered DF image 106. Thus, this iterative optimization at each layer is performed to achieve continuous refinement of the image across different layers. Continuous refinement may be performed based on the level of detail (LOD) associated with the visualization error value 210. At each layer, the image obtained by rendering the current set of DFPs for that layer may hereafter be referred to as the working canvas WC.

[0111] In some embodiments, the LOD includes the current resolution of the layered DF image (e.g., layered DF image 106 or layered DF image 211). Thus, method 212 includes converting an intensity image (e.g., intensity image 101 or intensity image 206) into a layered DF image using iterative optimization, which includes generating a current DF procedure for the current layer, which reduces the error between the current resolution of the received intensity image and the current intensity image reconstructed from a current sequence of layers including the previous layer (reconstructed intensity image 209, etc.) and the current layer determined by the previous iteration in the iterative optimization. Thus, different iterations in the iterative optimization use different resolutions of the received intensity image, which vary from a low resolution to a high resolution in proportion to the exponent of the iteration. For example, layer 1 may have a resolution R1, layer 2 may have a resolution R2, and so on.

[0112] Therefore, in some embodiments, method 212 may be performed repeatedly to determine the sequence of multiple layers of the layered DF image 211 and to initialize and repeatedly update the sequence of DFPs based on the LOD until a termination condition (equivalent to the previously disclosed stop condition) is met, so that each iteration may include selecting candidate regions of the received intensity image, determining a new DFP for the selected candidate regions, and combining the new DFP with the sequence of DFPs identified in the previous iteration. This can be further illustrated in the flowchart shown in Figure 2D.

[0113] Figure 2D shows a flowchart of method 212a for converting an intensity image to a layered DF image based on LOD to search for a set of DFPs until a termination condition is met. For the consideration of method 212a, the intensity image is represented by IMG206, the current set of DFPs is represented by DFPs216, the rendered image of each layer is the working canvas WC217, the level of detail is LOD218, the candidate regions in one or more locations are CR219, and the new set of DFPs obtained by updating the current set are NDFPs220.

[0114] Method 212a includes, in step 221, assigning DFPs to an initial configuration of IMG206. The initial configuration may be an empty set of DFPs. Alternatively, the initial configuration may be a set of DFPs determined by various means for establishing a starting point for the refinement of IMG206. These means include interpolation, regression, optimization, human editing, AI systems, search methods (such as random search, grid search, cell search, divide and conquer search, trial and error search, etc.) and combinations thereof. After assigning the DFPs to an initial configuration, Method 212a includes initializing WC in step 222. WC may be initialized by rendering the initial configuration of a set of DFPs. Thus, rendering may include rendering a blank image, a blank image, and a constant color image (e.g., the average color of the image, equal to the common color of the image).

[0115] The starting point for initializing WC can be determined by one or more of the following: interpolation, regression, search, filtering, etc. The starting point includes auxiliary data, which is stored with a set of DFPs to enable image reconstruction. The auxiliary data may be a compressed version of the image, a low-resolution version of the image, an image approximation, or image interpolation. Therefore, assigning a set of DFPs216 to the initial configuration and initializing WC217 is frequently adjusted (i.e., designed in bulk) to determine an appropriate starting point for the refinement steps. For example, one particularly effective adjustment design is to assign the initial configuration to an empty set and initialize WC to a small low-resolution version of image S, which is stored with a set of DFPs216 so that image reconstruction begins with rendering S, followed by rendering a set of DFPs216 on top of S.

[0116] Furthermore, in step 223, LOD218 is also initialized to the first level. Then, in step 224, a set of DFPs216 is repeatedly refined until the stopping condition is met. As previously mentioned, the stopping condition includes determining the error value between the current layered DF image and the original intensity image, and checking whether the error value has fallen below the error threshold. This will be further explained in relation to steps 225-229, which define the actions performed in each iteration for the refinement of the DFPs.

[0117] Candidate regions are selected in step 225. The selection of candidate regions is guided by an error criterion, which is determined by using methods such as greedy algorithms, optimization algorithms, trial-and-error methods, grid search methods to ensure coverage, cell search methods to ensure coverage, random search methods, or divide-and-conquer search methods. Thus, candidate regions are determined by the maximum error by the locations of substantial differences in a pair of locations, using either a local error criterion or a global error criterion. The global error criterion is increased by allowing branching steps or by other means including trial-and-error, grid search, cell search, random search, divide-and-conquer search, or a combination thereof.

[0118] The error criteria or error values ​​may correspond to measuring the proximity between a rendering of a set of distance field procedures (i.e., WC217) and an image (i.e., IMG206) using various criteria, including L1, L2, perception, AI-based, etc. In some embodiments, each criterion may further include a penalty term to encourage simpler and smaller representations (e.g., fewer distance field procedures) that appear as optimized choices or criteria. After selecting candidate regions in this manner, in step 226, a new set of DFPs for the candidate regions is determined using the previously stored set of DFPs216, WC217, and the original image IMG206. Then, in step 227, the new DFPs, NDFPs220, are combined with the previous DFPs216 to form an updated set of DFPs216. Next, in step 228, WC217 is updated based on the updated DFPs, and in step 229, LOD218 is incremented to the next level. These steps are then repeated by checking the stop condition in step 224. When the stop condition is met, method 212a is stopped and the final stage WC217 is rendered to give the output image.

[0119] Therefore, LOD218 is incremented at each level in a coarse-to-dense (or dense-to-sparse) scheme for each level to generate different degrees of coarse-to-dense (or dense-to-sparse) progression between successive elassification layers in conversion methods 212 and 212a. In some embodiments, the increment of LOD218 depends on one or more processes, which include, but are not limited to, being constant at the first and subsequent levels (i.e., zero increment), allowing a single level of detail (flat progression, zero progression, etc.), procedural determination, table determination, visual control, human guidance, AI module, image-independent, image-dependent, manual and / or procedural setting to control and / or realize specific criteria of the nature of the image progression (e.g., smoothness) and the magnitude and quality of representation, dynamic updates, statistical updates, etc.

[0120] In some embodiments, the initial configuration determined in step 221 may be an empty set, and WC217 may be initialized by rendering an initial configuration of a set of DFPs216 onto the working canvas. Subsequently, LOD218 may be initialized to a coarse level. Furthermore, a stopping criterion may be specified as a criterion in which densification stops when the difference between the image and the working canvas is less than an error threshold. For densification, a one-time processing of image IMG206 is performed to generate preprocessed image data including an edge map determined at subpixel resolution. The error value for defining the stopping condition may be selected as an L2 error criterion for measuring the difference between image IMG206 and the working canvas WC217.

[0121] Furthermore, candidate regions are selected by finding a small set L of locations in the image with the largest L2 error (i.e., by using a greedy method to select the locations with the lowest degree of agreement between the image and the working canvas). Next, a new set S of DFPs, i.e., NDFPs220, is obtained, which is located near each location P in L that improves the local L2 error criterion in the region surrounding P when S is combined with a set of DFPs. Each distance field procedure in S is derived from a stroke along the edge determined in the preprocessed image data that reduces the difference between the working canvas and the image in the region surrounding P. The stroke is determined by the location and color attributes of the curve fitting. The precision of the stroke is determined by the level of detail, with coarser detail levels using a large, soft brush and denser detail levels using a smaller (finer), harder, and more complex brush, resulting in a progressive reconstruction of the image. The distance field procedures in S represent the stroke at an appropriate level of detail by using a detail-oriented, resolution-independent distance field.

[0122] Furthermore, the working canvas is updated from a set of distance field procedures DFPs216, and the working canvas is updated by rendering a set of distance field procedures onto the working canvas. Additionally, LOD218 is incremented to the next level, which may be the final level.

[0123] In this way, method 212a can be implemented in the embodiments. Therefore, in some embodiments, NDFPs220 are combined with DFPs216 using an arithmetic procedure. Also, as described in the embodiments above, candidate regions are selected based on the error between the intensity of the selected candidate region in the received intensity image and the corresponding region in the reconstructed intensity image obtained from WC217.

[0124] In some embodiments, dividing the received intensity image into candidate regions involves dividing it into a grid of rectangular regions. For example, the received intensity image may be divided into a 16x16 square grid using a grid search method. Then, as described in Method 212a and the Examples, points are selected within each region of the grid where the reconstruction error is greater than the error threshold. For example, a random point P is identified within each square where the L2 error is greater than a specified threshold. The neighborhoods around each P constitute a collective candidate region. Next, using these selected points, the optimization problem is solved for each selected point using regions that generate a DF procedure for each selected point such that the algebraic combination of DF procedures for the selected points reduces the reconstruction error within the region. Thus, in different iterations of the iterative optimization, the grid size may be increased with each subsequent iteration. The grid size may be increased with each refinement step to ensure convergence to the solution (e.g., 32x32, 64x64, 128x128, etc.). Therefore, for the combining step 227, we solve an optimization problem for each square and find the union of a new set of detail-oriented, resolution-dependent distance-field procedures (NDFPs220) located near P that minimize the local L2 error criterion in the region surrounding P when S is arithmetically combined with a set of distance-field procedures (DFPs216). If the distance fields of the underlying S capture details equivalent to the size of the square to which they belong, larger squares capture wider and lower-frequency image components, smaller squares capture finer and higher-frequency image components, resulting in a smooth, visually appealing, gradual image reconstruction that progresses from high-level structures to fine and complex details.

[0125] In some embodiments, the received intensity image may be preprocessed to extract sub-pixel resolution preprocessed image data, and the received intensity image may be converted into a layered DF image using the preprocessed image data. Therefore, the preprocessed image data may include one or a combination of edge maps, gradient maps, Laplacian maps, zero-crossing maps, filtered versions of the received intensity image, frequency domain versions of the received intensity image, wavelet decompositions of the received intensity image, and intensity statistics of the received intensity image.

[0126] In some embodiments, after the stopping condition is met, intensity calculations for intensity images reconstructed from layered DF images are estimated and stored in order to further reduce the error with the received intensity image.

[0127] Therefore, methods 212 and 212a are configured to achieve the objective of sequential image refinement using layer-by-layer transformations, where the convergence of the optimization problem—which optimizes computation, complexity, size, quality, and storage in the image transformation process—is dependent on the convergence of the solution. The convergence is achieved by using a greedy optimization method to minimize the error value and select candidate regions, while the reduced computational complexity is achieved by dividing the entire computation into parts that process each layer in order to select candidate regions of appropriate size and achieve sequential refinement. Such sequential refinement is illustrated by the embodiment shown in Figure 2E.

[0128] Figure 2E shows a schematic diagram of the layer-by-layer DF transformation 230, implemented by the method shown in Figures 2C and 2D, using an embodiment. The layer-by-layer transformation acts sequentially on layers 230a, 230b, ..., 230n. The output of each layer is used as input for the refinement of the subsequent layers, and this sequential refinement continues until a stopping condition is met. For example, the output image rendered in layer 230a is used as the input image for layer 230b, and so on.

[0129] At each layer, the transformation is performed according to methods 119 and 123 shown in Figures 1E and 1F, respectively, and methods 212 and 212a shown in Figures 2C and 2D, respectively. Processing at each layer is initiated for a series of refinements, progressing from top to bottom, i.e., from layer 230a to layer 230n, and from left to right, i.e., from stage or frame 230a to 230an. Each frame contains a new set of distance field procedures NDFPs 220 for the refinement step, and the working canvas WC after the refinement step shows the effect of the determined NDFPs. At each step, the convergence rate of the refinement and the smooth, gradual, and detail-oriented multi-resolution gradually increase.

[0130] Thus, continuous refinement may be configured to select a set of operations (e.g., Boolean, blend, arithmetic, etc.) that act on a set of distance fields located at a set of locations determined from the candidate region. The set of locations may be associated with the candidate region by either being near the candidate region or being centered on the candidate region. The distance fields can then take various forms, including strokes, filled regions, textured regions, gradients, edges, parameterized templates (all kinds for matching regions of an image, including noise (with different amplitudes, orientations, and frequencies), textures, etc.), detail-oriented distance fields, regularly sampled distance fields, procedural distance fields, analytical distance fields, and distances stored in memory.

[0131] In some embodiments, solutions to various operations on the distance field may be derived by error criteria using one of the following methods: a greedy method, an optimization method, or a greedy stepwise optimization method that avoids solving higher-dimensional problems and instead solves one dimension in one step (at each step of each layer) and combines the steps. Furthermore, continuous refinement may be carried out by one of the following methods: using a curve fitting method, using the location of the maximum error, and identifying the location of the distance field at a pair of locations using the location of the substantial difference, such as using a local error criterion, a global error criterion, or allowing a distributed step.

[0132] In some embodiments, curve fitting may be performed to resemble the intuitive brush strokes of a painting process. The brush strokes may be fixed width, variable width, fixed density, variable density, solid, textured, procedural, statistical, or dynamic, generating positive and negative values ​​that enable the modeling of both drawing and erasing on a surface, and the strokes may include a series of splines, polynomials, procedures, functions, points, curves, etc., along with relevant attributes (location, pressure, width, color, luminance value, chrominance value, opacity, contour, texture data, time data, etc.).

[0133] The sequential refinement of an image in the manner described in all previous embodiments provides a smooth, natural, and gradual viewing of the reconstructed image without unexpected jumps and visual artifacts. This is achieved by allowing the expressiveness of a new set of DFPs to increase with each refinement step. Expressiveness is determined by the number of operations, the number of distance fields, the number of locations, the complexity of the operations, etc. In fact, this is done to mimic the actions used by artists who often start painting with a rough sketch using a large, soft brush and then add detail with a series of smaller (i.e., finer) and harder brushes. As a result, fine brushstrokes are used only when necessary to refine the paint, and the rest remain coarse. At the same time, the gradual refinement and search of NDFPs provides a graceful (i.e., smooth, expected, predictable) gradual visual refinement of the image without visual artifacts and unexpected changes accompanying the progression, and can ensure that specified error tolerances are smoothly met where essential (e.g., medical applications). Thus, using the above method, extremely high-quality images can be obtained.

[0134] Some embodiments are based on the understanding that converting intensity images to layered DF images also provides the flexibility to transform the images to achieve desired performance and optimization feature levels, including but not limited to compression, texture, resolution, and detail. Therefore, the image processing system 102 may be configured to provide a function to specify desired performance feature levels.

[0135] Figure 3A shows an image processing system 102 configured to receive a conversion command 301 for converting an input image to a desired performance and optimized feature level. The image processing system 102 has already been described in a previous embodiment.

[0136] As shown in Figure 3A, the image processing system 102 includes an input interface 107, which may further have a first input interface 107a and a second input interface 107b. The first input interface 107a is configured to receive an input image, which may be a layered DF image 106. The input interface 107a may include an interface configured to receive external inputs from a user, an external source, a remote server, a database, etc. However, the input interface 107a may also be an internal interface, such as a system bus, that transfers inputs within the image processing system 102. The layered DF image 106 is received via either an internal or external source. The layered DF image 106 includes an ordered sequence of multiple layers, as shown in Figure 2E, and each layer of the layered DF image 106 includes a DF procedure that defines DF values ​​at all locations in the received intensity image and rules for mapping these DF values ​​to the intensity values ​​of that layer.

[0137] The input interface 107 also includes a second input interface 107b configured to receive a conversion command 301. The conversion command 301 may be received by the user from an external source, an internal source, etc. Therefore, the second input interface 107b may also be an internal input interface or an external input interface. The second input interface 107b may receive the conversion command 301, convert the layered DF image 106 based on the conversion command, and output the reconstructed intensity image 103 obtained from the converted layered DF image through the output interface 108.

[0138] Therefore, the conversion instruction 301 may specify one or a combination of the following: (1) a compression parameter that causes the converted layered DF image to include compression of the layered DF image; (2) a texture mapping parameter that causes the converted layered DF image to include a layered DF image with a modified texture; (3) a scaling instruction that causes the converted layered DF image to include a scaled version of the layered DF image; or (4) an algebraic instruction that causes the converted layered DF image to include the result of an algebraic operation on the layered DF image.

[0139] The compression parameters can define the compression level of the layered DF image 106. The layered DF image may be repeatedly transformed to generate a transformed layered DF image in order to compress the layered DF image until the compression level specified in the transformation command is satisfied. The desired compression level is satisfied when the compression level of the transformed layered DF image is less than or equal to the desired compression level. Therefore, in order to achieve the desired compression level, a set of DFPs for the layered DF image in a certain layer may define lossless compression parameters to generate a compressed layered DF image. Furthermore, in situations where the compression level is not satisfied, the compressed layered DF image of the upper level is removed from the consecutive layers until the compression level is satisfied in order to generate a transformed layered DF image.

[0140] In an alternative embodiment, to achieve a desired compression level, a set of DFPs for a layered DF image in a certain layer may define parameters for lossy compression to generate a compressed layered DF image. Also in these situations, if the compression level is not satisfied, the compressed layered DF images at higher levels are removed until the compression level is satisfied in order to generate a transformed layered DF image. Therefore, the compression of the DF procedure may be set to be resolution-independent. A resolution-independent DF procedure is added to the top or final layer of the layered DF image 106 to generate a transformed layered DF image.

[0141] Compressing layered DF images 106 in this manner is essential in many applications, such as modern electronic devices like microscopes, which generate massive amounts of data—gigabytes per second, terabytes per day—all in image form. Storing and processing this data presents significant challenges. In these applications, the compressed representation described in the previous paragraph is advantageous because it not only compresses the data but also achieves two other goals: 1) allowing direct manipulation of the representation (e.g., direct rendering, direct querying) without performing a decompression step, and 2) ensuring the representation adapts to the complexity of the data content. Such adaptive representations are also detail-oriented, overcoming the storage and processing challenges inherent in previously known solutions by concentrating more expressive power in the high-frequency range of the data and less expressive power in the low-frequency range.

[0142] The conversion command 301 may also specify texture mapping parameters. Correspondingly, the texture mapping parameters are stored in the memory of the image processing system 102 in the form of a set of texture DF procedures that define different textures for at least a portion of the layered DF image. Furthermore, the conversion command 301 indicates a desired texture, and this desired texture is used to select a texture DF procedure indicated by the desired texture from the set of texture DF procedures and to add the selected texture DF procedure to the top layer of the layered DF image. Alternatively, the DF procedure of the top layer of the layered DF image 106 may be replaced with the selected DF procedure. Therefore, the image processing system 102 may execute the method 302 shown in Figure 3B to convert the layered DF image 106 according to the desired conversion features specified in the conversion command 301.

[0143] Method 302 includes a layered distance field (DF) image in step 303, which includes an ordered sequence of multiple layers. The layered DF image may be received at a first input interface 107a. As previously stated, each layer of the layered DF image includes a DF procedure that defines the DF values ​​at all locations in the received intensity image, and rules for mapping these DF values ​​to the intensity values ​​of that layer. In addition, a transformation instruction 301 is also received at a second input interface 107b. The transformation instruction 301 specifies at least one of the following: a compression parameter, a texture mapping parameter, a scaling instruction, an algebraic instruction, or a combination thereof.

[0144] Furthermore, in step 304, the layered DF image is transformed based on the transformation command 301. The transformation is performed by selecting a set of DFPs along with transformation parameters and adding or replacing the top layer DFP of the layered DF image with the transformed DFP. Based on this transformation, in step 305, the transformed layered DF image may be output at the output interface 108.

[0145] Therefore, Figure 3C shows another flowchart of Method 306 for transforming a layered DF image into a transformed image based on a transformation command. For the purposes of Method 306, the intensity image is represented by IMG307, the current set of DFPs is represented by DFPs308, the rendered image of each layer is the working canvas WC309, the level of detail is LOD310, the candidate regions in one or more locations are CR312, and the new set of DFPs obtained by updating the current set is NDFPs313. These are as already described for Method 212a. However, Method 306 also includes data related to Schedule 311, which specifies a transformation command to select a transformation schedule 323 to transform a set of DFPs based on the transformation command. The resulting image can be seen as the transformed IMG324.

[0146] Method 306 includes assigning DFPs to an initial configuration in step 314. The setting of the initial configuration has already been described in relation to Figure 2D. Next, in step 315, WC309 is initialized. For example, if the initial configuration is an empty set, WC309 may also be a blank or black image. Then, LOD310 and schedule 311 are also set to their initial values. In some examples, LOD310 may vary from coarse to dense. Also, LOD310 may be initialized according to schedule 311.

[0147] Schedule 311 may be configured to control how a set of DFPs 308 avoids step-by-step and / or per-DF additional data storage by being refined step-by-step in a data-efficient manner independent of the distance field procedure. Thus, Schedule 311 allows for step-specific and DFP-specific data processing. Furthermore, Schedule 311 specifies instructions for processing the layered DF image in each layer, progressing from coarse to dense (or dense to coarse) in various increments to perform progressive compression that progresses from coarse to dense (or dense to coarse) to varying degrees in the image. Thus, in some embodiments, the processing of the layered DF image proceeds at a constant level of detail that results in a single LOD 310 and non-progressive (flat progressive, zero progressive) compression. The LOD 310 may be determined, for example, by any procedure, by a table, by visual inspection, by human guidance, by an AI component, may be image-independent or image-dependent, may be dynamically updated or statistically updated. Once the DFP308 is initialized, WC309, LOD310, and schedule311 are performed in this manner, and in step 317, the refinement process of the DFPs begins, which is carried out incrementally until a stopping criterion is met. The stopping criterion may, in one example, determine the size and quality level of the compressed image. The stopping criterion is determined based on a trade-off between the size and quality level of the compressed image. In some embodiments, the stopping criterion is set manually and / or procedurally to control and / or achieve specific criteria for the size and quality level of the compressed image.

[0148] The refinement of IMG307 and DFPs308 continues from steps 318 to 322 until the stopping criteria are met. In each refinement, DFPs308 are updated according to the conversion schedule 323 and the converted IMG324, which are used to identify the quality (or size or similar constraints) of the converted IMG324. This quality is then compared to the quality (or size or similar constraints) specified by the stopping criteria. When the desired quality set by the stopping criteria is reached, the refinement of the DFPs is stopped; otherwise, steps 318-322 continue.

[0149] In step 318, CR312 is selected from WC309, and a pair of NDFPs313 are selected from a pair of DFPs308 for WC309. Next, in 320, the DFPs308 and NDFPs313 are combined (by substitution or addition, etc.), and based on this update, WC309 is updated in step 321. Furthermore, in step 322, LOD310 is incremented according to the conversion schedule 311. For example, if the conversion schedule specifies a conversion instruction that specifies compression parameters, LOD310 for the next layer may be set to the next compression level. Then, steps 310-322 are repeated until a stop criterion is reached that defines the desired level of compression. The compression can be lossy, lossless, or otherwise.

[0150] Therefore, image transformations based on an updated set of DFPs are more accurate and provide more essential redundancy, which helps to self-fill any missing data in image compression because, in DFPs, the distance values ​​of the locations surrounding the distance values ​​or the distance values ​​surrounding the locations are missing.

[0151] Similarly, if the transformation instruction specifies texture mapping parameters, the stopping criterion may determine the desired level of texture in the final output image. For any image, the detail supported by an image is determined by its resolution. This detail provides information about the image's texture. Obtaining sufficient detail in every location may require extremely large images, particularly in terms of space within limited GPU memory. Some well-known image reconstruction techniques, such as structured vector graphics (SVG), provide edge detail at arbitrary magnifications, but are overly simplified, lack rich detail, are less general, and are more complex to encode, evaluate, and filter.

[0152] However, using schedule 311, which defines a set of DFP texture mapping information for each layer of the image, provides a resolution-independent distance-field procedure that significantly improves the quality, memory utilization, and performance characteristics of image synthesis. In addition, the single-instruction, multiple-data (SIMD) nature of the determined resolution-independent distance-field procedure allows method 306 to provide efficient rendering to the display and various processing tasks imposed on the texture map.

[0153] In some embodiments, the conversion command specifies a scaling command for generating a layered DF image based on a scaled version of the layered DF image associated with a scaling level. The scaling level includes an infinite upper limit.

[0154] In some embodiments, the transformation can be applied to any type of image, such as a font image or an image associated with a geographical area map.

[0155] Therefore, using methods 302 and 306, a desired level of transformation that satisfies any desired output characteristics, such as compression or texture, can be obtained for an image with efficient processing and fewer storage requirements. For this reason, the image processing system 102 may be implemented as a remote computing module that can be accessed from any client device that may be a light client, accessing the features of the image processing system 102 without excessively heavy processing.

[0156] Figure 4 shows such an architecture for the image processing system 102. The image processing system 102 is configured to perform all the operations described so far in the previous embodiments. The image processing system 102 may also be able to communicate with the user layer computing device 400, the image processing system 102 itself is in the application or service layer, and further interacts with the image processing server 402 in the interaction server layer. Layers such as the user layer, application layer and server layer are used to represent the isolation between different computing components in the image processing ecosystem, which can be facilitated by the image processing system 102.

[0157] The image processing system 102 may be accessed from a computing device 400, which may be any dedicated or general-purpose computing device. The computing device 400 can provide images such as the original intensity image 101 that need to be converted into a high-quality DF layered image by the image processing system 102. For example, the computing device 400 may be a navigation device, and the intensity image 101 may be an image of a map that will be displayed for a navigation service.

[0158] The image processing system 102 includes an input interface 107, which receives an intensity image 101 from a computing device 400 and converts it into a high-quality layered DF image by accessing an image processing application programming interface (API) 401. To this end, the processor 102a may be configured to generate API calls to the image processing API 401. The image processing API further encapsulates various functions that help convert the intensity image 101 into a high-quality layered DF image. Thus, the API calls may be directed to an image processing server 402 that stores various libraries corresponding to the various types of conversions that can be performed on the intensity image 101 to obtain a layered DF image containing an ordered sequence of multiple layers, as previously described. Each layer includes a DF procedure for defining DF values ​​at multiple locations in the received intensity image 101 and a set of rules for mapping the DF values ​​to the intensity values ​​of each layer. The DF procedure and the set of rules may be stored in a DFP library 403 on the image processing server. Each DF procedure is associated with a layer in a sequence of multiple layers of a layered DF image, and each DF procedure contains instructions to initialize and repeatedly update the DF procedure until a termination condition is met. API calls are configured to send image features to the image processing server 402 in the form of standard API messages and to retrieve the functions necessary for image transformation from the DFP library 403.

[0159] Subsequently, the processor 102a may be configured to perform layer-by-layer transformations, which depend on the LOD and stop functions, by executing the instructions or processes specified in the received functions, as described in the previous embodiment. For this reason, the processor 102a is configured to receive a response from the image processing server 402, which includes one or more functions necessary to obtain the layered DF image. The layered DF image may then be rendered on the output interface 108.

[0160] One or more of the above functions may include an intensity reconstruction function for reconstructing the received intensity image after transformation by combining the mapped intensities of each layer according to their order in the sequence of layers. One or more functions may also include a function for transforming the received intensity image by finding an error value associated with the difference between the received intensity image and the intensity image reconstructed from the layered DF image. Furthermore, the error value is compared with an error threshold, and the reconstructed intensity image is updated based on this comparison. The transformation may be performed repeatedly until the error between the received intensity image and the intensity image reconstructed from the layered DF image by combining the intensity values ​​of each level in their corresponding order is less than the error threshold. The error may be a visualization error that includes the sum of the distances between the intensities of corresponding pixel pairs at corresponding locations in the received intensity image and the reconstructed intensity image.

[0161] In some embodiments, the API call further includes a transform instruction, which is used to transform a layered DF image based on the transform instruction. The transform instruction includes one or a combination of the following: (1) a compression parameter, which causes the transformed layered DF image to include a compression of the layered DF image; (2) a texture mapping parameter, which causes the transformed layered DF image to include a layered DF image with a modified texture; (3) a scaling instruction, which causes the transformed layered DF image to include a scaled version of the layered DF image; and (4) an algebraic instruction, which causes the transformed layered DF image to include the result of an algebraic operation on the layered DF image.

[0162] The image processing server 402 stores various function libraries, including, but not limited to, an arithmetic library 404 containing functions for performing algebraic and other operational calculations on distance fields in layered DF images, a brushstroke library 405 containing functions for realizing DF procedures as a series of brushstrokes, a curve fitting library 406 containing curve fitting functions, a primitive library 407 for realizing a set of primitives, and a database 408 for storing images, mathematical constants and formulas, training data, and other arbitrary data.

[0163] The DFP library 403 may include functions for implementing various types of procedures, functions for selecting candidate regions, determining new DF procedures for the selected regions, iteratively updating DF procedures, and combining DF procedures. The DFP library 403 may also include adaptive DF procedures based on an adaptive sampling rate associated with the intensity image 101.

[0164] The brushstroke library 405 may include functions for implementing asymmetric stroke procedures associated with spline curves. For example, an API call includes pen type data for defining an asymmetric stroke procedure, and the pen type data includes at least pin stroke pen type options, pressure-sensitive pen type options, customizable pen type options, scalable pen type options, and textured pen type options, so that the corresponding associated functions are retrieved from the brushstroke library 405. The spline curve associated with the corresponding distance field, and the asymmetric stroke procedure, define rules for mapping the distance field of the spline curve to different gradients of intensity changes on different sides of the central axis of the spline curve, so that the intensity of the spline curve changes in a direction relative to its central axis. Such procedures are further illustrated in Figure 6.

[0165] The asymmetric stroke procedure may also be associated with a real-time curve fitting operation performed on a spline curve, which may be defined by a function stored in the curve fitting library 406. Therefore, the image processing server 402 provides the necessary functions from different libraries as a response to an API call to implement a method for converting an intensity image 101 into a layered DF image (such as the layered DF image 106 shown in Figure 1B). This method includes receiving the intensity image 101 and generating an API call to convert the received intensity image 101. The API call is sent to the image processing server 402 to convert the intensity image 101 into a layered distance field (DF) image. The layered DF image includes an ordered sequence of multiple layers. Each layer in the ordered sequence includes a DF procedure for determining DF values ​​at multiple locations in the received intensity image, and a set of rules for mapping the DF values ​​to the intensity values ​​of the respective layer. The image processing server 402 responds to API calls with a response message, which includes one or more functions obtained from different libraries to obtain a layered DF image, which is obtained by transforming an intensity image. Once the transformation is complete, the layered DF image may be rendered (in the form of a working canvas WC, or as a reconstructed intensity image if a stop condition has been reached).

[0166] Therefore, the image processing server 402 can provide various libraries to achieve high-quality, highly efficient, resolution-independent image transformation and distributed computing by separately storing large amounts of data on the image processing server 402 and accessing this data from the image processing system 102 via API calls as needed.

[0167] The image processing system 102 is also configured to provide resolution-independent channel-level image transformations, as shown in the method of Figure 5.

[0168] Figure 5 shows a flowchart of a method 500 for determining a resolution-independent transformation for an image 501 according to several embodiments. Method 500 includes decomposing the image 501 into a set of channels 502 in step 503.

[0169] Next, in step 504, channel 502 is used to configure a synthesis engine 506 for a pair of channels 502, and the parameters P1, P2, P3, P4, ..., Pn-1, Pn of the synthesis engine 506 determine a pair of resolution-independent DFPs such that rendering a pair of resolution-independent DFPs directly reconstructs a pair of channels 502. The synthesis engine may be a module in the processor 102a of the image processing system 102, specifically configured to execute instructions for realizing resolution-independent channels and their transformations, as described in this method 500.

[0170] Method 500 also includes training a synthesis engine 506 to configure various parameters in step 505. Thus, this configuration may further include identifying resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, Pn in step 508. Next, in step 509, DFPs corresponding to the resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, Pn are determined. Next, in step 510, the determined DFPs are rendered, and in step 511, the channels 502 are reconstructed from the rendered DFPs. Furthermore, this reconstruction process is provided to the training block 505, and the image may be reconstructed by a trained synthesis engine 507 based on the trained synthesis engine 506, as revealed in steps 512-515. Steps 512-515 are identical to steps 508-511, in which step 511 identifies resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, and Pn. Next, in step 513, DFPs corresponding to the resolution-independent parameters P1, P2, P3, P4, ..., Pn-1, and Pn are determined. Next, in step 514, the determined DFPs are rendered, and in step 515, Channel 502 is reconstructed from the rendered DFPs.

[0171] Therefore, training the synthesis engine 506 on a pair of channels 502 is performed to adjust the parameters of the synthesis engine 506 to determine a resolution-independent representation of image 501 by a pair of resolution-independent DFPs 509 and 513 on a pair of channels 502.

[0172] Therefore, configuring the synthesis engine 506 for a pair of channels 502 504 is done to determine a pair of resolution-independent DFPs 509 such that rendering a pair of resolution-independent DFPs reconstructs a pair of channels 503. This reconstruction may be done directly from the channels, for example, through a linear or nonlinear mapping from distance to intensity.

[0173] Therefore, the decomposition 503 determines the luminance channel of image 501, the chrominance channel of image 501, multiple chrominance channels, luminance and multiple chrominance channels, performs a color space conversion from the image's color space to a set of RGB (HSV, HSB, YUV, LAB, LUV, CMYK, etc.) channels, uses the unchanged image channels as a set of channels, and the above decomposition converts image 501 to the frequency domain of channels, the wavelet domain of channels, etc. Furthermore, the decomposition 503 may perform a one-time preprocessing on a set of channels 502 to generate preprocessed channel data, which includes edge maps, gradient maps, Laplacian maps, zero-crossing maps, filtered versions of the set of channels, statistics, etc., where the edges, gradients, Laplacian, and zero-crossing maps are obtained at sub-pixel resolution, etc. Decomposition 503 may further include applying a filtering operation to image 501 once before decomposing image 501 into a pair of channels 502, the filtering operation performing sharpening, denoising, etc. (to compensate for (or complement) a pair of resolution-independent rendering features of DFPs 509).

[0174] The synthesis engine 506 may be implemented as a procedure, a neural network, or a synthesis of procedures. The synthesis may be static or dynamic during training, and each procedure in the synthesis of procedures determines some elements (or a subset of elements) of a set of resolution-independent DFPs, and each procedure in the synthesis of procedures determines some elements (or a subset of elements) of a set of resolution-independent DFPs, the part being a set of operations (e.g., Boolean, blend, arithmetic) acting on a set of distance fields or a set of locations for a set of distance fields.

[0175] In some embodiments, configuring 504 and training 505 are performed separately for each channel of a set of channels, and this configuring (training) is performed once for a set of channels 502, using pre-processed channel data to perform its function.

[0176] Therefore, training 505 uses renderings of a set of resolution-independent DFPs 509 or images 501, or both, at a set of viewpoints to tune the parameters of the synthesis engine 506. To tune the parameters of the synthesis engine 506, viewpoint-dependent versions of the set of resolution-independent DFPs (and images) are used by methods such as method 123 or 212a, which are guided by an error criterion. The viewpoint determines scale, translation, rotation, etc., and a single viewpoint is used during training in a viewpoint-independent manner to tune the parameters of the synthesis engine 506. Various means are used to tune the parameters of the synthesis engine 506, including trial and error, range enumeration, grid search, cell search, random search, divide and conquer search, or a combination thereof. Therefore, to minimize the size of the set of resolution-independent DFPs, training may be inherently progressive (coarse to dense), inherently multi-resolution, inherently SIMD, etc.

[0177] Therefore, the objective of this training is to select a set of operations (e.g., Boolean, blend, arithmetic, etc.) and a subset of synthesis engine parameters that act on a set of distance fields, for each resolution-independent DFP in a set of resolution-independent DFPs509, such that the distance field takes various forms, including strokes, filled areas, textured areas, gradients, edges, and parameterized templates (all kinds for matching image regions, including noise (different amplitudes, orientations, and frequencies), textures, etc.). The distance field may be detail-oriented, periodically sampled, procedural, analytical, or stored in memory.

[0178] In one example, image 501 is decomposed into a set of channels in the LAB color space. Then, separate synthesis engines 506 for L and AB channels are trained 504 to constitute a synthesis engine 507 for the set of channels 502. The first layer of the synthesis engine 506 for L includes a set of fixed-size procedures L1, each procedure of L1 including a set of parameters that define detail-oriented, resolution-independent DFPs 509 capable of capturing the low-frequency components of image 501. The second layer of the synthesis engine 506 for L includes a set of fixed-size procedures L2, each procedure of L2 including a set of parameters that define detail-oriented, resolution-independent distance-field procedures 509 capable of capturing the frequency components of image 501 that are slightly higher than those modeled by L1. L2 receives the output of L1, and the procedures of L1 are combined in distance-field specific schemes (e.g., Boolean joins, blend joins) determined during training 505 to produce an output for L2 to consume. The layer synthesis continues in this manner until the final layer Ln is configured to capture the high-frequency components of image 501, and the output of Ln is a reconstruction of the L channel of image 501. A similar setup is repeated for the AB channel. The number of procedures for each layer L1, L2, ..., Ln can be adjusted to suit the needs of its application. More procedures per layer allow for a better overall reconstruction of image 501. The number of procedures per layer can also be determined dynamically during training rather than remaining static. While training the synthesis engine for one set of channels, the synthesis engine parameters for the L and AB channels are tuned individually. Correspondingly, an optimization method is used to adjust the L2 error criterion between image 501 and the rendering of a set of detail-oriented, resolution-independent distance-field procedures across multiple viewpoints.

[0179] In this way, the image processing system 102 may implement a method 500 for transforming an image based only on its channels.

[0180] Another possible image transformation is to mimic the painting process in the form of brushstrokes, as shown in method 600 in Figure 6. Figure 6 shows a flowchart of method 600 for transforming an image using a brushstroke-based procedure according to several embodiments. Functions for implementing method 600 may be provided in the brushstroke 405 library shown in Figure 4.

[0181] A brushstroke is a series of points with attributes, a path with attributes, a centerline with attributes, a sweep with attributes, a set of curves with attributes, or a set of distance field procedures with attributes. Attributes may include width, contour, distance field procedure, stamp, pressure, color, brightness value, chrominance value, opacity, noise level, texture data, time data, arc length data, etc., and a path includes movement to a command, lines, and curves. The centerline of a brushstroke is defined by a series of points, a path, or a set of curves, and the brushstroke generates positive and negative values ​​that allow for the modeling of both drawing and erasing on the working canvas.

[0182] Brushstroke-centric primitives are more easily converted into an equivalent set of brushstrokes (e.g., edges and gradients). These primitives are similar to brushstrokes and, when combined, produce the same type of primitive (in mathematical terms, a closed set of combinational operations, such as distance fields and Boolean operations that combine them). These primitives can take various forms, including edges, gradients, solid regions, textured regions, strokes, parameterized templates (all kinds for matching regions of an image, including noise (with different amplitudes, directions, and frequencies), textures, etc.), detail-oriented distance fields, regularly sampled distance fields, procedural distance fields, analytical distance fields, and distances stored in memory.

[0183] Method 600 includes an image 601 to a set of brush strokes BSTROKES616 such that rendering a set of brush strokes 616 progressively reconstructs the image. Method 600 includes assigning a set of DFPs 602 to an initial configuration in step 606. Next, in step 607, the working canvas WC is initialized. Furthermore, in step 608, the detail level LOD 604 is set to a first level.

[0184] Furthermore, in step 609, a set of DFPs 602 is iteratively refined based on images 601 and WC 603 until the stopping criteria are met. The iterative refinement is performed based on steps 610-614. In step 610, a candidate region CR 605 of image 601 is selected in a set of locations where the image is different from the working canvas WC 603. Furthermore, in step 611, a new set of distance field procedures NDFPs 617 is determined from the candidate region and a set of locations of detail levels defined by LOD 604. The NDFPs 617 define the brushstroke center primitives provided by BSTROKES 616. Next, in step 612, the NDFPs 617 are combined with DFPs 602 (by addition or substitution, etc.).

[0185] Subsequently, in step 613, WC603 is updated by rendering a set of distance field procedures DFPs602, and in step 614, LOD604 is incremented to the next level.

[0186] Furthermore, this iteration returns to step 609, where the stopping condition is checked. If the stopping condition is met, the brushstroke-centered primitives BSTROKES616 of a set of DFPs602 are processed to determine a set of brushstrokes, and rendering the set of brushstrokes progressively reconstructs image 601.

[0187] In step 615, in order to convert image 601 into a set of brush strokes, image 601 is first converted into a set S constrained by a distance field procedure, and then a set of brush strokes is determined from S. This indirect two-step process accesses more information (i.e., is information-rich) and is therefore more efficient and simpler than prior art methods that infer a set of brush strokes directly from the pixels of the image.

[0188] Brushstrokes possess several properties that make them an ideal candidate for image representation: 1) Because brushstrokes are inherently 1D, they render faster than more common and complex 2D forms such as triangles; 2) The points defining a brushstroke are strongly bound together, making them more effectively compressed; 3) The style can be changed from one form (e.g., pencil) to another (e.g., paintbrush) to achieve different looks and effects for various applications such as filmmaking and games; 4) Brushstrokes can be easily converted into distance fields, which are inherently SIMD, and then efficiently processed (e.g., rendered, selected, transformed, stylized) on the GPU by leveraging the structure of the fragment SIMD pipeline on the GPU; and 5) As determined by the present invention, they can be physically drawn on a canvas for artistic creation.

[0189] Next, various forms of processing may be performed on these brush strokes, including but not limited to edge following, smoothing, curve fitting, attribute fitting, region filling, template fitting, distance field fitting, image sampling, optimization, greedy search, greedy stepwise optimization, trial-and-error search, random search, divide-and-conquer search, and the like.

[0190] Such processing of brush strokes, and using brush strokes for rendering images by the image processing system 102, provides a more intuitive image transformation method that closely resembles the real-world painting process.

[0191] Another application of the image processing system 102 is to convert an image into a set of primitives, as shown in Figure 7.

[0192] Figure 7 shows a method 700 for converting image 701 into a set of primitive SOPs 703, where each primitive in the set contains a set of distance field procedures, and rendering the set of primitives reconstructs image 701. The primitives may include brush strokes, solid areas, textured areas, edges, gradients, etc.

[0193] Method 700 includes setting the working canvas WC702 to an initial state in step 706. The initial state may be a blank (i.e., empty) state, a constant color state (e.g., a common color of the image, equal to the average color of the image), etc. The setting may include various means such as interpolation, regression, optimization, search, filtering, etc., to determine the starting point of the working canvas, the starting point including auxiliary data. The auxiliary data is stored with a set of primitive SOPs703 to enable the reconstruction of image 701. The auxiliary data may be a compressed version of image 701, a low-resolution version of image 701, an approximation of image 701, an interpolation of image 701, etc.

[0194] Next, in step 707, the initial SOPs703 are determined. In some examples, when the working canvas is set to blank, a specific primitive is selected to approximate each pixel in image 701, and that specific primitive is added to SOPs703. Image 701 is decomposed into a set of regions that cover image 701, and a specific set of primitives is selected to approximate each region within the set of regions in order to determine the initial SOPs703, and that specific set of primitives is further added to SOPs703. Add to it.

[0195] In another example, if the working canvas WC702 is set to a non-blank state (for example, set to a lower-resolution version of the image as the starting point), a difference image is generated as the difference between image 701 and WC702. Next, in order to determine SOPs703, a specific primitive is selected to approximate each pixel in the difference image, and the specific primitive is added to a set of primitives, so that the difference image is decomposed into a set of regions that cover the difference image. Next, a specific set of primitives is selected to approximate each region within the set of regions, and the specific set of primitives is added to a set of primitives SOPs703.

[0196] Next, in step 708, SOPs703 are rendered onto the working canvas. Furthermore, from step 709, SOPs703 are refined until the stopping condition is met. Refinement includes, in step 710, selecting a candidate region CR705 in a set of locations where image 701 resembles the working canvas WC702. Next, in step 711, a set of primitives SSOP704 are discovered from CR705 and the set of locations mentioned above.

[0197] Subsequently, in step 712, a subset of primitives is merged to reduce the size of a set of primitives SOPs703. The merge combines sets of distance-field procedures corresponding to the primitives in the subset of primitives SSOPs704. The merge leverages the computational advantages of distance fields to perform various operations that are complex, difficult, slow, and in some cases impossible with other representations, including blend, Boolean, offset, sweep, and morph. The merge blends sets of distance-field procedures using, for example, Boolean operations, blend operations, arithmetic operations, and conditional operations. The merge further uses a search method to combine sets of distance-field procedures, which is guided by an error criterion, ensuring that the error criterion does not exceed a specified error tolerance during the merge. The search method may be a greedy method, an optimization method, a greedy stepwise optimization method, a curve fitting method, a trial-and-error method, a range enumeration method, a grid-based method, a cell-based method, a random method, a divide-and-conquer method, or a method for using a combination of these methods. The search method minimizes the size of a set of primitives while maintaining a specified error tolerance during merging.

[0198] Furthermore, in step 713, the working canvas WC702 is updated from SOPs703, and the rendering of SOPs703 reconstructs image 701.

[0199] In this manner, the above iteration is repeated until method 700 stops when the stopping condition is met. Method 700 starts with a large set of primitives S that have a high degree of agreement with image 701, and refines S into smaller sets while maintaining (i.e., not exceeding) a predetermined error tolerance (stopping condition) between S and image 701 until the stopping criterion is met.

[0200] The sequential refinement provided by Method 700 is guided by an error criterion, employing various methods such as greedy, optimization, trial and error, grid search to ensure coverage, cell search to ensure coverage, random search, divide and conquer search, etc. The error criterion may be a local error criterion, a global error criterion that allows branching steps to increase the global error criterion using simulated annealing temperature cooling, etc. The CR705 in iterative refinement is determined by a pair of locations, which are the locations with the least error, the locations with small differences, the locations with high similarity, or a combination thereof. Sequential refinement maps a set of points to a new set of points by applying refinement rules. The refinement rules typically increase the number of points and produce a smoother sequence. By being adaptive, the refinement rules can change their behavior based on, among other things, the properties of the points and their neighborhoods. For example, a refinement rule may insert a new point between consecutive points in a set of data points only if the distance between these consecutive points exceeds a threshold.

[0201] Method 700 may be implemented by the image processing system 102 to provide a highly efficient image transformation method based on a set of primitives derived from an image.

[0202] The various embodiments described above can also be realized using the computing system shown in Figure 8.

[0203] Figure 8 shows a block diagram of a computing system 800 used to implement various embodiments disclosed herein for converting an image into a layered DF image. Figure 8 will be described in relation to Figures 1A to 7. The computing system 800 may correspond to an image processing system 102 or a computing device 400. The computing system 800 may have several interfaces connecting the computing system 800 to one or more image rendering devices 813. For example, a network interface controller (NIC) 808 may be adapted to connect the computing system 800 to a network 812 via a bus 807. Through the network 812, either wirelessly or wired, the computing system 800 can receive an input intensity image 815. In addition, additional information associated with the input intensity image 815 can be received via an input interface 816. The input interface 816 can connect the computing system 800 to a keyboard and / or pointing device. As an example, the pointing device may include, among other things, a mouse, trackball, touchpad, joystick, pointing stick, stylus, or touchscreen. The input interface 816 may be connected to one or more sensors 801 that capture the input image 815. For this purpose, the one or more sensors may include a camera.

[0204] The computing system 800 includes a processor 802 configured to execute stored instructions, and a memory 810 that stores instructions executable by the processor 802. The processor 802 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 810 may include random-access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system. Furthermore, the computing system 800 may include a storage device 803 adapted to store different modules that store executable instructions for the processor 802, such as an image processing system 102. The storage device 803 may be implemented using a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof.

[0205] The storage device 803 is configured to store a set of DFPs 804 and their corresponding rules. In addition, the storage device 803 can store the working canvas WC at various stages of processing. The storage device 803 can also store the layered DF image IMG 806 at all layers of processing. In addition, the storage module 803 can also store data on the level of detail, a set of API-related data on API calls generated to the image processing API, error value-related data, and so on. Thus, the storage module 803 may be configured to allow the processor 802 to receive the input image 815 and perform all the functions of the image processing system 102 disclosed in a previous embodiment. As a result, the input image 815 can be converted into a layered DF image, and then the layered DF image can be reconstructed and rendered via the output interface 809. The output interface 809 may be configured to connect the computing system 800 to an image rendering device 813. For example, the image rendering device 813 includes, among other things, a computer monitor, a television, a projector, or a mobile device.

[0206] The computing system 800 may also be configured to implement other features, which will be described below in relation to Figures 9, 10, 11, and 12.

[0207] Figure 9 shows a curve fitting method 900 using an image processing system 102 according to one embodiment of the present disclosure.

[0208] In some embodiments, Method 900 may be a computer-implemented method for fitting a set of curves 945 to a set of data points 910. Method 900 includes, in step 950, filtering a set of data points DPs 910 to obtain a filtered set of data points FILTERED DPs 915. Furthermore, in step 955, Method 900 includes discovering a first set of stationary points STAT.PTs:1 920 from the filtered set of data points FILTERED DPs 915. Furthermore, Method 900 includes, in step 960, fitting a first set of curves CURVES:1 925 between consecutive elements of the first set of stationary points STAT.PTs:1 920, and in step 965, tessellating the first set of curves CURVES:1 925 to become a tessellated set of data points TESS.DPs 930. Next, in step 970, the tessellated set of data points TESS.DPs930 is filtered to find a second set of stationary points FILTERED TESS.DPs935 from the filtered and tessellated set of data points. Furthermore, in step 975, a second set of stationary points STAT.PTs:2 940 from FILTERED TESS.DPs935 is determined. Then, in step S980, these curves are fitted between consecutive elements of the second set of stationary points STAT.PTs:2 940 to obtain a second set of curves CURVEs:2 945. Thus, the second set of curves CURVEs:2 945 determines a set of curves that fit the second set of data points DPs910.

[0209] Each data point contains a set of attributes, which may include location, number, value, measurement, order, sequence number, width, contour, distance field procedure, stamp, pressure, color, luminance value, chrominance value, opacity, noise level, texture data, time data, arc length data, etc., and may be of any dimension (e.g., 2D, 3D). The set of attributes includes independent and dependent attributes, and a set of curves determines the relationship between the independent and dependent attributes of the set of data points.

[0210] Of a set of data points, those that are required to remain invariant (i.e., their attributes remain fixed throughout the fitting process) are called stationary points. Any point in a set of data points can be labeled as stationary before the fitting method begins (an example of this would be the start and end data points of a set of data points that identify an independent sequence, such as independent curves in a digital drawing).

[0211] One of the data points in a set of data points is in the fitting method A procedure can dynamically classify a data point as stationary, which detects features (performs feature detection), including corners, edges, maximal curvature points, inflection points, common feature templates, and combinations thereof.

[0212] Curves may include Bézier curves of various degrees (e.g., quadratic, cubic, quartic, etc.), splines of various degrees (e.g., quadratic, cubic, quartic, etc.), polynomials of various degrees, mathematical functions, procedures with parameters that give a set of input values ​​and produce a set of output values, a set of distance field procedures, combinations of these forms with and without constraints to achieve specific goals such as C1 and C2 continuity, vexels, and segmental variations of these forms to allow long sequences of data points to fit appropriately. A Bexel is a cubic Bézier curve C whose first and second off-curve points are subject to the following constraints: the perpendicular projection from the first off-curve point of C onto the line L between the first endpoint p0 and the second endpoint p1 of C intersects L at a distance of one-third the length of L from p0, and the perpendicular projection from the second off-curve point of C onto the line L intersects L at a distance of two-thirds the length of L from p0.

[0213] Tessellation approximates each curve in a set of curves with another primitive, typically of a lower order or less complexity. For example, in the tessellation of a cubic Bézier curve (of order 3) into a sequence of line segments (of order 1) that closely approximate the cubic Bézier curve, the primitives can include curves, lines, points, procedures, functions, and so on.

[0214] Most geometric smoothing methods have several problems. Perhaps the most serious is the shrinkage problem. When a geometric smoothing method is applied to a shape many times, the shape eventually collapses to a point. Perhaps the most popular geometric smoothing technique is Gaussian smoothing, which is performed by convolution of a set of data points P that define a curve C by a Gaussian filter G. The Gaussian method is known to cause shrinkage. In one embodiment of the present invention, a two-pass geometric smoothing method is used to reduce the shrinkage of a curve C defined by a set of data points P, where pass 1 calculates the smoothed curve G(C) and residual CG(C), where G(C) is the Gaussian filtered curve C, and pass 2 calculates G(C) + G(CG(C)) to obtain a smoothed set of data points with reduced shrinkage.

[0215] A corner can be defined as the intersection of two edges. Alternatively, a corner can be defined as a point where two different principal edge directions exist in the local neighborhood of that point. One approach to corner detection is as follows:

[0216] The formula for the dot product is: dot(A,B)=||A|| ||B||cos(angle).

[0217] A and B are normalized vectors around each candidate point p, excluding points near the start and end points of each independent sequence in a set of data points. Furthermore, a normalized vector around each candidate point p is obtained. To do this, the cos(angle) around candidate point p is calculated using the dot product formula. The result (as either true or false) is obtained by comparing this angle with a specified tolerance. Several other methods of corner detection include Harris corner detection, Shi-Tomasi corner detection, robust corner detection, and pre-trained neural networks.

[0218] In some embodiments, a set of data points is filtered, a non-shrinking smoothing operation is performed to obtain a filtered set of data points, a smoothing operation is performed, a convolution operation is performed, a noise reduction operation is performed, a moving average operation is performed, a corner-preserving smoothing operation is performed, a continuous refinement operation is performed, and a combination of these operations is performed.

[0219] In some embodiments, a continuous elaboration operation maps a set of data points to a new set of data points by applying an elaboration rule, the new set of data points replaces the set of data points, the elaboration rule resizes (e.g., increases) the set of data points, and connects the points within the set of data points to produce a smoother sequence. The elaboration rule may be static or adaptive. An adaptive elaboration rule is based on a feature of a point and its neighborhood. This feature may be distance, a difference between specific attributes of a set of attributes of a set of data points, etc.

[0220] Figure 10 shows a computer-implemented method 1000 for performing a continuous refinement operation on a set of data points for curve fitting, according to some embodiments of the present disclosure.

[0221] Method 1000 includes, in step 1050, filtering a set of data points DPs 1010 to obtain a filtered set of data points FILTERED DPs 1015. Method 1000 further includes, in step 1055, performing a continuous refinement operation on the filtered set of data points FILTERED DPs 1020 to obtain a set of continuously filtered data points R.FILTERED.DPs 1020. Next, Method 1000 includes, in step 1060, filtering a set of continuously filtered data points R.FILTERED.DPs 1020 to obtain a set of filtered and continuously refined filtered data points FRFILTERED.DPs 1025. Furthermore, in step 1065, a set of stationary points STAT.PTs 1030 is found from the filtered and continuously refined filtered data points FRFILTERED.DPs 1025. Finally, in step 1070, a set of curves CURVEs1035 is fitted between consecutive elements of a set of stationary points STAT.PTs1030.

[0222] In some embodiments, discovering a first (or second) set of stationary points from a filtered set of data points (a filtered, tessellated set of data points) involves performing a procedure to find the first (or second) set of stationary points, which detects features such as corners, edges, maximal curvature points, inflection points, common feature templates, and combinations thereof. This discovery identifies data points from the filtered set of data points (a filtered, tessellated set of data points) that are labeled as stationary in order to find the first (or second) set of stationary points.

[0223] In some embodiments, tessellating a first set of curves involves performing degree reduction (complexity reduction) on the first set of curves to obtain a tessellated set of data points, decomposing the first set of curves into a sequence of approximate line segments (approximate lower-order curves, approximate data points) to obtain a tessellated set of data points, and performing recursive subdivision on the first set of curves to obtain a tessellated set of data points. Degree reduction (decomposition, recursive subdivision) includes an error tolerance (the error is measured as the difference between the tessellation and the first set of curves), and the tessellation continues until the error tolerance is satisfied (typically until the tessellation closely approximates the first set of curves).

[0224] In some embodiments, fitting a curve to a first (second) set of stationary points between consecutive elements of the first (second) set of points involves first identifying a subset of filtered data points (filtered, tessellated data points) between each pair of consecutive elements of the first (second) set of stationary points, and then performing a fitting operation on the subset of filtered data points (filtered, tessellated data points) to obtain a fitted curve to add to the first (second) set of points.

[0225] The fitting operation may include linear regression, nonlinear regression, least squares, optimization, robust regression, and pre-trained neural network regression. First, a distance field is obtained from a subset of filtered data points (a filtered, tessellated set of data points). Second, a pair of candidate curves are transformed using the distance field to fit the subset of filtered data points (a filtered, tessellated set of data points). Third, the transformed pair of candidate curves are assigned to the fitted curve. Transforming the pair of candidate curves involves obtaining an error criterion by sampling the distance field. An error criterion may be used to move and reshape a set of candidate curves, and obtaining a distance field from a filtered set of data points (a filtered, tessellated set of data points) involves first constructing an approximation to a subset of the filtered set of data points (a filtered, tessellated set of data points), then obtaining a distance field from the approximation, where the approximation is a polyline connecting the subset of the filtered set of data points (a filtered, tessellated set of data points), where the approximation is a polyline connecting the continuous refinements of the subset of the filtered set of data points (a filtered, tessellated set of data points), and where the approximation is a set of curves connecting the subset of the filtered set of data points (a filtered, tessellated set of data points).

[0226] In some embodiments, filtering a tessellated set of data points involves performing an operation to obtain a filtered, tessellated set of data points. The operation may include smoothing, convolution, non-shrinking smoothing, noise reduction, moving average, corner-preserving smoothing, continuous refinement, or a combination of these operations.

[0227] In some embodiments, viewpoint-dependent fitting defines a subset of these steps (e.g., filtering and tessellation) that are viewpoint-dependent. Viewpoint-dependent fitting can offer distinct advantages for certain applications (e.g., faster performance and higher quality). The viewpoint determines the viewing characteristics (e.g., scale, rotation, translation, resolution, etc.) of how a set of data points is viewed. Examples of viewpoint dependency include: During tessellation, the viewpoint scale (i.e., scaling level) can be used to mitigate the fineness of the tessellation. As the viewpoint zooms in on a set of data points, the tessellation becomes finer, and as the viewpoint zooms out on a set of data points, the tessellation becomes coarser. Similarly, when filtering, the viewpoint scale can be used to mitigate how the filtering is performed; as the viewpoint zooms in on a set of data points, the filtering uses fewer points from the set of data points, and as the viewpoint zooms out on a set of data points, the filtering uses more points from the set of data points.

[0228] For example, in one embodiment, the type of curve may be a bexel, and each data point contains a tuple of three elements: x, y, and pen pressure for a digital drawing application. Filtering of a set of data points may include a non-shrinking smoothing filter. Furthermore, for filtering, a first set of stationary points of the filtered set of data points is identified to identify corners as stationary points using the dot product formula. Next, the first set of curves is fitted between the continuous elements of the first set of stationary points. For fitting, a bexel can be fitted directly to the data points between continuous elements in a single step by using its definition (no iterative regression method is required for bexels). Furthermore, tessellation is performed to fit the first set of curves to a tessellated set of data points by recursive subdivision of the bexel into line segments. The tessellated set of data points is filtered using a Gaussian smoothing filter. Furthermore, a second set of stationary points of the filtered, tessellated set of data points is identified. This is done by identifying corners as stationary points using the dot product formula. Furthermore, a second set of curves is fitted between the continuous elements of the second set of stationary points. For this purpose, a bexel can be fitted directly to the data points between continuous elements in a single step by using its definition (iterative regression is not required for bexels).

[0229] In some embodiments, a computer-implemented method is provided for incrementally fitting a set of curves to a series of data points.

[0230] Figure 11 shows a method 1100 for incrementally fitting such a set of curves CURVEs1105 to a series of data points DPs1110. Method 1100 may include setting the current steady point CUR.STAT.PT1115 to an initial value in step 1160. Then, in step 1162, the batch as a series of data points DPs1110 is processed one at a time until a stopping criterion is met. This batch contains a batch subset BATCH.SUBSET1120 of the series of data points. The batch processing further involves, in step 1164, discovering a pair of stationary points STAT.PTs1125 from a batch subset BATCH SUBSET1120 of a set of data points DPs1110 and the current stationary point CUR.STAT.PT1115; in step 1166, updating the current stationary point CUR.STAT.PT1115 from the pair of stationary points STAT.PTs1125; and in step 1168, fitting a new pair of curves NEW CURVEs1130 to each subset SUBSET.DPs1135 of a set of data points identified by consecutive pairs of elements of the pair of stationary points STAT.PTs1125.Fitting a new set of curves further involves, in step 1170, filtering a subset of data points SUBSET.DPs1135; in step 1172, fitting the first set of curves CURVEs:1 1140 to the filtered subset of data points FILTERED.SUBSET.DPs 1145; in step 1174, tessellating the first set of curves CURVEs:1 1140 to a tessellated set of data points TESS.DPs1150; in step 1176, filtering the tessellated set of data points TESS.DPs1150 to obtain filtered, tessellated data points FILTERED TESS.DPs1155; and in step 1178, fitting a new set of curves NEW CURVEs1130 to a filtered, tessellated set of data points FILTERED This involves applying the data to TESS.DPs1155 and, in step 1180, adding a new set of curves, NEW CURVEs1130, to a set of curves, CURVEs1105, which corresponds to the set of data points processed up to that point.

[0231] Several embodiments provide an interactive incremental curve fitting component required for digital drawing systems. In such systems, data points are generated by pen pressure as the artist draws using a tablet. The data points are typically processed and a curve is fitted to small batches (e.g., 5 points) at a time to provide the artist with immediate visual feedback while drawing. The post-processing and curve fitting of all data points can be computationally intensive depending on the system, and therefore it is necessary to divide the data points into small subset batches. Each batch of data points is curve-fitted and integrated into a previously determined curve obtained from the previous batch. Curve fitting begins during the pen-down event and ends during the pen-up event. In such a process, the first point of a series of data points is set to null. A series of data points terminates when certain termination conditions are met, such as the occurrence of a pen-up event, exceeding a time limit, exceeding the maximum number of iterations, satisfying an error tolerance, or failure to converge across a series of processing steps. These conditions can be, for example, data-independent, data-dependent, dynamically updated, static, visually inspected, human-instructed, AI-controlled, procedurally determined, or determined by a table.

[0232] In some embodiments, a set of stationary points is discovered from a batch subset of a series of data points, and updating the current stationary point from the set of stationary points is done using a state machine. Generally, a state machine is any device that stores the status of something at a given point in time, and by interacting with inputs, it can change the status and / or trigger actions or outputs for any given change. More formally, a state machine can be described as an initial state or record of something stored somewhere, a set of possible input events, a set of new states that may arise from the input events, and a set of possible actions or output events that may arise from the new states.

[0233] In some embodiments, updating the current steady point from a set of steady points involves assigning the current steady point to the last steady point in the set of steady points.

[0234] Some embodiments provide a method for rendering an ordered set of primitives using a depth buffer, where each primitive in the ordered set has a composite shape determined by a set of Boolean operations performed on a set of distance fields.

[0235] Figure 12 shows a method 1200 for rendering an ordered set of primitives using a depth buffer 1205. This method 1200 may be implemented by a processor configured to perform the operation of method 1200, which includes, in step 1250, setting the depth buffer 1205 to an initial state. This operation further includes, in step 1255, dividing the numerical range of the depth buffer 1205 into an ordered set of subranges 1215. The operation of method 1200 further includes, in step 1260, assigning the ordered set of subranges 1215 to each of the ordered set of primitives in a one-to-one sorting manner. Furthermore, this operation includes, in step 1265, rendering each primitive of the ordered set of primitives 1210 in order, where each primitive has a subrange 1220, a set of Boolean operations 1225, and a set of distance fields 1230. Method 1200 further includes, in step 1270, using a depth buffer 1205 to determine a composite shape 1235 of a primitive from a subrange 1220, a set of Boolean operations 1225, and a set of distance fields 1230, and rendering the composite shape 1235.

[0236] The depth buffer includes a memory buffer and a processor operating on the memory buffer. The processor may be a CPU, a GPU, etc. The depth buffer may include a dynamic z-buffer of the GPU, which for example has a numerical range and includes addressable elements (e.g., fragments, pixels), performs Boolean operations on the addressable elements, performs arithmetic operations on the addressable elements, and operates in parallel on the addressable elements. The Boolean operations include addition, multiplication, and subtraction, which are decomposed into a pair of minimum and maximum operations, and the depth buffer operates in a SIMD manner.

[0237] Embodiments disclosed herein reformulate Boolean operations and a set of distance field ranges by dividing the numerical range of the depth buffer into a set of subranges, typically one subrange for each primitive, thereby avoiding clearing the depth buffer for each primitive in a set of ordered primitives, and defining primitives to operate strictly within their subranges in such a way that when rendering the previous primitive in a set of ordered primitives in the requested order, all previous values ​​assigned to the depth buffer are ignored. This significantly improves computational efficiency by eliminating the penalty imposed on clearing the depth buffer, as done in solutions of existing techniques.

[0238] In some embodiments, the numerical range of the depth buffer spans from a first value to a second value, where the first value is less than the second value, and the setting assigns the first value to 0 and the second value to 1. The setting can assign the depth buffer to the first value, where a subrange of an ordered set of subranges is a subset of the numerical range, and the subrange has a first subrange value and a second subrange value, where the first subrange value is less than the second subrange value.

[0239] In some embodiments, the above division involves dividing the numerical range of the depth buffer into N subranges, thereby dividing the numerical range of the depth buffer into an ordered set of subranges, where N is equal to the total number of primitives in the ordered set of primitives. The numerical range of the depth buffer may be divided into N numerically increasing subranges or N numerically decreasing subranges.

[0240] In some embodiments, determining the composite shape involves using a depth buffer to perform a set of Boolean operations on a set of metric fields within a sub-range. This determination reformulates the Boolean operations and the range of the set of metric fields, defining the primitive to operate strictly within its sub-range, in such a way that when rendering the previous primitive in an ordered set of primitives in the requested order, all previous values ​​assigned to the depth buffer are ignored.

[0241] In some embodiments, the rendering renders the composite shape using a depth buffer obtained by determining the composite shape. For example, an ordered set of four primitives A, B, C, and D may require rendering. Each of these consists of a Boolean sum operator, and these composite shapes can be determined from this operator. According to embodiments disclosed herein, the depth buffer is initially initialized to a numerical range of 0 to 1, holding the maximum value of any future operations applied to the fragments of the depth buffer. Primitive A may be assigned to a subrange starting at 0.00 and ending at 0.25, primitive B to a subrange starting at 0.25 and ending at 0.50, primitive C to a subrange starting at 0.50 and ending at 0.75, and primitive D to a subrange starting at 0.75 and ending at 1.00.

[0242] Furthermore, in this example, rendering proceeds as follows: The depth buffer is cleared once. Next, a set of distance fields for primitive A is programmed to calculate its composite shape by using a maximum operation with a depth buffer setting that holds the maximum value, so that it operates within its assigned subrange [0.00, 0.25]. Next, primitive A is rendered using the composite shape in the depth buffer. Furthermore, a set of distance fields for primitive B is programmed to calculate its composite shape by using a maximum operation with a depth buffer setting that holds the maximum value, so that it operates within its subrange [0.25, 0.50]. Note that all distance values ​​determined by primitive B are greater than any distance values ​​of primitive A during processing. As a result, the distance values ​​for primitive A that were previously rendered do not affect the calculation of the composite shape for primitive B and therefore do not need to be cleared before processing primitive B. Next, primitive B is rendered using the composite shape in the depth buffer. The same steps for primitive C and primitive D enable the gradual rendering of the primitives.

[0243] The above description provides only specific embodiments and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the above description of specific embodiments provides a description that enables the realization of one or more specific embodiments. The intention is to describe various modifications that can be made to the function and configuration of the elements without departing from the spirit and scope of the disclosed subject matter as described in the appended claims.

[0244] Specific details are provided in the above description to ensure a full understanding of the embodiments. However, those skilled in the art will understand that embodiments can be carried out even without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in the form of block diagrams to avoid obscuring the embodiments with unnecessary details. In other examples, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. Furthermore, similar reference numbers and names in different drawings refer to similar elements.

[0245] Furthermore, individual embodiments may be described as processes shown as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be performed in parallel or simultaneously. Moreover, the order of operations is interchangeable. A process may terminate when its operations are complete, but it may have other steps that are not discussed or included in the diagrams. Furthermore, not all operations in any process specifically described may occur in all embodiments. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, the termination of the function may correspond to returning that function to the calling function or the main function.

[0246] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, manually or automatically. Manual or automatic implementation may be performed, or at least assisted, through the use of a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. If implemented with software, firmware, middleware, or microcode, the program code or code segments for performing the required tasks may be stored on a machine-readable medium. A processor may perform the required tasks.

[0247] The various methods or processes outlined herein may be encoded as software executable on one or more processors employing any one of a variety of operating systems or platforms. In addition, such software may be written using any of several suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functions of program modules may be combined or distributed as required in various embodiments.

[0248] Embodiments of this disclosure may be implemented as methods, and an example thereof is provided. The order of operations performed as part of this method may be determined in any suitable manner. Thus, embodiments may be configured such that operations are performed in an order different from the order illustrated, which may include performing some operations simultaneously, even if they are shown as a series of operations in the illustrated embodiments. While this disclosure has been described with reference to several preferred embodiments, it should be understood that various other adaptations and modifications can be made within the spirit and scope of this disclosure. Therefore, it is an aspect of the appended claims to cover all such variations and modifications that fall within the true spirit and scope of this disclosure.

Claims

1. An image processing system, wherein the image processing system comprises an input interface configured to receive an intensity image, and at least one processor, which, by executing computer-executable instructions, is configured to convert the intensity image into a layered distance field (DF) image, the layered DF image including an ordered sequence of a plurality of layers, wherein each layer in the ordered sequence includes a DF procedure for determining DF values at a plurality of locations in the received intensity image, and a set of rules for mapping the DF values to intensity values of each layer, and the image processing system further comprises an output device configured to render the layered image. An image processing system.

2. The image processing system according to claim 1, wherein each of the plurality of locations corresponds to a respective candidate image among a plurality of candidate images in the received intensity image.

3. The image processing system according to claim 1, wherein the layered DF image includes an intensity reconstruction function for reconstructing the received intensity image by combining the mapped intensities of each layer in the sequence of the layers according to their order.

4. The at least one processor is further configured to determine an error value associated with a difference in intensity between the received intensity image and the intensity image reconstructed from the layered DF image, compare the error value with an error threshold, and update the reconstructed image based on the comparison. The image processing system according to claim 1.

5. The image processing system according to claim 1, wherein the ordered sequence of the plurality of layers is represented as an iteration of the DF procedure in the layered DF image.

6. The image processing system according to claim 1, wherein each layer in the layered DF image is associated with a different DF procedure.

7. The image processing system according to claim 6, wherein the DF procedure includes at least one of a parameterized DF procedure, an analytical DF procedure, and a sampled DF procedure.

8. Each layered DF image is associated with a Backus-Naur Form (BNF) grammar, and the BNF grammar of each structure of the layered DF image includes a plurality of operations. The plurality of operations includes at least one of a distance field operation, a unary operator operation, a composite distance field calculation operation, a combination operation, a DF image reconstruction operation, a DF image generation operation, a blend operation, and a Porter-Duff composite blend operation. The image processing system according to claim 1.

9. The distance field operation includes at least one of a distance map operation, an adaptive distance field calculation operation, an analytical distance field calculation operation, a procedural distance field calculation operation, an in-memory distance / distance field calculation operation, a stroke distance field calculation operation, an intra-region distance field calculation operation, and a unary operator distance field calculation operation. The image processing system according to claim 8.

10. The unary operator operation includes at least one of a CSM operation, an offset operation, an inset operation, and a probability map calculation operation. The image processing system according to claim 8.

11. The composite distance field calculation operation includes at least one of a distance field calculation operation and a combination operation. The image processing system according to claim 8.

12. The combination operation includes at least one of a Boolean operation, an implicit blend operation, a linear combination operation, and an arithmetic combination operation. The image processing system according to claim 8.

13. The DF image reconstruction operation includes at least one of a DF / map-intensity operation and a mask-based map-intensity operation. The image processing system according to claim 8.

14. The DF image generation operation includes at least one of a reconstruction DF operation and a blend operation. The image processing system according to claim 8.

15. The blend operation includes at least one of an addition operation, a subtraction operation, a replacement operation, a darkening operation, a brightening operation, and a Porter-Duff composite blend operation. The image processing system according to claim 8.

16. The Porter-Duff composite blend operation includes at least one of a src operation, an over operation, and a dest operation. The image processing system according to claim 8.

17. At least one DF procedure includes an asymmetric stroke procedure associated with a spline curve, the spline curve being associated with a corresponding distance field, and the asymmetric stroke mapping the distance field of the spline curve to different gradients of intensity change on different sides of the central axis of the spline curve such that the intensity of the spline curve changes in a way perpendicular to its central axis, the information processing system according to claim 1, which defines a rule for this.

18. The intensity of the spline curve changes gradually, the image processing system according to claim 17.

19. The gradient for changing the intensity is different on different sides of the central axis, the image processing system according to claim 18.

20. The spline curve is shifted with respect to the central axis, the image processing system according to claim 19.

21. At least one DF procedure includes DF visualized with masked stepwise intensity, the masked stepwise intensity including a null intensity value at a specific location in the layered DF image, the image processing system according to claim 1.

22. The null intensity value in a certain layer of the layered DF image does not correct the intensity of the previous layer at the corresponding location, the image processing system according to claim 21.

23. Different layers of the layered DF image represent elements corresponding to different resolutions of the received intensity image, the image processing system according to claim 1.

24. Each subset of the DF procedures selected from the beginning of the sequence of DF procedures reconstructs the received intensity image at different resolutions, but the range of reconstruction errors is the same, the image processing system according to claim 14.

25. The processor is configured to obtain the distance field image based on multi-scale normalization that repeatedly estimates each layer of the layered DF image by changing optimization parameters in different iterations, the image processing system according to claim 1.

26. A method for image processing, the method comprising: receiving an intensity image; converting the received intensity image into a layered DF image, the layered DF image including an ordered sequence of a plurality of layers; each layer in the ordered sequence: a DF procedure for determining DF values at a plurality of locations in the received intensity image; including a set of rules for mapping the DF values to the intensity values of the respective layers; The method further includes outputting the layered DF image for rendering on an output device. [

27. ] A non-transitory computer-readable storage medium having a program executable by a processor for performing a method for image processing, the method comprising: receiving an intensity image; converting the received intensity image into a layered DF image, the layered DF image including an ordered sequence of a plurality of layers; each layer in the ordered sequence includes: a DF procedure for determining DF values at a plurality of locations in the received intensity image; a set of rules for mapping the DF values to the intensity values of the respective layers; The non-transitory computer-readable storage medium, wherein the method further includes outputting the layered DF image for rendering on an output device.

Images