System and method for detecting, suppressing, and correcting background regions in scanned documents - Patents.com

Abstract

To provide an image processing device and an image processing method that properly correct a background part.SOLUTION: An image processing device 10 includes an input device 40 which receives image adjustment selections from an associated user interface device 43. Memory 30 stores a GUI generator 48. An image adjustment unit 12 includes a processor device 34 to generate: a GUI (background adjustment selector 44) to be displayed to a user on the user interface device; a background adjustment component which, for each of multiple pixels of an input image, computes adjusted color values, as a function of at least one of a background adjustment factor computed for the respective pixel and a background class derived from the computed background adjustment factor, the background adjustment factor being a function of the background strength of the pixel and the luminance strength of the pixel; and an image output component that outputs an output image derived from the adjusted color values for the multiple pixels.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) Cross-reference is made to co-pending U.S. Patent Application Publication No. 16 / 830,479, filed March 26, 2020, by Metcalfe, entitled "SYSTEM AND METHOD FOR PROCESSING AND ENHANCING ACHROMATIC CHARACTERS OF SCANNED DIGITAL DOCUMENTS," and to co-pending U.S. Patent Application Publication No. 16 / 830,419, filed March 26, 2020, by Metcalfe et al., entitled "SYSTEM AND METHOD TO DETECT AND ADJUST IMAGE BACKGROUND," the disclosures of which are incorporated herein by reference in their entireties.

[0002] FIELD OF THEINVENTION Exemplary embodiments relate to image adjustment and find particular application in systems and methods for background suppression and background correction. [Background technology]

[0003] Digital image capture devices such as scanners and cameras capture images as an array of pixels, with each pixel assigned a set of color values ​​in a multi-dimensional color space, such as the RGB (Red, Green, Blue) color space, referred to herein as the input color space. The processed image captured by the digital device is generally expressed in a L2D color space, referred to herein as the processing color space. * a * b * The process is performed in a multi-dimensional color space having a larger gamut, such as a color space. The processed image may then be converted to a multi-dimensional output color space suitable for rendering the processed image, such as CMYK (cyan, magenta, yellow, and black) for printing.

[0004] During image processing, various adjustments may be made to the image, which are often performed sequentially and may include image resolution adjustment, color correction, removal of unwanted artifacts, cropping, and background suppression.

[0005] Background suppression is a useful feature provided by the image path in copiers and multifunction systems that removes or integrates background color in a digitally acquired image. Background suppression removes the background or makes the background uniform so that the electronic image appears more uniform and consistent. Background suppression is more difficult for input images that contain poor contrast between foreground and background regions. Undesirable backgrounds can be present for many reasons, such as aged, discolored, and / or dirty document(s) that are scanned to generate the input image. In addition, the original document may be printed on colored substrates or recycled paper that customers generally do not want reproduced. Thin stocks can also be problematic because they tend to increase the probability of show-through caused by detecting and rendering content from the other side of a duplex print. Users generally do not want to have extraneous dots or background reproduced in their copies, but rather have a faithful delivery of the actual content contained in the original print. Background suppression helps improve the contrast between foreground and background regions. Background suppression also helps minimize compressed file sizes in applications such as scanning, document archiving, etc.

[0006] To achieve acceptable throughput, the complexity of background suppression algorithms is generally limited by the processing power of image processing devices such as printer processors. Algorithms have been developed that segment an image into foreground and background regions and then apply corrections only to the background regions. A threshold is set with the goal of producing uniform background region(s), typically white (no color), without undesirably affecting the foreground region(s). The threshold can be close to, but below, a maximum value. For example, on the luminance channel, which is a scale of 0-255, where 0 is black and 255 is white, a threshold of 250 can be set and all pixel values ​​above the threshold can have a gain applied to bring them to 255, i.e., white. However, pixels below 250 are not adjusted and remain gray. Chrominance channels can be adjusted similarly. As a result, in many existing threshold-based segmentation classification algorithms, abrupt switching artifacts can be produced, which are visible in the output image as non-uniform foreground or background regions. These are often referred to as "punch-through" artifacts in the halftone and highlight areas.

[0007] Furthermore, advances in the number of bits per pixel (bpp) available in output devices such as marking engines, such as the change from 1 bpp to 8 bpp, mean that these artifacts are more noticeable. In recent years, advances have been made in processing hardware and software that allow more complex algorithms to be used without affecting throughput.

[0008] A system and method for background adjustment is disclosed that can leverage the processing power of more advanced processors to minimize artifacts that can occur during background suppression and provide background color selection. (Incorporated by reference)

[0009] The following references, the disclosures of which are incorporated herein by reference in their entireties:

[0010] U.S. Patent Application Publication No. 20170255849(A1), published on September 7, 2017 by Xing Li et al. and entitled "DOCUMENT BACKGROUND SUPPRESSION PERFORMED DURING COLOR SPACE CONVERSION," describes performing background suppression correction on an image after it has been converted from a first color space to a second color space, and then converting it to a third color space used by a printing device. The process of converting the image to the third color space includes sampling nodes of the second color space values, classifying the nodes as background nodes or non-background nodes, multiplying the second color space values ​​of the background nodes by a percentage less than 100%, and interpolating values ​​between the nodes after multiplication. Summary of the Invention [Problem to be solved by the invention]

[0011] According to one aspect of the exemplary embodiment, an image processing device includes an input device that receives an image adjustment selection from an associated user interface device. The memory stores a user interface generator that generates a background adjustment selector for presentation to a user on the user interface device, and a background adjustment component that calculates an adjusted color value for each of a plurality of pixels of the input image as a function of at least one of (a) a background adjustment factor calculated for the respective pixel, and (b) a background class derived from the calculated background adjustment factor. The background adjustment factor is a function of a background intensity of the pixel, and a luminance intensity of the pixel. The memory also stores an image output component that outputs an output image derived from the adjusted color values ​​for the plurality of pixels. A processor implements the background adjustment component and the image output component.

[0012] In accordance with another aspect of the exemplary embodiment, an image processing method includes receiving an input image including a plurality of pixels. For each of the plurality of pixels, the method includes calculating a background intensity of the pixel, calculating a luminance intensity of the pixel, and calculating adjusted luminance and chrominance values ​​of the pixel as a function of a user selected highlight color and the background intensity and luminance intensity of the pixel. The method further includes outputting an output image derived from the adjusted luminance and chrominance values ​​for the plurality of pixels.

[0013] One or more steps of the method may be performed by a processor.

[0014] According to another aspect of the exemplary embodiment, the image processing device includes an image adjustment unit that receives an input image, a selected highlight color, and a selected global adjustment factor. The image adjustment unit includes a background adjustment component that, for each of a plurality of pixels of the input image, calculates a background intensity of the pixel, calculates a luminance intensity of the pixel, and calculates adjusted luminance and adjusted chrominance values ​​of the pixel as a function of the background intensity and luminance intensity of the pixel, the selected highlight color, and the selected global adjustment factor. An image output device receives an output image derived from the adjusted luminance and chrominance values ​​of the plurality of pixels, and renders the output image by printing. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a functional block diagram of an electronic transaction device in accordance with an aspect of an exemplary embodiment.

[0016] [Diagram 2] FIG. 2 is a flow chart illustrating a software image processing path according to another aspect of the exemplary embodiment.

[0017] [Diagram 3] FIG. 3 illustrates one embodiment of a background adjustment component in the device of FIG.

[0018] [Figure 4] FIG. 4 illustrates background level determination by trilinear interpolation.

[0019] [Diagram 5] FIG. 5 graphically illustrates changes to image pixels during background adjustment by white point mapping in one embodiment of an exemplary method.

[0020] [Figure 6] FIG. 6 graphically illustrates changes to image pixels during background adjustment by grayscale highlight mapping in another embodiment of an exemplary method.

[0021] [Figure 7] FIG. 7 graphically illustrates changes to image pixels during background adjustment by direct highlight mapping in one embodiment of an exemplary method.

[0022] [Figure 8] FIG. 8 shows an exemplary user interface in the system of FIG.

[0023] [Figure 9] FIG. 9 illustrates a window for region selection within the exemplary user interface of FIG.

[0024] [Figure 10] FIG. 10 illustrates a window for text selection within the exemplary user interface of FIG.

[0025] [Figure 11] FIG. 11 is a schematic block diagram of a background adjustment kernel of the background adjustment component of FIG. 3, according to one aspect of the exemplary embodiment.

[0026] [Figure 12] FIG. 12 shows an exemplary conversion function for calculating the background intensity of a pixel.

[0027] [Figure 13] FIG. 13 shows an exemplary conversion function for calculating the luminance intensity of a pixel.

[0028] [Figure 14] FIG. 14 illustrates an example white point mapping component of the background adjustment kernel of FIG.

[0029] [Figure 15] FIG. 15 illustrates an example direct mapping component of the background adjustment kernel of FIG.

[0030] [Figure 16] FIG. 16 illustrates an example grayscale mapping component of the background adjustment kernel of FIG.

[0031] [Figure 17] FIG. 17 is a flow diagram of a method for image adjustment according to another aspect of the exemplary embodiment.

[0032] [Figure 18] FIG. 18 is a flow chart showing background adjustment in the method of FIG.

[0033] [Figure 19] FIG. 19 shows direct highlight editing with different global adjustment factors. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Systems and methods are described that provide image processing incorporating background highlight mapping. Exemplary background highlight mapping may form a step in a software image pathway (SWIP) as described herein or may be performed as a stand-alone operation. Exemplary systems and methods identify potential background pixels and process them based on highlight mapping instructions. The highlight mapping instructions may be pre-set or may be selected by a user, such as a customer or other person who enters instructions for image processing.

[0035] Background highlight mapping, as used herein, is a method for modifying the color of background pixels in an image. The color modification is tunable, allowing the user to fine-tune the amount of background modification applied to the document. In one embodiment, the customer can select the amount of background modification via a user interface. For documents or copy jobs originally printed on colored stock, some of the background color of the document can be preserved, but with reduced saturation levels. This feature can also be incorporated as an economical print mode, for example, to reduce the amount of marking material, such as toner, used for any given copy job. This feature can also be incorporated as a scan file size minimization method when compressing images for scanning and for electronic distribution or archiving.

[0036] In one embodiment of highlight mapping, referred to herein as grayscale highlight mapping, the extent to which any identified background pixel is driven towards a selected highlight color is based on the pixel's "background intensity" (or neutral intensity) (as opposed to being fully labeled / classified as "background" vs. "non-background") and, optionally, also based on a global adjustment factor.

[0037] In another embodiment, referred to as direct highlight mapping, the extent to which identified background pixels are driven towards a user-selected highlight color is a function of a global adjustment factor.

[0038] In the absence of a user-selected highlight color, the identified background pixels are mapped to a default, referred to herein as white point mapping (i.e., L * a * b * = 255, 128, 128) towards "pure white". In one embodiment, the default background adjustment can be based on the "background intensity" of the pixel, similar to the grayscale mapping method (as opposed to being completely labeled / classified as "background" vs. "non-background").

[0039] In one embodiment, a user (or an automatic region selector) can select an identified background pixel or a region of text characters within the identified background pixel to be processed according to one of the background highlighting methods (e.g., direct, grayscale, or default), while a second region of the identified background pixel or text characters within the identified background pixel is processed according to another of the background highlighting methods.

[0040] The present system and method enables accurate segmentation and classification of background and foreground regions of a document, thereby significantly improving the image quality of the resulting printed document, which improvement is particularly significant and noticeable not only in low frequency halftone regions, but also in other highlight regions where the paper background characteristics are close to the foreground content of the document.

[0041] The highlight mapping process allows the background regions of a scanned document to be precisely altered and set to a pre-set or user-selectable color while leaving the remaining foreground regions unchanged. For example, background regions of a document printed on colored stock can be completely suppressed or otherwise slightly adjusted to save toner or maintain color content but with reduced saturation.

[0042] As used herein, "image output device" may include any device for rendering an image on a print medium, such as a copier, laser printer, bookbinding machine, facsimile machine, or multi-function machine (including one or more functions such as scanning, printing, archiving, email, and fax).

[0043] A "print medium" may be a physical sheet of paper, plastic, or other suitable physical print medium substrate on which an image is carried. For example, the print medium may be virtually any type of medium on which a marking engine can print, such as high quality bond paper, low quality "copy" paper, overhead transparencies, high gloss paper, colored paper, etc. A "job" or "document" refers to one or more sheets copied from an original job sheet(s) or electronic document page images from a particular user or from other related users. In accordance with the systems and methods herein, a "job" may be a print job, a copy job, a scan job, etc.

[0044] "Original image" or "input image" is used herein to mean an electronic (e.g., digital) record of information. The original image may include image data in the form of text, graphics, or bitmaps.

[0045] As used herein, a "pixel" refers to the smallest segment into which an image can be divided. A received pixel of an input image is associated with a color value that is defined in terms of a color space, such as color, intensity, lightness, brightness, or some mathematical transformation thereof. A pixel color value may be, for example, a luminance value (L * ) and chrominance (a * b * ) value, for example, RGB-L * a * b * Using a converter, * a * b * The pixels can be converted to a chrominance luminance space such as RGB or L * a *b * It should be understood that the number of bits may be represented by values ​​other than .

[0046] L * a * b * The color space has L dimensions of lightness, and opponent color dimensions (i.e., chrominance), a and b, which are based on nonlinearly compressed coordinates. * a * b * The color space contains all perceptible colors, which means that its gamut exceeds that of the RGB and CMYK color spaces, but the L * a * b * - The color space is device-independent, which means that its colors are defined independently of the nature of their production, or the device on which they are output (displayed or rendered).

[0047] 1 shows an exemplary image processing device 10. The image processing device includes an image adjustment unit 12 and, optionally, an image output device 14. The image adjustment unit 12 receives an original digital image 16, such as a scanned image, in a first (input) color space, such as RGB. The image adjustment unit 12 converts the original image 16 into a second color space, specifically, L * a * b *, where image adjustments are made to form an adjusted digital image 18. The adjustments may include background adjustments. The image adjustment unit 12 may convert the adjusted digital image 18 to an output digital image 20 in a third (output) color space, such as CMYK, in which the output device 14 operates. An exemplary output device 14 includes a marking device, which renders the output digital image 20 on a print medium, such as paper, using a marking material, such as ink or toner, to form a rendered (typically hardcopy) image 22. The image processing device 10 may further include or be communicatively connected to a source 24 of the original digital image, such as a scanner or computing device. In some embodiments, such as a scan-to-copy image processing path, the output image 20 may be stored in a local or remote memory, such as the source 24 of the digital image. The components 12, 14, 24 of the image processing device 10 may be communicatively connected by a wired link 26, such as a wire, a local area network, or a wide area network, such as the Internet, or a wireless link 28.

[0048] Image adjustment unit 12 includes a memory 30 that stores software instructions 32 for performing processing steps to generate adjusted image 18 and output image 20. A processor device ("processor") 34 in communication with memory 30 executes these instructions. Image adjustment unit 12 also includes one or more input / output (I / O) devices 36, 38, 40 for communicating with external devices, particularly for receiving original image 16, for outputting output image 20, and for receiving image adjustment selections 42 from a local or remote user interface device 43. The input / output (I / O) devices 36, 38, 40 enable image adjustment unit 12 to communicate with other devices over a computer network, such as a local area network (LAN) or wide area network (WAN), or the Internet, and may each include a modulator / demodulator (modem), a router, a cable, and / or an Ethernet port.

[0049] The user interface device 43 displays a graphical user interface, such as the illustrated background adjustment selector 44. The user interface device 43 may be a display device incorporating a display screen and one or more user input components, such as a touch screen, a keyboard, a keypad, a cursor control device, a voice command receiver, etc. The user interface device 43 may be communicatively coupled to the input / output device 40 by a wired or wireless link, whereby the GUI 44 is transmitted to the user interface device 43 and user selections are received from the user interface device.

[0050] The hardware components 30 , 34 , 36 , 38 , 40 of the image control unit 12 may communicate via a data / control bus 46 .

[0051] The image control unit 12 may include one or more computing devices such as a microprocessor, a PC such as a desktop, laptop, palmtop computer, personal digital assistant (PDA), server computer, mobile phone, tablet computer, combinations thereof, or other computing device capable of executing instructions to perform the exemplary methods.

[0052] Memory 30 may represent any type of non-transitory computer-readable medium, such as random access memory (RAM), read-only memory (ROM), magnetic disk or tape, optical disk, flash memory, or holographic memory. In one embodiment, memory 30 includes a combination of random access memory and read-only memory. In some embodiments, processor 34 and memory 30 may be combined within a single chip. Memory 30 stores instructions for executing the exemplary methods and processes data and may include one or more memory devices.

[0053] The digital processor device 34 may be variously embodied by a single-core processor, a dual-core processor (or more generally a multi-core processor), a digital processor and associated mathematical co-processors, a digital controller, and the like. In addition to executing the instructions 32, the digital processor 34 may also control the operation of the image output device 14. In one embodiment, the processor may be or may include a dedicated processor specialized for processing image data, and may include an application specific integrated circuit (ASIC) specialized for handling image processing operations, processing image data, calculating pixel values, and the like. The processor may include a raster image processor (RIP), which raster images the job using the original image description. Thus, for a print job, the print instruction data is converted into a printer-readable language. The print job description is generally used to generate a print-ready file. The print-ready file may be a compressed file that can be accessed repeatedly for multiple (and subsequent) passes.

[0054] As used herein, the term "software instructions" or simply "instructions" is intended to encompass any collection or set of instructions executable by a computer or other digital system to configure the computer or other digital system to perform a task for which the software is intended. As used herein, the term "software" is intended to encompass such instructions stored on a storage medium such as RAM, hard disk, optical disk, etc., and is also intended to encompass so-called "firmware," which is software stored in ROM or the like. Such software may be organized in a variety of ways and may include software components organized as libraries, Internet-based programs stored on a remote server or the like, source code, interpreted code, object code, directly executable code, etc. It is contemplated that the software may invoke system level code or calls to other software resident on a server or elsewhere to perform certain functions.

[0055] The illustrated instructions 32 include a graphical user interface (GUI) generator 48, a first (input image) transformation component 50, optionally one or more image pre-processing components 52, a page background detection component 54, a background adjustment component 56, optionally one or more image post-processing components 58, a second (adjusted image) transformation component 60, an image output component 62, and an optional parameter learning component 64. As will be appreciated, one or more of components 48, 50, 52, 54, 56, 58, 60, 62, and 64 may be separate or combined and may be software or hardware components.

[0056] A GUI generator 48 generates a graphical user interface (GUI) for display on the UI 43. The GUI includes a background adjustment selector 44 that is provided for a user to make a background adjustment selection 42 on the user interface 43, for example, via a touch screen, keypad, etc. incorporated within or otherwise associated with the device 43.

[0057] 2, assuming the input image 16 is not already in a luminance-chrominance color space, a first transform component 50 receives the original image 16 in a first color space and transforms pixels 66 of the original image 16 to pixels 68 to form a transformed image 70 (FIG. 1) in a second (luminance-chrominance) color space. * a * b * 2 by conversion to the second color space. As will be appreciated, if the original image 16 is already in the second color space, the first conversion component 50 may be omitted. This conversion may be performed using other methods, such as with a look-up table or using an appropriate conversion equation.

[0058] Optional image pre-processing component(s) 52 may perform various pre-processing operations on the converted image pixels 68 in the second color space. These operations may include one or more of cropping, scaling, padding, etc. As will be appreciated, one or more of these operations may additionally or alternatively be performed in a software image path. The output of the pre-processing is an optionally pre-processed image 72 that includes pre-processed image pixels 74 in the second color space.

[0059] The optional page background detection component 54 may detect the background color 75 of each page and segment a sequence of incoming pages into a set of pages with similar background colors. The component 54 may incorporate a page-based or leading edge background detection kernel to assign a single background color to the entire page. This allows the component 56 to process pages with a "white" background differently than colored backgrounds (non-white). In this way, the page background can be dynamically adjusted on a document-by-document (or page-by-page) basis based on the current scanner and paper media attributes. This allows full or partial background suppression capabilities when a mixture of both color and standard white paper stocks are included in the same copy job. The component may perform simple detection of the color 75 of background areas such as the margins of a page (i.e., outside the normal print area), so that all pages within a preselected color value range are classified as "white background," while other pages are classified according to the average L color of the non-white background. * a * b * The background value can be used to classify the page. In the following example, a page with a detected pink and yellow background is shown.

[0060] Overall print image quality can be improved by the page background detection component 54; accurate background detection profiles can be dynamically adjusted for each document based on current scanner and paper media attributes, allowing full or partial background suppression capabilities when a mixture of both colored and standard white paper stocks are included in the same copy job.

[0061] The background adjustment component 56 applies a background adjustment to pixels 74 of the page image 70 or 72 according to any user selection via the background adjustment selector 44, or by using a default selection. The output of the background adjustment operation is background-adjusted image pixels 76, i.e., pixels of a background-modified image 78, some of which may be adjusted for the purpose of providing improved image quality and / or background highlighting.

[0062] Optional image post-processing component(s) 58 may perform various post-processing operations on pixels of background-adjusted image 78 in the modified second color space to form adjusted image 18. If no post-processing is performed, background-adjusted image 78 functions as adjusted image 18.

[0063] A second transformation component 60 transforms the adjusted image 18 into pixels 80 of the output image 20 in a third color space, such as CMYK.

[0064] The images 16, 18, 20, 70, 72, 78 may be temporarily stored in memory 30 or in a separate memory during processing.

[0065] An image output component 62 outputs the output image 20 to the image output device 14, for example, via the I / O device 38. Alternatively, the output image may be stored in memory or output to a remote computing device and / or memory storage device.

[0066] During the image processing described herein, the original image 16 may be converted into a bitmap image that indicates the value to print at each pixel in the image.

[0067] An optional parameter learning component 64 learns one or more parameters of the system, such as a lookup table, a scalar parameter, etc.

[0068] 3, an exemplary background adjustment component 56 is shown. The input to the illustrated component 56 is 8 bpp for each channel for each pixel of the image, i.e., L * a* a and b values, each of which is a sequence of 8 bits (each bit can be 0 or 1) corresponding to a range from 0 to 255. However, other scales are contemplated.

[0069] Background adjustment component 56 includes a background segmentation component 82 and a background adjustment kernel 84. In one embodiment, background segmentation component 82 assigns a background level (or an average background level) to every pixel in the image, which is an indication of how likely each pixel is to be a background pixel. In another embodiment, background segmentation component 82 segments the image into background and foreground regions and identifies all pixels in the background region as background pixels.

[0070] The illustrated background segmentation component 82 includes a background level determination component 90 and an averaging component 92. Component 90 includes a planar-to-pixel aggregation component 96 and a background level output component 98 that can operate via trilinear interpolation.

[0071] The background level determination component 90 outputs a background level 100, which is essentially the probability that a given pixel is a background pixel. In an exemplary embodiment, L is closer to 0 than the second pixel. * The first pixel with a value (i.e. black on a scale of 0 to 255) is the a * value and b * It receives a lower background level value than the second pixel, assuming the values ​​are the same.

[0072] In the illustrated embodiment, the background level 100 is an 8-bit output, i.e., each pixel has a background level of 0 to 255 on a decimal scale, but this could alternatively be a scale of 0 to 1 or 0 to 100, or other non-binary scale with a range of possible values. The background level 100 determination is made for each pixel and determines its L without reference to surrounding pixels. * , a * , and b * It is a function of value.

[0073] In the illustrated embodiment, aggregation component 96 aggregates input value L to facilitate calculation by background level determination component 90. * , a * , b * into a single Lab channel value 101. The output is L * , a * , and b * For example, L * , a * , and b * are (1,0,0,0,0,1,0,0), (0,0,0,0,0,1,0,1), and (0,0,0,0,0,0,0,0,0,1), the aggregation component outputs (1,0,0,0,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1) as the aggregated Lab value 101.

[0074] The background level output component 98 receives the Lab values ​​101 and outputs the background level 100 as a function thereof. The background level output component 98 essentially functions as a look-up table (LUT). To reduce memory requirements, trilinear interpolation can be used to calculate intermediate values ​​between a set of stored grid points. In this embodiment, the most significant bits are used to access grid points in the 3D color gamut (forming 8 points of a cube around the interpolation point corresponding to the pixel). The number of significant bits used can be selected to provide a trade-off between image quality and computation time. Each grid point is associated in the LUT with a corresponding background level. Interpolation is then performed to determine the background level of the pixel. In an exemplary embodiment, trilinear interpolation is performed in the X, Y, Z color space, although other color spaces are contemplated. In one embodiment, the trilinear interpolation LUT includes 3,375 grid point integer values ​​(i.e., a 15×15×15 LUT), although smaller or larger LUTs are contemplated. It will be appreciated that if memory size were not a constraint, the interpolation could be replaced with a larger LUT, which in some cases could achieve slightly higher image quality.

[0075] FIG. 4 graphically illustrates the operation of an exemplary background level output component 98. * a * b * The values ​​are mapped to corresponding CIE XYZ tristimulus values ​​using respective mapping LUTs 102, 103, 104. The XYZ values ​​defined on orthogonal axes are 111 , P 110 , P 101 , P 100 , P 011 , P 010 , P 001 , P 000 The inputs are input to a lattice point LUT 105 which identifies the eight closest lattice points 106, denoted by x, y, z. A trilinear interpolation kernel 107 calculates the Euclidean distance from pixel P to each lattice point. As shown, this involves generating a path from the pixel to the lattice point using three perpendicular X, Y, and Z axes, and calculating the Euclidean distance (shortest path) from there. A background level output is then assigned to each of the lattice points as an inverse function of the Euclidean distance, e.g., a function of the background level:

number

[0076] The trilinear interpolation component 98 programs the input remapping LUTs 102, 103, 104 and the 3D grid points 106 to determine which L * a * b * A flexible and robust method is provided for determining whether a triplet value or color gamut region should be classified as a "background" pixel.

[0077] The LUTs 102, 103, 104, 105 (FIG. 4) used by the background level output component 98 may be generated to optimize image quality for a given type of image set being processed. In one embodiment, two or more LUTs 105 may be stored and the user is provided with the option of selecting one of the stored LUTs based on personal preference. To calculate the background level in the LUT, the neutral magnitude of a pixel may be derived from equation (1) below:

number

[0078] An optional averaging component 92 (FIG. 3) adjusts the background level 100 of a given pixel output by the background level determination component 90 to take into account the background levels 100 of a set of local (e.g., surrounding) pixels. In one embodiment, component 92 outputs an average background level 108 for each pixel of the image. This provides the ability to segment relatively low frequency halftone screens / regions that would otherwise be classified simply as background. Taking into account the attributes of neighboring pixels can reduce abrupt segmentation switching artifacts that may otherwise occur in a pure point process classification scheme. In one embodiment, the average background level is the average of 9 pixels that form a box with the given pixel as the center pixel, with the 8 closest surrounding pixels. In other embodiments, a larger number of surrounding pixels may be considered. In some embodiments, for example, different weights may be applied to the surrounding pixels depending on their proximity to the given pixel. The averaging component 92 may be any suitable box filter, such as a 3×3, 5×5, or 7×7 box filter. The result of the averaging is that the background level of a given pixel is shifted closer to the background levels of the surrounding pixels.

[0079] In another embodiment suitable for directing highlight mapping, the background level 100 or average background level 108 may be thresholded or used to access a lookup table that generates a binary "yes" or "no" decision of whether a pixel is classified as background or not. Other methods of generating a "yes / no" decision for a background pixel are also contemplated, as will be described in more detail with reference to FIG. 11.

[0080] The background adjustment kernel 84 (FIG. 3) applies the original (or pre-processed) L * , a * , b *The values ​​70, 72 are taken as inputs and a background adjustment is applied. In one embodiment, the adjustment is a function of the background level 100 (or 108 if calculated), such that pixels with a high background level 100 (or 108) receive a larger adjustment than pixels with a relatively low background level 100 (or 108). The adjustment range and the direction of the adjustment in color space are also a function of the background adjustment selection 42 input from the background adjustment selector 44 and / or the detected background color 75. In another embodiment, the adjustment range and the adjustment direction in color space are primarily a function of the background adjustment selection 42 input from the background adjustment selector 44 and / or the detected background color 75 for those pixels classified as background.

[0081] In one embodiment (FIG. 3), the user input 42 includes one or more of a region / text selection 110, a background highlight color selection 112, a mode selection 114, and a global adjustment factor (GAF) selection 116. The region / text selection 110 specifies the background region that the user wants to be processed and / or textified within the background region that he or she wishes to highlight. The background highlight color selection 112 specifies the color to be used in the highlight. The mode selection 114 specifies one of several highlight modes (e.g., white point, grayscale, or direct mapping to background pixels of the background region or the entire image). The global adjustment factor (GAF) selection 116 specifies the global adjustment factor to be used in each mode (which may be 0 or 1 by default). The GAF is applied uniformly to each pixel receiving the adjustment, and is therefore independent of the luminance and chrominance values ​​of the pixel.

[0082] The background adjustment kernel 94 may have multiple operating modes, such as two, three, or more modes, which may be specified in a mode selection 114. The first of these modes, referred to as white point mapping, causes any background pixel to be colored "pure white" (i.e., L * a * b *The range in which the pixel is driven towards (=255, 128, 128) is based on the (average) background level 108 of the pixel in question, as opposed to being fully labelled / classified as "background" versus "non-background". In a second mode, called greyscale highlight mapping, instead of driving the background pixels towards "pure white" as in white point mapping, the background pixels are driven towards a user selected colour value which may be specified in background highlight selection 112. The selected colour may be a colour value similar to the actual background colour, e.g., slightly pink for a pink background, or a completely different colour such as yellow. In both the first and second modes, the new pixel colour may also be a function of the global adjustment factor 116, i.e. a factor that is applied uniformly to the identified background pixels. A single mode may combine the white point mapping and greyscale mapping modes, where the selected highlight colour is selected from white and other colours. In a third mode, called direct highlight mapping, the identified background pixels are replaced with a user selected color value 112 or a color that is a function of the user selected color value and a global adjustment factor 116 .

[0083] The background adjustment selector 44 may allow a customer to completely suppress the background level(s) on any document printed on a wide range of colored stocks using the accurate page background detection component 54. Similarly, a customer may decide to retain some of the background color of the document, but with reduced saturation. This feature may also be combined with "economy print" and / or "draft mode" to reduce the amount of toner deposited on the printed document for any given copy job.

[0084] FIG. 5 shows the white point mapping mode, where arrows 120, 122, and 124 indicate the pixel A , Pixel B , and Pixel C Three example pixels, denoted by A is a * b* The component is L * Because it is close to the neutral axis, Pixel B and Pixel C Similarly, Pixel B is a * b * The component is L * Because it is close to the neutral axis, Pixel C It is more tuned towards pure white than the L * In addition to moving closer to the axis, the pixel a * b * Values ​​are also in pixels. A Pixel B and Pixel C is shifted more towards the neutral value (128, 128).

[0085] The L shown in Figure 5 * a * b * The background (neutral) region 126 of the color gamut is defined and generated by appropriately programming the trilinear interpolation grid points and the input remapping LUT. * a * b * Any area of ​​the gamut that is not within the gamut is classified as background. The remainder of the gamut is considered to be foreground. The perimeter 128 of the background region defines the boundary between the neutral background pixels and the foreground image pixels. Pixels whose values ​​lie on or within the boundary 128 are candidates for adjustment (i.e., they may be adjusted, but are not required to be, while pixels outside the boundary do not receive adjustment). In this example, the background region 126 of the gamut is circular and L * a in the lighter areas of the axis * b * It is centered on the neutral axis, which is suitable for most documents printed on white paper.

[0086] However, for copies printed on yellow stock, for example, the background areas are colored yellow. * a * b *The amount of background adjustment can be accommodated and programmed within the region(s) so that further adjustments to these (yellow background) pixels can be achieved using the same variable processing method, as shown, for example, in FIG. 6. The degree of background adjustment can thus be adjusted based on customer preferences and / or other factors (such as reduced toner usage). For example, in a copy of a highly colored "yellow paper", the background can be suppressed to achieve a much lighter yellow color to preserve at least some of the background content of the original print. Alternatively, the yellow background areas may be removed while still maintaining the integrity of the print content. The "pure white" L used to derive the adjustment values ​​can be adjusted based on customer preferences and / or other factors (such as reduced toner usage). For example, in a copy of a highly colored "yellow paper", the background can be suppressed to achieve a much lighter yellow color to preserve at least some of the background content of the original print. Alternatively, the yellow background areas may be removed while still maintaining the integrity of the print content. * a * b * By providing the ability to change / program the values, similar but different background adjustment effects can be achieved. For example, the detected L * a * b * (Pink) Value 75 or selected L * a * b * A pixel adjustment value based on the (pink) value 112 can be created. Figure 6 illustrates the grayscale highlight mapping mode, with arrows 130, 132, 134 indicating changes to corresponding R, G, and B components of pixels in the neutral background region, and automatic correction of exemplary pink 136 and yellow 138 portions of the color gamut to pink and yellow pixels.

[0087] 7 illustrates a direct highlight mapping mode, with arrows 140, 142, 144 indicating changes to corresponding R, G, and B components of pixels in a neutral background region, as well as modifications to those pixels in an exemplary pink 146 and yellow 148 portion of the color gamut based on a user-selected global adjustment factor (GAF). In the illustrated embodiment, a user may select a "pink" GAF ranging from 0 to 1 for a background color that is detected as having a color value within a predetermined range of the "pink" color 75 and / or may select a "yellow" GAF ranging from 0 to 1 for a background color that falls within a "yellow" range of background values. A GAF of 0 will, on average, retain more background color than a GAF closer to 1.

[0088] 8, an exemplary background adjustment selector 44 is shown. A user can use a "Select Background Color" button or icon 150 to select a background color. A window 152 opens to provide a color wheel or other color selector. The current (detected) color 75 of the background can be displayed in the window 152, allowing the user to select the same color or change to a new color 112. If the user does not select a background color 112, "pure white," or the detected background color 75, can be selected by default.

[0089] A second icon or button 154 provides a "highlight area selection" function. This function may be used in combination with the background color selection function 150. When the button 154 is selected, a scan 156 of the page to be printed may be displayed in the window 152, allowing the user to draw or otherwise define a bounding box 158, as shown in FIG. 9. The bounding box is shown as a rectangular bounding box, although other regular or free-form bounding boxes are contemplated. Background pixels within the bounding box 158 are replaced with or driven towards the selected background color 112, while background pixels outside the bounding box are driven towards pure white color (or vice versa) in this example. The window 152 may also display a simulated scan 160 (an approximation) of what the page will look like when the modifications are made. 9 is shown in grayscale, but for ease of illustration, it should be understood that region 162 of simulated scan 160 (corresponding to bounding box 158) appears in window 152 as having, for example, a yellow background color, while the (non-background) word "quality" has the same color as scan 156. Similarly, in region 164, pixels identified as background are shown as "white," while the word "quality" has the same color as scan 156.

[0090] A third icon or button 166 (FIG. 8) allows the user to select the text to be highlighted. A window 152 (FIG. 10) may open when this function is activated, which allows the user to type the character string to be highlighted, for example the word "quality" in a text entry box 168. The scan 16 is then OCRed to identify instances within the text of the selected character string. A simulated scan 170 may be displayed in the window, with the identified instance 171 highlighted in a user selected color 112. This highlight may take the form of a rectangle of colored pixels tightly surrounding the instance 171. Alternatively or additionally, the pixels of the instance 171 itself may be colored based on the selected color 112.

[0091] A fourth icon or button 172 (FIG. 8) provides a GAF selection function, which controls the extent to which background pixels are driven toward the background highlight color 112. A higher GAF value results in a given background pixel being driven more toward the selected background highlight color (or pure white) than if a lower GAF value were selected. The user may be presented with a slider or other input mechanism for selecting the GAF. A simulated scan similar to that shown in FIG. 9 may be presented in window 152 visualizing the effects of changing the GAF.

[0092] A fifth icon or button 174 provides a "page type selector" function that allows the user to enter different instructions for different pages. For example, the user may select to treat pages that have a "white" background, as identified by page background detection component 54, differently than pages that have a "pink" or "yellow" background.

[0093] As will be appreciated, various features of the functions 150, 154, 166, 172, 174 may be combined.

[0094] Referring now to Figure 11, a functional block diagram of one embodiment of the background adjustment kernel 84 of Figure 3 is shown. The background adjustment kernel 84 processes incoming L * a * b * The image 72 is processed to output background-adjusted image pixels 76. The illustrated kernel 84 includes a white point mapping component 180, a grayscale mapping component 182, a direct mapping component 184, a background adjustment factor (BAF) calculation component 186, a highlight / region identifier 188, a blending component 190, and a background sweep function component 192. However, it should be understood that the background adjustment kernel 84 may operate with fewer, more, or different components.

[0095] The optional white point mapping component 180 maps the input values ​​72 of the background pixels to white point background adjustment pixels (WBA L * , a * , and b * ) 196. This may be performed using the methods of the above-mentioned patent applications, U.S. Patent Application Publication No. 16 / 830,479 or 16 / 830,419. Briefly, the method involves adjusting each pixel as a function of the likelihood that the pixel is a background pixel, as represented by a background adjustment factor 198 output by the BAF computation component 186. The adjustment may also be a function of a global adjustment factor (GAF) 116.

[0096] The BAF calculation component 186 calculates the (average) background level (ABL) 108 calculated by the averaging component 92 (FIG. 3) and the pixel luminance value L * 72 as an input and calculates a background adjustment factor 198 for a pixel as a function thereof. In the illustrated embodiment, component 186 includes a background intensity LUT 200, a luminance intensity LUT 202, and an aggregation factor 204. The background intensity LUT 200 converts the average background level 108 to a background intensity according to a first non-linear function. The luminance intensity LUT 202 converts the luminance (L * ) values ​​to luminance intensities according to a second non-linear function. In an exemplary embodiment, luminance intensity LUT 200 and background intensity LUT 202 are each 256-element, floating-point LUTs, for example as described in co-pending U.S. Patent Application Publication No. 16 / 830,419. Programmable aggregation factor 204 aggregates (e.g., multiplies) the luminance and background intensities to generate background adjustment factor 198. Figures 12 and 13 show examples of non-linear functions that may be applied by LUTs 200 and 202. Background adjustment factor 198 is thus a function of the average background level of a pixel, which is a function of the luminance and chrominance values ​​of the surrounding pixels, and the L value of the pixel, which is independent of the luminance values ​​of the surrounding pixels. * The aggregated luminance and chrominance values ​​of the pixel are changed to reflect the * a * b *) value. For direct mapping, the BAF 198 can be converted to a binary value 199 (or class) corresponding to whether the pixel is considered to be background or not.

[0097] FIG. 12 shows an exemplary first non-linear function that may be applied by the LUT 200. For a first range R1 of background levels, e.g., 0-90 on a scale of 0-255, i.e., if the probability of the pixel being background is very low, the background intensity is set to 0 on a scale of 0-1 (or a very low value close to 0). For a second range R2 of background levels, e.g., 90-220 on a scale of 0-255, i.e., if the probability of the pixel being background is medium, the background intensity increases with increasing background level, e.g., exponentially up to a maximum background intensity (1 in the exemplary embodiment). For example, all pixels in a third region R3 having an average background level between 220 and 255 are automatically assigned a maximum background intensity of 1. As will be appreciated, the three ranges are adjustable and fewer or more than three ranges may be used. In other embodiments, different functions may be applied.

[0098] FIG. 13 shows an exemplary second nonlinear function that may be applied by the LUT 202. * For a first range of values ​​R1, for example 0 to 120 on a scale of 0 to 255, i.e. if the probability that the pixel is background is very low, the luminance intensity is set to 0 (or a very low value close to 0) on a scale of 0 to 1. * For a second range of values ​​R2, for example 120 to 240 on a scale of 0 to 255, i.e., where the pixel has a medium probability of being background, the luminance intensity is L * It increases with increasing values, e.g., exponentially, up to the maximum background intensity (1 in the exemplary embodiment). For example, L *A maximum luminance intensity of 1 is automatically assigned to all pixels within the third region R3 having a value. As will be appreciated, the three ranges are adjustable. In other embodiments, different functions may be applied. In the following, it is assumed that the background intensity and luminance intensity 256 element LUTs each have an output floating point in the range 0.0 to 1.0.

[0099] Thus, the background adjustment factor 198 is a function of the background intensity and luminance intensity output by the corresponding LUT 200, LUT 202. In one embodiment, the aggregation function 204 is a multiplication function such that the background adjustment factor 198 is the product of the background intensity and the luminance intensity. As a result, if at least one of the background intensity and the luminance intensity is zero, the background adjustment factor 198 is zero, and if both the background intensity and the luminance intensity are non-zero, the background adjustment factor 198 is greater than zero with a maximum value of 1. The 256 element LUTs 200, 202 of background intensity and luminance intensity have output floating point values ​​in the range of 0.0 to 1.0, which are multiplied together to form the composite background adjustment factor value 198.

[0100] The BAF computation component 186 can use the BAF 198 to calculate a background class 208 for a pixel. Pixels having a BAF 198 that exceeds a threshold 194 (e.g., 0 or 0.1) receive a background class 208 of 1 (background), while pixels at or below the threshold 194 receive a background class of 0 (non-background).

[0101] As shown in FIG. 14, when white point mapping is selected as the mode to be used for some or all of the image pixels, the input L * , a * , b * The value 72 can be processed by the white point mapping component 180. At 210, a maximum L of 255 * value

number

number

number

number

[0102] Similarly, the same background adjustment factor is applied to the adjusted chrominance values

number

number

number

[0103] As an example, if the BAF is 0.9 and the input L* , a * , and b * If the value is (240, 123, 127), then the three background adjustment values ​​196 are (254, 128, 128), i.e. close to pure white but not exactly pure white. If the BAF is lower (less likely to be a background pixel), then the adjustment will be less.

[0104] In an exemplary embodiment, the difference value ΔL * , Δa * , and Δb * is determined as the Cartesian distance to pure white (255, 128, 128), although another point in the gamut may be chosen.

[0105] As a result, if the calculated background intensity and luminance intensity are high enough (based on the LUT profile), the pixel's composite white point background adjustment value(s) 196 will be close to or exactly equal to "1.0", which means the maximum delta value for each L * a * b * The composite background adjustment value of the pixel is equal to "0.0", which is applied to the component, thereby forcing the pixel to a pure white value. If the pixel falls outside or outside the "background" region, as defined and derived within the programmable trilinear grid point table, the composite background adjustment value of the pixel is equal to "0.0", which is applied to the input L pixel, unmodified for output. * a * b * On the other hand, if the pixel falls close to or within the boundary between "background" and "non-background", the composite background adjustment range varies between 0.0 and 1.0, thereby adjusting the L of the pixel based on the "neutrality" of the input pixel. * a * b * In effect, this neutralizes pixels classified as background and adjusts their L value to avoid and minimize the abrupt switching artifacts that are traditionally problematic in threshold-based segmentation algorithms. * a * b * Dynamically adjusts values ​​towards pure white.

[0106] Referring now to Figure 15, there is shown one embodiment of the direct mapping component 184 of Figure 11. The direct mapping component 184 includes:

number

number

number

[0107] The direct mapping component 184 applies the direct highlight mapping formula below to calculate the HBA value 230 for background pixels (pixels having a background class 199 corresponding to "background"):

number

[0108] Thus, for example, if a user selects {Lab}_{Highlights\}112 and GAF116 of 0.5, L of 83, 24, 4) * a * b * If we select the color pink having a value, the output value is calculated as follows:

number

[0109] Therefore, the output Lab HBA 230 is (169, 76, 66) (rounded to the nearest integer).

[0110] FIG. 15 shows how this can be accomplished in simplified steps. For example, at 240,

number

number

number

number

number

number

[0111] Referring now to Figure 16, there is shown one embodiment of the grayscale mapping component 182 of Figure 11. The grayscale mapping component 182 receives as input:

number

[0112] The grayscale mapping component 182 employs the following grayscale highlight mapping formula for each color channel:

number

[0113] Equation (6) is therefore identical to that for the direct highlight mapping component, except that each pixel is associated with a respective BAF 198 that is used in the calculation. For example, if the same user selected a pink color (83, 24, 4) and a GAF (0.5) as above, and considered the pixel to have a high probability of being in the background (e.g., a BAF of 0.9), then:

number

[0114] Therefore, the output Lab HBA 230 becomes (160, 71, 60) (rounded to the nearest integer). In contrast, if the pixel has a low probability of being a background pixel (e.g., a BAF of 0.1), the output Lab HBA 230 becomes (92, 29, 10). Thus, in the latter case, the pixel is only slightly driven towards the selected color at this stage, i.e., it retains most or all of its original color or is only slightly driven in order not to be considered background. Pixels that do not exceed a preselected threshold BAF (e.g. 0) are not adjusted at this stage.

[0115] As will be appreciated, the following set of equations can be used for the direct and grayscale methods, which are:

number

number

number

[0116] Note that unlike white point mapping, these formulas are * , a * , and b * Note that the value is not used.

[0117] Returning to FIG. 11, the highlight region identifier 188 determines which pixels should be modified based on the selected region 158 or text 162 and outputs a respective pixel class 250 for each pixel based on the selection. The classes may include white point mapping, grayscale highlight mapping, and direct highlight mapping. The blending component 190 receives the pixel classes 250 and outputs the adjusted (L * a * b * ) value 196 or 230. In one embodiment, if the BAF 198 is greater than a threshold 194 (e.g., 0) and the pixel is within the highlight region 158 or selected text 162, the blend component is the adjusted highlight background.

number

number

[0118] The optional background sweep function 192 is a function of the adjusted L * is the value,

number

number

number

number

[0119] The sweep threshold 254 may be programmable, for example, by a user. For example, a user may be able to select between two or more values ​​of the sweep threshold.

number

number

[0120] The adjusted values ​​for each pixel, optionally modified by the sweep function 192, are output from the background adjustment component 84, as shown at 76 in FIG. 3, and input to the post-processing component(s) 58, if present.

[0121] In an exemplary embodiment, all pixels of the image are processed by background adjustment component 56. However, it is contemplated that such processing may be limited to pixels within one or more regions of the image that together occupy less than the entire image, such as border regions of the image or text regions of the image. In one embodiment, identifying candidate background regions for processing by background adjustment component 56 is performed by a page background detection component 54, which is upstream of the background adjustment component.

[0122] Figure 17 shows an image processing method that may be performed by the device of Figure 1. The method begins at S100.

[0123] At S102, parameters of the adjustment unit are provided, which may include learning and / or setting parameters of the LUT (or other suitable data structure generally referred to as a LUT) and other parameters in parameter learning component 64, as described further below.

[0124] At S104 , one or more original image(s) 16 , such as a scanned image of a hardcopy document, may be received by input device 36 and stored in memory 30 .

[0125] At S106, provision is made for a user to input background adjustment selections 42 via a background adjustment selector (GUI) 44. These may include one or more of a selected highlight color 112, a selected mapping mode 114, a selected region 158 and / or text 162 to be highlighted, a global adjustment factor (GAF) 116, an aBAF threshold 194, a background sweep threshold 254, a selected LUT 200, 202, pages to be treated differently, etc. As will be appreciated, some of these selections may be pre-set, for example for a detected background that is pink, such that a pink highlight color 112 is automatically selected.

[0126] At S108, each original image 16 is converted by a first transformation component 50 into a suitable color space for processing.

[0127] At S110, each transformed image 70 may be preprocessed by component 52 with one or more preprocessing operations to generate a preprocessed digital image 72.

[0128] At S112, a background color 75 of the page is identified by component 54, for example from the boundaries of a copy page or other areas predicted to be likely background. At this stage, page images having similar background colors may be identified by component 54. The background color 75 may be an average or other function of the colors of the pixels in the predicted background areas. This step may be performed on the original image 16, the transformed image 70, or the preprocessed image 72. The background color may be displayed to a user, for example, via user interface 44, and / or may be used to define a selected background color 150 in an automated system / mode.

[0129] In S114, background adjustment is performed by component 56 on the optionally preprocessed page image(s) based on user selection, and optionally on the background color detected in S112. Further details of the background adjustment step are described below with reference to FIG.

[0130] At S116, the background-adjusted image may be post-processed by component 58 with one or more post-processing operations.

[0131] At S118, the background-adjusted, optionally post-processed image is converted by component 60 to an output image in the output device color space, such as CMYK (for a printer), RGB (for a color monitor), etc.

[0132] At S120, the output image is output by component 62.

[0133] At S122, the output image may be rendered in hardcopy form by printing. Alternatively, or additionally, at S124, the output image may be stored in memory, for example for later viewing on a screen or for subsequent processing.

[0134] The method ends at S126.

[0135] Referring to FIG. 18, a flow chart summarizing the background adjustment process of S114 of FIG. 17 is shown.

[0136] In S200, the received user selection 42 is used by component 188 to identify the user selected (or automatically selected) highlight color 112, if present (default is pure white or detected background color).

[0137] In S202, if the user selects a region or text to be highlighted, the corresponding region / text is identified.

[0138] If the user selected a mapping method in S204, then the appropriate mapping component 180, 182, or 184 is selected for each pixel.

[0139] In S206, for each pixel, if the pixel is identified as text to be highlighted, the method proceeds to S214, otherwise the method proceeds to S208.

[0140] In S208, the BAF 198 is calculated for each remaining pixel (or only for pixels in the highlight region, if selected), as described with respect to Fig. 11. Specifically, a background intensity is calculated for each pixel based on its (average) background level. A luminance intensity is also calculated for each pixel based on its luminance value. The two values ​​are aggregated (e.g., multiplied) to calculate the BAF.

[0141] At S210, if the BAF is greater than the BAF threshold 194, the method proceeds to S214, otherwise, the method proceeds to S212.

[0142] At S212, the pixel retains its input color as the output color.

[0143] At S214, the selected mapping method is applied to the other pixels. The blending component can then call the appropriate mapping component for each pixel. Each pixel is processed according to its mapping class to the adjustment value 196 or 230 generated by the identified mapping component, as described above with respect to Figures 11 and 14-16.

[0144] If the user selects text highlighting, then by default it may be the background pixels surrounding the text that receive white point mapping, while the highlighted pixels receive direct highlight mapping.

[0145] If the user selects grayscale highlight mapping for the entire image, this may result in all pixels being classified as background pixels in the image with the pixel color adjusted according to equation (6) based on the selected highlight color and the GAF.

[0146] If the user selects grayscale highlight mapping in a specific region (smaller than the entire image), this may result in all pixels within the selected region being classified as background pixels with the pixel color adjusted according to equation (6), and pixels outside the region receiving either no adjustment or a white point adjustment.

[0147] If the user selects direct highlight mapping for the entire image, this may result in all pixels being classified as background pixels in the image with the pixel colors adjusted according to equation (5).

[0148] If the user selects direct highlight mapping for only a region of an image (less than the entire image), this may result in all pixels being classified as background pixels within the region of the image having pixel colors adjusted according to equation 5. Pixels outside the region may receive no adjustment, or may receive a white point adjustment.

[0149] If the user selects white point mapping, this may result in all pixels being classified as background pixels in the image with the pixel colors adjusted according to equations (2-4).

[0150] At S216, for each pixel that undergoes adjustment mapping, a background sweep threshold (or thresholds) 254 may be applied to the adjustment value 196 or 230. Once the threshold(s) 254 are met, the pixel is replaced with pure white (in the case of white point mapping) or a selected color (in the case of direct highlight mapping or grayscale highlight mapping).

[0151] At S218, the background-adjusted image pixels are output for optional post-processing and conversion to a rendering color space.

[0152] The methods illustrated in Figures 17 and 18 may be implemented in a computer program product that can be executed on a computer. The computer program product may include a non-transitory computer-readable recording medium on which a control program is recorded (stored), such as a disk, a hard drive, etc. Typical forms of non-transitory computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic storage medium, a CD-ROM, a DVD, or any other optical medium, a RAM, a PROM, an EPROM, a FLASH-EPROM, or any other memory chip or cartridge, or any other tangible medium that can be read and used by a computer from within. The computer program product may be integral to image adjustment unit 12 (e.g., an internal hard drive in RAM), or may be separate (e.g., an external hard drive operatively connected to unit 12), or may be separate and accessed via a digital data network such as a Local Area Network (LAN) or the Internet (e.g., as a Redundant Array of Inexpensive / Independent disks (RAID) or other network server storage indirectly accessed by unit 12 via the digital network).

[0153] Alternatively, the method may be implemented in a transitory medium such as a transmittable carrier wave in which the control program is embodied as a data signal using a transmission medium such as acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0154] The exemplary method may be implemented in one or more general purpose computers, special purpose computers, programmed microprocessors or microcontrollers, and peripheral integrated circuit elements, ASICs or other integrated circuits, digital signal processors, wired electronic or logic circuits such as discrete element circuits, programmable logic devices such as PLDs, PLAs, FPGAs, graphic card CPUs (GPUs), or PALs. In general, any device capable of implementing a finite state machine capable of sequentially executing the flowcharts shown in FIG. 17 and / or FIG. 18 may be used to implement the method for image adjustment. As will be appreciated, the steps of the method may all be computer-implemented, although in some embodiments, one or more of the steps may be performed at least in part manually. It will also be appreciated that the steps of the method need not all proceed in the order described, and fewer, more, or different steps may be performed.

[0155] Further aspects of the systems and methods are now described.

[0156] As discussed above, various LUTs and tuning parameters are used in the exemplary system, which can be programmed and tuned to optimize results for the particular printer, paper type, and / or colorant being used.

[0157] Although not intended to limit the scope of the exemplary embodiments, the following examples illustrate applications of the systems and methods. EXAMPLES

[0158] Image adjustment unit 10 is instantiated as either a CPU or GPU resource within the software image path(s), thereby providing a mechanism to optimize or "balance" the overall image processing throughput rate.

[0159] The software image path used to evaluate the method includes a background adjustment component 56, a first transformation component 50, a second transformation component 60, and various pre-processing components 52 and post-processing components 58. These include symmetric filters, as well as components for cropping, scaling, padding, neutral (gray) edge enhancement, error diffusion, and compression. The first transformation (RGB to L * a * b * ) is performed by a 3×3 color space conversion component. The second conversion (L * a * b * to CMYK) is performed by trilinear interpolation. In the background adjustment component 56, the trilinear interpolation component 98 and the (3x3, 5x5, or 7x7) box filter 92 are compiled using Intel precompiled GPU kernels. The background adjustment kernel 84 is generated using the OpenCL language using vectorized SIMD intrinsics (compiled on the Apollo Lake™ GPU). The first transform, neutral edge enhancement, and second transform are performed using GPU kernels (Intel precompiled GPUs). Each kernel is multi-threaded and processes the image on a tile-by-tile basis to accelerate the overall rendering time. The three input remapping LUTs 102, 103, 104, the luminance intensity LUT 202, and the background intensity LUT 200 (as shown in Figures 4 and 11) are each 256-element, floating-point LUTs.

[0160] Both highlight mapping methods offer unique rendering properties due to the accuracy with which the background kernel can segment and detect what is defined and measured as "background." The simple direct highlight mapping method provides consistent and uniform background CMYK marking values, whereas when processing an image via the grayscale highlight mapping method, pixel gray multi-level (8 bpp) information can be used to match natural paper material variations contained in the original scanned document.

[0161] Figure 19 shows the L generated via the software model to demonstrate the direct highlight mapping method under various levels of the Global Adjustment Factor (GAF). * a * b * Illustrated is an image snippet. The source document is a scanned text document originally printed on yellow stock and processed with a global adjustment factor (GAF) ranging from 0.00 to 1.00 in increments of 0.20 via a global background pink highlight mapping across the entire document. Although shown in black and white for ease of illustration, the background is in fact different levels of pink depending on the GAF selected. In the image snippet, the background is properly detected and corrected without producing any associated artifacts in the foreground (text) regions. This is primarily due to leveraging the full power of the 3D trilinear interpolation kernel to accurately segment regions of the color gamut to detect and correct areas classified as background.

[0162] It will be understood that variations of the above-disclosed and other features and functions, or alternatives thereof, may be combined into other different systems or applications. Various substitutions, modifications, variations, or improvements therein, not presently foreseen or anticipated, may be made by those skilled in the art, which are also intended to be encompassed by the following claims.

Claims

1. 1. An image processing device, comprising: an input device for receiving image adjustment selections from an associated user interface device; A processor; A memory storing processor-executable instructions that, when executed by the processor, a user interface generator that generates a background adjustment selector for presentation to a user on the user interface device; A background adjustment component, comprising: a background adjustment factor calculated for each pixel, the background adjustment factor being a function of the pixel's background intensity and the pixel's luminance intensity; and a background adjustment component that calculates an adjusted color value for each of a plurality of pixels of an input image as a function of at least one of: and an image output component that outputs an output image derived from the adjusted color values ​​for the plurality of pixels.

2. The image processing device of claim 1 , wherein the background intensity of each of the plurality of pixels is derived from a luminance value of the pixel.

3. 3. The image processing device of claim 2, wherein the background intensity of each of the plurality of pixels is derived from an aggregate background level derived from a luminance value of the pixel and a set of luminance values ​​of local pixels.

4. The image processing device of claim 3 , wherein the background intensity for each of the plurality of pixels is calculated as a non-linear function of the aggregate background level.

5. 2. The image processing device of claim 1, wherein the luminance intensity of each of the plurality of pixels is calculated as a non-linear function of the luminance value of the pixel.

6. 2. The image processing device of claim 1, wherein the color values ​​include luminance and chrominance values, and for each of the plurality of pixels, the background adjustment component calculates an adjusted luminance value and an adjusted chrominance value as a function of the background adjustment factor calculated for the respective pixel.

7. 2. The image processing device of claim 1, wherein the color values ​​include luminance and chrominance values, and for each of the plurality of pixels, the background adjustment component calculates an adjusted luminance value and an adjusted chrominance value as a function of the background class calculated for the respective pixel.

8. The image processing device of claim 1 , wherein for each of the plurality of pixels, the background adjustment component calculates the background adjustment factor as a product of the background intensity and a luminance intensity of the pixel.

9. The image adjustment selection includes a selected highlight color, and for each of at least a portion of the plurality of pixels, the background adjustment component adjusts an adjusted luminance value. [0010] and adjusted chrominance values [0025] of, [0030] or calculate according to that function, During the ceremony, B is the background adjustment factor calculated for each pixel and is between 0 and 1, or B is the background class selected from 0 to 1; [0045] are the luminance and chrominance values ​​of the selected highlight color, [0050] are the luminance and chrominance values ​​of white, 2. The image processing device of claim 1, wherein GAF is a global adjustment factor that is independent of luminance and chrominance values ​​of the pixel.

10. The image processing device of claim 9 , wherein the image adjustment selection comprises the global adjustment factor.

11. The image processing device of claim 1 , wherein the image adjustment selection includes a highlight region, and the plurality of pixels form the highlight region.

12. 2. The image processing device of claim 1, wherein the image adjustment selections include a text highlight color and a character string to be highlighted, and the memory further includes processor-executable instructions for identifying an occurrence of the character string within an image as a function of the text highlight color.

13. 2. The image processing device of claim 1, wherein for each of the plurality of pixels, the background adjustment component determines whether the adjusted luminance value exceeds a sweep threshold, and if the adjusted luminance value exceeds the sweep threshold, the adjusted luminance value and adjusted chrominance value of the pixel are set to the adjusted color value, and otherwise the adjusted luminance value and adjusted chrominance value of the pixel are left unchanged.

14. The memory includes: a first transformation component that transforms the plurality of pixels of the input image from an input color space to a luminance-chrominance color space; 2. The image processing device of claim 1, further storing processor-executable instructions to cause the device to perform a second transformation component that transforms the adjusted luminance and chrominance values ​​of the plurality of pixels to an output color space.

15. The memory includes: a pre-processing component that performs pre-processing of the plurality of pixels in the luminance-chrominance color space prior to background adjustment; 15. The image processing device of claim 14, further storing processor executable instructions to cause said device to implement: a post-processing component that performs post-processing of said plurality of pixels in said luminance-chrominance color space after said background adjustment.

16. The image processing device of claim 1 , further comprising an image output device for rendering the output image.

17. The image output device includes: a printer for rendering the output image on a print medium; and a display device for displaying the output image on a display screen.

18. 1. An image processing method, comprising: Receiving an input image comprising a plurality of pixels; For each of the plurality of pixels, calculating a background intensity of said pixel; calculating the luminance intensity of said pixel; calculating a background adjustment factor that is a function of the background intensity of the pixel and the luminance intensity of the pixel; calculating an adjusted luminance value and an adjusted chrominance value for the pixel as a function of a user selected highlight color and the background adjustment factor for the pixel; and outputting an output image derived from the adjusted luminance values ​​and the adjusted chrominance values ​​for the plurality of pixels.

19. 1. An image processing device, comprising: an image adjustment unit receiving an input image, a selected highlight color, and a selected global adjustment factor; A processor and a memory storing processor-executable instructions, the processor-executable instructions, when executed by the processor, a background adjustment component for each of a plurality of pixels of the input image, Calculate the background intensity of said pixel; Calculating the luminance intensity of said pixel; calculating a background adjustment factor that is a function of the background intensity and the luminance intensity of the pixel; an image adjustment unit for calculating adjusted luminance and adjusted chrominance values ​​of the pixels as a function of the background adjustment factor, the selected highlight color, and the selected global adjustment factor; an image output device that receives an output image derived from the adjusted luminance and chrominance values ​​of the plurality of pixels and renders the output image by printing.

Images