A method for transforming integral images and its application, medium and device

Abstract

This invention discloses an improved integral image that can support larger original images than conventional integral images and has practical value in the field of image processing. The transformed integral image is generated by performing a biased transformation on the pixels of the original image and then accumulating the transformed image. Specifically, the transformed integral image is accumulated after adding or subtracting an offset from the original image pixels. The offset can be set to the average value of the original image pixels or a value close to it. This results in an integral image with more balanced pixel sizes, capable of supporting larger original images. In applications, the functionality of a conventional integral image can be achieved with minimal additional computational effort.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for transforming integral images and its application, medium and device. Background Art

[0002] In the field of image processing, integral images are important computing tools that can be used for region accumulation calculations, feature calculations, etc. A conventional integral image is an image obtained by cumulative calculation of the original image. The value of any pixel ii(x,y) is the accumulation of pixels in the rectangular area Rect(-1,-1,x,y) enclosed by the upper left corner pixel oi(0,0) and the pixel oi(x,y) of the original image. The relationship between the conventional integral image and the original image is:

[0003] (E1)ii(x,y)=∑0 <x’≤x,0<y’≤yoi(x’,y’)

[0004] in,

[0005] x≥0, y≥0;

[0006] oi(x',y') represents the pixel of the original image;

[0007] ii(x,y) represents the pixel of the regular integral image;

[0008] The upper left corner pixel of the original image is the origin, and its coordinates are (0,0).

[0009] Conventional integral images suffer from several drawbacks.

[0010] Defect 1: When the number of storage bits of conventional integral image pixels is fixed, it can only support original images of smaller size.

[0011] Typically, the pixel values ​​of the original image are greater than or equal to zero. For example, in grayscale images, the average pixel value of the original image is significantly different from zero. The average pixel value of the original image is represented by d1, and the absolute value of d1 is d1_a. Assuming the width of the original image is W_OI and the height is H_OI, the theoretical difference between the maximum and minimum pixels of a conventional integral image is d2_a = d1_a * W_OI * H_OI. The difference between the maximum and minimum values ​​of the conventional integral image pixel representation range should theoretically be greater than or equal to d2_a. Therefore, the larger d1_a and the larger the original image size, the larger the pixel representation range of the conventional integral image is required to be. A larger pixel representation range of a conventional integral image requires more bits in its storage unit, occupying more resources.

[0012] With the development of technology, the resolution of various images is getting higher and higher, that is, the image size is getting larger and larger, and conventional integral images are gradually unable to be applied to large-size original images.

[0013] Defect 2: When the pixels of a conventional integral image use floating-point numbers, the conventional integral image generated from a large-size original image has excessively large errors.

[0014] Although floating-point numbers with a certain number of digits can have a wide range of values, including very large absolute values, the significant digits will not cover the lower digits when the absolute value is very large. Addition and subtraction of two floating-point numbers require alignment to ensure that the exponents of the two floating-point numbers are the same. In computers, alignment is done by aligning the lower order to the higher order. Therefore, when the absolute values ​​of two numbers differ significantly, the decimal point of the number with the smaller absolute value needs to be moved higher during alignment, potentially causing data loss at the end of the mantissa.

[0015] The main purpose of using integral images is to save computational effort, so the generation of integral image pixels generally adopts a fast calculation method, for example,

[0016] (E2)ii(x,y)=-ii(x-1,y-1)+ii(x,y-1)+ii(x-1,y)+oi(x,y), where ii(x,y) represents a regular integral image pixel and oi(x,y) represents an original image pixel. When oi(x,y) is far from the origin, ii(x-1,y-1), ii(x,y-1), and ii(x-1,y) are usually numbers with relatively large absolute values, while oi(x,y) is a number with a relatively small absolute value. As a result, large errors will occur during the addition and subtraction operations to generate ii(x,y).

[0017] Excessive errors in conventional integral images will cause excessive calculation errors in the application process and reduce the accuracy of application calculations. In order to avoid excessive errors, we must try to limit the absolute value of the integral image pixels. Summary of the Invention

[0018] The present invention makes the following agreements.

[0019] 1. Original image

[0020] OI, the original image used to generate the integral image, includes all types of images that can generate conventional integral images, such as grayscale images and binary images. It can be an image or a rectangular area within an image. The grayscale image used as the original image for the integral image can be an image directly captured by an image acquisition device, or it can be an image converted from a certain type of image, such as a grayscale image converted from an RGB color image by weighted summing of each color component, or a grayscale image converted from a remote sensing image by weighted summing of each band component. If a certain image pixel has multiple components, a grayscale image composed of a common component for each pixel can also be used as the original image for the integral image, such as a grayscale image that retains only the color component R for each pixel in an RGB image, or a grayscale image that retains only the common band for each pixel in a remote sensing image.

[0021] 2. Conventional integral image

[0022] Abbreviated as II, it is the integral image that directly accumulates the pixels of the original image.

[0023] 3. Transform the image

[0024] The transformed image is referred to as TI. The width and height of the transformed image are the same as those of the original image. The transformed image uses the same coordinate system as the original image. The relationship between TI and the original image OI is:

[0025] (E3)ti(x,y)=F(oi(x,y),…),

[0026] Among them, ti(x,y) is the pixel of the transformed image, and oi(x,y) is the pixel of the original image.

[0027] The transformed image can be a physical image or a non-physical equivalent object of the physical transformed image. In the process of generating IIWT from the original image, TI can be generated or not.

[0028] Unless otherwise specified, each transformed image mentioned in the present invention refers to a transformed image of the original image currently mentioned at the mentioned position.

[0029] 4. Transform integral image

[0030] IIWT for short is the improved integral image of the present invention.

[0031] Unless otherwise specified, each transformed integral image mentioned in the present invention refers to the transformed integral image of the original image currently mentioned at the mentioned position.

[0032] 5. Integral Image

[0033] It includes a conventional integral image and a transformed integral image. Compared with the original image, the integral image extends one column and one row outside the top-left pixel. The width and height of the original image are denoted as W_OI and H_OI respectively, and the width and height of the integral image are denoted as W_II and H_II respectively. The size relationship between the original image and the integral image is W_II = W_OI + 1, H_II = H_OI + 1.

[0034] VI. The main region and extended region of the integral image

[0035] According to the relationship between the integral image and the original image defined by the integral image, the rectangular region in the integral image corresponding to the original image is the main region. For IIWT, the following relationship formula (E4) is satisfied between any rectangular region in the main region and the corresponding rectangular region of the original image. The extended region is a row Row(-1) and a column Col(-1) that are extended outside the origin of the main region and adjacent to the main region.

[0036] VII. For various units related to the length and area of an image, such as the width, height, area, etc. of the image and any rectangular region thereof, in the present invention, pixels are used as the measurement unit for these objects.

[0037] VIII. Coordinates (x, y)

[0038] It means that the X-axis coordinate is x, the Y-axis coordinate is y, and both x and y are integers.

[0039] IX. Row Row(y)

[0040] It means the row with the Y-axis coordinate of y.

[0041] X. Column Col(x)

[0042] It means the column with the X-axis coordinate of x.

[0043] XI. Rectangular region Rect(x1, y1, x2, y2)

[0044] The rectangular region Rect(x1, y1, x2, y2) is a rectangular region Rect enclosed by two points (x1 + 1, y1 + 1) and (x2, y2), where x1 < x2, y1 < y2. The rectangular region includes the two pixels (x1 + 1, y1 + 1) and (x2, y2), but does not include the pixel (x1, y1), as Figure 3 .

[0045] XII. Pixel

[0046] The pixel at coordinates (x, y) in the original image OI is represented by oi(x, y). The lowercase abbreviations of different image types are used to represent pixels of corresponding image types. Image types include original images, transformed images, regular integral images, and transformed integral images. In this description of the present invention, unless otherwise specified, a pixel refers to the value of that pixel.

[0047] 13. Image Coordinate System

[0048] The IIWT main area is regarded as an image, and the coordinate system of the integral image is described with the integral image main area as the image. The coordinate systems of various images, including the original image, the transformed image, and the integral image main area, include several types such as (AS01) to (AS06), with (0,0) as the origin coordinate.

[0049] (AS01) The first pixel in the upper left corner of the image is taken as the origin, the horizontal direction is the X-axis, the right direction is the positive direction, and the vertical direction is the Y-axis, the downward direction is the positive direction;

[0050] (AS02) The origin is the first pixel in the lower left corner of the image, the horizontal direction is the X-axis, the right direction is the positive direction, and the vertical direction is the Y-axis, the upward direction is the positive direction;

[0051] (AS03) The origin is the first pixel in the upper right corner of the image, the horizontal direction is the X-axis, the left direction is the positive direction, and the vertical direction is the Y-axis, the downward direction is the positive direction;

[0052] (AS04) The origin is the first pixel in the lower right corner of the image, the horizontal direction is the X-axis, the left direction is the positive direction, and the vertical direction is the Y-axis, the upward direction is the positive direction;

[0053] (AS05) A coordinate system obtained by translating one of the coordinate systems (AS01) to (AS04);

[0054] (AS06) A coordinate system obtained by interchanging the X axis and the Y axis of one of the coordinate systems (AS01) to (AS05).

[0055] When the integral image adopts the coordinate system described in (AS01) to (AS04), the integral image uses Row (-1) and Col (-1) as the expansion area.

[0056] Unless otherwise specified, each image described in the present invention, including the integral image main area, is based on the coordinate system (AS01) by default.

[0057] The various relations and equations of the present invention are still valid when each image, including the integral image main body area, adopts one of the coordinate systems (AS02) to (AS04).

[0058] For each relation and formula of the present invention, after converting a certain image coordinate system from one of the coordinate systems (AS01) to (AS04) to another coordinate system, each coordinate, coordinate range, relation, traversal order, etc. are converted according to the coordinate conversion formula of the converted coordinate system, and each relation and formula still holds after conversion. The converted relations and formulas fall within the protection scope of the present invention. For example, if the image was originally based on the coordinate system described in (AS01), and this coordinate system is moved 3 units to the right and 5 units down, then the pixel with the coordinate (0, 0) in the original coordinate system becomes (-3, -5) in the new coordinate system, the rectangular area Rect(3, 2, 7, 8) in the original coordinate system becomes Rect(0, -3, 4, 3) in the new coordinate system, and the range of the X-axis coordinate values [0, 10] in the original coordinate system becomes [-3, 7] in the new coordinate system. If the coordinate system described in (AS01) is used as the original coordinate system and the coordinate axes are interchanged, then the coordinate (3, 8) in the original coordinate system becomes (8, 3) in the new coordinate system, and the traversal method in the original coordinate system that uses the Y-axis coordinate as the large loop and the X-axis coordinate as the small loop becomes the traversal method in the new coordinate system that uses the X-axis coordinate as the large loop and the Y-axis coordinate as the small loop.

[0059] XIV. Support of the integral image for the original image

[0060] According to the definition of the integral image, a high-precision integral image can be generated for the original image.

[0061] XV. The precision of the integral image is determined by the relation between the integral image and the original image defined by the integral image. For any rectangular area, the smaller the error between both sides of the equation, the higher the precision of the integral image.

[0062] XVI. Summation symbol

[0063] The summation symbol is ∑, which represents the summation of an expression. The subscript of ∑ represents the range of values of the variable in the expression. For example, ∑(0 < x ≤ ⑤, 0 < y ≤ ⑤) oi(x, y) represents the cumulative summation of the pixels in the rectangular area (0, 0, 5, 5) of the original image.

[0064] In view of the defects of the conventional integral image, the present invention proposes three improvement methods, and any number of these methods can be adopted.

[0065] Improvement 1

[0066] To make the pixel average value of the input image of the integral image closer to 0, or to make the precision of the integral image higher, perform a bias transformation and / or a scaling transformation on the pixels of the original image.

[0067] Improvement 2

[0068] After the pixels of the original image are transformed, they may be negative. In order to adapt to the situation where the accumulation of regions containing negative pixels may be negative, the pixels in the main region of the integral image can take negative, zero, or positive values.

[0069] Improvement Three

[0070] All pixels in the extended region of the conventional integral image are zero, while the pixels of Row(-1) or Col(-1) in the extended region of the improved integral image can take the same value that is negative, zero, or positive.

[0071] To implement various improvement methods, the present invention provides an improved integral image, a medium, and a device, which will be described separately herein.

[0072] I. Transformed Integral Image.

[0073] The improved integral image is a transformed integral image, abbreviated as IIWT. The generation method of the transformed integral image is to perform accumulation after transforming the pixels of the original image, and it satisfies the relational expression (E4) with the original image.

[0074] (E4) ∑x1<x’≤x2,y1<y’≤y2F(oi(x’,y’),…) = iiwt(x1,y1) - iiwt(x2,y1) - iiwt(x1,y2) + iiwt(x2,y2), where

[0075] iiwt represents the pixel of IIWT;

[0076] oi(x’,y’) represents the pixel of the original image OI;

[0077] x1 ≥ -1, y1 ≥ -1;

[0078] x1 < x2, y1 < y2;

[0079] F(oi(x’,y’),…) represents the transformation performed on the pixel oi(x’,y’) of the original image, and the parameters of the F function can be any number of terms;

[0080] ∑x1<x’≤x2,y1<y’≤y2F(oi(x’,y’),…) represents the accumulation of the rectangular region Rect(x1,y1,x2,y2) of the transformed image of the original image.

[0081] (E4) can be understood as that the accumulation of the transformed image in any rectangular region Rect(x1,y1,x2,y2) can be obtained by performing addition and subtraction operations on the pixels iiwt(x1,y1), iiwt(x2,y1), iiwt(x1,y2), and iiwt(x2,y2) at the four corners of this rectangular region in IIWT.

[0082] An equivalent relation of (E4) is that

[0083] iiwt(x2, y2) = -iiwt(x1, y1) + iiwt(x2, y1) + iiwt(x1, y2) + ∑x1<x’≤x2,y1<y’≤y2F(oi(x’, y’), …). This equivalent relation can be understood as a method for generating IIWT. When x1 + 1 = x2 and y1 + 1 = y2, ∑x1<x’≤x2,y1<y’≤y2F(oi(x’, y’), …) represents a pixel in the transformed image, that is, ti(x2, y2), or it can be understood as the transformation of the original image pixel oi(x2, y2) according to F(oi(x2, y2), …). Therefore, the method for generating IIWT can be to generate each pixel in the order of traversing by row and in ascending order of the X-axis coordinate and Y-axis coordinate. A calculation method for the value of the current pixel iiwt(x, y) is

[0084] iiwt(x, y) = -iiwt(x - 1, y - 1) + iiwt(x, y - 1) + iiwt(x - 1, y) + F(oi(x, y), …),

[0085] Similarly, IIWT can also be generated in the order of traversing by column.

[0086] The transformation included in F(oi(x, y), …) described in (E4) includes any number of terms of (T1) and (T2), or (T3).

[0087] (T1) Bias transformation, subtracting or adding a value as a bias amount to the original image pixel or its transformed value;

[0088] (T2) Scaling transformation, multiplying or dividing the original image pixel or its transformed value by a scaling factor;

[0089] (T3) Other transformations containing bias transformation and / or scaling transformation.

[0090] When there are two or more transformations, each transformation is a combined transformation combined together. The way of combining each transformation is to perform multiple transformations in any order. Take the original image pixel as the input value for the first transformation to obtain a result value, take the result value as the input value for the second transformation to obtain a second result value. If there are other types of transformations, and so on, perform other types of transformations until all types of transformations in this transformation combination are completed.

[0091] The specific forms of F(oi(x, y), …) include several types such as (F1) to (F5). Other forms that can be easily thought of according to the present invention all fall within the protection scope of the present invention.

[0092] (F1)oi(x,y)-Os;

[0093] (F2)oi(x,y)*Rt;

[0094] (F3)(oi(x,y)-Os)*Rt;

[0095] (F4)oi(x,y)*Rt-RO;

[0096] (F5) A combination of one of (F1) to (F4) with other transformations, including (oi(x,y)-Os)^2, etc.

[0097] In (F1) to (F5),

[0098] Os is the offset;

[0099] Rt is the scaling factor;

[0100] RO is the product of Rt and Os.

[0101] When F(oi(x,y),…) contains a bias transformation, if Os is taken as the mean of the original image pixels, the difference between the maximum pixel and the minimum pixel of the generated IIWT is smaller. Compared with the conventional integral image, when the pixel storage unit length is the same, IIWT can support larger original images.

[0102] Taking (F1) as an example, when Os takes the mean of the original image pixels, theoretically, the cumulative value of all the pixels in the original image after transformation is zero, and the difference between the maximum and minimum pixels of the IIWT is small. Only a smaller length storage unit is needed to support a larger original image.

[0103] The principle of selecting the offset Os is to minimize the difference between the maximum pixel and the minimum pixel of IIWT. In general, in order to save computational effort, only a rough calculation of Os is required.

[0104] In (F1) to (F5), the methods for obtaining Os include the following:

[0105] (GA) The median of the pixel value range of the original image or its approximate value. The median of the value range is the average of the maximum and minimum values ​​in the value range;

[0106] (GB) the mean value of all pixels in the original image;

[0107] (GC) is the mean value of pixels uniformly sampled from the original image;

[0108] (GD) If Os obtained by methods (GA) to (GC) is a decimal, it should be rounded or truncated to retain only an integer or a small number of significant figures;

[0109] (GE) The value of Os can be manually set as needed.

[0110] The value range of Os is between the maximum and minimum pixels of the original image.

[0111] In (F1) to (F5), the value of Rt can be determined in the following ways:

[0112] (GF) To support large-size original images, Rt can be set to a number less than 1;

[0113] (GG) To obtain a higher precision IIWT, Rt can be set to a number greater than 1;

[0114] (GH) is rounded to a whole number or a decimal number with a relatively small number of significant figures;

[0115] (GI) is taken as 2^N, where N is an integer other than 1. The multiplication and division operations on Rt can be replaced by shift operations in the computer to reduce the amount of calculation.

[0116] The advantage of taking integers or having only a small number of significant figures for Os and Rt is that the significant figures of the lower digits are removed, which reduces errors in the exponential operation in the calculation process involving large numbers, thereby reducing errors in the calculation process of generating and applying IIWT.

[0117] IIWT's extended area pixel value methods include the following:

[0118] (GJ) all take a value, which is the median of the IIWT pixel value range, that is, the mean of the upper and lower boundaries of the value range or its approximate value;

[0119] (GK) Row(-1) or Col(-1) all take a value that is the median of the IIWT pixel value range or a value close to it;

[0120] (GL)IIWT When the pixel is a signed number, Row(-1) or Col(-1) is all zero;

[0121] (GM) Manually set as needed.

[0122] F(oi(x,y),…) may be a combination of a bias transformation and / or a scaling transformation with other transformations. For example, when the other transformation is a square transformation in a power transformation, it may be:

[0123] (oi(x,y)-Os)^2.

[0124] The calculation of IIWT includes two aspects: the generation and application of IIWT. The application of IIWT includes the cumulative calculation of any rectangular area of the original image and its transformed image, and the feature calculation in any feature detection window, etc.

[0125] The generation method of IIWT is as follows:

[0126] First, set the extended area, which can all be set to the median value or its approximate value within the pixel value range of IIWT. Secondly, starting from the origin, traverse the original image and IIWT row by row synchronously. Taking iiwt(x,y) as the current pixel of IIWT, perform calculation (C1) or (C2) for each IIWT pixel.

[0127] (C1) iiwt(x,y) = -iiwt(x - 1,y - 1) + iiwt(x,y - 1) + iiwt(x - 1,y) + F(oi(x,y),…),

[0128] (C2) iiwt(x,y) = -iiwt(-1,y - 1) + iiwt(-1,y) + iiwt(x,y - 1) + ∑0≤x’<xF(oi(x,y),…) + F(oi(x,y),…). In (C1) and (C2),

[0129] x ≥ 0, y ≥ 0

[0130] The -iiwt(-1,y - 1) + iiwt(-1,y) + ∑0≤x’<xF(oi(x’,y),…) in (C2) can be replaced by a variable SumL. Before calculating the first pixel iiwt(0,y) of each row of IIWT, set SumL to -iiwt(-1,y - 1) + iiwt(-1,y). After calculating each pixel from the first pixel to the penultimate pixel of each row of IIWT, update SumL by adding the transformation value F(oi(x,y),…) of the corresponding pixel in the original image for the current pixel. When all pixels of Col(-1) of IIWT are the same, before calculating the first pixel iiwt(0,y) of each row of IIWT, set SumL to 0 because -iiwt(-1,y - 1) + iiwt(-1,y) = 0.

[0131] The application method of IIWT is as follows:

[0132] 1). Calculate the cumulative of the rectangular area of the transformed image. The formula for calculating the cumulative TIS of the rectangular area Rect(x1,y1,x2,y2) of the transformed image of the original image is as follows:

[0133] (C3)TIS=iiwt(x1,y1)-iiwt(x2,y1)-iiwt(x1,y2)+iiwt(x2,y2),

[0134] 2). Calculate the features of the transformed image in the feature detection window Fw,

[0135] An example of a feature detection window is Figure 4 , assuming that Fw consists of N rectangular areas, the calculation method of the feature TIFt of the transformed image is,

[0136] (C4)TIFt=∑0≤n <N(Wn*TISn),

[0137] in,

[0138] n represents the serial number, starting from 0;

[0139] The weight of the nth rectangular area is Wn;

[0140] The area of ​​the nth rectangular region is An;

[0141] The accumulation of the nth rectangular area in the original image is Sn;

[0142] The accumulation of the nth rectangular area in the transformed image is TISn, which is calculated according to (C3) based on the vertex coordinates of the nth rectangular area.

[0143] 3). The method for calculating the accumulation of the rectangular area of ​​the original image and the features of the feature detection window is described separately in the detailed implementation section according to different forms of F(oi(x,y),…).

[0144] 2. The medium of transforming the integral image.

[0145] The medium stores an IIWT generation program, an IIWT application program, or IIWT of several original images. The medium may include a memory, a mechanical hard disk, a solid-state hard disk, or other types of media.

[0146] 3. A device for transforming an integral image.

[0147] The first device uses any several items of the IIWT, IIWT generation method and IIWT application method of the present invention during the production process.

[0148] A second device uses any of the programs of the IIWT, IIWT generation method, and IIWT application method of the present invention during operation. One type of such device is a computer system, comprising the following components:

[0149] (U1) image acquisition module;

[0150] (U2) image preprocessing module;

[0151] (U3) IIWT generation and / or application module;

[0152] (U4) machine vision module;

[0153] (U5) Image stitching module;

[0154] (U6) 3D reconstruction module;

[0155] (U7) Other image application modules, including:

[0156] (U1), (U2), (U4), (U5), (U6), (U7) are optional modules.

[0157] (U3) contains any number of items from (U31) to (U33),

[0158] (U31) Any number of IIWTs;

[0159] (U32) Procedure for generating method of IIWT;

[0160] (U33) Procedure for application of IIWT method. BRIEF DESCRIPTION OF THE DRAWINGS

[0161] They are listed below:

[0162] Figure 1 is an example diagram of the original image and conventional integral image data, and the shaded part is the extended area of ​​the conventional integral image;

[0163] Figure 2 is based on Figure 1 The transformed image and transformed integral image data examples obtained when Os is 164. The shaded area is the expanded area of ​​the transformed integral image.

[0164] Figure 3 is a schematic diagram of the rectangular area Rect(2,3,7,8) in the image, which is represented by the non-shaded area;

[0165] Figure 4 This is a schematic diagram of the feature detection window. From top to bottom and from left to right, the top two are feature detection windows containing two rectangular areas, the third is a feature detection window containing three rectangular areas, and the fourth is a feature detection window containing four rectangular areas. The number in each rectangular area represents the weight of the rectangular area.

[0166] Figure 5 This is an example of a floating-point original image and its regular integral image. The table in the figure shows the data of a rectangular area in the center of the original image and its corresponding regular integral image data;

[0167] Figure 6 This is an example diagram of a floating-point type transform image and its transform integral image. The table in the figure shows the data of a rectangular area in the center of the transform image and its corresponding transform integral image data;

[0168] Figure 7 1 is a structural diagram of an apparatus including IIWT generation and / or application functions of the present invention, wherein modules indicated by dotted boxes are optional modules. DETAILED DESCRIPTION

[0169] The purpose of the embodiments is to make this patent application easy to understand, rather than to limit the scope of protection of the invention. Any combination of multiple embodiments, or easily conceivable changes and extensions to the embodiment methods are all within the scope of protection of the invention.

[0170] This section describes implementation examples for each specific form of F(oi(x,y),…).

[0171] IIWT generated based on (F1) and its application

[0172] When this transformation form is adopted, it is mainly to enable the IIWT of a certain length of pixel storage unit to support an original image of a larger size.

[0173] The extended area of ​​an IIWT should generally be set to the median of the IIWT pixel value range, that is, the average of the maximum and minimum values ​​in the range. If the IIWT pixel data type is a signed number, all pixels in the extended area can be set to zero or its approximate value. If the IIWT data type is an unsigned number, all pixels in the extended row and column can be set to the median of the range represented by this unsigned number or its approximate value.

[0174] The pixels of the conventional integral image increase monotonically with the increase of the coordinates, while the pixels of the IIWT sometimes increase and sometimes decrease with the increase of the coordinates.

[0175] like Figure 1 This is a data diagram of the original image and the conventional integral image. The pixel mean of the original image is 16246 / (11*9)=164.1010, which is rounded to 164. Figure 2 Yes Figure 1 The transformed image and transformed integral image obtained by taking 164 of the original image according to Os.

[0176] Os can also be obtained by uniformly sampling the original image and taking the average value, that is, taking a value every few pixels in the X-axis and Y-axis directions, and calculating the average value of all sampled values ​​as Os. The calculated Os will have a certain deviation from the average value of all pixels, but it saves calculations. Figure 1The original image is uniformly sampled, starting from the upper left corner, and sampled every two pixels in the horizontal and vertical directions. That is, the coordinates of the sampled pixels are (2*M, 2*N), where M and N are integers. The average value of the sampled pixels is 164.0000, which is rounded to 164 and used as Os. It can be seen that the Os obtained by sampling at this sampling rate is very close to the mean value of all pixels in the original image. The higher the sampling rate, the closer the obtained Os is to the mean value of all pixels. The lower the sampling rate, the smaller the amount of calculation. In specific applications, a suitable sampling rate can be formulated according to actual conditions.

[0177] like Figure 4 , for all feature detection windows listed in the image, for each feature detection window, the areas of the several rectangular regions contained are equal, and the sum of the products of the weights and areas of each rectangular region is 0.

[0178] like Figure 5 and Figure 6 , taking a grayscale image as an example, its pixels are floating point type, Os is 223.25, and taking the data of a 11*4 rectangular area in the middle of the image, which is the non-shaded area in the figure, as an example, the original image has a cumulative value of 4174.49170 in this area. The cumulative value calculated by conventional integral image is,

[0179] 5.233947e+008-5.244731e+008-5.279604e+008+5.290430e+008=4200,

[0180] By transforming the integral image, the cumulative

[0181] 9.18303e+006–9.221989e+006–8.917984e+006+8.951287e+006+223.25*11*4=4167. It can be seen that for any rectangular area, the accumulation calculated by transforming the integral image is closer to the accumulation of the original image and has a smaller error.

[0182] The cumulative S calculation formula of the original image in the rectangular area Rect(x1,y1,x2,y2) is:

[0183] (CA)S=iiwt(x1,y1)-iiwt(x2,y1)-iiwt(x1,y2)+iiwt(x2,y2)+(x2-x1)*(y2-y1)*Os,

[0184] For the cumulative calculation of rectangular areas of equal area, the (x2-x1)*(y2-y1)*Os part only needs to be calculated once, and the result is retained for future use.

[0185] An arithmetic formula for calculating a feature Ft of an original image in a feature detection window including N rectangular regions Rn (0 ≤ n < N) is as follows.

[0186] (CB) Ft = ∑0≤n<N (Wn * Sn),

[0187] Where,

[0188] Sn can be calculated according to the coordinates of Rn through (CA).

[0189] When ∑0≤n<N (Wn * An) is zero,

[0190] Ft = ∑0≤n<N (Wn * Sn) = ∑0≤n<N (Wn * TISn) + ∑0≤n<N (Wn * An * Os) = ∑0≤n<N (Wn * TISn),

[0191] That is, Ft is equal to TIFt, and the used (CB) can be simplified to (C4). It can be seen that when F(oi(x, y), …) only contains bias transformation, in the process of generating IIWT, for each pixel, the accumulation only has one more addition or subtraction operation compared with the conventional integral image. And in the feature calculation of the feature window based on ∑0≤n<N (Wn * An) = 0, which accounts for most applications, it is the same as the application method of the conventional integral image, which can greatly increase the supported size of the original image.

[0192] Such as Figure 1 and Figure 2 shown in each image, taking Fw as the feature detection window of two rectangular regions, the transformed image and the transformed integral image are generated from the original image based on (F1), Os is 164, and the rectangular regions are respectively

[0193] R1(2, 1, 6, 5), with a weight of -1;

[0194] R2(2, 5, 6, 9), with a weight of 1,

[0195] Using the conventional integral image to calculate the accumulation of the original image in R1 and R2,

[0196] The accumulation of R1 = ii(2, 1) - ii(6, 1) - ii(2, 5) + ii(6, 5) = 980 - 2229 - 2885 + 6634 = 2500,

[0197] The accumulation of R2 = ii(2, 5) - ii(6, 5) - ii(2, 9) + ii(6, 9) = 2885 - 6634 - 4834 + 11315 = 2732,

[0198] Using the transformed integral image to calculate the accumulation of the transformed image in R1 and R2,

[0199] Accumulation of R1 = iiwt(2,1) - iiwt(6,1) - iiwt(2,5) + iiwt(6,5) = (-4) - (-67) - (-67) + (-254) = -124,

[0200] Accumulation of R2 = iiwt(2,5) - iiwt(6,5) - iiwt(2,9) + iiwt(6,9) = (-67) - (-254) - (-86) + (-165) = 108,

[0201] Feature of the original image in Fw = -1 * Accumulation of R1 + 1 * Accumulation of R2 = -(2500) + (2732) = 232,

[0202] Feature of the transformed image in Fw = -1 * Accumulation of R1 + 1 * Accumulation of R2 = -(-124) + (108) = 232,

[0203] It can be easily verified that when Fw contains N rectangular regions and ∑0≤n<N(Wn*An) = 0 for each rectangular region, the features of the original image and the transformed image in Fw are the same, and the calculation formulas for calculating the feature of the original image in Fw using the conventional integral image and the transformed integral image are the same.

[0204] IIWT generated based on (F2) and its applications in the embodiments

[0205] When adopting this transformation form, when Rt < 1, the IIWT of a pixel storage unit with a certain length can support a larger - sized original image; when Rt > 1, the calculation accuracy of the IIWT application can be improved, and more accurate calculation results can be obtained when calculating the regional accumulation and the feature of the feature detection window.

[0206] The calculation formula for the accumulation S of the original image in the rectangular region Rect(x1,y1,x2,y2) is,

[0207] (CC)S = (iiwt(x1,y1) - iiwt(x2,y1) - iiwt(x1,y2) + iiwt(x2,y2)) / Rt,

[0208] The calculation formula for the feature of the original image in the feature detection window containing N rectangular regions Rn(0≤n<N) is,

[0209] (CD)Ft = ∑0≤n<N(Wn*TISn) / Rt.

[0210] IIWT generated based on (F3) and its applications in the embodiments

[0211] When adopting this transformation form, it is possible to simultaneously improve the support ability for original images of larger sizes and improve the accuracy of calculations based on IIWT.

[0212] The cumulative S calculation formula for the original image in the rectangular region Rect(x1, y1, x2, y2) is as follows.

[0213] (CE)S = (iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2)) / Rt + (x2 - x1) * (y2 - y1) * Os

[0214] A calculation formula for the features of the original image in the feature detection window containing N rectangular regions Rn (0 ≤ n < N) is (CB). Among them, Sn can be calculated according to the coordinates of Rn through (CE). When ∑0≤n<N (Wn * An) is zero, Ft = ∑0≤n<N (Wn * Sn) = ∑0≤n<N (Wn * TISn) / Rt + ∑0≤n<N (Wn * An) * Os / Rt = ∑0≤n<N (Wn * TISn) / Rt.

[0215] That is, the used (CB) can be simplified to (CD).

[0216] Embodiment: IIWT generated based on (F4) and its applications

[0217] This transformation form is an equivalent deformation of (F3). When the absolute value of Rt is relatively large, RO is relatively large. When generating IIWT pixels, the coincidence degree of the significant digit sections of the two numbers for addition and subtraction, that is, the calculation result of oi(x, y) * Rt and RO, is large, reducing errors.

[0218] The cumulative calculation formula for the original image in the rectangular region Rect(x1, y1, x2, y2) is as follows.

[0219] (CF)S = (iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2) + (x2 - x1) * (y2 - y1) * RO) / Rt

[0220] A calculation formula for the features of the original image in the feature detection window containing N rectangular regions Rn (0 ≤ n < N) is (CB). Among them, Sn can be calculated according to the coordinates of Rn through (CF). The same as the case of IIWT based on (F3), when ∑0≤n<N (Wn * An) is zero, the used (CB) can be simplified to (CD).

[0221] For fast counting of non-zero values ​​in a rectangular region of a binary image whose pixel values ​​are only 0 and 1, Rt = 2 and RO = 1 can be used. This eliminates decimals and produces accurate results. For a binary image whose pixel values ​​are 0 and 255, 255 can be treated as 1.

[0222] For grayscale images converted from color images, where pixels are decimals, the IIWT can be used in integer pixels rather than floating-point pixels to represent the size of larger grayscale images. Rt can be set to 2^4, or 64. For example, if the approximate average value of the grayscale image is 195.52, RO is set to 195.52*64=12513.28. When generating the IIWT, the original image pixels are first multiplied by 64 (a computer-generated left shift of 4 bits), and then RO is subtracted. When calculating the cumulative value of a rectangular region of the original image based on the generated IIWT, the cumulative value of that rectangular region in the transformed image is divided by 64 (a computer-generated right shift of 4 bits). Because shift operations are computationally less complex than multiplication and division, using shift operations saves computational effort in generating and applying the IIWT.

[0223] When calculating the features of the feature detection window, if the exact value is not required, such as interest point detection, the features of the transformed image can be used directly without using the features of the original image.

[0224] Specific form of embodiment 1 (F5)

[0225] Taking the other transformation in (F5) as a square transformation as an example, the specific form of F(oi(x',y'),…) is (oi(x',y')-Os)^2.

Claims

1. A method for transforming integral images, characterized in that: The relationship between the transformed integral image and the original image is: x1<x’≤x2,y1<y’≤y2 F(oi(x',y'),…)=iiwt(x1,y1)-iiwt(x2,y1)-iiwt(x1,y2)+iiwt(x2,y2), where, iiwt represents the pixel of the transformed integral image; oi(x’, y’) represents the pixel of the original image OI; x1 < x2, y1 < y2; F(oi(x’, y’), …) represents the transformation performed on the pixel oi(x’, y’) of the original image, The specific form of the said F(oi(x’, y’), …) is one of (F1) to (F5): (F1) oi(x’, y’) - Os, (F2) oi(x’, y’) * Rt, (F3) (oi(x’, y’) - Os) * Rt, (F4) oi(x’, y’) * Rt - RO, (F5) (oi(x’, y’) - Os)^2, In (F1) to (F5), Os is the offset, Rt is the scaling factor, and RO is the result of multiplying Rt by Os.

2. The method for transforming integral images according to claim 1, wherein: Among the specific forms (F1) to (F5) of the said F(oi(x’, y’), …), The method for obtaining Os is one of (GA) to (GE): (GA) The median value of the value range of the pixels of the original image or its approximation, and the median value of the value range is the average of the maximum and minimum values of the value range, (GB) The average value of all pixels of the original image, (GC) The average value of the pixels uniformly sampled from the original image, (GD) If the Os obtained by the methods (GA) to (GC) is a decimal, round it to an integer, (GE) Manually set the value of Os according to needs; The method for obtaining the value of Rt is one of (GF) to (GH): (GF) To support large-sized original images, set Rt to a number less than 1, (GG) To obtain a transformed integral image with higher precision, set Rt to a number greater than 1, (GH) Take it as 2^N, where N is an integer other than 1, and replace the multiplication and division operations of Rt with shift operations in the computer; The method for obtaining the values of the pixels in the extended area of the transformed integral image is one of (GJ) to (GL): (GJ) All take the middle value of the value range interval of the transformed integral image, that is, the average of the upper and lower boundaries of the value range interval, (GK) When the pixels of the transformed integral image are signed numbers, all take zero, (GL) Manually set according to needs; In (F4), for the fast statistics of non-zero values in the rectangular area of a binary image with pixel values of only 0 and 1, take Rt = 2, RO = 1, and no decimals will appear in the calculation process, and the result is accurate.

3. The method for transforming integral images according to claim 1, wherein: The method for applying the said transformed integral image is: For any F(oi(x’, y’), …), the formula for calculating the cumulative TIS of the rectangular area R(x1, y1, x2, y2) of the transformed image of the original image is, (C3) TIS = iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2), Assume that the feature detection window Fw consists of N rectangular areas, and the method for calculating the feature TIFt of the transformed image TI is, <h2 style=";text-align:left;direction:ltr">(C4)TIFt=∑<h2 style=";text-align:left;direction:ltr"> 0≤n<N <h2 style=";text-align:left;direction:ltr"> (Wn*TISn), Where n represents the serial number, starting from 0, the weight of the nth rectangular area is Wn, the area of the nth rectangular area is An, the accumulation of the original image in the nth rectangular area is Sn, and the accumulation of the transformed image in the nth rectangular area is TISn. TISn is calculated according to the coordinates of the nth rectangular area according to (C3); The method for applying the transformed integral image corresponding to (F1) is as follows: Application 1. Calculate the accumulation of the rectangular area of the original image. The calculation formula for the accumulation S of the original image in the rectangular area R(x1, y1, x2, y2) is (CA) S = iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2) + (x2 - x1) * (y2 - y1) * Os, Application 2. The calculation formula for the feature Ft of the original image in the feature detection window containing N rectangular regions Rn (0 ≤ n < N) is (CB) Ft = ∑ 0≤n<N (Wn * Sn), Where Sn is calculated according to the coordinates of Rn according to (CA), When 0≤n<N When (Wn*An) is zero, Ft is equal to TIFt, and (CB) is simplified to (C4); The method for applying the transformed integral image corresponding to (F2) is as follows: Application 1. The calculation formula for the accumulation S of the original image in the rectangular area R(x1, y1, x2, y2) is (CC) S = (iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2)) / Rt, Application 2. The calculation formula for the feature of the feature detection window including N rectangular areas Rn (0 ≤ n < N) of the original image is (CD), (CD)Ft=∑ 0≤n<N (Wn*TISn) / Rt; The application method of the transformed integral image corresponding to (F3) is: Application 1. The calculation formula for the accumulation S of the original image in the rectangular area R(x1, y1, x2, y2) is (CE) S = (iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2)) / Rt + (x2 - x1) * (y2 - y1) * Os, Application 2. The calculation formula for the feature of the feature detection window including N rectangular areas Rn (0 ≤ n < N) of the original image is (CB), Sn is calculated according to the Rn coordinates according to (CE), when ∑ 0≤n<N When (Wn*An) is zero, Ft=TIFt / Rt, (CB) is simplified to (CD); the application method of the transformed integral image corresponding to (F4) is: Where Application 1. The calculation formula for the accumulation S of the original image in the rectangular area R(x1, y1, x2, y2) is (CF) S = (iiwt(x1, y1) - iiwt(x2, y1) - iiwt(x1, y2) + iiwt(x2, y2) + (x2 - x1) * (y2 - y1) * RO) / Rt, Application 2. The calculation formula for the feature of the feature detection window including N rectangular areas Rn (0 ≤ n < N) of the original image is (CB), Sn is calculated according to the Rn coordinates according to (CF), when ∑ 0≤n<N When (Wn*An) is zero, (CB) is simplified to (CD).

4. The method for transforming integral images according to claim 1, wherein: Where The transformed integral image includes one or all of features (B1) to (B2), (B1) There are pixels with negative values in the main area; 5. A storage medium, characterized in that: (B2) All pixels of Row(-1) or Col(-1) in the extended area are set to the same non-zero value.

6. A device based on the method for transforming integral images according to any one of claims 1 to 4, characterized in that: A program storing the transformed integral image, the method for generating the transformed integral image, or the method for applying the program as described in any one of claims 1-4.

7. The device according to claim 6, characterized in that During use, the program for generating the transformed integral image and / or the program for applying the method are run. The device is a computer system, including the following parts: (U1) Image acquisition module, (U2) Image preprocessing module, (U3) Transform integral image generation and / or application module, (U4) Machine Vision Module, (U5) Image stitching module, (U6) 3D reconstruction module, (U7) Other image application modules, in, (U1), (U2), (U4), (U5), (U6), (U7) are optional modules. (U3) contains at least one of (U31) to (U33), (U31) at least one transformed integral image, (U32) A computer program for a method of generating a transform integral image, (U33) A computer program for applying the method of transforming integral images.

Images