Systems and methods for providing an image classifier

Abstract

Systems and methods are provided for image classification using a histogram of oriented gradients (HoG) in combination with a trainer. By first establishing a bitmap that identifies a subset of pixels in a HoG window as including relevant foreground information, and limiting the HoG computation and comparison process to only include pixels in the bitmap, the efficiency of the process is greatly increased.

Description

[0001] This application is a divisional application of the original application filed in the China Patent Office on July 17, 2018 (international filing date of June 22, 2017) with the application number 201680079255.9 and titled “System and method for providing image classifiers.” TECHNICAL FIELD

[0002] The present application relates generally to computer vision processing, and more specifically to efficient image classification. BACKGROUND

[0003] The field of computer vision generally requires classifiers that are trained to detect objects such as faces and people, given the implementation of applications that interact with people and real-world objects. There are various classifiers because computer vision researchers have been seeking more resource-efficient methods to accurately locate and identify various objects in images.

[0004] Figure 1 One known method of identifying objects of a particular class described in the art uses Histograms of Oriented Gradients (HoG) in combination with training images and learning systems. HoG has been used to detect humans, as well as faces, animals, vehicles, and other objects, for a variety of backgrounds. Because HoG uses a relatively compact reference descriptor, it has been successfully used to locate objects in real-time classification of streaming video. It has also proven to enable robust detection in the presence of variations in rotation, scaling, and lighting conditions.

[0005] Figure 1 A process 100 known in the art for classifying objects in images using HoG in combination with a Support Vector Machine (SVM) algorithm, suitably referred to in the art as HoG / SVM, is shown. The described process uses the parameters identified by Dalal and Triggs in their 2005 article, “Histograms of oriented gradients for human detection,” International Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 886-893, June 2005, which is incorporated by reference in its entirety.

[0006] First, gradient values are calculated for each pixel within a particular cell (step 102 in Figure 1 As the process is shown in FIG. 1, the gradient values are calculated for each pixel in a 16x16 cell. The gradient values are then normalized to a range of 0 to 1. The normalized gradient values are then quantized to 9 bins, each representing a range of gradient values. The quantized gradient values are then histogrammed to produce a 36x36 histogram for each cell. The 36x36 histogram is then flattened to produce a 1x1296 vector for each cell. Figure 2As shown, the defined HoG detection window 202 is applied to a portion of an image, which divides the pixels into discrete cells 204. For example, the HoG cells 204 can include 8 pixels on each side, for a total of 64 (8 by 8) pixels per cell 204, although larger or smaller cell sizes can be selected in some implementations. For each cell 204, the magnitude and orientation of the gradient is computed. Various filters can be applied to compute these values. For example, as one implementation, the magnitude of the gradient |G| can be given in terms of the intensity values of its neighboring pixels as:

[0007] |G| = |G x + |G y + |G x+1 + |G x-1 + |G y+1 + |G y-1 .

[0008] And, the orientation θ can be given in terms of the tangents of the horizontal intensity x and the vertical intensity y as:

[0009] θ = arctan(|G y+1 – G y-1 | / |G x+1 – G x-1 |)

[0010] To create a histogram, the orientation angle θ is broken into a number of bins. In this example, the range of 0° to 180° is broken into 9 bins, each of 20°. Each intensity value |G| is added to the bin associated with its orientation angle θ (step 104 in Figure 1 ). The resulting HoG cell descriptor (as shown in 206 in Figure 2 ) has 9 values, each of which has a minimum value of zero and a maximum value of 128 times the maximum pixel intensity value.

[0011] Each of the cell descriptors 206 is then aggregated into a block descriptor 210 based on each 2 by 2 block 208 of four cells (step 106 in Figure 1 ). Because each block 208 of cells is used, cells 204 that are not on the edge of the window 202 will appear in four different blocks 208, and thus their descriptors 206 will be included in four different block descriptors 212.

[0012] Each block descriptor 210 is normalized according to the descriptors in that block (step 108 in Figure 1Each block descriptor 210 includes a descriptor 206 for each of the four cells 204 in the block 208. Various normalization algorithms can be used, many of which are discussed in the above-cited Dalal and Triggs, 2005 article. The result of this process is a normalized block descriptor 212 for each block 208, i.e., a set of histogram data representing 36 data elements for each block. Because normalization depends on the values of the four descriptors 206 in a particular block descriptor 210, the normalized value associated with a particular cell 206 can be different in each normalized block descriptor 212 that includes that cell.

[0013] For a 64 by 128 pixel window 202, the complete HoG descriptor 214 representing the normalized block descriptors 212 includes 105 normalized histogram data blocks: a total of 3780 data values. This complete descriptor 214 is fed to the SVM classifier previously evaluated on training images according to the same parameters (step 110 in FIG. 1). Figure 1 The training images can be any suitable set of training data for the object being evaluated, e.g., the MIT and INRIA image data sets described in the Dalal and Triggs, 2005 article. Other publicly available or proprietary training images can be used.

[0014] HoG computation is performed by repeatedly stepping across a 64 pixel wide by 128 pixel high window in the illustrated example, across the source image frame, and computing HoG descriptors as set forth in the previous section. Because HoG computation does not contain an intrinsic sense of scale, and objects can appear at multiple scales within an image, HoG computation is stepped across each level of a scale pyramid, and repeated across each level of the scale pyramid.

[0015] Figure 3 A window 302 is shown stepped across each level 304 of a scale pyramid 306. Each level 304 represents a further reduced copy of the image being scanned. The scaling factor between each level in the scale pyramid is typically 1.05 or 1.2 between one level and the next. The image is repeatedly reduced until the scaled source frame can no longer accommodate a complete HoG window.

[0016] The closed form for the number of pixels in the scaled pyramid is given by the expression based on s. s is the scale multiplier for the total number of pixels in the scale pyramid:

[0017]

[0018] a is a scaling factor used between pyramid levels. m = log(W / H) / log(a), where W and H are the width and height of the respective input image / video frame. Thus, the total number of pixels to be considered in the scale pyramid is s*W*H.

[0019] As Figure 2 As can be seen in the and shown in the above calculation, the HoG descriptor of the system using 9D histograms for each of the 7 by 15 4 by 4 patches in the image produces 3.78 kB descriptors for each 64 by 128 window examined in the incoming image.

[0020] The images used to train such classifiers are typically rectangular, as a byproduct of the 2D image sensor array used to capture the images. In addition to this, it is simple to step a rectangular descriptor across a rectangular source image and convolve to detect matches, and it is easy to understand why this paradigm has taken root. While some objects (e.g. furniture) can in fact be square or rectangular, most objects of interest in classification are not easily represented with simple geometric shapes. Thus, the matching of rectangular reference images to these objects is poor. In fact, the use of rectangular reference images means that a great deal of additional work must be done to convolve pixels that are irrelevant to the matching task, and furthermore these pixels mean that some of the background surrounding the object of interest is aliased into the descriptor for the matching image, confusing and degrading the accuracy of the matching operation.

[0021] The computational cost of each HoG dataset is very high. Dziri, Chevobbe and Darouich made an estimate in their 2013 article "Gesture recognition on smart camera", CEA LIST - Embedded Computing Laboratory, 2013. For example, applying HoG to a region of interest of 42 by 42 pixels requires the following operations: 11664 additions, 1296 multiplications, 5200 divisions, 16 square roots and 5184 arctangents. The computation requires many expensive and complex mathematical operations, such as divisions, square roots and arctangents, which take multiple cycles to implement in software on a traditional sequential processor. The computation also requires a large number of more common mathematical operations, such as additions and multiplications, which are typically performed in one clock cycle or less. The computational cost is complicated by the fact that a brute force search is performed by stepping the HoG template to compare over the entire image, which is computationally more expensive depending on the resolution of the input image. Furthermore, in a scene where objects can be seen at a range of distances, it is often necessary to search candidate windows of different sizes, which further increases the computational cost.

[0022] HoG / SVM is a very expensive operation. Many optimizations from changing scale factors to modifying the HoG window stride across the scaled source image step size of the blocks can be used to tailor the search space, thus limiting the computational effort. The combination of these factors means that robust real-time HoG is limited to very high-spec desktop systems that often offload the computation to a high-performance general-purpose processing unit (GPU). This makes the power cost far beyond the bounds of mobile devices such as cellphones, tablets, and mobile robots.

[0023] While it is possible to subsample the input image and perform a series of optimizations for mobile platforms, this often comes at a huge loss in matching accuracy, making the mobile implementation very limited in its practicality. Despite this, further optimizations are needed to limit the computational cost of the HoG process. SUMMARY

[0024] According to the disclosed subject matter, systems and methods are provided for image classification using HoG in combination with a trainer. By first establishing a bitmap that identifies a subset of pixels in a HoG window as including relevant foreground information, and limiting the HoG computation and comparison process to only include pixels in the bitmap, the efficiency of the process is greatly increased.

[0025] The disclosed embodiments include a computer-implemented method for image classification. The method includes generating, from a set of training images, a non-rectangular contour representing a relevant portion of an image window; generating, from the contour, a non-rectangular bitmap identifying less than all cells within a rectangular detection window, the rectangular detection window having a predetermined cell width and a predetermined cell height; receiving an image for classification, the image including a cell width and a cell height; applying the rectangular detection window to a portion of the image having the cell width and the cell height of the rectangular detection window, wherein applying the detection window includes identifying cells within the portion of the image that match the identified cells of the non-rectangular bitmap; generating, for each of the identified cells within the portion of the image, a histogram of orientations of pixels within the cell; establishing overlapping cell blocks, each cell block having a plurality of contiguous identified cells from the portion of the image; generating, for each cell block, a descriptor, the descriptor including a histogram of each of the identified cells in the cell block, wherein each histogram within the descriptor is normalized to all histograms within the descriptor; and evaluating the generated descriptor against the set of training images to determine whether to identify the portion of the image as including a class of objects associated with the set of training images.

[0026] In some embodiments, the method further includes rescaling the image to a different cell width and a different cell height; and applying the rectangular detection window to a portion of the rescaled image having the cell width and the cell height of the rectangular detection window.

[0027] In some embodiments, the method further comprises generating the non-rectangular contour by one of: averaging the set of training images, applying a computer vision algorithm, or applying deep learning to determine relevant portions of the image window.

[0028] In some embodiments, the method further comprises generating the non-rectangular contour by: determining, for each training image from the set of training images, intensity values of all pixels in each location; summing the intensity values of all pixels for each location from the set of training images to generate a result value; and dividing the result value for each location from the set of training images by a number of training images from the set of training images.

[0029] In some embodiments, the method further comprises generating, for each of the identified cells within the image portion, a histogram of orientations of pixels within the cell by: computing gradients, each gradient having a magnitude and a corresponding orientation; associating each corresponding orientation with one of a plurality of bins; and adding each magnitude to the one of the plurality of bins associated with the corresponding orientation.

[0030] In some embodiments, the method further comprises providing each cell patch as a two-by-two contiguous identified cell from the image portion.

[0031] In some embodiments, the method further comprises generating, for each cell patch, a descriptor by concatenating the histograms of each of the identified cells in the cell patch.

[0032] In some embodiments, the method further comprises determining that the object class is one of: a human; a face; a non-rectangular geometric shape; or a non-geometric shape.

[0033] The disclosed embodiments include a computing device for image classification. The computing device includes a memory configured to store: a set of training images; a non-rectangular contour representing a relevant portion of an image window from the set of training images; and a non-rectangular bitmap identifying less than all cells within a rectangular detection window, the rectangular detection window having a predetermined cell width and a predetermined cell height. The computing device also includes a hardware accelerator coupled to the memory, the hardware accelerator configured to: receive, as input, an image for classification, the image comprising a cell width and a cell height; apply the stored rectangular detection window to a portion of the image having the cell width and the cell height of the rectangular detection window by identifying cells within the portion of the image that match the identified cells of the non-rectangular bitmap; generate and store, for each of the identified cells within the portion of the image, a histogram of orientations of pixels within the cell; and establish overlapping cell blocks, each cell block having a plurality of contiguous identified cells from the portion of the image. The computing device also includes a processor coupled to the hardware accelerator and the memory, the processor configured to: generate, for each cell block, a descriptor, the descriptor comprising the histogram of each of the identified cells in the cell block, wherein each histogram within the descriptor is normalized to all histograms within the descriptor; and evaluate the generated descriptor against the stored set of training images in order to determine whether the portion of the image is identified as including a class of objects associated with the set of training images. The memory, the hardware accelerator, and / or the processor can also be configured to perform any and all of the steps described with respect to any of the above-described embodiments.

[0034] The disclosed embodiments include a non-transitory computer readable medium. The non-transitory computer readable medium has executable instructions operable to cause a data processing apparatus to: generate, from a set of training images, a non-rectangular contour representing a relevant portion of an image window; generate, from the contour, a non-rectangular bitmap identifying less than all cells within a rectangular detection window, the rectangular detection window having a predetermined cell width and a predetermined cell height; receive an image for classification, the image comprising a cell width and a cell height; apply the rectangular detection window to a portion of the image having the cell width and the cell height of the rectangular detection window by identifying cells within the portion of the image that match the identified cells of the non-rectangular bitmap; generate, for each of the identified cells within the portion of the image, a histogram of orientations of pixels within the cell; establish overlapping cell blocks, each cell block having a plurality of contiguous identified cells from the portion of the image; generate, for each cell block, a descriptor, the descriptor comprising a histogram of each of the identified cells in the cell block, wherein each histogram within the descriptor is normalized to all histograms within the descriptor; and evaluate the generated descriptor against the set of training images in order to determine whether the portion of the image is identified as comprising a class of objects associated with the set of training images. The executable instructions can be further operable to cause the data processing apparatus to perform any and all of the steps described with respect to any of the above-described embodiments.

[0035] The present application will now be described in greater detail with reference to specific embodiments of the application as illustrated in the accompanying drawings. While the following detailed description references implementing techniques in the context of certain embodiments and examples, the disclosure is not limited to these. Indeed, additional implementation examples of this disclosure can be derived from the teaching provided by this disclosure that can be readily implemented by persons of ordinary skill in the art, and that are supported in the claims. BRIEF DESCRIPTION OF DRAWINGS

[0036] The disclosed subject matter can be more completely understood in consideration of the following detailed description in connection with the accompanying drawings, in which like reference numerals designate like elements, and in which: The figures are schematic representations for purposes of the explanation and are not necessarily drawn to scale. For purposes of clarity, not every component can be labeled in every figure. In some instances, only the components necessary for an understanding of the disclosed subject matter have been shown. Every embodiment of the disclosed subject matter is not necessarily shown to scale in the drawings.

[0037] Figure 1 is a flowchart depicting an implementation of a HoG / SVM classification process as known in the art.

[0038] Figure 2An implementation of the HoG / SVM process as practiced in the art in the absence of a bitmap is shown.

[0039] Figure 3 A pyramid scaling scheme as known in the art is shown.

[0040] Figure 4 An outline produced by averaging over a set of training images according to an embodiment of the application is shown.

[0041] Figure 5 An implementation of the bHoG / SVM process applying a bitmap according to an embodiment of the application is shown.

[0042] Figure 6 is a flowchart depicting an implementation of the HoG / SVM classification process according to an embodiment of the application.

[0043] Figure 7 is a schematic diagram depicting hardware configured according to an embodiment of the application.

[0044] Figure 8 is a schematic diagram depicting hardware configured according to an embodiment of the application.

[0045] Figure 9 is a schematic diagram depicting hardware configured according to an embodiment of the application.

[0046] Figure 10 is a timing diagram depicting an operating bitmap scheduler for a typical bitmap according to an embodiment of the application.

[0047] Figure 11 is a flowchart depicting an implementation of the HoG / SVM classification process according to an embodiment of the application. DETAILED DESCRIPTION

[0048] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the systems and methods of the disclosed subject matter and of various ways in which they can be practiced. However, it will be apparent to those skilled in the art that the specific subject matter can be practiced without these specific details. In other instances, well-known features are not described in detail to avoid obscuring the specific subject matter. Furthermore, examples provided below are examples only and one can conceive of other systems and methods within the scope of the disclosed subject matter.

[0049] The present invention utilizes conditional processing using a bitmap that encodes those portions of an image that are used to train a classifier. The bitmap is derived by any suitable means. For example, the bitmap can be derived by averaging the images in a training data set. As another example, a computer vision algorithm or deep learning can be used to derive the bitmap to determine the location and extent of objects to be detected in the images. These means produce an outline of the objects, which is then thresholded to produce the bitmap. The bitmap can also take into account the underlying cell structure used in the Histogram of Oriented Gradients (HoG) computation.

[0050] Figure 4 A plot is shown with an outline generated from the average values according to the INRIA Pedestrian Detection image dataset (described in relation to the article by Dalal and Triggs, 2005, cited above). The plot shows a 48 by 96 pixel image 400, which is divided into 4 by 4 cells, for a total of 288 cells (48 / 4 by 96 / 4 = 12 by 24 = 288 4 by 4 cells). Of the 288 cells, only 92 cells are included in the foreground outline 402 produced by averaging the images in the dataset, representing 32% of the entire area of the image (92 / 288).

[0051] The foreground outline can be generated in any number of suitable ways. For example, the outline can be generated by summing the intensity values of all pixels in each location within the training images, and then dividing the resulting values by the number of images. The following article describes an "average image" that this process can generate: Average Explorer: Interactive Exploration and Alignment of Visual Data Collections, by Jun-Yan Zhu et al., ACM Transactions on Graphics, Volume 33, Issue 4, August 2014. For example, the MATLAB functions imadd() and imdivide() can be used to perform these functions on the data set, and output the results to a bitmap file for viewing. The outline shape can be automatically generated based on the results of averaging the data set pixel by pixel, computer vision, deep learning, or other suitable means or combination of means.

[0052] The outline is then converted to a HoG detection bitmap 500 as shown in Figure 5 wherein reference numerals 502-514 are relative to Figure 2The process shown is similar to that used above using reference numerals 202-214. Here, only 74 of the original 128 cells included in the HoG detection window 502 are included as foreground cells on the bitmap 500, representing 58% of the total area of the window 502 (74 / 128). Furthermore, when creating the histogram data 506, six orientation bins of 30° each are used, rather than nine of 20°, to further reduce the total size of the complete HoG descriptor 514. While covering the entire detection window 502 requires 105 overlapping 2x2 blocks, covering the bitmap requires only 74 such 2x2 blocks 508.

[0053] As Figure 5 shown, the complete bHoG descriptor includes only 1776 total reference values (74x24-D), a 3x reduction in the size of the descriptor compared to the entire HoG descriptor described above. Comparing these descriptors to reference images of similar size to the silhouette 400 by a SVM or another suitable learning algorithm yields a further efficiency improvement. Figure 4

[0054] By comparing only the relevant regions of the current image to only the relevant regions of the reference image, a significant reduction in processing cost and complexity can be achieved. In one experimental example, the bHoG / SVM classifier was trained using only those portions of the INRIA image database that do not contain background information, and compared to a HoG / SVM classifier trained in the conventional manner. The results showed a 75% reduction in the effort to perform pedestrian detection using the INRIA image set referenced above. A similar reduction in effort was found using the Daimler pedestrian segmentation benchmark dataset introduced in "PedCut: an iterative framework for pedestrian segmentation combining shape models and multiple data cues" by F. Flohr and D. M. Gavrila, Proc. of the British Machine Vision Conference, Bristol, UK, 2013.

[0055] The results shown apply without regard to higher level optimizations that can be implemented to reduce the dimensionality of the search space, and are easily accelerated in hardware, resulting in a potential 4x speedup in a hardware accelerated bHoG / SVM detector, with a corresponding 4x reduction in memory storage, memory bandwidth and power consumption.

[0056] Figure 6 ​is a flowchart depicting an implementation of the HoG / SVM classification process as described herein. Process 600 begins at step 602 with classifying an image. A detection window containing an arbitrary shape bitmap having an arbitrary shape contour is applied to a portion of the image. For example, a rectangular detection window containing a non-rectangular bitmap having a non-rectangular contour is applied to a portion of the image. The detection window is composed of cells and has a predetermined cell width and a predetermined cell height. The bitmap identifies less than all of the cells within the regular detection window. The contour, and thus the bitmap, can be any suitable arbitrary shape, including any geometric shape (non-rectangular shape), non-geometric shape, or a combination of geometric and non-geometric shapes. The application process only identifies cells within the image portion that match the identified cells of the bitmap. Next, at step 604, the process computes the gradient values (magnitude and orientation angle) for each pixel within the identified cells within the image portion. At step 606, the process then decomposes the orientation angles into a suitable number of bins, and adds each magnitude to the bin associated with its orientation angle. Each resulting HoG cell descriptor is then aggregated into a block descriptor at step 608. At step 610, each block descriptor (including its descriptor) is normalized using any suitable normalization algorithm. The resulting complete HoG descriptor representing the normalized block descriptor is then fed into the SVM classifier in step 110. In Figure 6 Step 604, 606, 608, 610, and 612 each perform similar functions as described above in connection with corresponding steps 102, 104, 106, 108, and 110 in Figure 1 but only for data corresponding to the identified cells of the bitmap corresponding to the contour, resulting in the optimizations, efficiency improvements, and reduction in process cost and complexity as described above.

[0057] Figure 7 A possible hardware block 700 for accelerated bHoG computation as described herein is shown. An input frame comprising an array of intensity values for a given cell and all neighboring cells is sent to a hardware bHoG accelerator 702, which includes modules for performing all necessary functions for computing gradients and generating orientation bins as described earlier. Certain modules can address and modify a plurality of HoG descriptors and associated bitmaps (i.e., bHoG descriptors) stored in system memory 704. The remaining steps of the bHoG / SVM process (generating descriptor blocks, block normalization, and SVM classification of bHoG data) can be performed by a VLIW processor 706 referencing the bHoG descriptors in system memory 704.

[0058] In Figure 8In this case, a possible hardware block for accelerating bHoG computation and histogram binning is shown, where the angle computation is performed and packed into memory for subsequent histogram binning in software. Integer divider 802 divides the theta angle computed for each pixel by the degrees associated with each histogram bin, in order to output a bin number - an integer between 0 and N, where N is the number of histogram bins. This output is then used to select from a number of accumulators 804 in the histogram bin to be updated at the next clock edge. The contents of these accumulators can then be read out at the end of the bHoG computation's bin via a memory-mapped processor interface 806.

[0059] This hardware can be extended by adding a bitmap scheduler 900 to support sparse bitmaps as shown in Figure 9 The bitmap counter 902 counts up to a max_count limit - the number of bits in the bitmap corresponding to the selected cell size. The output of the counter is "count" and is used to look up the bits loaded via the bitmap input at the start of the circuit operation from an n-bit bitmap array 904. The output of the bitmap array lookup 904 is the "HoG-enable" variable used to control the HoG computation bin, and the "X_adr" and "Y_adr" outputs used to look up the pixels required for the HoG operation in memory by enabling the output of AND gates 906 and 908. The inputs to these AND gates are generated by looking up "X_rel" and "Y_rel" relative addresses from a lookup table 910 loaded via the "lut_data" input at the start of the circuit operation. The "X_rel" and "Y_rel" addresses are added to "X_base" and "Y_base" inputs via a pair of adders 912 and 914.

[0060] Figure 10 is a timing diagram 1000 depicting the operation of a bitmap scheduler for an exemplary bitmap 1010. A bit stream 1012 is shown for the bitmap 1010, with the bits representing the first row of the bitmap 1010 labeled as a first row 1014 on the bit stream 1012. The portion of the timing diagram 1000 corresponding to the first row 1014 is labeled as a first timing portion 1002. As shown, a clock signal "clk" is used as a timer by sending alternating high and low signals as clock ticks. In this context, various other signals representing different variables are sent. Most notably, and in accordance with the Figure 9The block diagram of Figure 9, the "HoG_enable" signal as described above is used as a mask for the "X_adr" signal and the "Y_adr" signal. If the "HoG_enable" signal is a high value, corresponding to a "bitmap" of 1 value at that location, then the "X_adr" signal will be the sum of the "X_rel" signal and the "X_base" signal, and the "Y_adr" signal will be the sum of the "Y_rel" signal and the "Y_base" signal. Otherwise, if the "HoG_enable" signal is low, corresponding to a "bitmap" of 0 value at that location, then the "X_adr" signal and the "Y_adr" signal will also be low (i.e., signaling a zero value or no value).

[0061] Figure 11 Figure 11 is a flowchart describing an implementation of the HoG / SVM classification process as described herein. The process 1100 begins at step 1102, where the process generates an arbitrary shape (e.g., non-rectangular) contour representing a relevant portion of an image window from a set of training images. At step 1104, the process generates an arbitrary shape (e.g., non-rectangular) bitmap identifying less than all cells within a rectangular detection window having a predetermined cell width and a predetermined cell height from the contour. At step 1106, the process receives an image for classification, the image including a cell width and a cell height. At step 1108, the process applies a rectangular detection window to a portion of the image having the cell width and the cell height of the rectangular detection window. This includes identifying cells within the image portion that match the identified cells of the non-rectangular bitmap. At step 1110, for each of the identified cells within the image portion, the process generates a histogram of orientations of pixels within the cell. At step 1112, the process establishes overlapping cell blocks, each cell block having a plurality of contiguous identified cells from the image portion. At step 1114, the process generates a descriptor for each cell block, the descriptor including a histogram of each of the identified cells in the cell block. Each histogram within the descriptor is normalized to all histograms within the descriptor. At step 1116, the process evaluates the generated descriptor against the set of training images in order to determine whether the image portion is identified as including a class of objects associated with the set of training images.

[0062] To minimize the memory requirements associated with bitmap storage, if determined to be advantageous, run length encoding can be used to compress each of the runs in each row of the bitmap. In this case, a hardware and / or software run length decoder will be used to reconstruct the bitmap from the run length encoded version.

[0063] It should be understood that while several different arrangements have been described herein, features of each arrangement can be combined in various forms to achieve the advantages of the present application.

[0064] In the foregoing specification, the application has been described with reference to specific examples. It is evident, however, that various modifications and changes can be made thereto without departing from the broader spirit and scope of the application as set forth in the appended claims. For example, the connections can be any type of connection suitable to transfer signals from or to the respective nodes, units or devices, for example direct connections, indirect connections, optical connections, wired connections, wireless connections etc. The disclosure as set forth here can encompass multiple distinct

[0065] It should be understood that the architectures depicted herein are merely exemplary and that in fact many other architectures can be implemented which achieve the same functionality. In an abstract, but still definite sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermediate components. Likewise, any two components so associated can also be viewed as being "operably connected," or "operably coupled," to each other to achieve the desired functionality.

[0066] Furthermore, to the extent that the terms "including", "includes", "having", "has", "with", or variants thereof are used in either the detailed description and the claims, such terms are intended to be inclusive in a manner similar to the term "comprising" as an aid in

[0067] However, other modifications, changes and alternatives are also possible. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

[0068] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word “comprising” does not exclude the presence of elements or steps other than those listed in a claim. Further, the word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. Furthermore, as used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. The singular encompasses the plural, vice versa, and other similar pronouns. The use of the term “about” accompanying an entity or property shall not be construed as excluding the entity or property itself, unless otherwise indicated. The use of the term “or” in the claims is used to mean “and / or” unless otherwise indicated. In addition, the indefinite articles “a” and “an” are defined herein to mean one or more unless indicated otherwise. The use of the terms “first” and “second” does not limit the quantity and / or extent of those elements. The use of the terms “first” and “second” herein does not limit the quantity and / or extent of those elements. The modification “one or more of’ followed by a listing of elements does not limit the number of elements to one in every instance in which the modification is used. The mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A computer-implemented method for image classification, the method comprising: generating, from a set of training images, a non-rectangular foreground profile representing a relevant portion of an image window; generating, from the profile, a non-rectangular bitmap that identifies fewer than all cells within a rectangular detection window, the rectangular detection window having a predetermined cell width and a predetermined cell height; applying the rectangular detection window to a portion of an image to be classified having the cell width and cell height of the rectangular detection window, wherein applying the detection window comprises identifying cells within the image portion that match the identified cells of the non-rectangular bitmap; generating, for the identified cells within the image portion, a histogram of orientations of pixels within the cells; establishing overlapping cell blocks having a plurality of contiguous identified cells from the image portion; generating, for each cell block, a descriptor comprising a histogram of each of the identified cells in the cell block, wherein each histogram within the descriptor is normalized to all histograms within the descriptor; and evaluating the generated descriptors for the set of training images to determine whether the image portion is identified as including a class of objects associated with the set of training images.

2. The method of claim 1, further comprising: rescaling the image to a second cell width and a second cell height; and applying the rectangular detection window to a portion of the rescaled image having the cell width and the cell height.

3. The method of claim 1, further comprising: averaging the set of training images; and storing the averaged set of training images as the non-rectangular foreground profile.

4. The method of claim 1, further comprising: determining intensity values of pixels of the training images; summing the intensity values of pixels of the training images to generate a resultant value; dividing the resultant value by a number of the training images; and storing the divided resultant value as the non-rectangular foreground profile. generating the histogram of orientations of pixels within the cells for the identified cells within the image portion comprises: computing a gradient having a magnitude and a corresponding orientation; associating the orientation with a respective bin of a plurality of bins; and 5. The method of claim 1, wherein, adding the magnitude to a respective bin of the bins associated with a corresponding one of the orientations.

6. The method of claim 1, further comprising determining whether the class of objects is one of: a human; a face; a non-rectangular geometric shape; or a non-geometric shape. each cell block comprises two-by-two contiguous identified cells from the image portion.

8. A computing device for image classification, comprising: a memory comprising:

7. The method of claim 1, wherein, a non-rectangular foreground profile representing a relevant portion of an image window generated based on a set of training images, and a non-rectangular bitmap that identifies fewer than all cells within a rectangular detection window, the non-rectangular bitmap being generated from the profile, the rectangular detection window having a predetermined cell width and a predetermined cell height; ​ ​ ​ a hardware accelerator to: apply the rectangular detection window to a portion of an image to be classified having a cell width and a cell height of the rectangular detection window, wherein applying the detection window includes identifying cells within the image portion that match the identified cells of the non-rectangular bitmap; generate, for the identified cells within the image portion, a histogram of orientations of pixels within the cells; establish overlapping cell blocks having a plurality of contiguous identified cells from the image portion; generate, for each cell block, a descriptor including the histogram of each of the identified cells in the cell block, wherein each histogram within the descriptor is normalized to all histograms within the descriptor; and evaluate the generated descriptors against the set of training images to determine whether the image portion is identified as including an object class associated with the set of training images.

9. The computing device of claim 8, wherein, the hardware accelerator to: rescale the image to a second cell width and a second cell height; and apply the rectangular detection window to a portion of the rescaled image having the cell width and the cell height.

10. The computing device of claim 8, wherein, the hardware accelerator to: average the set of training images; and store the averaged set of training images as a non-rectangular foreground profile.

11. The computing device of claim 8, wherein, the hardware accelerator to: determine intensity values of pixels of the training images; sum the intensity values of pixels of the training images to generate a resultant value; divide the resultant value by the number of training images; and store the divided resultant value as the non-rectangular foreground profile. generating the histogram of orientations of pixels within the cells for the identified cells within the image portion includes:

12. The computing device of claim 8, wherein, computing a gradient having a magnitude and a corresponding orientation; associating the orientation with a respective bin of a plurality of bins; and adding the magnitude to a respective bin of the bins associated with a corresponding one of the orientations. the hardware accelerator to determine whether the object class is one of:

13. The computing device of claim 8, wherein, a human; a face; a non-rectangular geometric shape; or a non-geometric shape. each cell block includes two-by-two contiguous identified cells from the image portion.

14. The computing device of claim 8, wherein, 15. A non-transitory computer-readable medium including instructions that, when executed, cause a computing device to at least: generate, from a set of training images, a non-rectangular foreground profile representing a relevant portion of an image window; generate, from the profile, a non-rectangular bitmap identifying fewer cells than all cells within a rectangular detection window having a predetermined cell width and a predetermined cell height; applying the detection window includes identifying cells within the image portion that match the identified cells of the non-rectangular bitmap; applying the rectangular detection window to a portion of the image to be classified having the cell width and the cell height of the rectangular detection window, wherein generate, for the identified cells within the image portion, a histogram of orientations of pixels within the cells; establish overlapping cell blocks having a plurality of contiguous identified cells from the image portion; ​ generating a descriptor for each cell block, the descriptor including a histogram for each of the identified cells in the cell block, wherein each histogram within the descriptor is normalized to all histograms within the descriptor; and evaluating the generated descriptors for the set of training images to determine whether to identify the image portion as including an object class associated with the set of training images.

16. The non-transitory computer-readable medium of claim 15, wherein, the instructions, when executed, cause the computing device to: rescale the image to a second cell width and a second cell height; and apply the rectangular detection window to a portion of the rescaled image having the cell width and the cell height.

17. The non-transitory computer-readable medium of claim 15, wherein, the instructions, when executed, cause the computing device to: average the set of training images; and store the averaged set of training images as the non-rectangular foreground profile.

18. The non-transitory computer-readable medium of claim 15, wherein, the instructions, when executed, cause the computing device to: determine intensity values of pixels of the training image; sum the intensity values of the pixels of the training image to generate a result value; divide the result value by a number of the training images; and store the divided result value as the non-rectangular foreground profile. generating a histogram of orientations of pixels within an identified cell within the image portion includes:

19. The non-transitory computer-readable medium of claim 15, wherein, computing a gradient, the gradient having a magnitude and a corresponding orientation; associating the orientation with a respective bin of a plurality of bins; and adding the magnitude to a respective bin of the bins associated with a corresponding one of the orientations. the instructions, when executed, cause the computing device to determine whether the object class is one of:

20. The non-transitory computer-readable medium of claim 15, wherein, a person; a face; a non-rectangular geometric shape; or a non-geometric shape. ​

Images