Image classification

Abstract

A method of classification of a digital image is provided, the method comprising scanning pixel locations in the digital image; collecting a regular grouping of pixels in the vicinity of each scanned pixel location; providing the collection of pixels to a fully convolutional neural network; and training the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels.

Description

[0001] IMAGE CLASSIFICATION

[0002] FIELD OF THE DISCLOSURE

[0003] Embodiments of the present invention relate generally to the classification of images and to the classification of an object in an image.

[0004] BACKGROUND

[0005] In the area of image classification and pixel prediction, neural networks may be used to classify images and to identify objects in images. However, the convergence time of models can be long and / or the accuracy of models may be insufficient. It is therefore desirable to provide image classification in which model convergence time and / or accuracy of the model may be improved.

[0006] SUMMARY

[0007] According to an embodiment of a first aspect, there is provided a method of classification of a digital image, the method comprising: scanning pixel locations in the digital image; collecting a regular grouping of pixels in the vicinity of each scanned pixel location; providing the collection of pixels to a fully convolutional neural network; and training the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels

[0008] Thus, a fully convolutional neural network may be trained using grouped pixels rather than single pixels. Such training using grouped pixels may provide richer contextual and textural information relating to target (center) pixels. This may benefit model convergence time and accuracy. For example, the model convergence time and accuracy may be improved compared with models in which the network is trained with single pixels. The trained network may be used to classify a target pixel in the image (e.g. a center pixel of a group of pixels) which may be used to determine the material of an object in the image. Pixel locations in the digital image may be scanned so as to provide scanned pixel locations. In other words, one or more pixel locations in the digital image may be scanned to provide one or more scanned pixel locations. Every pixel location in the image may be scanned.

[0009] A regular grouping of pixels may be collected for each of the scanned pixel locations. If all pixel locations are scanned, then a regular group for every pixel location may be collected. The grouping of pixel locations may provide a group or grouping of pixels in the vicinity of the pixel location. The grouping of pixels may be regular m x n groups of pixels.

[0010] After the groups of pixels have been collected, they may be provided to a fully convolutional neural network so that they may be used in the training of the network. More specifically, the fully convolutional neural network may be trained using samples of grouped pixels so as to provide a target classification of a center pixel of the grouping of pixels. The target classification may be the classification of the center pixel in the group of pixels. The classification may be output to a user and / or may be provided to another system for further processing. Multiple groups of pixels may be used to train the model, and then the trained model may be applied to a single group to classify the centre pixel of the group.

[0011] The classifier may comprise analyzing neighboring pixels to help make a determination about the material of an object in the image. This is in contrast to, for example, analyzing an image to determine what type of object the image is of. The neighboring pixels may be incorporated in the grouping of pixels in the vicinity of the scanned pixel locations. The method of classification of a digital image may also be referred to as an image classifier and / or a texture classifier.

[0012] The method may comprise pruning the fully convolutional neural network after training. The method may comprise analyzing neighboring pixels of the grouping of pixels after pruning. The method may comprise determining a material type depicted in the digital image based on the analyzing.

[0013] The network may therefore be pruned after the training. The classification may then comprise analyzing neighboring pixels of the grouped pixels and determining a material type in the image. For example, determining the material of an object in the image. Pruning the network after training may reduce the network size while minimizing accuracy loss. A reduced sized network may have a lower latency and throughput, which may enable more efficient sorting.

[0014] Collecting the regular grouping of pixels may comprise identifying pixels from one or more regions of the digital image depicting one or more material types. Collecting the regular grouping of pixels may comprise assigning labels to each of the identified pixels. Collecting the regular grouping of pixels may comprise extracting adjacent pixels from around each of the labelled pixels.

[0015] Thus, pixels may be grouped in accordance with material types identified in the image, e.g. materials of objects in the image. Pixels associated with different material types may be identified and labelled, and adjacent pixels surrounding each of the pixels may be extracted to provide the regular grouping of pixels.

[0016] This may be considered as a method in which the image is labelled. Since the training is performed on small groups of pixels, it may not be necessary to label the entire image. Instead, small regions within the image may be labelled and grouped pixels may be extracted, along with their labels, from these regions.

[0017] The method may comprise converting the labels to one-hot encoding. This may be performed in accordance with practices for training multi-class classifiers.

[0018] Extracting adjacent pixels may comprise extracting 3x3 patches of pixels from around each of the labelled pixels. That is, a 3x3 grid of pixels may be extracted such that a total of 9 pixels are extracted for each labelled pixel, with the labelled pixel at the centre of the 3x3 grid. In other words, the labelled pixel and the immediate surrounding pixels may be extracted.

[0019] Providing the plurality of pixels to the fully convolutional neural network may comprise providing the plurality of pixels to a 2 or more layer, fully convolutional neural network. That is, the neural network may comprise 2 or more layers. The network may be fully convolutional, which means that it may be applied directly to any image size. The size of the image may therefore not negatively impact the method. Training the fully convolutional neural network may comprise creating a plurality of repetitions of the regular grouping of pixels. Training the fully convolutional neural network may comprise applying a plurality of filters to the repetitions. Training the fully convolutional neural network may comprise performing a depthwise convolution on the grouping of pixels with the filters. The plurality of repetitions may be created from the extracted regular grouping of pixels.

[0020] The neural network may therefore be trained through a process of repetition, applying filters and performing convolution. Alternatively, creating a plurality of repetitions and applying a depthwise convolution may be replaced with a single, standard convolution. However, the weights in this convolution would be sparse and the number of parameters associated with it would be much larger. Therefore, it may be preferable to create a plurality of repetitions and applying a depthwise convolution.

[0021] The plurality of repetitions may comprise 8 repetitions. The plurality of filters may comprise 8 filters. Fast application of each of the 8 filters to each channel via a depthwise convolution may therefore be possible.

[0022] The plurality of filters may comprise: i) Gaussian blur filter with sigma equal to one; ii) Gaussian blur filter with sigma equal to two; iii) Laplacian of Gaussian filter; iv) Sobel filter with respect to an X direction; v) Sobel filter with respect to a Y direction; vi) Second derivative Hessian filter with respect to an X direction; vii) Second derivative Hessian filter with respect to a Y direction; and viii) Second derivative Hessian filter with respect to XY directions. The filters may be applied in any order and in any combination, such that the plurality of filters may comprise one or more of the listed filters.

[0023] The features extracted from each channel of the image, via these filters, may be standard filters in computer vision and image processing for inferring textural, color and shape information from that image.

[0024] The method may comprise normalizing an output feature map by a predetermined mean of the feature map and standard deviation of the feature map. Feature normalization prior to feature classification may prevent divergence during training by ensuring input values are kept within a small hypersphere centered at the origin. Model weights applied to these input values during training may be less likely to produce exploding gradients. The feature map may be the result of applying the 1st layer to the training data.

[0025] Training the fully convolutional neural network may comprise training the fully convolutional neural network with back propagation. That is, back propagation may be used to train the fully convolutional neural network.

[0026] The method may comprise pruning the fully convolutional neural network after training. Pruning the fully convolutional neural network comprises pruning to a sparsity up to 95%. The sparsity may be between 90% and 95%.

[0027] Loss in accuracy may be acceptable up to a sparsity of 95%. Generally, sparsity has a direct relationship with throughput. Balancing throughput with accuracy is a fundamental component in classifier consideration. Thus, an appropriate balance of throughput and accuracy may be achieved by pruning to a sparsity up to 95%. That is, the throughput may be improved and an acceptable accuracy may be maintained by pruning to a sparsity up to 95%.

[0028] According to an embodiment of a second aspect, there is provided an apparatus for classification of a digital image, the apparatus comprising: means for scanning pixel locations in the digital image; means for collecting a regular grouping of pixels in the vicinity of each scanned pixel location; means for providing the collection of pixels to a fully convolutional neural network; and means for training the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels.

[0029] The apparatus may comprise means for pruning the fully convolutional neural network after training. The apparatus may comprise means for analyzing neighboring pixels of the digital image after pruning. The apparatus may comprise means for determining a material type depicted in the digital image based on the analyzing.

[0030] According to an embodiment of a third aspect, there is provided an apparatus for classification of a digital image, the apparatus comprising a processor and a memory, the processor configured to: scan pixel locations in the digital image; collect a regular grouping of pixels in the vicinity of each scanned pixel location; provide the collection of pixels to a fully convolutional neural network; and train the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels.

[0031] According to an embodiment of a fourth aspect, there is provided a computer readable medium comprising instructions thereon which when executed by a processor cause the processor to carry out carry out a method of scanning pixel locations in the digital image; collecting a regular grouping of pixels in the vicinity of each scanned pixel location; providing the collection of pixels to a fully convolutional neural network; and training the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels.

[0032] Features and sub-features of the method aspects may be applied in any combination to the apparatus and computer-readable medium aspects and vice versa. An apparatus or computer- readable medium according to preferred embodiments may comprise any combination of the method aspects. Methods or computer-readable mediums according to further embodiments may be described as computer-implemented in that they require processing and memory capability. Aspects relating to computer-readable mediums may also be considered as computer programs. For example, according to an embodiment of an aspect there is provided a computer program which when executed carries out a method of classification of a digital image according to an embodiment of an aspect. The computer-readable medium may be a non-transitory computer-readable medium.

[0033] The apparatus according to preferred embodiments is described as configured or arranged to or simply ‘to’ carry out certain functions. This configuration or arrangement could be by use of hardware or middleware or any other suitable system. The configuration or arrangement may be by software.

[0034] Thus according to one aspect there is provided a program which, when loaded onto at least one computer configures the computer to become the apparatus according to any of the preceding apparatus definitions or any combination thereof. According to a further aspect there is provided a program which when loaded onto the at least one computer configures the at least one computer to carry out the method steps according to any of the preceding method definitions or any combination thereof. In general the computer may comprise the elements listed as being configured or arranged to provide the functions defined. For example this computer may include memory, processing, and a network interface.

[0035] The apparatus may comprise one or more processors and processes of the apparatus may be performed by a single processor or by multiple processors in combination. A single processor may therefore perform one or more of the processes of the apparatus. The memory and the one or more processors are communicably connected, for example, via a bus. The one or more processors may store and retrieve information from the memory, such as for example, intermediate data generated when performing the processes of the apparatus.

[0036] Embodiments of aspects may be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Embodiments of aspects may be implemented as a computer program or computer program product, i.e., a computer program tangibly embodied in a non-transitory information carrier, e.g., in a machine-readable storage device, or in a propagated signal, for execution by, or to control the operation of, one or more hardware modules.

[0037] A computer program may be in the form of a stand-alone program, a computer program portion or more than one computer program and may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a data processing environment. A computer program may be deployed to be executed on one module or on multiple modules at one site or distributed across multiple sites and interconnected by a communication network.

[0038] Method steps according to embodiments of aspects may be performed by one or more programmable processors executing a computer program to perform functions of the aspects by operating on input data and generating output. An apparatus according to embodiments of aspects may be implemented as programmed hardware or as special purpose logic circuitry, including e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0039] Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer may be seen as a processor for executing instructions coupled to one or more memory devices for storing instructions and data.

[0040] The invention is described in terms of particular embodiments. Other embodiments are within the scope of the following claims. For example, the steps of the invention may be performed in a different order and still achieve desirable results.

[0041] Elements have been described using the terms ‘memory’, ‘processor’, etc. The skilled person will appreciate that such terms and their equivalents may refer to parts of the system that are spatially separate but combine to serve the functions defined. Equally, the same physical parts of the system may provide two or more of the functions defined. For example, separately defined means may be implemented using the same memory and / or processor as appropriate.

[0042] BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Exemplary embodiments will now be described, by way of example only, with reference to the following drawings, in which:

[0044] Fig- 1 is a table showing an example of a network before and after pruning; and

[0045] Fig. 2 is a hardware diagram illustrating hardware that may be used to implement invention embodiments.

[0046] DETAILED DESCRIPTION

[0047] Embodiments of the present disclosure and the various features and advantageous details thereof are explained more fully with reference to the non-limiting examples that are described and / or illustrated in the drawings and detailed in the following description. It should be noted that the features illustrated in the drawings are not necessarily drawn to scale, and features of one embodiment may be employed with other embodiments as the skilled artisan would recognize, even if not explicitly stated herein. Descriptions of well-known components and processing techniques may be omitted so as to not unnecessarily obscure the embodiments of the present disclosure. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments of the present may be practiced and to further enable those of skill in the art to practice the same. Accordingly, the examples herein should not be construed as limiting the scope of the embodiments of the present disclosure, which is defined solely by the appended claims and applicable law.

[0048] It is understood that the embodiments of the present disclosure are not limited to the particular methodology, protocols, devices, apparatus, materials, applications, etc., described herein, as these may vary. It is also to be understood that the terminology used herein is used for the purpose of describing particular embodiments only, and is not intended to be limiting in scope of the embodiments as claimed. It must be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural reference unless the context clearly dictates otherwise. The words “comprises / comprising” and the words “having / including” when used herein with reference to embodiments of aspects are used to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0049] Unless defined otherwise, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art to which the embodiments of the present disclosure belong. Preferred methods, devices, and processes are described, although any methods, devices and processes similar or equivalent to those described herein may be used in the practice or testing of the embodiments.

[0050] Embodiments of aspects may provide a method, apparatus and computer readable medium of classification of a digital image. The classification of the image may relate to determining a material type depicted in the image. That is, the image may be classified by a material of an object in the image.

[0051] Embodiments of aspects therefore relate to image classification. They may also be referred to as an image classifier and / or a texture classier. Other classifiers may extract spatial features from an image and pass the spatial features to a classical machine learning type algorithm for pixel prediction. In contrast, according to embodiments of aspects, the entire process is embedded into a single network and pruning may be applied for feature selection post-training.

[0052] The texture classifier is a fully convolutional network, meaning the network does not contain dense layers. Instead, the network comprises of a series of convolutional layers in which the input is convolved with an MxN filter. This may allow the network to accept a varying sized input and may minimize the number of network parameters. A fully connected network, by contrast, consists of a series of dense or fully connected layers in which a linear operation is applied to the input of each layer. Linear operations require a fixed sized input, meaning the target image size would need to be predetermined before training the classifier. The texture classifier according to embodiments of aspects may therefore provide advantages in complexity, flexibility and efficiency.

[0053] The process according to embodiments of aspects may be described as follows.

[0054] For training:

[0055] 1. Get (numerical) pixel-level labels of the objects in the image. For 3 A data, the pixel labels would just be the object label. Convert these to one-hot-encoding.

[0056] 2. Extract 3x3 patches around each of the labelled pixels. Pass these to a small, shallow fully connected network.

[0057] 3. The first few layers of this network do the following:

[0058] 1. “convld_2”: takes the 6 channel image and repeats it 8 times. This is so that the 8 filters may be be applied individually to each channel of the image via a depthwise convolution in the next layer.

[0059] 2. “depthwise_conv2d_3”: apply 8 pre-determined filters to each channel of the image individually - the channels are not linked at this stage. This allows for easier separation of ‘important’ channels in the pruning stage. The raw data is passed to the network and the network is allowed to decide what to do. There is enough power in it to extract eigenvalues of the structure tensor (Sobel derivative matrix) and the Hessian matrix as well as a difference of Gaussian if they are of value. The filters are: i. Gaussian blur with sigma=l, ii. Gaussian blur with sigma=2, iii. Laplacian of Gaussian, iv. Sobel w.r.t X, v. Sobel w.r.t. Y, vi. Hessian - 2nd derivative w.r.t. X vii. Hessian - 2nd derivative w.r.t. Y viii. Hessian - 2nd derivative w.r.t. XY

[0060] 3. “depthwise_conv2d_4”: normalizes the output of the last layer by some predetermined mean and standard deviation. This is the only normalization layer within the network.

[0061] 4. All weights in the first 3 layers are not trainable.

[0062] 5. The network is trained with the Adam optimizer.

[0063] 6. Labels are converted to one-hot-encoding prior to training and the categorical cross entropy loss is used for optimization.

[0064] 7. The balanced sparse categorical accuracy metric may be coded in tensorflow backend and used to test for convergence.

[0065] 8. Once the network has been trained, it is possible to safely prune to 90-95% sparsity without loss of accuracy. Several neurons will be completely eliminated, and the architecture can be modified accordingly e.g. in Fig. 1, 15 features are extracted in the pruned network instead of 48 in the original. This pruning step may be seen as a replacement for Genetic Algorithm which could be used as a feature selection device. Pruning is also faster than the GA approach and makes more sense as the objective function for the GA relies on the balanced accuracy from a simple logistic regression classifier - it is not possible to train a network at each stage of the GA. The logistic regression accuracy from the optimal feature set from the GA was often vastly separated from the accuracy found by training the network with that feature set. It was assumed there was a linear trend between the two.

[0066] Because the network is fully convolutional, it will accept any input with a height / width greater than 3 pixels. The input image will need one layer of padding before being passed to the network to ensure the same sized output. Fig. 1 is a table showing an example of a network before and after pruning. In Fig. 1, for the 3 A, 6 channels are considered. The table on the left of Fig. 1 is the network before pruning and the table on the right is the network after pruning. Inference may be ~lms per (3x1000x6) line.

[0067] According to embodiments of aspects, the classifier is developed to do it in one pass through a neural network which may allow for simpler compute systems and to perform faster and more efficiently. This classifier extracts spatial features from an image and passes them to a classical machine learning type algorithm for pixel prediction. The entire process has been embedded into a single network with pruning for feature selection post-training.

[0068] The method may be is trained on 3x3 tiles of pixels. The first few layers of the network may not be trainable. The method may not use NSCT or DCNN. The method may be for pixel classification rather than image classification. The layer in the network may not be fully connected - the network is fully convolutional. Since the network is fully convolutional, it does not expect a fixed input size.

[0069] A form of dimensionality reduction is performed but it is done via pruning post-training. Semisupervised learning, support vector machines, mNLDR, ensemble learning or self-organizing networks may not be used.

[0070] The method may use one 2D convolution followed by ID convolutions. The method may not use residual layers. The method may not involve random fixed optimization to optimize classification result. The method may prune the network post-training.

[0071] The method may not use the inverse Hessian for pruning. The method may not use synthetic images or image augmentation. The method may use spatial information. The method may not use auto-encoders. The method may not compare input / test images to training data but rather the method may build a network with the training data and apply it directly to the test image for pixel classification.

[0072] The method may support supervised training or unsupervised training. The method may comprise pruning for optimization. The method may include use of a backlight channel for foreground segmentation. The method may comprise foreground erosion. The method may comprise thresholding the target probability of the classified pixel. Fig. 2 is a block diagram of a computing device incorporating resources suitable for implementing methods of image classification, which may embody aspects, and which may be used to implement some or all of the steps of a method embodying aspects, and perform some or all of the tasks of an apparatus of an embodiment. The computing device comprises a processor 993, and memory 994. Optionally, the computing device also includes a network interface 997 for communication with other computing devices, for example with other computing devices of embodiments of aspects.

[0073] For example, an embodiment may be composed of a network of such computing devices. Optionally, the computing device may also include one or more input mechanisms 996 and a display unit 995. The input mechanisms 996 may enable a user to input data and instructions to the computing device. The components are connectable to one another via a bus 992.

[0074] The memory 994 may include a computer-readable medium, which term may refer to a single medium or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to carry computer-executable instructions or have data structures stored thereon. Computer-executable instructions may include, for example, instructions and data accessible by and causing a general purpose computer, special purpose computer, or special purpose processing device (e.g., one or more processors) to perform one or more functions or operations. Thus, the term “computer- readable storage medium” may also include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methods of the present disclosure. The term “computer-readable storage medium” may accordingly be taken to include, but not be limited to, solid-state memories, optical media and magnetic media. By way of example, and not limitation, such computer-readable media may include non-transitory computer-readable storage media, including Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid state memory devices).

[0075] The processor 993 is configured to control the computing device and execute processing operations, for example executing code stored in the memory to implement the various different functions described here and in the claims. The memory 994 stores data being read and written by the processor 993, such as the inputs, interim results and results of the processes referred to above. As referred to herein, a processor may include one or more general purpose processing devices such as a microprocessor, central processing unit, or the like. The processor may include a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. The processor may also include one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. In one or more embodiments, a processor is configured to execute instructions for performing the operations and steps discussed herein.

[0076] The display unit 995 may display a representation of data stored by the computing device and may also display a cursor and dialog boxes and screens enabling interaction between a user and the programs and data stored on the computing device. The input mechanisms 996 may enable a user to input data and instructions to the computing device. The display unit 995 and input mechanisms 996 may form the output.

[0077] The network interface (network I / F) 997 may be connected to a network, such as the Internet, and may be connectable to other such computing devices via the network. The network I / F 997 may control data input / output from / to other apparatus via the network. Other peripheral devices such as microphone, speakers, power supply unit, etc. may be included in the computing device.

[0078] Methods embodying aspects may be carried out on a computing device such as that illustrated in Fig. 2. Such a computing device need not have every component illustrated in Fig. 2 and may be composed of a subset of those components. A method embodying the aspects may be carried out by a single computing device in communication with one or more data storage servers via a network. The computing device may be a data storage itself storing the input content before and after processing and thus for example, the dialogue and / or trained model. A method embodying aspects may be carried out by a plurality of computing devices operating in cooperation with one another. One or more of the plurality of computing devices may be a data storage server storing at least a portion of the data.

[0079] Other hardware arrangements, such as laptops, iPads and tablet PCs in general could alternatively be provided. The software for carrying out the method of embodiments of aspects as well as input content, and any other file required may be downloaded, for example over a network such as the internet, or using removable media. Any dialogue or trained model may be stored, written onto removable media or downloaded over a network.

[0080] Embodiments of aspects may be applied to any field which uses image classification and pixel classification.

[0081] Variations to the disclosed embodiments may be understood and effected by those skilled in the art in practicing the principles and techniques described herein, from a study of the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored or distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.

[0082] The above-described embodiments of aspects may advantageously be used independently of any other of the embodiments or in any feasible combination with one or more others of the embodiments.

Claims

CLAIMS1. A method of classification of a digital image, the method comprising: scanning pixel locations in the digital image; collecting a regular grouping of pixels in the vicinity of each scanned pixel location; providing the collection of pixels to a fully convolutional neural network; and training the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels.

2. The method of claim 1, comprising: pruning the fully convolutional neural network after training; analyzing neighboring pixels of the grouping of pixels after pruning; and determining a material type depicted in the digital image based on the analyzing.

3. The method of claim 1 or 2, wherein collecting the regular grouping of pixels comprises: identifying pixels from one or more regions of the digital image depicting one or more material types; assigning labels to each of the identified pixels; and extracting adjacent pixels from around each of the labelled pixels.

4. The method of claim 3, comprising converting the labels to one-hot encoding.

5. The method of claim 3 or 4, wherein extracting adjacent pixels comprises extracting 3x3 patches of pixels from around each of the labelled pixels.

6. The method of any preceding claim, wherein providing the plurality of pixels to the fully convolutional neural network comprises providing the plurality of pixels to a 2 or more layer, fully convolutional neural network.

7. The method of any preceding claim, wherein training the fully convolutional neural network comprises: creating a plurality of repetitions of the regular grouping of pixels; applying a plurality of filters to the repetitions; andperforming a depthwise convolution on the grouping of pixels with the filters.

8. The method of claim 7, wherein the plurality of repetitions comprises 8 repetitions; and the plurality of filters comprises 8 filters.

9. The method of claim 7 or 8, wherein the plurality of filters comprises: i) Gaussian blur filter with sigma equal to one; ii) Gaussian blur filter with sigma equal to two; iii) Laplacian of Gaussian filter; iv) Sobel filter with respect to an X direction; v) Sobel filter with respect to a Y direction; vi) Second derivative Hessian filter with respect to an X direction; vii) Second derivative Hessian filter with respect to a Y direction; and viii) Second derivative Hessian filter with respect to XY directions.

10. The method of any preceding claim, comprising: normalizing an output feature map by a predetermined mean of the feature map and standard deviation of the feature map.

11. The method of any preceding claim, wherein training the fully convolutional neural network comprises training the fully convolutional neural network with back propagation.

12. The method of any preceding claim, comprising: pruning the fully convolutional neural network after training, wherein pruning the fully convolutional neural network comprises pruning to a sparsity up to 95%.

13. An apparatus for classification of a digital image, the apparatus comprising: means for scanning pixel locations in the digital image; means for collecting a regular grouping of pixels in the vicinity of each scanned pixel location; means for providing the collection of pixels to a fully convolutional neural network; andmeans for training the fully convolutional neural network with samples of grouped pixels providing a target classification of a center pixel of the grouping of pixels.

14. The apparatus of claim 13 further comprising: means for pruning the fully convolutional neural network after training; means for analyzing neighboring pixels of the digital image after pruning; and means for determining a material type depicted in the digital image based on the analyzing.

15. A computer readable medium comprising instructions thereon which when executed by a processor cause the processor to carry out the method of any of claims 1 to 12.

Images