Image classification

Abstract

A method of classification of a digital image is provided, the method comprising scanning pixel locations in the digital image; collecting regular groups of pixels near each scanned pixel location; providing the set of pixels to a full convolutional neural network; and training a fully convolutional neural network with the grouped samples of the pixels, thereby providing target classification of the grouped center pixel of the pixels.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the classification of images and the classification of objects in images. Background Technology

[0002] In the fields of image classification and pixel prediction, neural networks can be used to classify images and identify objects within them. However, the convergence time of these models can be long and / or the accuracy can be insufficient. Therefore, there is a desire to provide image classification solutions that can improve both model convergence time and / or model accuracy. Summary of the Invention

[0003] According to an embodiment of the first aspect, a method for classifying a digital image is provided, the method comprising: scanning pixel locations in the digital image; collecting regular groups of pixels near each scanned pixel location; providing the set of pixels to a fully convolutional neural network; and training the fully convolutional neural network with samples of the grouped pixels to provide a target classification of the center pixel of the pixel group.

[0004] Therefore, grouped pixels can be used instead of individual pixels to train a fully convolutional neural network. This training with grouped pixels provides richer contextual and texture information related to the target (center) pixel. This can benefit model convergence time and accuracy. For example, compared to a model where the network is trained with individual pixels, convergence time and accuracy can be improved. The trained network can be used to classify target pixels in an image (e.g., the center pixel of a group of pixels), which can be used to determine the material of an object in the image.

[0005] It is possible to scan pixel locations in a digital image to provide the scanned pixel locations. In other words, it is possible to scan one or more pixel locations in a digital image to provide one or more scanned pixel locations. It is possible to scan every pixel location in the image.

[0006] For each scanned pixel location, rule groups of pixels can be collected. If all pixel locations are scanned, then rule groups can be collected for each pixel location. Pixel location grouping can provide groups or sets of pixels near that pixel location. Pixel grouping can be an m×n set of pixel rules.

[0007] After collecting the pixel groups, they can be fed into a fully convolutional neural network (LCN) for training. More specifically, the LCN can be trained using samples of grouped pixels to provide target classification for the center pixel of each group. The target classification can be the classification of the center pixel within a pixel group. The classification can be output to a user and / or provided to another system for further processing. The model can be trained using multiple groups of pixels, and then the trained model can be applied to a single group to classify the center pixel of that group.

[0008] A classifier can include analyzing neighboring pixels to help determine the material of an object in an image. This contrasts with, for example, analyzing an image to determine what type of object it contains. Neighboring pixels can be grouped into groups of pixels near the scanned pixel location. Methods for classifying digital images can also be called image classifiers and / or texture classifiers.

[0009] This method may include pruning a fully convolutional neural network after training. It may also include analyzing neighboring pixels of groups of pixels after pruning. Based on this analysis, it may determine the type of material depicted in the digital image.

[0010] Therefore, the network can be pruned after training. Classification can then include analyzing the neighboring pixels of the grouped pixels and determining the type of material in the image. For example, determining the material of an object in the image.

[0011] Pruning the network after training can reduce its size while minimizing accuracy loss. A smaller network can have lower latency and throughput, enabling more efficient sorting.

[0012] Regular grouping of collected pixels may include identifying pixels from one or more regions of a digital image depicting one or more material types. Regular grouping of collected pixels may include assigning a label to each of the identified pixels. Regular grouping of collected pixels may include extracting neighboring pixels from each of the labeled pixels.

[0013] Therefore, pixels can be grouped based on the material type identified in the image (e.g., the material of objects in the image). Pixels associated with different material types can be identified and labeled, and neighboring pixels surrounding each pixel can be extracted to provide regular grouping of pixels.

[0014] This can be considered a method in which images are labeled. Since training is performed on small groups of pixels, it is not necessary to label the entire image. Instead, small regions within the image can be labeled, and the grouped pixels and their labels can be extracted from these regions.

[0015] This method may include converting the labels into one-hot codes. This can be done based on the practice of training multi-class classifiers.

[0016] Extracting neighboring pixels can include extracting a 3x3 patch of pixels from the area surrounding each marked pixel. That is, a 3x3 grid of pixels can be extracted such that a total of 9 pixels are extracted for each marked pixel, with the marked pixel located at the center of the 3x3 grid. In other words, the marked pixel and its immediate surrounding pixels can be extracted.

[0017] Feeding multiple pixels to a fully convolutional neural network can include feeding multiple pixels to a fully convolutional neural network with two or more layers. That is, the neural network can include two or more layers. The network can be fully convolutional, which means it can be directly applied to any image size. Therefore, the image size does not negatively affect the method.

[0018] Training a fully convolutional neural network can include creating multiple repetitions of regularly grouped pixels. Training a fully convolutional neural network can include applying multiple filters to the repetitions. Training a fully convolutional neural network can include performing depthwise convolution on the groups of pixels using filters. Multiple repetitions can be created from the extracted regularly grouped pixels.

[0019] Therefore, a neural network can be trained by repeating the process, applying filters, and performing convolutions. Alternatively, creating multiple repetitions and applying depthwise convolutions can be replaced by a single standard convolution. However, the weights in this convolution will be sparse, and the number of parameters associated with it will be much larger. Therefore, creating multiple repetitions and applying depthwise convolutions may be preferred.

[0020] Multiple repetitions can include up to eight repetitions. Multiple filters can include up to eight filters. Therefore, it is possible to quickly apply each of the eight filters to each channel via depthwise convolution.

[0021] The plurality of filters may include: i) a Gaussian blur filter with sigma equal to one; ii) a Gaussian blur filter with sigma equal to two; iii) a Gaussian-Laplace filter; iv) a Sobel filter with respect to the X direction; v) a Sobel filter with respect to the Y direction; vi) a second-derivative Hessian filter with respect to the X direction; vii) a second-derivative Hessian filter with respect to the Y direction; and viiii) a second-derivative Hessian filter with respect to the XY direction. The filters may be applied in any order and in any combination, such that the plurality of filters may include one or more of the listed filters.

[0022] The features extracted from each channel of an image via these filters can be standard filters in computer vision and image processing, used to infer texture, color, and shape information from the image.

[0023] This method may include normalizing the output feature map using a predetermined mean and standard deviation of the feature map.

[0024] Feature normalization prior to feature classification can prevent divergence during training by ensuring that the input values ​​are kept within a small hypersphere centered at the origin. The likelihood of gradient explosion caused by model weights applied to these input values ​​during training is reduced. The feature map can be the result of applying the first layer to the training data.

[0025] Training a fully convolutional neural network can include using backpropagation. That is, backpropagation can be used to train a fully convolutional neural network.

[0026] This method may include pruning the fully convolutional neural network after training. Pruning the fully convolutional neural network involves pruning it to a sparsity of up to 95%. The sparsity can be between 90% and 95%.

[0027] Accuracy loss at sparsity up to 95% is acceptable. Generally, sparsity is directly related to throughput. Balancing throughput and accuracy is a fundamental component considered in classifiers. Therefore, by pruning to sparsity up to 95%, a proper balance between throughput and accuracy can be achieved. That is, by pruning to sparsity up to 95%, throughput can be improved while maintaining acceptable accuracy.

[0028] According to an embodiment of the second aspect, an apparatus for classifying digital images is provided, the apparatus comprising: a component for scanning pixel locations in the digital image; a component for collecting regular groups of pixels near each scanned pixel location; a component for providing a set of pixels to a fully convolutional neural network; and a component for training the fully convolutional neural network with samples of the grouped pixels to provide a target classification of the center pixel of the pixel group.

[0029] The apparatus may include components for pruning a fully convolutional neural network after training. The apparatus may include components for analyzing neighboring pixels of a digital image after pruning. The apparatus may include components for determining the type of material depicted in the digital image based on the analysis.

[0030] According to an embodiment of the third aspect, an apparatus for classifying digital images is provided, the apparatus including a processor and a memory, the processor being configured to: scan pixel locations in the digital image; collect regular groups of pixels near each scanned pixel location; provide the set of pixels to a fully convolutional neural network; and train the fully convolutional neural network with samples of the grouped pixels to provide a target classification of the center pixel of the pixel group.

[0031] According to an embodiment of the fourth aspect, a computer-readable medium having instructions thereon is provided, the instructions, when executed by a processor, causing the processor to perform a method comprising: scanning pixel locations in a digital image; collecting regular groups of pixels near each scanned pixel location; providing a set of pixels to a fully convolutional neural network; and training the fully convolutional neural network with samples of the grouped pixels to provide a target classification of the center pixel of the grouped pixels.

[0032] Methodological features and sub-features can be applied to apparatus and computer-readable media in any combination, and vice versa. An apparatus or computer-readable medium according to a preferred embodiment may include any combination of methodological aspects. Methods or computer-readable media according to further embodiments may be described as computer-implemented because they require processing and memory capabilities. Aspects relating to computer-readable media can also be considered as computer programs. For example, according to one aspect of an embodiment, a computer program is provided that, when executed, performs a method for classifying digital images according to one aspect of an embodiment. The computer-readable medium may be a non-transitory computer-readable medium.

[0033] The apparatus according to the preferred embodiment is described as being configured or arranged, or simply "to", to perform certain functions. This configuration or arrangement can be implemented using hardware or middleware or any other suitable system. The configuration or arrangement can also be implemented in software.

[0034] Therefore, according to one aspect, a program is provided that, when loaded onto at least one computer, configures the computer to become an apparatus according to any one or any combination thereof of the preceding apparatus definitions. According to a further aspect, a program is provided that, when loaded onto at least one computer, configures the at least one computer to perform method steps according to any one or any combination thereof of the preceding method definitions. Generally, a computer may include the listed elements configured or arranged to provide the defined functions. For example, the computer may include memory, processing, and a network interface.

[0035] The device may include one or more processors, and the processes of the device may be executed by a single processor or by a combination of multiple processors. Thus, a single processor can execute one or more processes within the device's operations. A memory is communicatively connected to the one or more processors, for example, via a bus. The one or more processors can store and retrieve information from the memory, such as intermediate data generated, for example, during the execution of the device's processes.

[0036] Embodiments of the various aspects can be implemented in digital electronic circuit systems, or in computer hardware, firmware, software, or combinations thereof. Embodiments of the various aspects can be implemented as computer programs or computer program products, that is, computer programs tangibly implemented as non-transitory information carriers (e.g., in machine-readable storage devices) or implemented as propagating signals, for execution or control of the operation of one or more hardware modules.

[0037] A computer program can be a standalone program, part of a computer program, or more than one computer program. It can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a data processing environment. A computer program can be deployed to execute on a single module or on multiple modules at a single site, or distributed across multiple sites and interconnected via a communication network.

[0038] The method steps according to the embodiments of the various aspects can be executed by one or more programmable processors executing a computer program to perform the functions of the various aspects by manipulating input data and generating output. The apparatus according to the embodiments of the various aspects can be implemented as programmable hardware or a special-purpose logic circuit system, such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application-Specific Integrated Circuit).

[0039] Processors suitable for executing computer programs include, for example, both general-purpose and special-purpose microprocessors, as well as any one or more processors in any kind of digital computer. Generally, a processor receives instructions and data from read-only memory or random access memory, or both. The basic element of a computer can be viewed as a processor for executing instructions, coupled to one or more memory devices for storing instructions and data.

[0040] This invention has been described with reference to specific embodiments. Other embodiments are within the scope of the following claims. For example, the steps of the invention can be performed in a different order and still achieve the desired result.

[0041] The components have been described using the terms "memory," "processor," etc. Those skilled in the art will recognize that such terms and their equivalents can refer to spatially independent but combined components in a system that implement the defined functions. Similarly, the same physical components in a system can provide two or more of the defined functions. For example, separately defined components may be implemented using the same memory and / or processor, where appropriate. Attached Figure Description

[0042] Exemplary embodiments will now be described by way of example only with reference to the following figures, wherein:

[0043] Figure 1 This is a table showing examples of networks before and after trimming; and

[0044] Figure 2 This is a hardware diagram illustrating hardware that can be used to implement embodiments of the present invention. Detailed Implementation

[0045] Embodiments of this disclosure and their various features and advantageous details are explained more fully with reference to non-limiting examples described and / or illustrated in the accompanying drawings and detailed in the following description. It should be noted that the features shown in the drawings are not necessarily drawn to scale, and features of one embodiment may be employed in conjunction with other embodiments, as those skilled in the art will recognize, even if not explicitly stated herein. Descriptions of well-known components and processing techniques may be omitted so as not to unnecessarily obscure the embodiments of this disclosure. The examples used herein are intended only to facilitate an understanding of how embodiments of the invention can be practiced and to further enable those skilled in the art to practice such methods. Accordingly, the examples herein should not be construed as limiting the scope of embodiments of this disclosure, which is defined only by the appended claims and applicable law.

[0046] It should be understood that the embodiments of this disclosure are not limited to the specific methods, protocols, devices, apparatuses, materials, applications, etc., described herein, as these can vary. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claimed embodiments. It must be noted that, as used herein and in the appended claims, the singular forms “a,” “an,” and “the” include plural references unless the context clearly specifies otherwise. When used herein with reference to embodiments of various aspects, the words “comprises / comprising” and “having / including” are used to specify the presence of a said feature, integer, step, or component, but do not exclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this disclosure pertain. While preferred methods, apparatuses, and processes have been described, any methods, apparatuses, and processes similar or equivalent to those described herein may be used in the practice or testing of the embodiments.

[0048] Various embodiments may provide a method, apparatus, and computer-readable medium for classifying digital images. Image classification may involve determining the type of material depicted in the image. That is, an image can be classified based on the material of objects in the image.

[0049] Therefore, embodiments of the various aspects relate to image classification. They may also be referred to as image classifiers and / or texture classifiers. Other classifiers can extract spatial features from an image and pass these spatial features to classic machine learning-type algorithms for pixel prediction. In contrast, embodiments of the various aspects embed the entire process into a single network and pruning can be applied after training for feature selection.

[0050] The texture classifier is a fully convolutional network, meaning it does not contain dense layers. Instead, it consists of a series of convolutional layers where the input is convolved with an MxN filter. This allows the network to accept inputs of varying sizes and minimizes the number of network parameters. In contrast, fully connected networks consist of a series of dense or fully connected layers where linear operations are applied to the input of each layer. Linear operations require fixed-size inputs, meaning the target image size needs to be pre-determined before training the classifier. Therefore, texture classifiers, according to various embodiments, can offer advantages in terms of complexity, flexibility, and efficiency.

[0051] The process according to various embodiments can be described as follows.

[0052] For training:

[0053] 1. Obtain the (numerical) pixel-level labels of objects in the image. For 3A data, the pixel labels will simply be object labels. Convert these to one-hot encoding.

[0054] 2. Extract 3x3 patches from around each marked pixel. Pass these to a small, shallow, fully connected network.

[0055] 3. The first few layers of this network perform the following operations:

[0056] 1. “conv1d_2”: Takes a 6-channel image and repeats it 8 times. This is so that the 8 filters can be applied individually to each channel of the image via depthwise convolution in the next layer.

[0057] 2. “depthwise_conv2d_3”: Applies 8 predefined filters individually to each channel of the image—channels are unlinked at this stage. This allows for easier separation of “important” channels during the trimming stage. The raw data is fed to the network, allowing it to decide what to do. It has sufficient capability to extract eigenvalues ​​from the structure tensor (Sobel derivative matrix) and Hessian matrix, as well as Gaussian differences (if they are valuable). The filters are:

[0058] i. Gaussian blur with sigma=1

[0059] ii. Gaussian blur with sigma=2,

[0060] iii. Gauss-Laplace,

[0061] iv. Sobel on X

[0062] v. Sobel concerning Y,

[0063] vi. Hessian - the second derivative with respect to X

[0064] vii. Hessian - the second derivative with respect to Y

[0065] viii. Hessian - the second derivative with respect to X and Y.

[0066] 3. “depthwise_conv2d_4”: Normalizes the output of the previous layer using a predetermined mean and standard deviation. This is the only normalization layer in the network.

[0067] 4. All weights in the first 3 layers are not trainable.

[0068] 5. Train the network using the Adam optimizer.

[0069] 6. Before training, convert the labels to one-hot encodings and use categorical crossentropy loss for optimization.

[0070] 7. Balanced sparse class accuracy metrics can be decoded in the TensorFlow backend and used to test convergence.

[0071] 8. Once the network has been trained, it is possible to safely prune it to 90-95% sparsity without loss of accuracy. A few neurons will be completely eliminated, and the architecture can be modified accordingly, for example, in... Figure 1In this approach, 15 features are extracted from the pruned network to replace the 48 features in the original network. This pruning step can be seen as an alternative to a genetic algorithm, which can be used as a feature selection device. Pruning is also faster and more meaningful than the GA method because the objective function of GA depends on a balanced accuracy derived from a simple logistic regression classifier—it is impossible to train the network at every stage of GA. The logistic regression accuracy of the optimal feature set from GA often differs significantly from the accuracy found by training the network using that feature set. Assume a linear trend exists between the two.

[0072] Because the network is fully convolutional, it will accept any input with a height / width greater than 3 pixels. The input image will require a padding layer before being passed to the network to ensure the same output size.

[0073] Figure 1 This is a table showing examples of networks before and after trimming. Figure 1 In the case of 3A, consider 6 channels. Figure 1 The table on the left shows the network before pruning, and the table on the right shows the network after pruning. Inference can be performed in ~1ms per (3x1000x6) rows.

[0074] According to various embodiments, the classifier is developed to be performed in one pass via a neural network, which allows for a simpler computational system and enables faster and more efficient execution. This classifier extracts spatial features from an image and feeds them to a classic machine learning algorithm for pixel prediction. The entire process is embedded in a single network and pruned after training for feature selection.

[0075] This method can be trained on 3x3 patches of pixels. The first few layers of the network can be non-trainable. This method does not need to use NSCT or DCNN. This method can be used for pixel classification rather than image classification. The layers in the network do not need to be fully connected—the network is fully convolutional. Because the network is fully convolutional, it does not expect a fixed input size.

[0076] Perform a form of dimensionality reduction, but it is done through post-training pruning. This can be done without using semi-supervised learning, support vector machines, mNLDR, ensemble learning, or self-organizing networks.

[0077] This method can use a 2D convolution followed by a 1D convolution. It can eliminate the need for residual layers. It can avoid randomized fixed optimization to improve classification results. The network can be pruned after training.

[0078] This method can perform pruning without using inverse Hessian. This method can also avoid using synthetic images or image augmentation. This method can utilize spatial information. This method can be performed without using an autoencoder. This method can avoid comparing the input / test images with the training data; instead, it can use the training data to build a network and apply it directly to the test image for pixel classification.

[0079] This method can support supervised or unsupervised training. It may include pruning for optimization. It may include foreground segmentation using a backlight channel. It may include foreground erosion. It may include thresholding the target probability of classified pixels.

[0080] Figure 2 This is a block diagram of a computing device incorporating resources suitable for implementing image classification methods. It can implement aspects of the methods and can be used to implement some or all of the steps of the methods implementing the aspects, and perform some or all of the tasks of the apparatus of the embodiments. The computing device includes a processor 993 and a memory 994. Optionally, the computing device also includes a network interface 997 for communicating with other computing devices (e.g., with other computing devices of the embodiments of the aspects).

[0081] For example, an embodiment may consist of a network of such computing devices. Optionally, the computing device may also include one or more input mechanisms 996 and a display unit 995. The input mechanisms 996 enable a user to input data and instructions into the computing device. Components may be interconnected via a bus 992.

[0082] Memory 994 may include computer-readable media, a term that may refer to a single or multiple media (e.g., a centralized or distributed database and / or associated caches and servers) configured to carry computer-executable instructions or store data structures thereon. Computer-executable instructions may include, for example, instructions and data accessible and caused to perform one or more functions or operations by a general-purpose computer, a special-purpose computer, or a special-purpose processing device (e.g., one or more processors). Therefore, the term "computer-readable storage medium" may also include any medium capable of storing, encoding, or carrying a set of instructions executable by a machine and causing the machine to perform any or more of the methods disclosed herein. Accordingly, the term "computer-readable storage medium" may be understood to include, but is not limited to, solid-state memory, optical media, and magnetic media. For example, and not limited to, such computer-readable media may include non-transitory computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), optical disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, and flash memory devices (e.g., solid-state memory devices).

[0083] Processor 993 is configured to control computing devices and perform processing operations, such as executing code stored in memory to implement the various functions described herein and in the claims. Memory 994 stores data read and written by processor 993, such as inputs, intermediate results, and results of the processes referenced above. As mentioned herein, the processor may include one or more general-purpose processing devices, such as microprocessors, central processing units, etc. The processor may include complex instruction set computing (CISC) microprocessors, reduced instruction set computing (RISC) microprocessors, very long instruction word (VLIW) microprocessors, or processors implementing other instruction sets or combinations of instruction sets. The processor may also include one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. In one or more embodiments, the processor is configured to execute instructions for performing the operations and steps discussed herein.

[0084] The display unit 995 can display a representation of data stored by the computing device, and can also display a cursor, dialog boxes, and screen, thereby enabling interaction between the user and programs and data stored on the computing device. The input mechanism 996 allows the user to input data and instructions into the computing device. The display unit 995 and the input mechanism 996 can form an output.

[0085] The network interface (network I / F) 997 can be connected to a network (such as the Internet) and can be connected to other such computing devices via that network. The network I / F 997 can control data input / output with other devices via the network. The computing device may include other peripheral devices, such as microphones, speakers, power supply units, etc.

[0086] Implementing various methods can be done in ways such as Figure 2 It is executed on the computing device shown. This computing device does not need to have… Figure 2 Each component shown can be composed of a subset of those components. The methods for implementing these aspects can be performed by a single computing device communicating with one or more data storage servers via a network. The computing device itself can be a data storage device that stores input content before and after processing, thus, for example, storing dialogue and / or trained models.

[0087] The methods for implementing these aspects can be performed by multiple computing devices that operate collaboratively with each other. One or more of these computing devices can be data storage servers that store at least a portion of the data.

[0088] Generally, other hardware arrangements, such as laptops, iPads, and tablet PCs, can be provided alternatively. The software for performing the methods of the embodiments of the various aspects, as well as input content and any other required files, can be downloaded, for example, via a network such as the Internet or using removable media. Any dialogue or trained model can be stored, written to removable media, or downloaded via a network.

[0089] Implementation examples of each aspect can be applied to any field that uses image classification and pixel classification.

[0090] In practicing the principles and techniques described herein, those skilled in the art will understand and implement variations of the disclosed embodiments by studying the accompanying drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude multiple. A single processor or other unit may perform the functions of several items recited in the claims. The fact that certain measures are recited in mutually different dependent claims does not indicate that combinations of these measures cannot be advantageously used. Computer programs may be stored or distributed on suitable media, such as optical storage media or solid-state media provided with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems. Any reference numerals in the claims should not be construed as limiting the scope.

[0091] The embodiments described above can be used advantageously independently of any other embodiment, or in any feasible combination with one or more other embodiments.

Claims

1. A method for classifying digital images, the method comprising: Scan the pixel positions in a digital image; Collect regular groups of pixels near each scanned pixel location; The set of pixels is fed into the fully convolutional neural network; as well as The fully convolutional neural network is trained using samples of grouped pixels to provide target classification for the center pixel of the pixel group.

2. The method of claim 1, comprising: Prune the fully convolutional neural network after training; After trimming, analyze the neighboring pixels of the pixel group; as well as The analysis determines the type of material depicted in the digital image.

3. The method of claim 1 or 2, wherein the regular grouping of collected pixels comprises: Identify pixels from one or more regions of a digital image depicting one or more material types; Assign a marker to each of the identified pixels; as well as Extract neighboring pixels from each of the labeled pixels.

4. The method of claim 3, further comprising converting the tag into a one-hot code.

5. The method of claim 3 or 4, wherein extracting neighboring pixels comprises extracting a 3x3 patch of pixels from each of the surrounding pixels of the marked pixels.

6. The method of any of the preceding claims, wherein providing the plurality of pixels to the fully convolutional neural network comprises providing the plurality of pixels to a fully convolutional neural network of two or more layers.

7. The method of any of the preceding claims, wherein training the fully convolutional neural network comprises: Multiple repetitions of creating rule-based groupings of pixels; Apply multiple filters to the repetition; as well as Depthwise convolution is performed on groups of pixels using filters.

8. The method of claim 7, wherein The multiple repetitions include 8 repetitions; and The plurality of filters includes eight filters.

9. The method of claim 7 or 8, wherein the plurality of filters comprises: i) A Gaussian blur filter with sigma equal to 1; ii) A Gaussian blur filter with sigma equal to 2; iii) Gaussian-Laplace filter; iv) Sobel filter with respect to the X direction; v) Sobel filter in the Y direction; vi) Second-order derivative Hessian filter with respect to the X direction; vii) Second-order derivative Hessian filter with respect to the Y direction; as well as viii) Second-order derivative Hessian filter with respect to the XY direction.

10. The method of any of the preceding claims, comprising: The output feature map is normalized using a predetermined mean and standard deviation of the feature map.

11. The method of any of the preceding claims, wherein training the fully convolutional neural network comprises training the fully convolutional neural network using backpropagation.

12. The method of any of the preceding claims, comprising: After training, the fully convolutional neural network is pruned, where Pruning a fully convolutional neural network involves pruning it to a sparsity of up to 95%.

13. An apparatus for classifying digital images, the apparatus comprising: A component used to scan the pixel positions in a digital image; A component used to collect regular groups of pixels near each scanned pixel location; A component used to provide a set of pixels to a fully convolutional neural network; as well as This component is used to train a fully convolutional neural network with samples of grouped pixels, thereby providing a target classification of the center pixel of the pixel group.

14. The apparatus of claim 13, further comprising: Components used to prune fully convolutional neural networks after training; A component used to analyze neighboring pixels of a digital image after cropping; as well as Components used to determine the type of material depicted in the digital image based on the analysis.

15. A computer-readable medium having instructions thereon, which, when executed by a processor, cause the processor to perform the method as claimed in any one of claims 1 to 12.

Images