Object classification

The neural network training method for maritime vessel classification improves accuracy by diversifying prototypes using stochastic probability and inverse attention weighting, effectively handling pose variations and enhancing classification performance.

WO2026239102A1PCT designated stage Publication Date: 2026-11-19BAE SYSTEMS PLC
0 Cites 0 Cited by

Patent Information

Application Number
PCT/GB2026/050824
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-15
Filing Date
2026-05-13
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Current computer vision models struggle with fine-grained vessel recognition tasks and pose variation in maritime vessels, failing to match the accuracy of human classification.

Method used

A neural network training method that includes feature extraction, inverse attention weighting, and prototype learning with stochastic probability, power law transformation, and diversity loss to enhance classification accuracy.

Benefits of technology

The method achieves high accuracy in fine-grained object classification of marine vessels by diversifying prototype sets and addressing pose variations, outperforming existing methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure GB2026050824_19112026_PF_FP_ABST
    Figure GB2026050824_19112026_PF_FP_ABST
Patent Text Reader

Abstract

There is described a computer implemented method of training a neural network for object classification. The method comprises inputting a set of training images comprising objects to be classified into the neural network, wherein the network is configured to perform feature extraction on the set of training images, and learn a set of prototypes (14) based on the feature extraction, and learn gradients for prototype activation of the set of protypes based on inverse attention weighting.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] XA25076 Draft for filing

[0002] - 1 -

[0003] OBJECT CLASSIFICATION

[0004] FIELD

[0005] The present invention relates to a computer implemented method for object classification. In particular, the present invention relates to classification techniques to identify maritime vessels (or other vehicles).

[0006] BACKGROUND

[0007] Ship classification and maritime applications of computer vision is of increasing importance. However, current computer vision models of ship classification struggle with fine grained vessel recognition tasks, and dealing with inherent pose variation in vessels.

[0008] In essence, there is a disconnect between human approaches to vessel classification and computer vision approaches. Humans can readily deal with edge cases of classification, or distinguish between specific vessel designations within a particular class of vessel. Computer vision currently cannot.

[0009] A new approach to object classification which is tailored to fine grain recognition, particularly in the context of maritime computer vision, is highly desirable.

[0010] SUMMARY

[0011] Embodiments of the present invention are directed towards solving problems (identified above or otherwise) with fine grained object detection in objection classification tasks. In particular, embodiments of the present invention provide high accuracy object classification of marine vessels.

[0012] According to an aspect of the present invention, there is provided a computer implemented method of training a neural network, NN, for object classification. The method comprises inputting a set of training images comprising objects to be classified into the neural network. The NN is configured to perform feature extraction on the set of training images, and learn a set ofXA25076 Draft for filing

[0013] -2 -

[0014] prototypes based on the feature extraction, and learn gradients for prototype activation of the set of protypes based on inverse attention weighting.

[0015] In an example, learning the set of prototypes may comprise activating a previously learned prototype in the set, or activating a new prototype to be included in the set, based on a stochastic probability distribution.

[0016] In an example, a probability of activating a previously learned prototype in the set may be based on a frequency of activation of the previously learned prototype.

[0017] In an example, a damping factor may be applied to the probability of activating a previously learned prototype.

[0018] In an example, a prototype may be considered activated based on a layerwise relevance propagation analysis.

[0019] In an example, the inverse attention weighting may comprise a power law transformation.

[0020] In an example, training the NN comprises applying a loss function comprising a measure for diversity loss.

[0021] In an example, the diversity loss is based on a divergence between a current batch prototype activation distribution and a target distribution.

[0022] In an example, training the NN may comprise forward training on the set of training images to determine the set of prototypes, and back propagation to learn gradients for the determined set of prototypes based on the inverse attention weighting.

[0023] In an example, the method may further comprising determining whether a prototype in the learned set of prototypes is polysemantic.

[0024] In an example, determining whether a prototype in the learned set of prototypes is polysemantic may comprise determining gradient flows across layers of the NN.

[0025] In an example, determining whether a prototype in the learned set of prototypes is polysemantic may comprise clustering activated neurons based on the determined gradient flows.

[0026] XA25076 Draft for filing

[0027] - 3 -

[0028] In an example, the method may further comprise suppressing an undesirable semantic.

[0029] In an example, the machine learning model may be configured to classify an object as a type of marine vessel.

[0030] In an example, the learned prototypes may be component parts of marine vessels.

[0031] In a related aspect of the present invention, there is provided a computer implemented method of inference on a neural network trained as above.

[0032] As will be appreciated by one skilled in the art, the present techniques may be embodied as a system, method or computer program product. Accordingly, present techniques may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0033] Furthermore, the present techniques may take the form of a computer program product embodied in a computer readable medium having computer readable program code embodied thereon. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[0034] Computer program code for carrying out operations of the present techniques may be written in any combination of one or more programming languages, including object oriented programming languages and conventional procedural programming languages. Code components may be embodied as procedures, methods or the like, and may comprise sub-components which may take the form of instructions or sequences of instructions at any of the levels of abstraction, from the direct machine instructions of a native instruction set to high-level compiled or interpreted language constructs.

[0035] XA25076 Draft for filing

[0036] - 4 -

[0037] Embodiments of the present techniques also provide a non-transitory data carrier carrying code which, when implemented on a processor, causes the processor to carry out any of the methods described herein.

[0038] The techniques further provide processor control code to implement the above-described methods, for example on a general purpose computer system or on a digital signal processor (DSP). The techniques also provide a carrier carrying processor control code to, when running, implement any of the above methods, in particular on a non-transitory data carrier. The code may be provided on a carrier such as a disk, a microprocessor, CD- or DVD-ROM, programmed memory such as non-volatile memory (e.g. Flash) or read-only memory (firmware), or on a data carrier such as an optical or electrical signal carrier. Code (and / or data) to implement embodiments of the techniques described herein may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as Python, C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language such as Verilog (RTM) or VHDL (Very high speed integrated circuit Hardware Description Language). As the skilled person will appreciate, such code and / or data may be distributed between a plurality of coupled components in communication with one another. The techniques may comprise a controller which includes a microprocessor, working memory and program memory coupled to one or more of the components of the system.

[0039] It will also be clear to one of skill in the art that all or part of a logical method according to embodiments of the present techniques may suitably be embodied in a logic apparatus comprising logic elements to perform the steps of the abovedescribed methods, and that such logic elements may comprise components such as logic gates in, for example a programmable logic array or applicationspecific integrated circuit. Such a logic arrangement may further be embodied in enabling elements for temporarily or permanently establishing logic structures in such an array or circuit using, for example, a virtual hardware descriptor language, which may be stored and transmitted using fixed or transmittable carrier media.

[0040] XA25076 Draft for filing

[0041] - 5 -

[0042] In an embodiment, the present techniques may be realised in the form of a data carrier having functional data thereon, said functional data comprising functional computer data structures to, when loaded into a computer system or network and operated upon thereby, enable said computer system to perform all the steps of the above-described method.

[0043] The methods described above may be wholly or partly performed on an apparatus, i.e. an electronic device, using a machine learning or artificial intelligence model. The model may be processed by an artificial intelligence-dedicated processor designed in a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be obtained by training. Here, "obtained by training" means that a predefined operation rule or artificial intelligence model configured to perform a desired feature (or purpose) is obtained by training a basic artificial intelligence model with multiple pieces of training data by a training algorithm. The artificial intelligence model may include a plurality of neural network layers. Each of the plurality of neural network layers includes a plurality of weight values and performs neural network computation by computation between a result of computation by a previous layer and the plurality of weight values.

[0044] As mentioned above, the present techniques may be implemented using an Al model. A function associated with Al may be performed through the nonvolatile memory, the volatile memory, and the processor. The processor may include one or a plurality of processors. At this time, one or a plurality of processors may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an Al-dedicated processor such as a neural processing unit (NPU). The one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule or artificial intelligence (Al) model stored in the non-volatile memory and the volatile memory. The predefined operating rule or artificial intelligence model is provided through training or learning. Here, being provided through learning means that, by applying a learning algorithm to a plurality of learning data, a predefined operating rule or Al model of a desired characteristic is made. The learning may be performed in aXA25076 Draft for filing

[0045] - 6 -

[0046] device itself in which Al according to an embodiment is performed, and / or may be implemented through a separate server / system.

[0047] The Al model may consist of a plurality of neural network layers. Each layer has a plurality of weight values, and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[0048] The learning algorithm is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0049] BRIEF DESCRIPTION OF THE FIGURES

[0050] Embodiments of the invention will now be described by way of example only with reference to the following figures, in which:

[0051] Figure 1 shows an example electronic device;

[0052] Figure 2 shows an example method for training a neural network;

[0053] Figure 3 shows an example image for training a neural network, including examples of prototypes.

[0054] DETAILED DESCRIPTION

[0055] Figure 1 is a block diagram of a computing device 100 configurable to execute embodiments of the invention. The device will normally comprise, or be associated with, at least one processor 102, memory 104 and a communications interface 106. In some embodiments the computing device 100 may comprise, or be otherwise connected to, an image sensor 108.

[0056] XA25076 Draft for filing

[0057] - 7 -

[0058] The at least one processor 102 may comprise one or more of: a microprocessor, a microcontroller and an integrated circuit. The memory 104 may comprise volatile memory, such as random access memory (RAM), for use as temporary memory, and / or non-volatile memory such as Flash, read only memory (ROM), or electrically erasable programmable ROM (EEPROM), for storing data, programs, or instructions, for example. The communications interface 106 can provide data communication between the device and other devices / components, e.g. via a wireless or wired internet connection, a cellular network connection, or the like. The image sensor 108 may be configured to capture images (still or video) of an environment in view of one or more image capturing optics. The image may be captured on an integrated circuit, for example a charge coupled device pixel array or a complementary metal oxide semiconductor pixel array, or similar. Other components and features of the device, such as a housing, power source / supply, display, audio output, user interface, etc, will be well-known to the skilled person and need not be described herein in detail.

[0059] In some embodiments the computing device 100 may comprise a constrained-resource device, but which has at least the minimum hardware capabilities required to use a trained neural network / ML model. The device may be: a smartphone, tablet, laptop, computer or computing device, virtual assistant device, a connected camera, etc. The example device may be portable, or integrated into a vehicle such as a marine vessel. It will be understood that this is a non-exhaustive and non-limiting list of example devices.

[0060] Figure 2 shows an example method of training a neural network for object classification that can be performed by means of software instructions being executed by the computing device 100, according to an embodiment. However, in some embodiments, one or more of the steps may be performed by a remote computing device, such as a server or a cloud service, that is in communication with the device 100. It will also be appreciated that some of the steps described herein may be re-ordered or omitted, additional steps may be performed, and although the steps are shown as being performed in sequence, in alternative embodiments some of them may be performed concurrently, possibly on different processors or cores.

[0061] XA25076 Draft for filing

[0062] - 8 -

[0063] It will also be understood that embodiments can be implemented using any suitable software, programming language, data editors, etc, and may be represented / stored / processed using any suitable data structures and formats. The method can be initiated in any suitable manner, such as an application being opened by a user or called by another process. The steps may be executed by a stand-alone application or may be provided as part of a multi-function application.

[0064] At least some machine learning models discussed herein may be configured to perform an object classification operation. In an example, the machine learning may be optimised for object classification to identify marine vessels in a target image of a maritime scene. Herein, a maritime scene may be an image comprising at least one or a combination of sea, sky, islands, or parts thereof, one or more vehicles, such as maritime vessels, aircraft, and the like, as could be expected to be observed from a shoreline, a vessel, and so on. Vessels that may be identifiable by the present techniques include, for example, pleasure vessels, cargo vessels, fishing vessels, and military vessels.

[0065] At step 202, the method comprises inputting training data to the neural network. In the following, it is assumed that a training data set to be used to train the machine learning models for their intended purpose is readily available for such use. That is, for example, that suitable images for training machine learning models for object classification are available. Suitably, step 202 may be considered, if needed, to be prepended with a step of obtaining such training data. In an example, obtaining training data may comprise downloading the training data from a publicly available repository. In another example, obtaining the training data may comprise generating a bespoke data set using suitable techniques as will be familiar to those in the art. For example, obtaining training data may involve capturing training images using the image sensor 108 as part of the computing device 100. By way of example, in the present techniques a training dataset may comprise the Marvel dataset, or a subset thereof.

[0066] An example of a suitable image 10 for training maritime object classification is shown in Fig. 3 (overlaid with bounding boxes on prototype parts, which may or may not be included in the training images). Here the image 10 comprises a vessel 12 to be classified. The classification will be based on features extracted from the image 10 by the neural network. It will be appreciateXA25076 Draft for filing

[0067] - 9 -

[0068] that image 10 will be one of a plurality of images used to train the neural network. In some examples the neural network may be trained on separate batches of suitable images 10.

[0069] In an example the image 10 may be stored in the memory 104, and the step of obtaining the image 10 may comprise retrieving the image 10 from the memory 104 by the processor 102. In another example, obtaining the image 10 may comprise controlling the image sensor 108 to capture the image 10 to be stored in the memory 104 for later analysis or analysed substantially straight away in real time. In yet another example obtaining the image 10 may comprise receiving the image 10 through the communications interface 106.

[0070] In an example, the obtained image 10 comprises pixel data corresponding to the visible light spectrum. Or put another way, the image 10 was captured using an image sensor which is configured to capture light in the visible spectrum. Here the visible spectrum is taken to be from about 380 nm to about 700 nm. It will of course be appreciated that the obtained image 10 may comprise pixel data corresponding to other light spectrums; for example infrared wavelengths from about 700 nm to about 1 mm.

[0071] At step 204, the neural network performs feature extraction to learn a set of prototypes on which the object classification (at inference) will be based. Here prototypes takes its usual meaning in the field of prototype based object classification, as will be appreciated in the art. That is, broadly, a prototype may be taken to mean a characteristic part of an image which is used as a comparator by the neural network when performing feature extraction. Fig. 3 shows examples of prototype parts 14 which may be identified during training of a suitable network according to the techniques presented herein.

[0072] The goal of the present techniques is to learn a set of prototype parts which corresponds to parts of an object that are likely to be used in human based classification approaches, while maintaining high classification accuracy.

[0073] To that end, it is desired to achieve as diverse a set of (simple) prototypes as possible. By contrast, existing techniques for generating prototypes in prototype based image classification tend to collapse to an un-diverse set of few (complex) prototypes.

[0074] XA25076 Draft for filing

[0075] - 10 -

[0076] Suitably, in the present techniques, the set of prototypes may be learned on the basis of a stochastic probability distribution. The distribution may be divided into a probability of associating an extracted feature with an existing prototype in the set, and learning a new prototype. Moreover, the probability distribution may be weighted toward favouring the generation of new prototype parts, for example by having a damping factor applied to the probability of activating an existing prototype in the set. That is, a probability of activating a previously learned prototype in the set may be based on a frequency of activation of the previously learned prototype.

[0077] In more detail, consider a classification problem with K classes and a training set T containing N labelled images (x(1),y(1)),...,(x(N),y(N)) e X x Y. Assume each class is characterized by an underlying set of prototypical parts {(fci, Pi), ( Pn)}

[0078]

[0079] anassociated distribution representing the probability of a part’s presence in a random image of that class.

[0080] This can be modelled as the distribution p(y|x) = p(y|fc)p(fc|x)p(fc) =

[0081]

[0082] where p(fc|x) represents the probability of prototype k given input x, and p(y\k~) is the probability of class y given prototype k. Traditional approaches (for example PIPNet) assume that p(fc) is uniformly distributed across batches, which is often not the case due to factors such as variations in object pose, viewpoint, or inherent class imbalance.

[0083] To generate a diverse prototype set the present techniques attempt take the approach that classification may not necessarily based on a predetermined number of categories, but that new categories (i.e., prototypes) may be created as needed. Suitably, the present techniques apply a modified Chinese Restaurant Process (CRP).

[0084] In this process, features (customers in traditional CRP vernacular) extracted from the (training) activate a prototype (tables in traditional CRP vernacular); put more colloquially, the extracted image features may provided a correspondence, or association, a prototype (either existing or to be created). Each feature extraction activates a prototype with a probability proportional to the number of prior activations of that prototype, or activates a new prototype with a probability proportional to a predetermined concentration parameter a.

[0085] XA25076 Draft for filing

[0086] - 11 -

[0087] Formally, if z£represents the prototype activation for the ithfeature extraction, the conditional distribution of z£given previous assignments is

[0088]

[0089] where nkis the number of previous customers at table k. This formalisation for classical CRP exhibits a "rich-get-richer" property which can lead to uneven prototype utilization. To achieve more even partitioning of the prototype space, classical CRP may be modified to implement a power-law correction:

[0090] f i i ble k

[0091]

[0092]

[0093] , ces the rich-get-richer effect, and Z is a normalization constant. This modification encourages a more balanced exploration of the prototype space, ensuring that we discover and utilize a diverse set of prototypes rather than concentrating on a few dominant ones.

[0094] This may be further implemented mechanically as:

[0095] 1 for existing prototype i

[0096] t prototype ^previous activations} ™ i 3 >

[0097]

[0098] evenness in prototype utilization (typically set to 0.5 to encourage exploration, or else at least between 0.4 and 0.6, or between 0.3 and 0.7), and a controls the propensity to explore new prototypes.

[0099] In one example, frequency of activation may be determined by maintaining a running count of prototype activations. Here, a prototype may be considered "activated" when its similarity score with an input exceeds a threshold T (typically set to 0.3, although more generally between about 0.2 and about 0.4, or between about 0.1 and about 0.5). This formulation dynamically balances between exploiting established prototypes and exploring new ones, allowing the model to adapt to the inherent complexity of the data.

[0100] XA25076 Draft for filing

[0101] - 12 -

[0102] In another example, frequency of activation may be determined based on whether a prototype P makes a non-negligible contribution to a classification. This may be measured using a layer-wise relevance propagation (LRP) of subset of parts P c p with respect to class y when an accurate prediction is made.

[0103] It will be appreciated that the set of prototypes may be learned when performing a forward training pass of the input images.

[0104] At step 206, the method comprises learning gradients for prototype activation. This learning may be based on an inverse attention weighting scheme, which acts to counteract the natural tendency of gradient based learning techniques to focus on dominant patterns. In particular, the inverse attention weighting scheme may comprise applying a power law transformation which dampens the attention given to high frequency prototype activations when learning gradients.

[0105] More specifically, given the set of prototype embeddings P = (Pi> P2> ■■■ > Pm}> it is possible to determine c£, the activation frequency of prototype i during training, N =

[0106]

[0107] , the cumulative activation count, and / ■ =Ci / ^, the empirical activation distribution.

[0108] These parameters may then be used to apply a power law transformation which emphasizes rate prototypes:

[0109]

[0110] Where? > 0 controls the strength of the rare feature emphasis. For input x, the similarity between x and prototype p£produces activation a£(x), which may be modified to create an adaptive attention mechanism:

[0111] « (x) — H’s -«£(x) (5)

[0112] This approach effectively acts as an implicit regularizer that increases the gradient flow to underutilized prototypes during back-propagation, encouraging them to specialize in representing rare but potentially important features. Put another way, the step 206 may be taken to be performed during a back propagation step of training the neural network. From a Bayesian perspective,

[0113] XA25076 Draft for filing

[0114] - 13 -

[0115] this implements a form of posterior regularization that helps overcome the inherent bias toward frequent patterns.

[0116] In addition to the above, the training may also comprise a modified loss function compared to existing prototype based classification techniques.

[0117] Here, the standard loss function is extended with an explicit representation diversity term:

[0118]

[0119] The diversity loss may be implemented as a divergence between the current batch prototype activation distribution q and a target distribution that emphasizes rare prototypes:

[0120]

[0121]

[0122] at(xj) is the average activation of prototype i across the batch, and Ptargetj

[0123]

[0124] 'sthe target distribution that emphasises rare prototypes.

[0125] The divergence measures the information lost when using the distribution Ptarget to approximate the current prototype activation distribution q. Minimizing this divergence encourages the prototype activation pattern to match our desired target distribution that emphasizes rare features.

[0126] In practice, we compute the target distribution at the beginning of each epoch based on the cumulative activation statistics from previous epochs, ensuring that it reflects the current state of prototype utilization. The parameter [3 controls how aggressively we push toward uniform utilization, with higher values of [3 resulting in stronger emphasis on rare prototypes.

[0127] While the above processes theoretically allows for an infinite number of prototypes, practical implementations (e.g., resource limitations in terms of hardware capabilities, availability of source data, and so on) result in a finite set of prototypes being learned. This can lead to polysemanticity, where a single prototype captures multiple semantic concepts. This can cause problems withXA25076 Draft for filing

[0128] - 14 -

[0129] classification accuracy, trust in the mode, and downstream tasks which may desire (or require) mono semantic prototypes.

[0130] Suitably, the method may comprise an optional step 208 of identifying polysemantic prototypes in the set of learned prototypes; or put another way, the method may comprise determining whether a prototype in the set learned prototypes is polysemantic.

[0131] Here, it is assumed that if prototype k is polysemantic, then there exists at least one difference in the neural circuitry when processing different inputs that activate this prototype. In other words, the method may comprise identifying polysemantic prototypes, and determining the different semantics comprised in the prototype, based on analysis of neural activations.

[0132] To quantify these differences, we analyze the gradient flow through the network using an inputs x gradients technique under the assumption that different concepts have different gradient flows with respect to an image.

[0133] or a given active prototype k:

[0134]

[0135]

[0136] (x) is the Jacobian of prototype activations with respect to intermediate layer representations.

[0137] A weighted sum of gradients across multiple network layers may be

[0138]

[0139] where wtrepresents the layer-specific weight.

[0140] Gradient vectors may be normalized per layer to ensure clustering can be a combination of different layers based on a weighting vector W = w0,...,wk

[0141] Once gradient flows have been determined, it is then possible to cluster activated prototypes (that is, neurons corresponding to the prototypes) based on those gradient flows.

[0142] XA25076 Draft for filing

[0143] - 15 -

[0144] Here there is employed a two-stage approach to clustering using HDBSCAN and K-means clustering.

[0145]

[0146] Equation (9) corresponds to automatically determining the number of clusters without imposing constraints on cluster sizes, while equation (10) corresponds to refining the clusters using K-means with the HDBSCAN cluster assignments as initialization.

[0147] Suitably, with an understanding of what parts of the network activate for each of the semantics, it is then possible to modify the network response to a polysemantic prototype to e.g., suppress or split, learned activations based on the clustering; for example, by applying a damping factor based on a nearest neighbour analysis to the cluster corresponding to the semantic which it is desired to suppress.

[0148] To prove the efficacy of the results, we perform ablative studies against the models on the Fine-grained Marvel dataset (137 classes) and a subset of Marvel outlined in Janes Fighting ships (37 classes). Experiments are performed using prototype decorrelation, prototype focus, and completeness.

[0149] We tune the number of prototypes and loss components for each model using random search and Bayesian Optimization for each model tested. We follow previous studies by applying data augmentations (e.g. random rotation, skew, shear and left-right flip) on the full uncropped images at resolution 224 x 224 with online data augmentation. Since the labelling scheme is relatively high resolution, the minimum bounding box size is 10 x 10, the feature map resolution is 26 x 26. The hyperparameter for activation masks is kept as 95-th percentile for ProtoPNet based architectures and for PIPNet it is 0.3. In each scenario, we provide at least the minimum for each network to capture each prototype for Janes Marvel |C| = 37 and Military Marvel |C| = 137, with each class assumed to have 15 salient prototypes; the minimum number of total prototypes is 555 and 2055 respectively. To measure prototype activations for parts we use the intersection over union of a boundary box surrounding centered at the part (see e.g., Fig. 3).

[0150] XA25076 Draft for filing

[0151] - 16 -

[0152] For all models’ backbones we utilize both resnet50 and convnext26 backbones, pretrained on Imagenet dataset, and run experiments using a single RTX3090 GPU (24GB). When comparing against the current state of the prototypical parts-based models, we utilize the publicly available repositories when possible. For ST-ProtoPNet we chose to use one support and one trivial head as further heads caused instability. For the cases of PIPNet and CRP-PIPNet we used a batch size 24.

[0153] To highlight the present techniques ability to distinguish and model higher levels of prototypical diversity, we compare against state-of-the-art prototypical part-based models: PIPNet, SPARROW, ST-ProtoPNet. We perform experiments on the Jane only subset of the Military Marvel vessel dataset as well as the entire dataset.

[0154] Table 1

[0155]

[0156] Results can be seen in the tablel, where ptd is prototype decorrelation, how often different prototypes overlap (the highest score is each feature has only one prototype), Ptf is prototype focus, how many features each prototype represents (this is a measurement of prototypical purity), and Sac is completeness of sample description, how many features that we have labelled can we model.

[0157] The results indicate high accuracy throughout the prototype families, with PIPNet based models generally performing the best. We note that the number of effective prototypes modelled by all other methods, even when reducing the effects of sparsity, was narrow with CRP loss recording the highest results; however, this came at the expense of a reduced score. As a result, after applyingXA25076 Draft for filing

[0158] - 17 -

[0159] the splitting procedure, we were able to improve the score. Qualitatively speaking, the parts easily learned by all methods were mainly mast and structures visible from all viewpoints, examples of these can be seen in Fig. 3. Whereas CRP-PIPNet and to a lesser extent PIPNet, when adapted with the techniques herein, can model parts of increasing diversity.

[0160] In summary, the above has described a new technique for training an neural network for fine grained object classification, particularly as applied to vessel classification. The results show that the suggested framework outperforms existing methods.

[0161] Where, in the foregoing description, integers or elements are mentioned that have known, obvious, or foreseeable equivalents, then such equivalents are herein incorporated as if individually set forth. Reference should be made to the claims for determining the true scope of the present disclosure, which should be construed so as to encompass any such equivalents. It will also be appreciated by the reader that integers or features of the disclosure that are described as optional do not limit the scope of the independent claims. Moreover, it is to be understood that such optional integers or features, while of possible benefit in some embodiments of the disclosure, may not be desirable, and can therefore be absent, in other embodiments.

Claims

XA25076- 18 -CLAIMS1. A computer implemented method of training a neural network, NN, for object classification, the method comprising:inputting a set of training images comprising objects to be classified into the neural network, wherein the NN is configured to:perform feature extraction on the set of training images, and learn a set of prototypes based on the feature extraction, andlearn gradients for prototype activation of the set of protypes based on inverse attention weighting.

2. The method of claim 1, wherein learning the set of prototypes comprises activating a previously learned prototype in the set, or activating a new prototype to be included in the set, based on a stochastic probability distribution.

3. The method of claim 1 or 2, wherein a probability of activating a previously learned prototype in the set is based on a frequency of activation of the previously learned prototype.

4. The method of claim 3, wherein a damping factor is applied to the probability of activating a previously learned prototype.

5. The method of claim 3 or claim 4, wherein a prototype is considered activated based on a layer-wise relevance propagation analysis.

6. The method of any preceding claim, wherein the inverse attention weighting comprises a power law transformation.

7. The method of any preceding claim, wherein training the NN comprises applying a loss function comprising a measure for diversity loss.XA25076- 19 -8. The method of claim 7, wherein the diversity loss is based on a divergence between a current batch prototype activation distribution and a target distribution.

9. The method of any preceding claim, wherein training the NN comprises forward training on the set of training images to determine the set of prototypes, and back propagation to learn gradients for the determined set of prototypes based on the inverse attention weighting.

10. The method of any preceding claim, further comprising determining whether a prototype in the learned set of prototypes is polysemantic.

11. The method of claim 10, wherein determining whether a prototype in the learned set of prototypes is polysemantic comprises determining gradient flows across layers of the NN.

12. The method of claim 12, wherein determining whether a prototype in the learned set of prototypes is polysemantic comprises clustering activated neurons based on the determined gradient flows.

13. The method of any of claims 10 to 12, further comprising suppressing an undesirable semantic.

14. The method of any preceding claim, wherein the machine learning model is configured to classify an object as a type of marine vessel.

15. The method of claim 14, wherein the learned prototypes are component parts of marine vessels.