Object classification

Abstract

Examples described herein relate to methods and apparatus for classifying an object. In one example a method of classifying the object comprises obtaining a plurality of classifications for the object (110A – 110D), the classifications associated with different poses (136) (140); determining a pose similarity (865) between a first pose (836Y) associated with a first classification (870Y) of the plurality of classifications and a plurality of second poses (836W) associated with second classifications (870W) of the plurality of classifications, the first classification and the second classifications being different (142); and determining a common classification for the object dependent on the pose similarity (144).

Description

[0001]CLASSIFYING AN OBJECT Technical Field Examples of the present disclosure relate to visual object detection and classification. Background Visual object detection and classification may be useful in many fields including augmented or extended reality (XR), autonomous vehicles, robotics and surveillance systems. A geometric characterization or three dimensional (3D) representation of objects in an environment may be performed using simultaneous localization and mapping (SLAM) algorithms. This allows a device such as a Smartphone or extended reality devices such as Smart glasses to geometrically map the environment as 3D data and simultaneously localize itself within the environment. Machine learning models such as object detection and segmentation networks may then be used to semantically understand what those 3D data mean. Instance segmentation may be employed for classifying each pixel of an image into a set of object classes. Such an approach is described in Chilbane, Julian, et al. “Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation using Bounding Boxes.” Computer Vision-ECCV 2022: 17thEuropean Conference, Tel Aviv, Israel, October 23-27, Proceedings, Part XXXI. Cham: Springer Nature Switzerland, 2022. In an alternative approach, Pillai, S., & Leonard, J. (2015). “Monocular slam supported object recognition”. arXiv preprint arXiv:1506.01732 uses 2D images for semantic understanding, localizing the object in the image using bounding boxes. Object classification may be based on machine vision and machine learning technology including the use of deep learning models such as Mask Region Based Convolutional Neural Network (R-CNN) trained with large datasets to reach high classification accuracies. However, these models are still trained with limited data in which it may not be possible to generalize to recognize an object in every situation in real-world scenarios. This situation may be exacerbated in multi-view scenarios where there is significant variation in the object’s appearance across different views or if the object is occluded or partially visible in some views. This may cause subpar classifications in situations which they are not trained for, such as different viewpoints, lighting conditions, and object classes. Summary It is an object of the present disclosure to provide improved methods and apparatus for object classification. In one aspect there is provided a method of classifying an object as follows. Obtaining a plurality of classifications for the object, the classifications associated with different poses. Determining a pose similarity between a first pose associated with a first classification of the plurality of classifications and a plurality of second poses associated with second classifications of the plurality of classifications, the first classification and the second classifications being different. Determining a common classification for the object dependent on the pose similarity. Using the similarity (or dissimilarity) of poses as an input to classifying an object from multiple views improves the accuracy of classification. In another aspect there is provided an apparatus configured as follows. Obtain a plurality of 2D images of an environment, the obtained 2D images associated with different poses. Generate a first 2D display image based on a first obtained 2D image of the plurality of obtained 2D images and responsive to the pose of the apparatus within the environment corresponding to the pose associated with the first obtained 2D image. The first 2D display image is dependent on a first classification of an object detected in said first obtained 2D image. Subsequently to generating the first 2D display image, generate a second 2D display image based on a second obtained 2D image of the plurality of obtained 2D images and responsive to the pose of the apparatus within the environment corresponding to the pose associated with the second obtained 2D image. The second 2D display image is dependent on a second classification of an object detected in said second obtained 2D image, the first and second classifications being different; subsequently to generating the second 2D display image, generate a third 2D display image based on the first obtained 2D image of the plurality of obtained 2D images and responsive to the pose of the apparatus within the environment corresponding to the pose associated with the first obtained 2D image. The third 2D display image is dependent on the second classification. According to certain embodiments described herein there is also provided a computer program comprising instructions which, when executed on a processor, cause the processor to carry out methods described herein. The computer program may be stored on a non-transitory computer readable media. Brief Description of the Drawings For a better understanding of examples of the present disclosure, and to show more clearly how the examples may be carried into effect, reference will now be made, by way of example only, to the following drawings in which: Figure 1 is a schematic view of an object recognition and classification system for an environment and according to an example; Figure 2 illustrates different views for the environment of Figures 1, according to an example; Figures 3 – 7 illustrate sets of poses relative to an object with different classification associations, according to examples; Figure 8 illustrates determining a pose similarity between poses associated with different classifications, according to an example; Figure 9 is a flow chart for a method of classifying an object, according to an example; Figure 10 is a flow chart for a method of representing an object, according to an example; Figures 11A - D illustrate recognition and classification of objects in different views or from different poses, according to an example; Figure 12 illustrates different views of an object from different poses; and Figures 13 – 15 illustrate sets of poses associated with different classifications, according to examples. Detailed Description The following sets forth specific details, such as particular embodiments or examples for purposes of explanation and not limitation. It will be appreciated by one skilled in the art that other examples may be employed apart from these specific details. In some instances, detailed descriptions of well-known methods, nodes, interfaces, circuits, and devices are omitted so as not obscure the description with unnecessary detail. Those skilled in the art will appreciate that the functions described may be implemented in one or more nodes using hardware circuitry (e.g., analog and / or discrete logic gates interconnected to perform a specialized function, Application Specific Integrated Circuits (ASICs), Programmable Logic Arrays (PLAs), etc.) and / or using software programs and data in conjunction with one or more digital microprocessors or general purpose computers. Nodes that communicate using the air interface also have suitable radio communications circuitry. Moreover, where appropriate the technology can additionally be considered to be embodied entirely within any form of computer-readable memory, such as solid-state memory, magnetic disk, or optical disk containing an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein. Hardware implementation may include or encompass, without limitation, digital signal processor (DSP) hardware, a reduced instruction set processor, hardware (e.g., digital or analogue) circuitry including but not limited to application specific integrated circuit(s) (ASIC) and / or field programmable gate array(s) (FPGA(s)), and (where appropriate) state machines capable of performing such functions. Embodiments relate to classifying visual objects in an environment using a pose based approach to aggregating classifications from multiple views. This may improve the accuracy of object classification. Figure 1 is a schematic view of an object recognition and classification system for an environment and according to an example. A device or apparatus 120 moves through an environment 100 along a path indicated by 134. The environment comprises a number of objects 110A, 110B, 110C and the device 120 obtains a plurality of 2D images of the objects from different views 136. Each view 136 or 2D image is associated with a viewing position and orientation referred to herein as a pose within the environment. The device 126 may be configured to determine a 3D map of the environment, for example using a simultaneous localization and mapping (SLAM) algorithm. This will generate 3D data typically in the form of a point cloud in three dimensions of the environment, although a 2D map may be sufficient in some applications. Points in the point cloud may be clustered to determine objects within the 3D map. Different types of clustering may be employed, such as spatial clustering and / or texture or color based clustering. Feature recognition may also be utilized to determine the objects within the 3D data. In order to reduce subsequent processing, a sparse representation of these objects may be used. A sparse representation uses only some of the points that could be provided for in the 3D map, for example using a lower than maximum point resolution and / or object features such as edges. The device 120 may be configured to perform object recognition and classification – this may be performed on the 2D images to separately recognize objects in 2D, for example using segmentation or bounding box approaches. The recognized 2D objects may then be matched and mapped into a 3D representation of the objects. Segmentation for generating the object masks may be performed in any suitable manner, for example using a trained machine learning model such as the Mask R- CNN model. In other examples, bounding box techniques may be used for object recognition and classification, such as those described in Pillai. Alternatively, 3D object recognition and classification may be performed on dense or semi-dense 3D representations of the objects. The object recognition and 3D mapping functions may be performed in parallel; with their outputs used for further processing as described below. Whilst the device may perform these functions, in other examples, the device 120 may request object detection from an external source, such as a third party computing resource coupled to the device by a communications network. Similarly, the device may determine the 3D map of the environment by downloading this from an external source or using a non-SLAM algorithm. In a further alternative, a separate imaging device may be employed to obtain 2D images of the environment together with corresponding poses. The functionality of the device described below may also be distributed in other ways. The device 120 may display a 2D display image corresponding to an obtained 2D image, depending on the current pose of the device within the environment 100. The device 120 may be configured to display a representation of and / or object information about recognized objects 110A, 110B, 110C, 110D. The representation and / or object information may depend on how the recognized objects are classified. From some views 136, some objects may be partially occluded and so a representation or information about the object may be inaccurate. For example, a chair in which only one corner of the seat and a leg are seen from a first view may be incorrectly classified as a table and a representation of a table or information about a table displayed. From a second view, the chair may be fully seen and then correctly classified as a chair. A representation of a chair or information about a chair may then be displayed. As the device 120 moves about the environment 100, the representations of and / or object information about the object may change. In some examples, the device integrates the object recognition and classification data from multiple different views or poses 136 within the environment with geometric data determined from the 3D map. In some examples, points from an object determined within the 3D map may be projected into multiple obtained 2D images or object masks and matched with corresponding pixels in the 2D images or object masks. Those projected object points from the 3D map may then be associated with classifications associated with the matching pixels from the object masks. In some views or 2D images these classifications may be different. Alternatively, different classifications from different poses of a dense 3D representation of an object may be determined. A classification for each point of a determined or represented object or all points of the determined or represented object in the 3D map may then be determined based on a function of the corresponding classifications gathered from different poses. Figure 12 illustrates different poses relative to an object and corresponding views. The object 1210 is shown in plan view together with four poses 1236-1 – 1236-4 with different locations and orientations within the environment. A fifth pose 1236-5 is also illustrated which is similar to the first pose 1236-1. The other poses are quite different from each other, being rotated 90 or 180 degrees and on different sides of the object 1210. Views of the object from poses 1236-1 – 1236-4 are shown respectively in views 1250-1 – 1250-4. From the first pose 1236-1, based on view 1250-1, the object may be classified in many different ways given a lack of feature detail. For example, the object may initially be classified as a “knob”. However, as the views of the object 1210 move around poses 1236-2, 1236-3 and 1236-4, more detail is revealed in subsequent views 1250-2, 1250-3, 1250-4, and the object may be re-classified as a “helmet”. By accumulating classification information from different poses, a function may be used to allocate a common classification to the object 1210 based on the classifications associated with the various poses 1236-1 – 1236-4. This classification, such as “helmet” may therefore be retained even when returning to the first pose 1236-1 where the object was initially mis-classified as “knob”. However, the common classification may be negatively affected by inaccurate classifications from one or more of the poses. For example, if the object was initially correctly classified as “helmet” from pose 1236-3, then further incorrectly classified as ”knob” from poses 1236-1 and 1236-5, the incorrect classifications may override the initially correct classification. Another example is illustrated in Figures 11A – 11D, which show two views of an environment from two similar poses. The environment has various office furniture objects 1110A (table), 1110B (chair), 1110C (table), 1110D (chair), 1110E (divider), 1110F (chair) some of which are recognized and classified in each view. In an example, Mask R-CNN is used as the object detection model, which predicts the object labels, bounding boxes, segmentation mask and scores for each instance. Figures 11A and 11B are associated with a first pose or view of the environment. Figure 11A illustrates sparse reprojected points from a 3D map with respective classifications – crosses or “x” 1112T correspond to a “table” classification, small triangles 1112C correspond to a “chair” classification, small squares 1112L corresponds to a low prediction score or inconclusive classification and small circles 1116 correspond to noise. Figures 11B illustrates recognized objects with a classification for the view of Figure 11A. A prediction score for each of these recognized objects is also shown. The left most object 1110A is recognized and correctly classified as a “table” and the next object to the right 1110B is recognized and classified as a “chair”. Table 1110C is not recognized whilst the two additional chairs 1110D, 1110F are recognized. These are illustrated using the rectangular and rounded rectangular outlines 1112A and 1112B respectively. This may correspond with respective object masks for the recognized objects. Other unrecognized objects in the environment include a right-most chair and an office divider 1110F. Figures 11C and 11D are associated with a second, different pose or view of the environment. Figure 11C illustrates sparse reprojected points from the 3D map for this view and with respective classifications. Figure 11D illustrates recognized objects with a classification for the view of Figure 11C. The left most object 1110A is again recognized as a table but with a lower prediction score. This corresponds to less of the object being visible from this second pose. The next object to the right 1110B is again recognized and classified as a “chair”, with a higher prediction score. The table 1110C to the right of this chair is now recognized in this view, however chair 1110D is again recognized. The divider or partition 1110E is incorrectly recognized and classified as part of a “chair” – illustrated by the rounded rectangular outlines 1112E. As the divider was not recognized in the first view of Figures 11A, 11B, if the incorrect classification 1112E was used as the common classification for this object, the common classification would be incorrect and this may be used when considering other poses. This may occur by applying simple functions to accumulated classifications of an object, for example a majority decision function. Finally, chair 1110F is correctly recognized with an improved prediction score in this view, corresponding to more of this object being visible. More generally, regardless of the method for multi-view aggregation used, a common classification may not provide an accurate classification for an object, especially if there is significant variation in the object's appearance across different views or if the object is occluded or partially visible in some views. Where a spurious prediction such as 1112C is the only one recognized and classified among a number of views (for example five views), it may be propagated to other views where it was not seen before, regardless of the predicting score. In examples, improved methods, algorithms or functions for determining a common classification from a plurality of classifications from different views or poses may be employed to improve the accuracy of object classification. This may incorporate a number of factors such as pose similarity, prediction scores of the respective input classifications, as well as considering the impact of inconclusive classifications where a recognized object cannot be clearly classified. In some examples, a pose similarity between poses associated with multiple classifications for a determined object is used to determine an updated or common classification for the object. For example, an object may be classified as a “chair” from some poses and a “table” from other poses. The similarity between at least some of these poses may be used to determine a common classification for the object. In one example, the classification associated with one pose may be pruned from a set of classifications if this is different to a classification associated with a plurality of very similar poses. If a plurality of poses having a similar position and orientation relative to an object are associated with a “chair” classification, and a single similar pose is associated with a “table” classification, this indicates that the “table” classification is inaccurate. This one table classification may then be removed or pruned from the set of classifications. The updated set of classifications may then be used to determine a common classification for the object, for example using majority voting. On the other hand, if the “table” classification is associated with a very different pose, then pruning of this from the set of classifications may not be appropriate. Referring again to Figure 1, once a common classification has been assigned to the object determined in the 3D map, this may be used for various applications such as updating a display or the classification of all matching pixels in the object masks from the obtained 2D images. This may improve the semantic understanding of the environment, including for example the accuracy of object masks, the accuracy of classification of 3D object representations, improved displayed representations of the objects and / or displayed object information. In an example, the device 120 of Figure 1 comprises a processor 122, memory 124, a 2D image capture device 130 such as a camera, and a display 132. The memory 124 comprises instructions 126 and a 2D object recognition model 128. The memory may also comprise a SLAM algorithm. The instructions 126, when executed on the processor 122, may cause the processor to carry out a number of steps 140 – 144. At 140, the processor obtains a plurality of 2D images of an environment 100 and which are associated with different poses or views 136. A pose for a 2D image may be determined using accelerometers, gyroscopes, GPS coordinates, movement tracking, cameras and any other suitable mechanism associated with the device capturing the 2D image. At 142, the processor determines a pose similarity between a pose associated with a first classification of the plurality of classifications and a plurality of poses of a second classification of the plurality of classifications, where the first and second classifications are different. The pose similarity may be used to determine how to handle the first classification, for example whether to prune this from a set of classifications. At 144, the processor determines a common classification form the object dependent on the pose similarity. For example, the common classification may be derived from a set of classifications from which some classifications have been pruned based on the pose similarity between the pose associated with the pruned classification and other poses having a different classification. In one example, the common classification may be determined from the pruned set of classifications by using a majority decision, however other functions are possible as described in more detail below. These may include majority estimation likelihood as well as weighting the input classifications based on various factors such as the distance from their associated poses to the object or the number of pixels in a corresponding object mask. As noted, these instructions or functions may be performed within the device 120, or may be distributed across other devices such as external computing resources, data stores for 3D maps or 2D images and so on. Figure 2 illustrates a number of different 2D images 250-1 – 250-4 of an environment, the 2D images associated with different poses within the environment. These images are simplified for ease of explanation and show different representations of objects 210A, 210B, 210C, 210D. These objects may correspond to the office furniture shown in the views of Figure 11, but as not so limited and may represent any types of objects. Figure 3 illustrates the poses 336X, 336Y relative to object 210B. Object 210B in this example may correspond to the helmet of Figure 12. It can be seen that the four poses are similar, having a similar location within the environment and orientation towards the object. In this example, one of the poses 336Y is associated with a first classification – indicated with the black and white stripped fill – and three of the poses 336X are associated with a second classification – indicated with the black fill. Because all of these poses 336X, 336Y are similar and therefore all have similar views of the object 210B, it is likely that the minority classification associated with pose 336Y is incorrect and that the majority classification associated with the three other poses 336X is accurate, or at least more likely to be correct. In this situation, the minority classification may be pruned to improve the accuracy of the object classification obtained from the other poses. Figure 4 shows an alterative arrangement of poses 436X, 436Y and classifications for the object 210B from different views. Again, three poses 436X are similar and are each associated with the same classification. However, the fourth pose 436Y, whilst associated with a different classification, is not similar to the other three poses – being some distance from the three other poses 436X and having a different orientation towards the object. Because the pose similarity between the pose 436Y having the first classification and the other three poses 436X having the second classification is much lower than the situation in Figure 3, it is more likely that the first classification (e.g. helmet”) of pose 436Y may be accurate because it may have access to additional object detail not visible from the other three poses 436X which therefore classify the object according to a less accurate second classification (e.g. “knob”). Prediction scores of the different classifications may also be used to help allocate a common classification. For example, if the score for the first prediction or classification (e.g. “helmet”) is significantly higher than that of the second classifications (e.g. “knob”), the first classification (“helmet”) may be more accurate. However, if the score of the first classification is not significantly different from the scores of the second classifications, then the object may be more likely correspond with the second classification (“knob”). In this case, the first classification (“helmet”) may be pruned or rejected. Alternatively, if the prediction score of the first classification (“helmet”) is higher than that of second classifications (“knob”), then the first classification may be kept for further aggregation. After pruning or retaining of the first classification, classifications from additional poses (not shown) may be added to a set of classifications used for determining a common classification for the object. The classifications of all poses may be considered for potential pruning in this way, with the updated or remaining set of classifications then used for determining the common classification, for example using majority voting. Figure 5 shows an example arrangement of poses 536X, 536Y, 536Z and classifications for the object 210B from different views. Poses 536Z are associated with inconclusive predictions or classifications – indicated by white fill. These inconclusive classifications may have a prediction score below a threshold and may be allocated a classification of none. Similar to Figure 4, poses 536X correspond to the same second classifications (e.g. “knob”) and pose 536Y is associated with a different first classification (e.g. “helmet”). In this case the first classification may be retained if its prediction score meets a minimum threshold as the associated pose 536Y has a pose similarity below a threshold when compared with the poses 536X associated with the second classification. In other words pose 536Y is sufficiently different to poses 536X. On the other hand, if the pose similarity between pose 536Y and poses 536X is high, then the first classification may be pruned. The pose similarity may be determined based on difference metrics between pose 536Y and poses 536X, including for example location and / or orientation. If these difference metrics are large enough, this may correspond to a pose similarity threshold. A location based difference metric may correspond to a minimum distance between pose 536Y and poses 536X. Referring to Figure 6, a large number of nearby poses 636Z associated with inconclusive classifications may also influence pruning. For example, if inconclusive classifications associated with poses 536Z are nearby to pose 536Y the first (minority) classification may be pruned, as the high number of inconclusive classifications indicates classification uncertainty from this part of the environment. On the other hand, as shown in Figure 7, if the pose similarity between a pose 736Y associated with the first (minority) classification is low when compared with the poses 736Z associated with inconclusive classifications, then the first classification is more likely to be accurate and may be retained. Figure 8 illustrates one example of determining a pose similarity. A single or minority pose 836Y is associated with a first classification “table” 870Y and a plurality of poses 836X are associated with a second classification “chair” 870X. The convex hull 860 of the poses 836X associated with the second classification 870X is calculated. A minimum distance 865 between the pose 836Y associated with the first classification 870Y and the convex hull 860 is then calculated. This distance 865 may then be used as the difference metric or pose similarity between the poses 836X of the second classifications and the pose 836Y of the first classification. Alternatives to this difference metric may be used, for example the distance may be between pose 836Y and a centroid of the convex hull 860. Orientation may additionally or alternatively be considered. In the Figure, the orientations of poses 836X are approximately 180 degrees different to the orientation of pose 836Y. Figure 9 illustrates a method of classifying an object according to an example. The method 900 may be implemented by the device 120 or by any other suitable equipment. Reference is also made to Figures 13 – 15. At 905, the method 900 obtains a plurality of 2D images of an environment, for example using an image capture device such as a camera or downloading the 2D images. The 2D images are associated with different poses or views of the environment. The method may start with an initial set of classifications or poses, and iterate through the poses to update a common classification. The initial set of poses may be for the entire environment, or for parts of the environment, for example only including poses within a certain quadrant or within a certain distance of each other. In this later case the method may be repeated for each part of the environment. This may improve classifications associated with poses in similar areas. At 910, the method determines if the current classification is in a minority compared with the classifications of the initial set. Where the classification is in a minority compared to a majority classification of the initial set, the method moves to 915. The use of minority and majority classifications corresponds to the first and second classifications respectively, of Figures 3-8. If the current classification is in the majority (910N), then the method moves to 945 where a function of the majority classification is used to update the common classification. In one example this function may be a majority vote and the common classification would correspond to the majority classification. If the current classification is not in the majority (910Y), then the method moves to 915 where the prediction score of the current classification is considered. If the prediction score is low (915N), the method again moves to 945 where the common classification updated based on the function of the majority classification. If the prediction score is high (915Y), the method moves to 920. A prediction score threshold may be used to determine whether the prediction score is high or low. At 920, the method determines whether the other (majority) classifications in the set are inconclusive and / or have low prediction scores. If not (920N), then the method again moves to 945. However, if the other classifications do have inconclusive classifications and / or low prediction scores (920Y) then the method moves to 925. This may correspond to a threshold number or proportion of other classifications being inconclusive. This may additionally or alternatively correspond to a threshold number or proportion of other classifications having a prediction score below a threshold. Various other algorithms for determining this step could alternatively be used A predetermined criteria may be defined, which if meet, the first classification is pruned. This predetermined criteria may include multiple factors such as those of 910 – 920, however other numbers and types of criteria may alternatively be used. At 925, the method calculates a convex hull for the poses of the other (i.e. not current) views. This was described with reference to Figure 8. The method then moves on to 930 where a minimum distance is determined between the convex hull and the pose associated with the current classification. Alternative pose similarity approaches may be used, for example considering orientation. The method them moves on 935, where it is determined whether the calculated minimum distance exceeds a threshold. If this minimum distance threshold is not exceeded (935N) then the method proceeds to 945. This situation means that the pose associated with the current classification is similar to the poses associated with the other classifications - low pose similarity. In this case, the common classification again is updated to the majority classification. If on the other hand, the calculated minimum distance exceeds the threshold (935Y), the method moves to 940. At 940, the method updates the common classification to be the current classification. This indicates that the current classification is considered more accurate than the other or majority classifications. As an example, referring to Figure 12, the helmet may be more accurately classified from pose 1236- 3 than poses near to pose 1236-1 which cannot see the visor detail of the helmet, only the featureless back surface. The output from the method is an aggregated or common classification for the object in the views under investigation. In another example, the output may be a semantic descriptor – for example some classifications may be retained that have a sufficiently high prediction score or are not pruned, for example by having a pose with a high pose similarity to a number of inconclusive classifications or conflicting classifications. These multiple classifications may be weighted based on factors such as prediction scores, distance of associated pose to object or number of pixels in a corresponding object mask. In some examples the weighting may be adjusted depending on the proportion of the object that is visible from the associated pose – for example reducing this if the entire object is not visible. The weighting could also model that very similar poses are treated as redundant – for example very similar views may be treated as one view. In other words the weighitng may depend on the pose similarity of the associated second classifications. This may be implemented using clustering such as Density-based spatial clustering of applications with noise (DBSCAN) for example. These multiple classifications or semantic descriptors may then be applied to different algorithms for determining a common classification for an object. These different algorithms may even be combined to improve object classification accuracy. In some examples, different functions may be used for determining the common classification, for example maximum likelihood estimation. Other factors may also be used, for example weighting of the classifications depending on the distance between their associated pose and the object, or the number of pixels in an object mask corresponding to the classification. In some examples the weighting may be adjusted depending on the proportion of the object that is visible from the associated pose – for example reducing this if the entire object is not visible. The weighting could also model that very similar poses are treated as redundant – for example very similar views may be treated as one view. In other words the weighitng may depend on the pose similarity of the associated second classifications. This may be implemented using clustering such as Density-based spatial clustering of applications with noise (DBSCAN) for example. Referring to Figures 13 – 15, the method 900 may handle various scenarios as illustrated and as described below. The set of poses on the left of Figure 13 are associated with a majority classification (black fill) and a smaller number of inconclusive classifications. The set of poses may be updated to all be associated with the majority classification (915N). The set of poses on the right, require further investigation (915Y) as there are a higher number of inconclusive classifications. The two of poses on the right of Figure 14 which are not associated with an inconclusive classification have the same classification with high prediction scores, and therefore so no further investigation is needed (920N). The set of poses on the left of Figure 14 which are not inconclusive are not consistent (different classifications) and have a medium score (black fill) and a low score (stripped fill). This set requires further investigation (920Y). In the set of poses on the left of Figure 15, a large number of poses are associated with inconclusive classifications and only one pose (stripped fill) is associated with a classification with high score. The pose associated with the classification is close to poses associated with the inconclusive classifications and is therefore pruned (935N). In the set of poses on the right, the pose associated with a classification with high score is a much larger distance from the poses having an inconclusive classification. The common score of this set of poses may then be updated with this high scoring classification (935Y). Referring to Figure 10, another method according to an example is illustrated. This method 1000 may be implemented by the device 120 of Figure 1, or any other suitable apparatus. At 1005, the method 1000 obtains a plurality of 2D images of an environment, the 2D images associated with different poses. This corresponds with step 905 of method 900 in Figure 9 and can be implemented in similar manner. At 1010, the method 1000 generates or displays a first 2D display image based on a first obtained 2D image of the plurality of obtained images and responsive to the pose of the apparatus corresponding to the pose of the first obtained 2D image. The first 2D display image is dependent on a first classification of an object detected in the first obtained 2D image. Referring also to Figure 12, this may correspond with displaying view 1250-1 of pose 1236-1, and from which the object 1210 is initially classified as a knob. A representation of the object in the form of an outline or a stylisation of a knob may be superimposed or alternatively displayed separately. At 1015, the method generates or displays a second 2D display image based on a second obtained 2D image of the plurality of obtained images and responsive to the pose of the apparatus corresponding to the pose of the second obtained 2D image. The second 2D display image is dependent on a second classification of an object detected in the second obtained 2D image, wherein the first and second classifications are different. This may correspond with displaying view 1250-3 of pose 1236-3 of Figure 12, and from which the object 1210 is now classified as a helmet. A representation of the object in the form of an outline of or a stylisation of a helmet may be superimposed or alternatively displayed separately. At 1020, the method generates or displays a third 2D display image based on the first obtained 2D image of the plurality of obtained images and responsive to the pose of the apparatus corresponding to the pose of the first obtained 2D image. The third 2D display image is dependent on the second classification. This may correspond with again displaying view 1250-1 of pose 1236-1 of Figure 12. However, the classification of the object 1210 is retained as “helmet”. A representation of the object in the form of an outline of or a stylisation of a helmet may be superimposed over the object as shown in the first view 1250-1, or alternatively displayed separately. This approach retains improved classifications of an object even when the pose changes to a poorer view of the object which may otherwise lead to re-classification based on the poorer view. Examples provide a number of advantages. These can improve weak labels and prune incorrect ones, regardless of whether the most common, MLE or other methods are used. Images from significantly different poses can enhance the semantic information from other views. On the other hand, images from similar poses should produce similar semantic information, and may be corrected or pruned if they do not. The multi-view aggregation approach improves the prediction or classification from single views by enhancing weak labels and pruning incorrect ones, provides for identification of redundant views and detected objects, identification of needed / highly informative views and detected object. The examples also provide for the removal of spurious classifications and labels which may be important in many applications of multi-view object detection. For example in scenarios where a collaborative metric semantic map is constructed, the methods can remove the need for manual filtering of spurious classifications, decrease the storage requirements, decrease computational burden of applications using the output or the metric semantic map, and can be used to prioritize which data to include when updating the shared map. For map making a less “sequential” mode may be employed, for example first obtaining and saving all 2D images (905) and then process the classifications offline (910-940) with the 2D images already included in the sparse 3D point cloud. Enhancing weak labels also allows for more accurate systems to be built, which reduces creation and maintenance costs. It should be noted that the above-mentioned examples illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative examples without departing from the scope of the appended statements. The word “comprising” does not exclude the presence of elements or steps other than those listed in a claim or embodiment, “a” or “an” does not exclude a plurality, and a single processor or other unit may fulfil the functions of several units recited in the statements below. Where the terms, “first”, “second” etc are used they are to be understood merely as labels for the convenient identification of a particular feature. In particular, they are not to be interpreted as describing the first or the second feature of a plurality of such features (i.e. the first or second of such features to occur in time or space) unless explicitly stated otherwise. Steps in the methods disclosed herein may be carried out in any order unless expressly otherwise stated. Any reference signs in the statements shall not be construed so as to limit their scope.

Claims

Claims 1. A method of classifying an object, the method comprising: obtaining a plurality of classifications for the object, the classifications associated with different poses; determining a pose similarity between a first pose associated with a first classification of the plurality of classifications and a plurality of second poses associated with second classifications of the plurality of classifications, the first classification and the second classifications being different; determining a common classification for the object dependent on the pose similarity.

2. The method of claim 1, wherein the pose similarity is a difference metric between the pose of the first classification and the poses of the second classifications.

3. The method of claim 2, wherein the difference metric is a distance and / or an orientation difference between the pose of the first classification and the poses of the second classifications.

4. The method of claim 3, wherein the distance is a minimum distance between the pose of the first classification and the convex hull of the poses of the second classifications.

5. The method of any one of claims 1 to 4, wherein determining the common classification is dependent on respective prediction scores for the first and second classifications.

6. The method of any one of claims 1 to 5, wherein the second classifications comprise one or more inconclusive classifications.

7. The method of claim 6, wherein determining the common classification for the object is dependent on the number of inconclusive classifications in the second classifications.

8. The method of any one of claims 1 to 7, wherein determining a common classification for the object comprises selecting the first classification or determining a function of the second classifications.

9. The method of claim 7, wherein the function of the second classifications comprises a majority vote or a maximum likelihood estimation of the second classifications.

10. The method of claim 8 or 9, wherein the function of the second classifications comprises weighting the second classifications based on one or more of the following: distance to the object from the corresponding pose; number of pixels in an object mask corresponding to the classification; proportion of object visible from the corresponding pose; pose similarity of the second classifications.

11. The method of any one of claims 1 to 10, wherein determining that the common classification is a function of the second classifications is responsive to one or more of the following: the first classification having a prediction score below a first classification threshold; the number of second classifications having a prediction score above a second classification threshold; the number of second classifications having an inconclusive classification being below an inconclusive classification threshold; a distance between the pose of the first classification and the poses of the second classifications being less than a minimum distance threshold.

12. The method of any one of claims 1 to 10, wherein determining that the common classification is the first classification is responsive to one or more of the following: the prediction score of the first classification being above a first classification prediction score; the prediction scores of the second classifications being below a second classification prediction score; the number of second classifications having an inconclusive classification exceeding an inconclusive classification threshold; the distance between the pose of the first classification and the poses of the second classifications being greater than a minimum distance threshold.

13. The method of any one of claims 1 to 12, comprising: defining a set of classifications initially comprising the plurality of classifications, updating the set of classifications by removing the first classification in response to a predetermined criteria; and determining the common classification for the object dependent on the updated set of classifications.

14. The method of claim 13, wherein the predetermined criteria is one or more of the following: the first classification having a prediction score below a first classification threshold; the number of second classifications having a prediction score below a second classification threshold; the number of second classifications having an inconclusive classification; a distance between the pose of the first classification and the poses of the second classifications.

15. A computer program comprising instructions which, when executed on a processor, cause the processor to carry out the method of any one of claims 6 to 14.

16. A computer program product comprising non-transitory computer readable media having stored thereon a computer program according to claim 15. 17 An apparatus configured to: obtain a plurality of 2D images of an environment, the obtained 2D images associated with different poses; generate a first 2D image based on a first obtained 2D image of the plurality of obtained 2D images and responsive to the pose of the apparatus within the environment corresponding to the pose associated with the first obtained 2D image; wherein the generated first 2D image is dependent on a first classification of an object detected in said first obtained 2D image; subsequently to generating the first 2D image, generate a second 2D image based on a second obtained 2D image of the plurality of obtained 2D images and responsive to the pose of the apparatus within the environment corresponding to the pose associated with the second obtained 2D image; wherein the generated second 2D image is dependent on a second classification of an object detected in said second obtained 2D image, the first and second classifications being different; subsequently to generating the second 2D image, generate a third 2D image based on the first obtained 2D image of the plurality of obtained 2D images and responsive to the pose of the apparatus within the environment corresponding to the pose associated with the first obtained 2D image; and wherein the generated third 2D image is dependent on the second classification.

18. The apparatus of claim 17, wherein the generated first and second 2D images comprises respective representations of the object.

19. The apparatus of claim 17 or 18, comprising one or more of the following: a 2D image sensor, a 2D image display, a processor and a memory.

Images