Pliability estimation

Abstract

A method for estimating the pliability of environmental features, the method including: acquiring, from a mapping system including a mapping device moving along a trajectory through an environment, mapping data including: a 3D point cloud of the environment; and, a trajectory of the mapping device relative to the environment; obtaining, from an imaging device associated with the mapping system, image data indicative of images of the environment captured by the imaging device as the imaging device moves through the environment; analysing the image data to determine image feature categories by categorising one or more image features; labelling a subset of points of the point cloud with the image feature categories; using the labelled subset of points to identify an environmental feature; calculating a geometry of the environmental feature; using the geometry and image feature categories of the environmental feature to calculate a pliability estimate for the environmental feature.

Description

PLIABILITY ESTIMATIONBackground of the Invention

[0001] The present invention relates to a method and system for estimating pliability, and in one particular example, to a method and system for estimating the pliability of structures such as vegetation in an outdoor environment.Description of the Prior Art

[0002] The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that the prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

[0003] Navigation through outdoor natural environments is fundamental to robots working in a wide range of scenarios, such as search and rescue, bush fire mitigation, environmental monitoring, forestry and related industries. These autonomous vehicles are usually equipped with cameras and with a 3D mapping system that converts input sensor data (lidar, stereo camera, depth camera) into a 3D point cloud in real time. Unfortunately, these robot sensors cannot directly distinguish between pliable and hard objects, so the only available safe option is to avoid navigating through such obstacles altogether. Consequently, autonomous robots typically will not navigate through long grass, and will not brush past vines or tree branches, they will halt and require manual navigation through these areas. Other outdoor hazards such as mud, gravel and water pose similar problems, the direct sensor data lacks understanding of outdoor environments.

[0004] Attempts to reconstruct a 3D understanding of a robot’s surroundings are further hampered by the dynamic nature of the SLAM systems that generate 3D point clouds in real time. Past trajectories are continually re-estimated, and loop closures and place recognition cause abrupt changes in the trajectory and therefore the 3D map.Summary of the Present Invention

[0005] In one broad form, an aspect of the present invention seeks to provide a method for use in estimating the pliability of environmental features, the method including in one or more electronic processing devices: acquiring, from a mapping system including a mapping device moving along a trajectory through an environment, mapping data including: a 3D point cloud of the environment; and, a trajectory of the mapping device relative to the environment; obtaining, from an imaging device associated with the mapping system, image data indicative of images of the environment captured by the imaging device as the imaging device moves through the environment; analysing the image data to determine image feature categories by categorising one or more image features; labelling a subset of points of the point cloud with the image feature categories; using the labelled subset of points to identify an environmental feature; calculating a geometry of the environmental feature; using the geometry and image feature categories of the environmental feature to calculate a pliability estimate for the environmental feature.

[0006] In one embodiment the method includes, in the one or more processing devices, determining the subset of points of the point cloud by at least one of: subsampling the point cloud; performing a visibility check to determine points in the point cloud that are visible to a field of view of the imaging device; selecting points within a defined spatial extent of the imaging device.

[0007] In one embodiment the method includes, in the one or more processing devices: label points in the point cloud; and, selecting a subset of labelled points.

[0008] In one embodiment the method includes, in the one or more processing devices, analysing the labelled subset of points to distinguish different environmental features.

[0009] In one embodiment the method includes, in the one or more processing devices, analysing points of the labelled subset of points having a particular image feature category to distinguish different environmental features.

[0010] In one embodiment the method includes, in the one or more processing devices, analysing the points of the labelled subset of points having a particular image feature category to identify connectivity between points forming part of an environmental feature.

[0011] In one embodiment the method includes, in the one or more processing devices, analysing the points of the labelled subset of points using a shortest path algorithm to identify connectivity between points.

[0012] In one embodiment the method includes, in the one or more processing devices, estimating the geometry of a structure of the environmental feature using the connectivity.

[0013] In one embodiment the method includes, in the one or more processing devices: using the point cloud to construct a ground surface; and, extending a structure of the environmental feature to the ground surface.

[0014] In one embodiment the method includes, in the one or more processing devices, estimating the geometry by modelling the structure as a series of interconnected tapering cylinders.

[0015] In one embodiment the method includes, in the one or more processing devices, estimating the geometry using a tapering ratio based on at least one of: a structure length; and, the image feature type of points in the structure.

[0016] In one embodiment the method includes, in the one or more processing devices, estimating the pliability using an elastic modulus based on the image feature type of the points forming the environmental feature.

[0017] In one embodiment the image features correspond to image pixels.

[0018] In one embodiment the method includes, in the one or more processing devices: analysing the images data to categorise a feature type of pixels within the images; and, labelling the subset of points based on the feature type determined for corresponding pixels in one or more of images.

[0019] In one embodiment the method includes, in the one or more processing devices: determining a transformation between image features and the subset of points; and, labelling the subset of points using the transformation.

[0020] In one embodiment the method includes, determining the transformation based on at least one of: imaging device intrinsic properties; imaging device extrinsic properties; and, a known geometric relationship between the mapping device and imaging device.

[0021] In one embodiment the method includes, in the one or more processing devices, labelling the subset of points in accordance with: a time stamp indicative of a capture of each image; and, the trajectory of the mapping device.

[0022] In one embodiment the method includes, in the one or more processing devices, categorising one or more environmental features as one of: vegetative features, including one more of: bush; grass; tree foliage; tree trunk; and, non-vegetative features, including one or more of: log; dirt; gravel; mud; terrain; fence; object; rock; vehicle; and, structure.

[0023] In one embodiment the method includes, in the one or more processing devices, controlling movement of a vehicle through the environment in accordance with the pliability estimate.

[0024] In one embodiment the method includes, in the one or more processing devices: using the pliability estimate to calculate a traversal indicator indicative of the ability to traverse the environment; and, controlling the vehicle using the traversal indicator.

[0025] In one embodiment the method includes, in the one or more processing devices: generating a three dimensional occupancy grid using the point cloud; populating the occupancy grid with pliability estimates; and, using the occupancy grid to identify traversable parts of the environment.

[0026] In one embodiment the mapping data updates as the vehicle moves within the environment.

[0027] In one broad form, an aspect of the present invention seeks to provide a system for use in estimating the pliability of environmental features, the system including one or more electronic processing devices configured to: acquire, from a mapping system including a mapping device moving along a trajectory through an environment, mapping data including: a 3D point cloud of the environment; and, a trajectory of the mapping device relative to the environment; obtain, from an imaging device associated with the mapping system, image data indicative of images of the environment captured by the imaging device as the imaging device moves through the environment; analyse the image data to determine image feature categories by categorising one or more image features; label a subset of points of the point cloud with the image feature categories; use the labelled subset of points to identify an environmental feature; calculate a geometry of the environmental feature; use the geometry and image feature categories of the environmental feature to calculate a pliability estimate for the environmental feature.

[0028] It will be appreciated that the broad forms of the invention and their respective features can be used in conjunction and / or independently, and reference to separate broad forms is not intended to be limiting. Furthermore, it will be appreciated that features of the method can be performed using the system or apparatus and that features of the system or apparatus can be implemented using the method.Brief Description of the Drawings

[0029] Various examples and embodiments of the present invention will now be described with reference to the accompanying drawings, in which: -

[0030] Figure 1 is a flow chart of an example of a method for use in estimating pliability;

[0031] Figure 2 is a schematic diagram of an example of a vehicle;

[0032] Figures 3A and 3B are a flow chart of a further example of a method for use in estimating pliability;

[0033] Figure 4 is a schematic diagram of a software architecture for estimating pliability;

[0034] Figure 5 is a schematic diagram of a tree segment;

[0035] Figure 6 A is an image of an example of a stiffness heatmap; and,

[0036] Figure 6B is an image of an example of a stem-height projected onto a 2D grid-cell.Detailed Description of the Preferred Embodiments

[0037] An example of a process for use in estimating pliability will now be described with reference to Figure 1.

[0038] The pliability estimation is typically performed in the context of assessing the traversability of an unstructured environment, which could be a natural open environment, such as an outdoor area, including vegetation or other similar pliable structures and / or objects.

[0039] The pliability estimation could be performed by, or on behalf of, a vehicle navigating the environment, which is assumed to be any device capable of traversing an environment, and could include autonomous vehicles, robots, or the like. The vehicle could use a range of different locomotion mechanisms depending on the environment, and could include wheels, tracks, or legs. Accordingly, it will be appreciated that the term vehicle should be interpreted broadly and should not be construed as being limited to any particular type of vehicle. However, this is not essential and in practice the pliability estimation could be performed in any scenario in which an estimate of pliability of a structure, such as vegetation, is required.

[0040] For the purpose of illustration, it is also assumed that the process is performed at least in part using one or more electronic processing devices forming part of one or more processing systems, typically connected to or in communication with a mapping system. Whilst the system can use multiple processing devices, with processing performed by one or more of the devices, for the purpose of ease of illustration, the following examples will refer to a single device, but it will be appreciated that reference to a singular processing device should be understood to encompass multiple processing devices and vice versa, with processing being distributed between the devices as appropriate.

[0041] The mapping system is typically incorporated into the vehicle, although this is not essential and other arrangements could be used. The mapping system typically includes a mapping device, such as a range sensor, which could include a LiDAR sensor, stereoscopic vision system, or the like. One or more electronic processing devices are also typically provided, which are configured to receive signals from the mapping device and either process the signals, and / or provide these to a remote processing device for processing and analysis. This can be achieved in any suitable manner, but typically involves acquiring range data from the range sensor that is indicative of a distance between the range sensor and a sensed part of the environment, allowing the mapping system to analyse the range data to generate mapping data indicative of a point cloud of the environment and a trajectory of the mapping device relative to the environment. In one example, this is achieved using a SLAM (Simultaneous Localisation and Mapping) type algorithm to perform simultaneous localisation and mapping.

[0042] An imaging device, such as a camera, is typically associated with the mapping system, to capture images of the environment as the imaging device moves through the environment. The camera is typically separate from the range sensor, and optionally mounted thereto, provided in a known geometric arrangement with respect to the range sensor, allowing features identified within the images to be correlated with corresponding features in the point cloud. However, this is not essential and in other examples, the image sensor could be the mapping device. For example, the mapping device could include a single camera, with mapping data being generated using a technique such as Monocular SLAM, NERF (Neural Radiance Field) SLAM, Structure From Motion, or the like.

[0043] In this example, at step 100, the processing device acquires mapping data from the mapping system. As mentioned above, the mapping data typically includes a 3D point cloud of the environment and a trajectory of the mapping device relative to the environment, although this is not essential and raw range data could be acquired and the point cloud and trajectory derived as needed, depending on the preferred implementation.

[0044] The processing device also obtains image data indicative of images captured by the imaging device at step 110. The images are captured as the mapping device moves through the environment, with the location of captured images relative to the trajectory of the mappingdevice being determined in an appropriate manner, for example based on time stamps within the images and a known geometric relationship between the images and the mapping device, or the like.

[0045] At step 120, the processing device analyses the image data to determine image feature categories by categorising one or more image features. This can be achieved in any suitable manner, but in one example this involves using a segmentation algorithm to categorise parts of the image as corresponding to a particular object or material type. This can be done for individual pixels, or larger regions of an image, for example by identifying different environmental features within the image and categorising these as needed, depending on the preferred implementation.

[0046] At step 130, a subset of points of the point cloud are labelled with the image feature categories. In one example, this is achieved by determining a transformation between the images and the point cloud, and then labelling points within the point cloud based on the labelling assigned to corresponding image features. However, it will be appreciated that this is not required in the event that the imaging device is the mapping device. Labelling of points could be performed for the entire point cloud, with the point cloud then being subsampled or otherwise limited, for example by limiting to an attention region in the vicinity of the sensors, or alternatively the subset of points could be selected prior to labelling occurring. In either case, considering only a labelled subset of points of the point cloud in later stages can help reduce computational requirements, which can assist in allowing the pliability estimation to be performed in real time.

[0047] At step 140, the processing device uses the labelled subset of points to identify an environmental feature, and calculate a geometry of the environmental feature at step 150. This can be achieved in any suitable manner, such as by identifying proximate or adjacent points having a common image feature category as connected parts of an environmental feature, and then using these to construct the overall geometry of a structure of the environmental feature. This process can take into account geometric properties of the environmental feature as determined from the image feature category, for example assuming vegetation has a treelike structure that tapers from a base towards one or more tips.

[0048] At step 160, the geometry and image feature categories of the environmental feature are used to calculate a pliability estimate for the environmental feature. This can be achieved in any suitable manner, but typically involves assuming certain mechanical properties based on the image feature category, such as a certain elastic modulus, and then using this information in combination with the geometry of the structure, to derive a pliability.

[0049] Accordingly, the above-described arrangement uses a combination of images and a point cloud of the environment to features within the environment, and then use a geometry and type of the features to calculate a pliability. Specifically, the approach uses images to perform categorisation of image features, which are then overlaid on a point cloud, to categorise environmental features. A geometry environmental features is determined from the point cloud, with the geometry being used together with mechanical properties associated with the image feature categories being used to estimate the pliability of different parts of the environmental feature.

[0050] Once a pliability has been determined, this can be used in one or more optional downstream processes, for example to calculate a travers ability based on knowledge of a vehicle's traversal capabilities, which can then in turn be used in calculating a navigable path through the environment. Specifically, this can be used to allow a vehicle to move through a region occupied by vegetation by estimating the degree to which the vegetation can be deformed, and hence assess the ability of the vehicle to push through the vegetation. This allows vehicles to traverse parts of the environment that would traditionally be assessed as impassable.

[0051] A number of further features will now be described.

[0052] The processing device can determine a subset of points of the point cloud in a number of manners. This can include subsampling the point cloud, for example by reducing a point cloud density so that only some of the points are considered. This can also involve performing a visibility check to determine points in the point cloud that are visible to a field of view of the imaging device, allowing any obstructed points, or points beyond the field of view to be ignored, although this will not be required in the event that the mapping device is the imaging device. Additionally and / or alternatively, this can involve selecting points within a definedspatial extent of the imaging device, for example ignoring points beyond the spatial extent on the basis these are not important in immediate decision making regarding traversability. This approach allows the number of points in the point cloud that are analysed to be reduced, which in turn reduces the computational burden, making it possible to perform the pliability prediction in real time as a vehicle navigates through an environment.

[0053] In one example, the processing device labels points in the point cloud and then selects a subset of labelled points, for example by subsampling the labelled points, performing a visibility check or spatial restriction of the point cloud. However, this is not essential and alternatively, the subset of points could be selected prior to labelling. This latter approach can reduce computation, but may lead to reduced labelling accuracy, and so the approach used may depend on the preferred implementation and requirements of the usage scenario.

[0054] In one example, the processing device analyses the labelled subset of points to distinguish different environmental features. Thus, the processing device uses the labelling of the subset of points to help identify specific environmental features, for example by analysing points of the labelled subset of points having a particular image feature category to distinguish different environmental features. This allows the processing device to select a particular image feature category, such as vegetation, and then examine all points labelled as vegetation, using this to then identify individual plants.

[0055] In one example, this process involves analysing points of the labelled subset of points having a particular image feature category to identify connectivity between points forming part of an environmental feature. Thus, one set of connected points will typically form an individual plant, with different points being part of different plants. This approach leverages the categorisation of the subset of points to identify individual environmental features within the point cloud. Furthermore, once the connectivity has been identified, it is then possible to estimate the geometry of a structure of the environmental feature using the connectivity.

[0056] In one example, this is process is achieved by assuming points of the environmental feature will be connected using a shortest path. Accordingly, this allows a shortest path algorithm to be applied to points in the point cloud having a common image feature category to identify the connected points and hence estimate the geometry of the environmental feature.As part of this it can be assumed vegetation connects to ground, so the method can further involve using the point cloud to construct a ground surface and then extending the structure to the ground surface. This is particularly useful in ensuring a ground connection is established, even if this part of the environment hasn't actually been captured in images or as part of the point cloud.

[0057] In order to model the structure, the processing device can estimate the geometry by modelling the structure as a series of interconnected tapering cylinders. This relies on the assumption that the majority of vegetation can be modelled in this manner to give a sufficiently accurate estimate of the structure, which in turn allows the structure to be derived from the subset of points. In this regard, the density of the subset of points may not be sufficient to distinguish the diameter of the vegetation, so using the assumption allows the geometry of the structure to be derived, thereby helping to preserve accuracy whilst reducing computational requirements. However, it will be appreciated that other suitable approaches could be used. For example, vegetation could alternatively be modelled as a tapering series of standard (nontapering) cylinders, which while less accurate can be computationally more straightforward.

[0058] In one example, the method includes estimating the geometry using a tapering ratio based on at least one of a structure length and an image feature type of the points in the structure. In this regard, the tapering ratio for a given type of vegetation will be largely consistent, and so this allows an estimate of the physical dimensions to be determined with a relatively high degree of certainty providing a simple mechanism for calculating the diameter of different parts of the structure, without requiring a sufficient density of point cloud to allow this to be measured.

[0059] Once the dimensions of the structure have been determined, the processing device can estimating the pliability using an elastic modulus, with the magnitude of the elastic modulus being based on the image feature type of the points of the environmental feature. This allows the pliability along the length of the structure of the environmental feature to be estimated as will be described in more detail below.

[0060] As mentioned above, in one example the image features correspond to image pixels. In this example, the processing device can analyse the image data to categorise an image featuretype of pixels within the images and label the subset of points based on the image feature type determined for corresponding pixels in one or more of images. To achieve this, the processing device typically determines a transformation between image features and the subset of points, labelling the subset of points using the transformation. This might be required if the mapping device and image sensor are not be aligned and instead are offset, so that the transformation is used to account for the differences in the fields of view of the mapping device and imaging device, as well as other differences, for example arising from image lens distortion or the like. The transformation is therefore typically determined using a combination of imaging device intrinsic and extrinsic properties, including a known geometric relationship between the mapping device and imaging device. However, it will also be appreciated that this is not required if the imaging device is the mapping device.

[0061] The labelling process will also take into account a time stamp indicative of a capture of each image, allowing a capture location of images to be localised on the trajectory of the mapping device as it moves through the environment. However, it will be appreciated that other suitable techniques could also be used.

[0062] The above-described process is typically used to estimate the pliability of environmental features such as vegetative features. To achieve this, the processing device typically categorises one or more environmental features including vegetative features such as bush, grass, tree foliage, or tree trunks and non-vegetative features, such as logs, dirt, gravel, mud, terrain, fences, objects, rocks, and vehicles.

[0063] In one example, the resulting pliability estimates can be used to allow the processing device to control movement of a vehicle through the environment in accordance with the pliability estimate. For example, this can involve allowing the vehicle to traverse through structures having a pliability below a threshold amount, dependent on the traversal capabilities of the vehicle. To achieve this, in one example, the pliability estimate is used to calculate a traversal indicator indicative of the ability to traverse the environment, which in turn will be vehicle dependent. The traversal indicator can then be used in path planning, in turn allowing this to be used in controlling the vehicle. It will be appreciated that separating the pliability estimate and vehicle control processes in this fashion allows the pliability estimation to beperformed in the same manner regardless of the nature of the vehicle traversing the environment, with traversal and control being implemented by specific approaches that are tailored to the vehicle being controlled.

[0064] Such vehicle control processes are known, and the ability to navigate through environments including obstacles are described for example in WO2024 / 227229, the contents of which are incorporated herein by cross reference.

[0065] Thus, in one example, such an approach will involve generating a three-dimensional occupancy grid using the point cloud and populating the occupancy grid with pliability estimates. Vehicle capabilities can then be used to assess parts of the occupancy grid that are passable, in turn allowing path planning to be performed. Specifically, the occupancy grid can be populated with pliability estimates, with thresholding being performed so that voxels of the occupancy grid where the pliability falls below a threshold, these are deemed unpassable and hence occupied, whilst where the pliability is above a threshold, the voxels are deemed to be passable and hence unoccupied, with the threshold being vehicle dependent. Once this has been completed, regions between occupied voxels can be considered as gaps, and the gap planning approach described in WO2024 / 227229 can be applied.

[0066] Thus, in this instance, the path planning algorithm will generate a three-dimensional occupancy grid, including unoccupied voxels where vegetation or other environmental features are sufficiently pliable.

[0067] Following this, the path planning algorithm calculates a pose of the vehicle for at least some positions within the environment. The pose of the vehicle for any given position is generated based on the environment terrain as determined from the mapping data. Typically this is performed to take into account the terrain in the vicinity of the vehicle, and how the vehicle will sit on the terrain. Additionally, this also takes into account motion primitives indicative of available vehicle movements, thereby accounting for how the vehicle will move through the environment, and in turn how this will affect the pose.

[0068] Having calculated the pose for the vehicle, this information is used together with the occupancy grid to allow a proximity of the vehicle to the environment (occupied voxels) to bedetermined. Thus, projecting the shape of the vehicle onto the occupancy grid for the calculated pose, can allow a distance between surfaces of the vehicle and the environment to be calculated, and hence allow the proximity of the vehicle to the environment to be determined.

[0069] These steps are repeated for multiple positions, which in one example may correspond to different cells within the occupancy grid. This is typically achieved by selecting neighbouring positions, such as neighbouring cells, and assessing the proximity of the vehicle for the neighbouring positions, repeating this progressively for neighbouring positions heading towards a target on the other side of the vegetation or other environmental feature.

[0070] This allows the processing device to progressively assess the proximity of the vehicle to the environment as the vehicle moves through the environment towards a target, and hence allows a path to be calculated where proximity to the environment is optimised, for example maximising separation between the vehicle and the environment, to thereby avoid collisions.

[0071] In one particular example, this approach is performed using a planning algorithm, such as an A* algorithm, in which a cost function is used to represent movement of the vehicle between search nodes corresponding to cells in the occupancy grid, taking into account and optionally optimising the pose of the vehicle, to thereby maintain a sufficient spacing between the vehicle and the environment.

[0072] Once the path has been created, this can be used to control the vehicle, so that the vehicle navigates within the environment. In this regard, it will be appreciated that as the vehicle continues to move within the environment, additional range data is acquired, and this can be used to refine the above process, for example, updating the mapping data and hence occupancy grid, as more of the environment is sensed. In this regard, as the pliability estimate is based on the mapping data, when mapping data is updated this can result in recalculation of the pliability estimate, so the mapping data and pliability estimates might update as the vehicle moves through the environment.

[0073] Thus, in the above approach, the use of the three-dimensional occupancy grid populated by thresholding the pliability allows areas containing pliable objects to be successfully traversed.

[0074] An example of a vehicle is shown in more detail in Figure 2.

[0075] In this example, the vehicle 200 includes a chassis and body 210 having at least one electronic processing device 211 located on-board, which is coupled to a mapping system 212 configured to perform scans of the environment surrounding the vehicle in order to build up a 3D map (i.e. point cloud) of the environment. In one example, the mapping system includes a 3D LiDAR sensor such as a VLP-163D LiDAR produced by Velodyne. The processing device 211 may also be coupled to an inertial sensing device 213, such as an IMU (inertial measurement unit), a control system 214 to allow movement of the vehicle to be controlled, and one or more other sensors 215. This could include proximity sensors for additional safety control, or an imaging device, or similar, to allow images of the environment to be captured, for example, for the purpose of colourising point cloud representations of the environment.

[0076] The processing device 211 can also be connected to an external interface 216, such a wireless interface, to allow wireless communications with other vehicles, for example via one or more communications networks, such as a mobile communications network, 4G or 5G network, WiFi network, or via direct point-to-point connections, such as Bluetooth, or the like.

[0077] The electronic processing device 211 is also coupled to a memory 217, which stores applications software executable by the processing device 211 to allow required processes to be performed. The applications software may include one or more software modules, and may be executed in a suitable execution environment, such as an operating system environment, or the like. The memory 217 may also be configured to allow mapping data and frame data to be stored as required, as well as to store any generated map. It will be appreciated that the memory could include volatile memory, non-volatile memory, or a combination thereof, as needed.

[0078] It will be appreciated that the above-described configuration assumed for the purpose of the following examples is not essential, and numerous other configurations may be used. For example, although the vehicle is shown as a wheeled vehicle in this instance, it will beappreciated that this is not essential, and a wide variety of vehicles and locomotion systems could be used.

[0079] Additionally, in this example, the processing device is described as being on-board the vehicle. However, as outlined above, processing could be distributed between different processing devices, in which case, the vehicle may include a processing device for processing data from the mapping device, and generating the point cloud and trajectory, with these and associated images being provided to a remote processing device to enable to the abovedescribed pliability estimates to be performed.

[0080] An example process for estimating a pliability will now be described with reference to Figures 3 A and 3B.

[0081] In this example, at step 300 image data is acquired. The image data undergoes analysis at step 305 to thereby categorise image pixels with different image feature types. The categorisation can be performed in any suitable manner depending on the preferred implementation, and known approaches will be discussed in more detail below. Typically, however, this involves using a segmentation algorithm trained using machine learning on images captured for similar environments where vegetation and other features have been labelled. The algorithm then recognises similar features within the captured images and then labels these accordingly.

[0082] At step 310 mapping data is acquired, including a point cloud and mapping device trajectory. A transformation between the point cloud and image is determined at step 315, taking into account intrinsic and extrinsic parameters relating to the imaging device, including optical properties of the imagining device and the relative positioning of the imaging device and mapping device.

[0083] At step 320, the point cloud is labelled based on the image feature types in the image. Thus, the transformation is used to map individual pixels on to corresponding points in the point cloud, with the labels being used to paint the point cloud, thereby assigning image feature types to the points in the point cloud.

[0084] At step 325, attention and visibility of points in the point cloud are determined. Specifically, when calculating pliability, this is typically limited to points within a set spatial distance of the sensors, to limit the amount of processing involved, with non-visible features being excluded for similar reasons.

[0085] Additionally, at step 330, the remainder of the point cloud is further subsampled, to further reduce the number of labelled points that are considered for subsequent analysis. This helps reduce computational requirements, without a significant loss in accuracy for the resulting pliability estimates, enabling the processing to be performed in real time, which is important for navigation in real world scenarios.

[0086] At step 335, a ground plane is reconstructed from the point cloud, using well known approaches, as will be described in more detail below.

[0087] At step 340, point connectivity is identified. Specifically, this examines points having a common image feature label, and then using an algorithm, such as a shortest path algorithm to identify connected points extending to the ground plane. This in effect identifies individual vegetation structures by assuming that vegetation is a treelike structure that extends to ground.

[0088] At step 345, a geometry of an individual structure is calculated. In this regard, the subsampled point cloud typically does not have a density of points that allows a diameter of the vegetation to be determined, and so individual plants or other structures are modelled as a series of interconnected tapering cylinders.

[0089] Once the geometry has been calculated, the pliability along the length of the structure can be estimated. This is performed using an elastic modulus determined for the particular image feature type, so that, for example, a tree will have a greater elastic modulus than a grass. An example of the specific calculations performed will be described in more detail below.

[0090] At step 355, one or more output maps can be generated. This can include a labelled point cloud, in which the pliability estimates are applied to the individual points in the point cloud, and could include a ground map labelled with the pliability of the plants above the ground point. Additionally, and / or alternatively, an occupancy grid could be generated, containing the pliability of any structures within the relevant voxels of the grid.

[0091] At steps 360 to 370 optional downstream processes could be performed, including but not limited to performing a traversability assessment and path planning, as well as controlling the vehicle based on the determined pliability.

[0092] A further specific example of the process for performing pliability estimation will now be described.

[0093] The problem of traversing outdoor environments has traditionally been tackled directly using reinforcement learning supervised and self- supervised learning; using lidar-only, camera-only and hybrid approaches. In these methods, traversibility is the output, with any knowledge of the environment remaining secondary or obscured within the network. The above-described approach conversely favours a separation of concerns, whereby scene understanding is separate from traversibility. Thus, the approach focusses on a vehicle-agnostic scene understanding, leaving traversibility as one of several possible downstream tasks, which can use the scene information as an information-rich source for learning on the vehicle in question.

[0094] Scene understanding comprises understanding what the environment is made from and the geometry of these semantic classes. The first part is suited to semantic segmentation of camera data using a segmentation algorithm, whilst the second part relates to the geometric reconstruction of environment features, as will now be described.

[0095] Semantic segmentation is the task of assigning a class label to each pixel in an image. Deep learning-based methods have achieved tremendous success in this area. For semantic segmentation, there exist both convolution neural network (CNN) based models and transformer-based models. The fully convolutional network introduced in Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431-3440, was originally proposed for the semantic segmentation task. The insight of this approach is to take advantage of existing CNN classifiers that can learn hierarchies of features and transform them by replacing the fully connected layers with convolutional layers to produce coarse output maps.

[0096] Dilated convolution-based networks, introduced by Liang-Chieh, G. Papandreou, I. Kokkinos, K. Murphy, and A. Yuille, “Semantic image segmentation with deep convolutional nets and fully connected crfs,” in International conference on learning representations, 2015, enhance semantic segmentation by expanding the receptive field without losing resolution or coverage, enabling effective multi- scale context aggregation. The architecture integrates dilated convolutions, Atrous Spatial Pyramid Pooling for multi-scale feature capture, and Conditional Random Fields for refining segmentation boundaries, demonstrating significant advancements in handling spatial detail and context.

[0097] Transformer-based models represent a recent advancement in semantic segmentation, leveraging self-attention mechanisms to capture global context and resolve ambiguities at the level of image patches. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segm enter: Transformer for semantic segmentation,” in Proceedings of the IEEE / CVF international conference on computer vision, 2021, pp. 7262-7272 describes adapting a Vision Transformer (ViT) model for semantic segmentation tasks, extending its capabilities beyond image classification. Unlike traditional convolutional approaches, ViT enables the modeling of global context from the initial layers throughout the network. This is achieved by processing image patches into embeddings, which are then decoded into class labels using either a point- wise linear decoder or a mask transformer decoder.

[0098] Other popular architectures have demonstrated significant advancements in dense prediction tasks. However, their high computational complexity limits their efficiency when processing high-resolution images. Addressing this limitation, Cai, C. Gan, and S. Han, “Efficientvit: Enhanced linear attention for high-resolution low-computation visual recognition,” arXiv preprint arXiv:2205.14756, 2022, describe a new family of vision transformer models that have been specifically designed for efficient high resolution dense prediction. At its core this can provide a novel multi- scale linear attention module, which combines a global receptive field with multi- scale learning while maintaining hardware efficient operations.

[0099] The other part to scene understanding is reconstructing the geometry of these classified classes. Tree trunk reconstruction has been performed in real-time, as described for examplein FreiBmuth, M. Mattamala, N. Chebrolu, S. Schaefer, S. Leutenegger, and M. Fallon, “Online tree reconstruction and forest inventory on a mobile robotic system,” arXiv preprint arXiv: 2403.17622, 2024, but pliability requires understanding the full branch structure. Such branch geometry has been reconstructed using shortest path methods but only in slow, offline processes.

[0100] Grass has been estimated in real-time in a basic manner, by measuring the average height or volume of grass above a ground estimate.

[0101] Ground reconstruction can also be performed using the approach described in T. Lowe, P. Moghadam, E. Edwards, and J. Williams, “Canopy density estimation in perennial horticulture crops using 3d spinning lidar slam,” Journal of Field Robotics, vol. 38, no. 4, pp.598-618, 2021.

[0102] While point-based geometric reconstruction methods provide 3D shape information, they lack the fidelity to estimate the type of object. Exploiting both modalities avoids these limitations. Accordingly, the above-described approach uses a combination of image segmentation and geometric reconstruction to be performed more effectively, substantially in real time.

[0103] In one example, the above-described pliability estimation approach is implemented using a run-time library, together with an offline library comprising a set of existing open source tools. These can be used for analysing, adjusting and visualising the generated environment maps. The data flow is summarised in Figure 4 where the run-time library has three main interfaces, the sensor and mapping inputs on the left side, the interpreted environment map outputs on the right side, and file storage of the maps on the underside. These will now be described in turn, before describing the full method itself.A. Inputs

[0104] The system has three real-time inputs, the mapping device trajectory, the point cloud and camera data including images.

[0105] The trajectory is estimated from the up-stream mapping system, which may employ any method or sensor to estimate the trajectory. It is expected that a portion of the past trajectory will be provided at each update, depending on how much has been re-estimated. It is typically formatted as a list of nodes containing the rotation quaternion, translation vector and time stamp.

[0106] The point cloud is the most recent 3D point cloud representing all the new observations since the last point cloud message. Each 3D point is in the same spatio-temporal coordinate frame as the trajectory and is time stamped.

[0107] The camera data is a list of RGB camera images with time stamps. The camera intrinsics and extrinsics are assumed to be known and fixed, therefore the timestamps can be used to retrieve the camera pose from the trajectory for each image.B. Outputs

[0108] A square bounding box attention region is maintained around the sensor origin, in which the environment is estimated. The results of this estimation are available in outputs of a ground map, a point map and a tree map.

[0109] The ground map is an n x n grid of ground cells, where each cell has a height, semantic class, and undergrowth height and density if it is a vegetation class.

[0110] The point map is a point cloud where each point contains the painted colour, semantic class, and estimated strength value, which estimates the Hookean constant (deviation per Newton of force applied) at that point.

[0111] The trees map: this is an n x n grid of tree cells, each cell indexes a list of trees. These trees are a list of branch sections and each branch section contains a tip position, radius and the index of its parent branch section. When this index is - 1 it indicates the root branch section.

[0112] These data structures are generated within the attention region using the attention grid as an acceleration structure, and are made available as continually published ROS2 messages.

[0113] It will also be appreciated that other outputs may also be provided. For example, any intermediate output of the system could also be outputted, such as details of the semantic segmentation, the ground map, or the like.C. Storage format

[0114] After the attention region maps are generated, they are stored in memory as a list of data per trajectory time stamp. This ensures they are available as a historical record of the interpreted environment. This data is then streamed to disk as needed to avoid exceeding RAM limitations. On shut down the remaining map in RAM can also written to disk. This map can be retrieved on request of the user, in order to access the interpreted environment outside of the attention region, which can be important for path planning beyond the attention region. Storing to disk also allows offline post analysis of the environment, and allows a prior map to be available on robot start-up.

[0115] The map data can be stored in the sensor frame such that loop closure events and place recognition events automatically alter and distort the map, which can in turn lead to recalculation of the pliability estimates. Typically old map data is not reapplied in the attention region algorithm, which instead relies on the certainty of only data observed while inside the region.D. Semantic Segmentation

[0116] The semantic segmentation component is used to estimate a discrete semantic class for each pixel within each received image. These classes represent the type of object or material. In one example, the following classes are used, although it will be appreciated that the classes used could vary depending in the preferred implementation or intended use:• Dirt• Gravel• Mud• Other terrain• Bush• Grass• Log• Tree foliage• Tree trunk• Fence• Other object• Rock• Vehicle• Structure• Water

[0117] DO is reserved for points that have been unobserved by the camera or were observed but have unknown class. The minimal requirement for pliability estimation is that the list includes classes that cover the vegetation types likely to be encountered. So, for example, when navigating an outdoor environment including vegetation, that is likely to include at least classes 5, 6, 8 and 9. Thus these classes can be used to allow existing outdoor annotated image data to be used for training a third-party classifier. Several such classifiers are available for this task as mentioned above.E. Point Cloud Painting

[0118] Semantic information of 3D lidar points is obtained by transferring semantic information computed on images to the visible 3D lidar points. This process, referred to as live colourisation, requires the intrinsic parameters of the 3D projective model for the camera as well as the Euclidean transform, or extrinsic parameters, which relates the position and orientation of the camera to the lidar sensor.

[0119] The 2D semantic segmentation algorithm is applied to each camera image in order to produce a corresponding semantic image. The segmentation algorithm is applied such that only one of the 15 semantic classes described above is assigned to each pixel.

[0120] Each 3D lidar measurement captured within a temporal window of the acquired camera image is then projected on to the image using the intrinsic and extrinsic camera parameters. The semantic label at the projected location is then transferred back to the 3D lidar measurement to produce a labelled point cloud. In the event that more than one 3D pointprojects to the same pixel location, then only the semantic label is transferred only to the nearest 3D lidar measurement.F. Dynamic Point Cloud Subsampling

[0121] To assist in interpreting the 3D environment from the classified inflowing point cloud data, real-time management of the data being attended to can be employed. In one example, to achieve the data being processed is bounded in size, fairly even in density, does not forget about points no longer observable from the current viewpoint and supports the re-estimation of the past trajectory.

[0122] To support these needs, a form of continuous spatial sub-sampling of the data is performed as the point data arrives. This is done by maintaining a grid of cells within the attention region, that refer to one point per cell. All incoming point clouds are stored in the local frame of the pose estimate at that time, building up a list of local point clouds, one for each pose in the trajectory. The point is also projected onto the attention grid in the unlocalised frame, a two-way link is maintained between the point and its current overlapping grid cell. Whenever a re-estimated trajectory is supplied, the points within these changed nodes are removed from the attention grid and reprojected onto the grid.

[0123] In both cases of a grid cell being written to, it is first checked whether the cell already links to a point, if it does then that point is removed from the list.

[0124] This approach maintains a spatially subsampled grid of points that can be accessed spatially within the attention box, with O(1) random access cost. But it also maintains a history of these points, which can be altered whenever a trajectory is re-estimated, such as during loop closure or map merging. As the robot moves through the world, the attention grid moves with it, but the points in the list remain. The result is a spatially subsampled map that has a bounded size in the attention region and grows only with area covered outside of that region.G. Ground Reconstruction

[0125] The ground is reconstructed according to a dry sand model described in T. Lowe, P. Moghadam, E. Edwards, and J. Williams, “Canopy density estimation in perennial horticulture crops using 3d spinning lidar slam,” Journal of Field Robotics, vol. 38, no. 4, pp. 598-618,2021, whereby observed points p, are labelled as ground if there are no other points p, such that:p? - p; > fc J(p? - pj)2+ (p? - pj)2

[0126] This represents a maximum gradient of kgbeyond which the points are considered to be a feature on the ground (like a ledge) or an object on the ground. kg= 1 is used in the current example. The ground surface is then an interpolation of these labelled ground points.H. Point Connectivity

[0127] To reconstruct vegetation, the assumption is made that vegetation is a collection of acyclic graphs (tree structures) of slender branches, and that it generally follows a short path to the ground where it is anchored. The vegetation semantic classes can be reconstructed using a multi-root shortest path algorithm, such as Dijkstra’s algorithm described in W. Dijkstra, “A note on two problems in connexion with graphs,” in Edsger Wybe Dijkstra: His Life, Work, and Legacy, 2022, pp. 287-290.

[0128] Unlike offline methods for reconstructing trees using this algorithm, in real time mapping the full connection to the ground may not yet be visible, so root locations are lacking. This can be addressed by making each new point observation in an unoccupied cell a root point, and setting its distance to ground dgto kd (pz- gz), where pzis the point height, gzthe estimated ground height at that location and kd » 1 is a large number to allow the point to connect to the ground once more points are observed, such as 10.

[0129] The result is that every point has a connection to its parent in a tree structure, and stores its distance to the ground dg. Not every point is vegetation, this is only performed for connectivity structure for points of the appropriate semantic classes. All other points above the ground are considered as hard obstacles such as rocks or ledges.

[0130] Once the graph of connections is built, the points can be traced downwards from the leaves, storing a maximum path distance to an end point de. which can be used required for radius estimation.I. Radius Estimation

[0131] Estimating vegetation branch radius directly is difficult due to the slenderness of most branches, the subsampling and accuracy of lidar points, and the occlusions and disorder in real vegetated environments. Consequently, in one example, strong priors are employed in order to obtain reliable estimates. The main prior is that each tree has a taper ratio (radius per branch length) that is approximately constant across the tree. This is input as a parameter kt which is set to O. Olm / m in one example, but can be customised for different types of vegetation.

[0132] In order to estimate the radius at a node given the taper prior, the longest distance to an end point di can be found. For thin trees this is already calculated as de, however thicker stems are composed of many parallel acyclic paths to the ground. These nodes can be agglomerated to all point to the same maximum deat each point up the stems. This is done in the following fashion:1. for each point p£, recursively set the nearest neighbour j’s long path index to lj = z as long as | p, — p71 < 3ktde(i) and de(i) > de(lj)2. for each point z = lj (longest path points) store the set Si = {J\lj = i}. these are the other nodes at this segment of the stem.3. for each point i = lj calculate the segment centroid= Z ' j&siP; / l^z I and a mean radius from the centroid r£= SyesJP; “ Q| / I^I-4. for each point i = lj set taper ti = n / deand weight= fctde(|S£| — l)k^ / 2, which approximates the volume of that cylindrical segment for taper kt with full point coverage.

[0133] The weighted average tree taper is then:kw+ Szw£

[0134] where kt is the taper prior and wt= nk^k^ is its weight, based on a cylindrical tree volume of height kh.

[0135] This means a tree of this volume has equal influence from the taper prior and observed taper, with smaller influence from the observations for smaller and more sparsely observed trees.

[0136] The taper prior is therefore employed in two ways, firstly it provides a maximum taper within which to look for stem points (in this example using 3fc), and secondly it provides the taper for the smaller and more sparsely observed trees where the signal to noise ratio is lowest.

[0137] Such careful use of priors allows tree thicknesses to be well estimated where the data allows it, and retain sensible values where it doesn’t.J. Strength Estimation

[0138] The estimated geometry is now in the form of an acyclic graph of generalised cylindrical segments, where generalised refers to the segments having a different base and tip radius according to the tree’ s taper gradient. Under a uniform material assumption the pliability at each node can be calculated in a cumulative manner from the root upwards. This pliability is the scalar deviation per applied force, lateral to the branch direction.

[0139] The beam formula from the Euler-Bernoulli static beam equation can be written as:d2w > Mdx2El

[0140] Where E is the elastic modulus of the wood, which is assumed to be a constant per-tree, w is lateral deflection and x is the height up the beam, treating the beam as vertical, to represent a tree stem. M is the bending moment, which in equilibrium is the negative of the applied force F multiplied by the orthogonal distance between the force and the point x being calculated. I is the second moment of area of the cross section at height x, which is ^rx4the radius rxat heightx.

[0141] The equation therefore defines the curvature of the beam as a function of the moment applied to it. For unit E and a force of one Newton applied at a distance L from x we have:d2w 4Ldx2n(Tx)4

[0142] where T is the taper (radius per length).

[0143] Considering the tree segment displayed in Figure 5, for the case where I = 0, the force is applied at distance L = x - eI+iat each position x, and so the change in tilt g = dw / dx of the beam at the top of the section is the integral of the beam curvature:4(x - e£+1)^9i+l — dx7T(T%)42 h2(eL+ 2e£+1)3 neiei+iT4

[0144] The tilt g for every segment up the tree can be updated with the iteration:9t+i 9i + ^9i+l +

[0145] This last term is the change in tilt at the base a due to applying the force h further up than when gi was calculated. This is calculated iteratively:Ag _ &g_+dg_^i+l Alt dl i+i

[0146] where:1 _ 1_dg d fei+141 4e3+1efdli+1dl Je. TI(TX')43 TIT4

[0147] The change in displacement on a single section is a double integration of the beam curvature:p+1f 4(x - ei+14 / i3AW£+1dxdx ='eiJei< Tx)437refe£+1T4

[0148] To calculate the displacement w up at segment z+1, the tilt is accumulated at each stage gi, with the iteration:wi+1= w + hgt+ Awi+1+ Wi+1

[0149] This last term is the change in displacement due to the force being applied higher up than when Wi, gi were calculated:AwWi+1= Wt+— hAw Aw dwEl i+i AZ i dl£+i

[0150] where the last term is the differential with respect to I of the doubly integrated beam curvature:dw d Cei+1f 41— = — I I — - — ~ dxdxdl£+1dl L J n(Tx^)43ne? T4

[0151] Due to the assumption of a constant bending modulus E, the general displacement per unit force at node z is Wi / E, and the Hookean stiffness k is its reciprocal: ki = E / wi.

[0152] This method is based on small displacements of a vertical pole. For non-vertical branches it only approximates the pliability to a force perpendicular to the branch.

[0153] The bending modulus E depends on the material, which in vegetated environments will be some kind of wood or green shoots. The standard deviation for tree wood is approximately 22% of the mean modulus (around 10 GPa or lelO N / m2), with non-woody stems like wheat having a smaller bending modulus around 5GPa. So even without knowing the species, being able to distinguishing wood from non-wood stems is still effective. It is therefore possible to use the estimated sematic classes to select from an approximate E value. These per-classstiffnesses can be sought from existing databases, or can be calibrated offline or online for a particular environment if the robot is equipped with a pressure sensor. In one example, the default stiffnesses is used, that being E = kw for wood and E = kp for non- woody plant stems.

[0154] Example default parameter values are shown in Table 1 below.Table 1symbol description default value and units kd height scale for disconnected 10pointskg max ground gradient 1 m / mkh tree taper height 1 mkt tree taper prior 0.01 m / mkw grid cell width 0.15 mkw wood bending modulus 10 GPakp plant bending modulus 5 GPaK. Vegetation Height and Density

[0155] In addition to per-point strength estimation, it is useful to summarise the ground surface as a 2D array of cells. Each cell specifies the reconstructed ground height and modal semantic class. If it is a vegetation class then we can also add two more pieces of information, namely the height, which is easily extracted as the distance to end de(i) for the plant rooted at each ground cell, and the density, which is represented the exposed surface area per cubic metre of the plant rooted at each ground cell, which may be due to leaves or branches. This can be estimated using a ray length method used originally for viticulture:

[0156] where n are the number of rays entering each cell that the plant overlaps, m are the number of rays ending within the cell, and z are the ray lengths.

[0157] The multiplier 2 here represents uniformly distributed surface normals. These values are accumulated in each grid cell for each ray that passes through the attention region. It is then possible to further accumulate the values up the plant at each ground root node, to obtain a more stable density than just averaging the ρᵢ for each cell i in the plant. It is also possible to additionally accumulate the n, m and Σxᵢ values for the eight cells in the Moore neighbourhood of each ground cell at one tenth of the central weight. This also aids in the stability of this density estimation, which is susceptible to fluctuations for small n.

[0158] This density value is a statistical means of estimating the amount of vegetation on the ground, which cannot be estimated directly from the subsampled points as it uses the accumulation of many passing rays within each grid cell. In the case of grass it may be used as a multiplier on the stiffness estimation, as denser grass regions generally means more blades or wider blades, in both cases this provides more resistance to a force.

[0159] Example outputs are shown in Figures 6A and 6B. In this regard, Figure 6A shows stiffness heatmap representing the stiffness as force per metre deviation (pliability is the opposite- metre deviation per force). Figure 6B shows the stem-height projected to ground as a value per 2D grid-cell makes for a simpler way for the downstream nav system to plan routes.

[0160] Accordingly, the above presents a software library that enables a real-time 3D point cloud and associated camera data to be used to determine an estimation of:• ground geometry• ground and object type• undergrowth length and density• vegetation geometry• vegetation pliability (where vegetation includes trunks, branches, undergrowth and grass).

[0161] This ‘environmental map’ updates whenever the trajectory is re-estimated, and downstream navigation software can access this information both at and away from the robot’s current location. In one example, as the robot moves through the world, the attention grid moves with it, but the points in the list of subsampled points remain. The result is a spatially sub sampled map that has a bounded size in the attention region and grows only with areacovered outside of that region. Whilst in some cases it would be more accurate to un-cull prior culled points on re-estimating the trajectory, for a low degree of re-estimation, this approach is still effective. This method assumes a static environment, but will always add new points when a new obstacle appears or moves closer. The downside is that it does not remove points where the object has moved away. From the point of view of navigation that means that the robot avoids the full set of past positions of any obstacle. In some cases (such as a tree waving in the wind) this may be a sensible choice. In other cases such as an animal crossing the future path of the robot, it can be problematic.

[0162] The reliance on both a real-time point cloud and camera data reflects the two components that are understood together to interpret the environment, namely the environment material, which requires high fidelity colour data to distinguish by texture, such as between grass, wood, rock and mud; and, the environmental geometry, which requires reliable 3D points The material informs the geometry reconstruction of what sort of geometry to reconstruct, and the geometry informs the pliability estimation, which is scaled according to the estimated material type.

[0163] The above-described approach employs the hybrid use of 2D Machine Learning for material estimation with computationally efficient numerical algorithms for geometry reconstruction, to utilise the strengths of both camera and point cloud data. The approach provides a method to estimate pliability for vegetation and can be configured to provide a map format that is robust to trajectory re-estimation.

[0164] Throughout this specification and claims which follow, unless the context requires otherwise, the word “comprise”, and variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated integer or group of integers or steps but not the exclusion of any other integer or group of integers. As used herein and unless otherwise stated, the term "approximately" means ±20%.

[0165] Persons skilled in the art will appreciate that numerous variations and modifications will become apparent. All such variations and modifications which become apparent to persons skilled in the art, should be considered to fall within the spirit and scope that the invention broadly appearing before described.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:1) A method for use in estimating the pliability of environmental features, the method including in one or more electronic processing devices:a) acquiring, from a mapping system including a mapping device moving along a trajectory through an environment, mapping data including:i) a 3D point cloud of the environment; and,ii) a trajectory of the mapping device relative to the environment;b) obtaining, from an imaging device associated with the mapping system, image data indicative of images of the environment captured by the imaging device as the imaging device moves through the environment;c) analysing the image data to determine image feature categories by categorising one or more image features;d) labelling a subset of points of the point cloud with the image feature categories; e) using the labelled subset of points to identify an environmental feature;f) calculating a geometry of the environmental feature;g) using the geometry and image feature categories of the environmental feature to calculate a pliability estimate for the environmental feature.2) A method according to claim 1, wherein the method includes, in the one or more processing devices, determining the subset of points of the point cloud by at least one of:a) subsampling the point cloud;b) performing a visibility check to determine points in the point cloud that are visible to a field of view of the imaging device;c) selecting points within a defined spatial extent of the imaging device.3) A method according to claim 1 or claim 2, wherein the method includes, in the one or more processing devices:a) label points in the point cloud; and,b) selecting a subset of labelled points.4) A method according to any one of the claims 1 to 3, wherein the method includes, in the one or more processing devices, analysing the labelled subset of points to distinguish different environmental features.5) A method according to any one of the claims 1 to 4, wherein the method includes, in the one or more processing devices, analysing points of the labelled subset of points having a particular image feature category to distinguish different environmental features.6) A method according to any one of the claims 1 to 5, wherein the method includes, in the one or more processing devices, analysing the points of the labelled subset of points having a particular image feature category to identify connectivity between points forming part of an environmental feature.7) A method according to claim 6, wherein the method includes, in the one or more processing devices, analysing the points of the labelled subset of points using a shortest path algorithm to identify connectivity between points.8) A method according to claim 6 or claim 7, wherein the method includes, in the one or more processing devices, estimating the geometry of a structure of the environmental feature using the connectivity.9) A method according to claim 8, wherein the method includes, in the one or more processing devices:a) using the point cloud to construct a ground surface; and,b) extending a structure of the environmental feature to the ground surface.10) A method according to claim 8 or claim 9, wherein the method includes, in the one or more processing devices, estimating the geometry by modelling the structure as a series of interconnected tapering cylinders.11) A method according to claim 10, wherein the method includes, in the one or more processing devices, estimating the geometry using a tapering ratio based on at least one of: a) a structure length; and,b) the image feature type of points in the structure.12) A method according to any one of the claims 1 to 11, wherein the method includes, in the one or more processing devices, estimating the pliability using an elastic modulus based on the image feature type of the points forming the environmental feature.13) A method according to any one of the claims 1 to 12, wherein the image features correspond to image pixels.14) A method according to any one of the claims 1 to 13, wherein the method includes, in the one or more processing devices:a) analysing the images data to categorise a feature type of pixels within the images; and, b) labelling the subset of points based on the feature type determined for corresponding pixels in one or more of images.15) A method according to any one of the claims 1 to 14, wherein the method includes, in the one or more processing devices:a) determining a transformation between image features and the subset of points; and, b) labelling the subset of points using the transformation.16)A method according to claim 15, wherein the method includes, determining the transformation based on at least one of:a) imaging device intrinsic properties;b) imaging device extrinsic properties; and,c) a known geometric relationship between the mapping device and imaging device.17) A method according to any one of the claims 1 to 16, wherein the method includes, in the one or more processing devices, labelling the subset of points in accordance with: a) a time stamp indicative of a capture of each image; and,b) the trajectory of the mapping device.18) A method according to any one of the claims 1 to 17, wherein the method includes, in the one or more processing devices, categorising one or more environmental features as one of:a) vegetative features, including one more of:i) bush;ii) grass;iii) tree foliage;iv) tree trunk; and,b) non-vegetative features, including one or more of:i) log;ii) dirt;iii) gravel;iv) mud;v) terrain;vi) fence;vii) object;viii) rock;ix) vehicle; and,x) structure.19)A method according to any one of the claims 1 to 18, wherein the method includes, in the one or more processing devices, controlling movement of a vehicle through the environment in accordance with the pliability estimate.20) A method according to any one of the claims 1 to 19, wherein the method includes, in the one or more processing devices:a) using the pliability estimate to calculate a traversal indicator indicative of the ability to traverse the environment; and,b) controlling the vehicle using the traversal indicator.21) A method according to claim 20, wherein the method includes, in the one or more processing devices:a) generating a three dimensional occupancy grid using the point cloud;b) populating the occupancy grid with pliability estimates; and,c) using the occupancy grid to identify traversable parts of the environment.22) A method according to any one of the claims 19 to 21, wherein the mapping data updates as the vehicle moves within the environment.23) A system for use in estimating the pliability of environmental features, the system including one or more electronic processing devices configured to:a) acquire, from a mapping system including a mapping device moving along a trajectory through an environment, mapping data including:i) a 3D point cloud of the environment; and,ii) a trajectory of the mapping device relative to the environment;b) obtain, from an imaging device associated with the mapping system, image data indicative of images of the environment captured by the imaging device as the imaging device moves through the environment;c) analyse the image data to determine image feature categories by categorising one or more image features;d) label a subset of points of the point cloud with the image feature categories;e) use the labelled subset of points to identify an environmental feature;f) calculate a geometry of the environmental feature;g) use the geometry and image feature categories of the environmental feature to calculate a pliability estimate for the environmental feature.

Images