Velocity estimation

The method enhances velocity estimation by generating and aligning vector fields to identify and classify moving features, addressing computational inefficiencies and enabling accurate real-time analysis for vehicle navigation and feature recognition.

WO2026239082A1PCT designated stage Publication Date: 2026-11-19BAE SYSTEMS PLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/GB2026/050774
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2026-01-15
Filing Date
2026-05-08
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Estimating velocities of features from images captured by cameras of vehicles moving through an environment is computationally expensive and often fails to complete analysis in sufficient time, leading to missed features in busy environments.

Method used

A computer-implemented method that estimates a velocity of a moving feature by generating an expected vector field, calculating an actual vector field, identifying misaligned regions, and using differential vector fields to determine the velocity of the feature, incorporating classification and dimension analysis.

Benefits of technology

Improves velocity estimation efficiency by accurately identifying and classifying moving features, enabling real-time analysis and construction of three-dimensional point clouds for enhanced vehicle navigation and feature recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure GB2026050774_19112026_PF_FP_ABST
    Figure GB2026050774_19112026_PF_FP_ABST
Patent Text Reader

Abstract

The present invention relates to a computer-implemented method of estimating a velocity of a moving feature, the computer-implemented method comprising: receiving a plurality of images from a camera mounted to an entity moving through an environment; estimating an expected vector field for pixel motion in the plurality of images, the expected vector field representing an expected velocity of the environment relative to the entity; calculating an actual vector field from the plurality of images, the actual vector field representing an actual velocity of the environment relative to the entity; identifying a region in the actual vector field that is not aligned with a corresponding region in the expected vector field as a moving feature; identifying a differential vector field for the identified region and a dimension associated with the identified region; and estimating the velocity of the identified object based on the differential vector field and the dimension associated with the identified region.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] VELOCITY ESTIMATION

[0002] FIELD

[0003]

[0001] The subject-matter of the present disclosure relates to computer-implemented methods of estimating a velocity of a moving feature, transitory, or non-transitory, computer-readable media, and vehicles.

[0004] BACKGROUND

[0005]

[0002] Estimating velocities of features from images captured by cameras of vehicles moving through an environment can be difficult. Typically, this may be done by identifying all features and then monitoring their relative positions, and thus speed, relative to the vehicle over time. This can be computationally expensive and in busy environments it may not be possible to complete the analysis in sufficient time, e.g. a time between image frames, so some features may be missed.

[0006]

[0003] It is an aim of the present invention to address such problems and improve on the prior art.

[0007] SUMMARY

[0008]

[0004] According to an aspect of the present disclosure, there is provided a computer-implemented method of estimating a velocity of a moving feature, the computer-implemented method comprising: receiving a plurality of images from a camera mounted to an entity moving through an environment; estimating an expected vector field for pixel motion in the plurality of images, the expected vector field representing an expected velocity of the environment relative to the entity; calculating an actual vector field from the plurality of images, the actual vector field representing an actual velocity of the environment relative to the entity; identifying a region in the actual vector field that is not aligned with a corresponding region in the expected vector field as a moving feature; identifying a differential vector field for the identified region and a dimension associated with the identified region; and estimating the velocity of the identified object based on the differential vector field and the dimension associated with the identified region.

[0009]

[0005] In an embodiment, the computer-implemented method further comprises: estimating a divergence in the differential vector field, wherein estimating the velocity of the identified feature based on the differential vector field comprises: estimating the velocity of the identified feature based on the divergence in the differential vector field.

[0006] In an embodiment, the computer-implemented method further comprises: classifying the identified feature using a classification algorithm.

[0010]

[0007] In an embodiment, the dimension associated with the region is an estimated distance to the identified region, wherein estimating the distance to the identified region uses: the estimated distance to the region, i LrfeayaiLoUnfL is

[0011]

[0012] a dimension of the feature in the plurality of images, Lobjectis a dimension of the feature obtained using known information about the classified object, and is a camera specific scaling factor.

[0013]

[0008] In an embodiment, the computer-implemented method further comprises: discounting the region if the region has a smaller size than a classification threshold.

[0014]

[0009] In an embodiment, calculating a relative transverse speed of the identified object uses: vreaion= dr I e ayai toun I *

[0015]

[0016] > where d i rf e ayai LoUnLL is an uncalibrated relative transverse

[0017] “expected

[0018] speed of the object, vregion⊥is the expected relative transverse velocity of the object, dregion is the estimated distance to the region, and dexpectedis an expected distance to the region.

[0019]

[0010] In an embodiment, the computer-implemented method further comprises: calculating the uncalibrated relative transverse speed of the feature using: lregioni=

[0020]

[0021] where ^region! is the uncalibrated relative transverse speed of the object, Vexpectedis the expected vector field, and Vmeasuredis the actual vector field.

[0022]

[0011] In an embodiment, calculating a relative longitudinal speed of the identified region uses. Vj-egionii — ^expected ' ^region '

[0023]

[0024] > where ^region is a vector field divergence

[0025] “expected

[0026] ratio of the region, vexpected\s an expected relative longitudinal speed of the object,vregion,s anestimated relative longitudinal speed of the object, and dregionis the estimated distance to the region, and dexpectedis an expected distance to the region.

[0027]

[0012] In an embodiment, calculating the vector field ratio uses: Be

[0028]

[0029] where Region is th® vector field divergence ratio, Vexpectedis the expected vector field, ^measured is the actual vector field.

[0030]

[0013] In an embodiment, the computer-implemented method further comprising: updating the expected vector field periodically.

[0031]

[0014] In an embodiment, the computer-implemented method further comprises updating the expected vector field at a rate synchronised with updating the actual vector field.

[0032]

[0015] In an embodiment, the expected vector field and / or the actual vector field includes a direction and a magnitude for respective velocities of respective points in the environment over a time between image frames.

[0033]

[0016] In an embodiment, generating an expected vector field comprises: generating the expected vector field based on a dynamic parameter of the vehicle and a camera property.

[0034]

[0017] In an embodiment, the dynamic parameter of the vehicle is one of a speed of the vehicle, an acceleration of the vehicle, a heading of the vehicle, and a turn radius of the vehicle, and optionally wherein the camera property includes at least one of a frame rate and a lens.

[0035]

[0018] In an embodiment, generating an actual vector field comprises: identifying points in an image frame of the plurality of image frames; identifying the points in a next image frame of the plurality of image frames; and calculating a change in position of the points between the image frame and the next image frame.

[0036]

[0019] In an embodiment, the points include points selected from a list of points including: an edge, a corner, and an object feature and a colour.

[0037]

[0020] In an embodiment, the computer-implemented method further comprises: identifying sudden changes in differences in the identified region.

[0038]

[0021] According to an aspect of the present disclosure, there is provided a transitory, or non-transitory, computer-readable medium, having instructions stored thereon that when executed by at least one processor, causes the at least one processor to perform the computer-implemented method of any preceding aspect or embodiment.

[0039] BRIEF DESCRIPTION OF DRAWINGS

[0040]

[0022] The subject-matter of the present disclosure is best described with reference to the accompanying figures, in which:

[0041]

[0023] Figure 1 shows a schematic view of a vehicle according to at least one embodiment;

[0042]

[0024] Figure 2 shows a schematic view of an expected vector field representing an air environment into which the vehicle of Figure 1 is moving, according to at least one embodiment;

[0043]

[0025] Figure 3 shows a schematic view of an expected vector field representing a land environment into which the vehicle of Figure 1 is moving, according to at least one embodiment;

[0044]

[0026] Figure 4 is a schematic view of an actual vector field representing a land environment into which the vehicle is moving, and which includes a static feature, according to at least one embodiment;

[0045]

[0027] Figure 5 shows a schematic view of an actual vector field representing a land environment into which the vehicle is moving, an which includes a moving feature, according to at least one embodiment;

[0046]

[0028] Figure 6 is a flow chart summarising a computer-implemented method of identifying a static feature, according to at least one embodiment;

[0047]

[0029] Figure 7 is a flow chart summarising a computer-implemented method of identifying a moving feature, according to at least one embodiment;

[0048]

[0030] Figure 8 is a flow chart summarising a computer-implemented method of extracting a static feature from an image, according to at least one embodiment;

[0049]

[0031] Figure 9 is a flow chart summarising a computer-implemented method of estimating a velocity of a moving feature, according to at least one embodiment; and

[0050]

[0032] Figure 10 is a flow chart summarising a computer-implemented method of constructing a three-dimensional point cloud, according to at least one embodiment.

[0051] DESCRIPTION OF EMBODIMENTS

[0052]

[0033] Although the example embodiments have been described with reference to the components, modules and units discussed herein, such functional elements may be combined into fewer elements or separated into additional elements. Various combinations of optional features have been described herein, and it will be appreciated that described features may be combined in any suitable combination. In particular, the features of any one example embodiment may be combined with features of any other embodiment, as appropriate, except where such combinations are mutually exclusive.

[0053] Throughout this specification, the term “comprising” or “comprises” means including the component(s) specified but not to the exclusion of the presence of others.

[0054]

[0034] The embodiments described herein may be embodied as sets of instructions stored as electronic data in one or more storage media. Specifically, the instructions may be provided on a transitory or non-transitory computer-readable media. When executed by the processor, the processor is configured to perform the various methods described in the following embodiments. In this way, the methods may be computer-implemented methods. In particular, the processor and a storage including the instructions may be incorporated into a vehicle.

[0055]

[0035] Whilst the following embodiments provide specific illustrative examples, those illustrative examples should not be taken as limiting, and the scope of protection is defined by the claims. Features from specific embodiments may be used in combination with features from other embodiments without extending the subject-matter beyond the content of the present disclosure.

[0056]

[0036] With reference to Figure 1, a vehicle 10 includes at least one camera 12 mounted to a body of the vehicle. The camera captures a plurality of image frames of an environment into which the vehicle is moving. The vehicle 10 also includes a computer 14. The computer 14 includes the storage 16 and the least one processor 18. The vehicle may be a military vehicle, e.g. a tank. The vehicle may be an uncrewed vehicle, e.g. an autonomous vehicle. The vehicle may be a land vehicle (e.g. a tank), a watercraft (e.g. an underwater autonomous vehicle), or an aircraft (e.g. a drone).

[0057]

[0037] The instructions on storage may be considered as a computer-implemented method.

[0058]

[0038] The computer implemented method may be a computer-implemented method of identifying a static feature. A feature, whether static or moving / dynamic, may be an object, a sub-object, etc.

[0059]

[0039] With reference to Figure 2, the computer-implemented method comprises generating an expected vector field 20 representing a movement of an environment in front of a vehicle, relative to the vehicle. The expected vector field includes an origin 22, a horizon 24, and a plurality of vectors 26. The vectors represent changes in position over a time interval of points within the environment. The vectors may thus represent a relative speed of the environment relative to the vehicle. This relative speed matches a speed of the vehicle.

[0060]

[0040] The vectors include a magnitude, e.g. a speed, and a direction. The vectors may be in three-dimensions, e.g. horizontal, vertical, and depth. An origin vector associated with the origin 22 may be only in a depth direction. The direction may be represented as an angle relative to the origin vector. The direction may be a direction generally toward or away from the vehicle. The environment may be assumed to be vacant. When the environment is a sky or underwater, and does not include land extending between the vehicle and the horizon 24, the vectors may be divergent. In this way, the vector directions point generally toward the vehicle. If the vehicle 10 were reversing, the expected vector field 20 may be converging. In a converging field, the vector directions point generally away from the vehicle. In both converging and diverging vector fields, the magnitudes of the vectors increases with increasing distance from the origin. This is because the vehicle is not stationary.

[0061]

[0041] The expected vector field includes a direction and a magnitude for respective changes in positions of respective points in the environment over a time between image frames.

[0062]

[0042] Generating the expected vector field may be based on a dynamic parameter of the vehicle and a camera property. The dynamic parameter of the vehicle is one of a speed of the vehicle, an acceleration of the vehicle, a heading of the vehicle, and / or a turn radius of the vehicle. For example, the speed of the vehicle may match the relative speed of each of the vectors. The acceleration may help predict the relative speed at future time steps, e.g. subsequent image frames. The heading and turn radius of the vehicle may result in changes in direction of the vectors in the expected vector field.

[0063]

[0043] The camera property may include at least one of frame rate and a lens, or at least lens type. Frame rate may affect a length of the vectors because the vector shows a change in position of a point between frames.

[0064]

[0044] With reference to Figure 3, a land vehicle will have land 28 extending between the vehicle and the horizon 24. Points of the land 28, or ground, will be nearer to the vehicle than points in the sky 29. Therefore, the points on the land 28 will be moving relatively faster than those in the sky. Therefore, the magnitudes, e.g. the vector lengths, will be bigger (or longer) for the land vectors 26_1 compared to the sky vectors 26_2 above the horizon 24.

[0065]

[0045] With reference to Figure 4, the method also comprises generating, or calculating, an actual vector field 30, which may comprise: identifying points in an image frame of the plurality of image frames; identifying the points in a next image frame of the plurality ofimage frames; and calculating a change in position of the points between the image frame and the next image frame. The points include points selected from a list of points including: an edge, a corner, a feature of a feature (e.g. an object feature), and a colour. In other words, the points includes one or more of an edge, a corner, an object feature, or a colour.

[0066]

[0046] In the same way as the expected vector field, the actual vector field includes a plurality of vectors. Each of those vectors corresponds to a change in the point between the frames, e.g. the adjacent frames. This may be a positional change, as above. Since the frame rate of the camera is known, the change in position can provide a speed of the point, which may be a speed relative to the vehicle. The positional changes may be represented as the vectors, include a magnitude denoting speed, and a direction. The direction may point generally toward the vehicle for diverging fields and may point generally away for converging fields. The direction may be an angle compared to the origin vector which only has a depth direction.

[0067]

[0047] This method described above in relation to Figure 4 is described for colour image frames. A different approach is used when using hyperspectral image frames which will now be described.

[0068]

[0048] For instance, the plurality of image frames may be a plurality of hyperspectral image frames when the at least one camera 12 (Figure 1) is a hyperspectral camera. The actual vector field can then be calculated from the plurality of hyperspectral image frames.

[0069]

[0049] With reference to Figure 5, calculating the actual vector field of the environment from the plurality of hyperspectral image frames comprises obtaining a hyperspectral curve 40, 42, for each pixel in each hyperspectral image frame. The hyperspectral curve 40, 42, includes a plurality of frequency bands. The plurality of frequency bands may include 50 to 20 frequency bands in some examples, and in other examples there may be more or fewer frequency bands. The hyperspectral curve may be stored as a vector, e.g. one intensity value for each frequency band.

[0070]

[0050] The frequency bands are divided into three frequency ranges, e.g. an ultraviolet (UV) range 43, a visible range 44, and an infrared (I R) range 45.

[0071]

[0051] Next, the method may comprise grouping contiguous pixels together that have substantially the same hyperspectral signature to form a plurality of hyperspectral regions. This may be done in various ways.

[0072]

[0052] One way would be to classify each pixel into one of a plurality of classes, e.g. material classes. Then, contiguous pixels sharing the same class can be grouped into hyperspectral classes using a clustering algorithm. The clustering algorithm may be an unsupervised machine learning algorithm such as k-means clustering.

[0073]

[0053] Another way would be to calculate a distance between contiguous pixels. The distance may not be a geometric distance but a distance in terms of similarity. The distance may be a Euclidian distance, a cosine similarity, etc. In this way, the similarity between the vector of each pixel may be computed.

[0074]

[0054] Then, changes in the hyperspectral regions may be monitored between the hyperspectral image frames. This may comprise identifying a point on each hyperspectral region. The identified point may be one of a point of an edge, a corner, an inflection point, a centre, and a centroid of the hyperspectral region. Then, changes in position of the point are monitored between hyperspectral image frames.

[0075]

[0055] With further reference to Figure 4, the feature 32 is also a hyperspectral region, the actual vector field may then be calculated based on the monitored changes. In other words, then the point moves position between frames, its direction of movement and also its magnitude of movement can be determined. In this way, the actual vector field can represent velocities of points through image frames.

[0076]

[0056] The method also comprises calculating a scalar field of vector scaling factors, y, of the actual vector field with reference to the expected vector field. The vector scaling factor formula is provided below, and equates to the expected vector divided by the actual vector for a corresponding point in the respective vector fields. The scalar field is a collection of vector scaling factors across the entire image, or environment.

[0077]

[0057] The method also comprises grouping regions of the scalar field having substantially the same scaling factor. In other words, adjacent points having the same scaling factor may be grouped as a region. When performed across the entire field, there may be a plurality of regions covering the entire environment, or image, except for any regions which are moving (see below). Therefore, the regions may be joined, and every point that is static within the environment may belong to one of the regions.

[0078]

[0058] The method comprises identifying a region of the grouped regions having vectors substantially aligned between the actual and expected vector fields. The term substantially aligned may be taken to mean misaligned within an alignment tolerance. Completely aligned vectors will be at the expected distance. Misaligned vectors whichare within an alignment tolerance may be deemed to be static features, which are closer than the expected distance. If the alignment is above the alignment threshold, the features may not be static.

[0079]

[0059] The method also comprises classifying the identified region as a static feature. This may be achieved by labelling that region as static.

[0080]

[0060] The method may further comprise calculating a distance to the identified region based on an expected distance to the region and a scaling factor.

[0081]

[0061] Calculating the distance to the identified region based on the expected distance to the region and the scaling factor comprises using:

[0082] yd-region deXpected. y, where dregionis the actual distance to the region, riexpected is the expected distance to the region, and y is the vector scaling factor.

[0083]

[0062] Calculating the vector scaling factor, y, uses:

[0084]

[0085] where Vexpectedis the expected vector field, and Vmeasuredis the actual vector field.

[0086]

[0063] The expected distance to the region is known because the environment used to generate the expected vector field is known or predetermined, and all points within the environment have known distances.

[0087]

[0064] The method may comprise updating the expected vector field periodically. The actual vector period may also be updated periodically. Those vector fields may be updated in a synchronised manner, in other words using the same synchronised rate. The rate may match a frame rate of the camera. Therefore, each time a new image is captured, the expected vector field and the actual vector field can be updated. Since the expected vector field is predetermined, it may be extracted from a map or a repository and localised to the camera using known techniques.

[0088]

[0065] The method may further comprise identifying sudden changes in differences in the identified region. In this way, temporarily occluded features can be detected when they suddenly appear or suddenly disappear.

[0089]

[0066] There may also be provided an aspect of the present disclosure which uses this method of identifying a static feature to extract the feature so that it can be used for downstream models such as feature recognition. Therefore, there is also provided a

[0090] computer-implemented method of extracting a static feature from an image. The computer-implemented method comprises: identifying a static feature using the computer-implemented method described above; generating a semantic segmentation map to segment the static feature from the image frame; and extracting the segmented static feature from the image. The semantic segmentation map may be generated using a learned model, such as a machine learning model including, for example, a neural network. The extracted image can be input to a downstream model trained to perform a machine vision task, such as feature recognition where the feature can be classified semantically. Other tasks may include trajectory prediction, where the trajectory of the feature is predicted over time. This can feed into trajectory generation for the vehicle so the vehicle can take account of the predicted trajectory of the feature either to avoid the other feature or catch the other feature in a pursuer-evader or evader-pursuer scenario.

[0091]

[0067] Another downstream model may be a model for generating a three-dimensional point cloud. In an aspect, the model may perform a computer-implemented method such as constructing the three-dimensional point cloud, the method comprising: identifying a static feature using the foregoing computer-implemented method. The method also includes identifying a position of the static feature. This may involve calculating an actual distance to the region, and determining its horizontal and vertical positions in the image frame using conventional techniques. The method also comprises constructing the three-dimensional points cloud including a plurality of points at the position of the static feature. The plurality of points may be located at edges, corners, or other features of the region, at the depth associated with the actual distance. When the entire image consists of static features, the entire image is covered in regions described above. Those regions will be reflected in the point cloud across the entire environment, or image.

[0092]

[0068] With reference to Figure 6, in another embodiment, there is provided a computer-implemented method of identifying a moving feature. The method comprises generating an expected vector field representing the anticipated relative movement of an environment in front of a vehicle 10. The expected vector field is the same as the static feature identification method.

[0093]

[0069] The method also comprises capturing a plurality of image frames of the environment as the vehicle moves into the environment, in the same way as the static feature identification method. In addition, the method comprises calculating an actual vector field of the environment from the plurality of images, in the same way as the static feature identification method.

[0094]

[0070] Other aspects of the moving feature identification method are the same as those of the static feature detection method, aside from identifying a region and classifying the identified region as an feature.

[0095]

[0071] For the moving feature detection method, the method includes identifying a region in the actual vector field that is not aligned with the corresponding region in the expected vector field. As with the static feature identification method, corresponding region is one which positionally the same in the expected and actual vector fields. The vector fields not being aligned may be taken to mean that the angles of the vectors within the vector fields are different relative to the origin vector. Expressed alternatively, the vectors associated with points in the region in the expected vector field may not be parallel, or may have a relative angle greater than 0 compared, with corresponding vectors in the region in the actual vector field. The fact that the actual vector field is not aligned with the expected vector field means that the region is travelling at a different relative velocity to the environment. This can equate to a moving feature. The feature may be an object, a sub-object of an object, etc. This is because any moving feature is highly unlikely to be moving exactly in the same trajectory as the vehicle 10 for an extended period of time. Even on a very straight flat road, another vehicle in front will deviate lateral position with its lane and there is likely to be undulations in levelness of the ground. Any bends in a road, and the other vehicle will be travelling on a different trajectory. That is to say the method may, in some circumstances, incorrectly not classify a moving feature but that this state of error is highly unlikely to persist for a significant period of time.

[0096]

[0072] The method of extracting a moving feature may be performed using the foregoing method of identifying a moving feature and may be summarised as including: identifying a moving feature using the computer-implemented method described above; generating a semantic segmentation map to segment the moving feature from the image frame; and extracting the segmented moving feature from the image.

[0097]

[0073] The method of constructing a three-dimensional point cloud may be performed in the same way but with the moving feature rather than the static feature. When performing both methods in combination, the three-dimensional point cloud will include points for both static feature and moving features.

[0098]

[0074] In another aspect of the present disclosure, there is provided a computer-implemented method of estimating a relative velocity of a moving feature. The computer-implemented method comprises: receiving a plurality of images from a camera mountedto an entity moving through an environment; estimating, or generating, an expected vector field representing an environment in front of the entity, or at least representing an environment, or a relative velocity of an environment in front of the entity; calculating an actual vector field from the plurality of images, the actual vector field representing an actual velocity of the environment relative to the entity; and identifying a region in the actual vector field that is not aligned with a corresponding region in the expected vector field as an feature, as with the method of identifying the moving feature. The entity may be the vehicle.

[0099]

[0075] The method may further comprise: estimating a divergence in the differential vector field, wherein estimating the velocity of the identified feature based on the differential vector field comprises: estimating the velocity of the identified feature based on the divergence in the differential vector field. Divergence may refer to the vectors generally separating in a direction toward the camera.

[0100]

[0076] This method may further comprise classifying the identified feature using a classification algorithm.

[0101]

[0077] This method also comprises identifying a differential vector field for the identified region and a dimension associated with the identified region.

[0102]

[0078] The dimension associated with the region is an estimated distance to the identified region, wherein estimating the distance to the identified region uses:

[0103]

[0104]

[0080] where dregionis the estimated distance to the region, Lregionis a dimension of the feature in the plurality of images, Lobjectis a dimension of the feature obtained using known information about the classified feature, and β is a camera specific scaling factor.

[0105]

[0081] The known information may be obtained from a look-up table for a classified feature. The dimension of the feature may be one of height, width, and length. Those distances may be maximum distances or average distances. The height, width, and length may also be the dimensions of a bounding box surrounding the feature once it has been classified depending on the model used to classify the feature. The dimension of the feature in the plurality of images may be a real dimension, e.g. it’s measured height, width, or length, or may be a number of pixels used to represent the feature.

[0106]

[0082] The method further comprises discounting the region if the region has a smaller size than a classification threshold. The size may be the dimension of the feature

[0107] obtained using known information. For example, a fly has a small size, and so can be discounted. A building has a comparatively larger size and so should not be discounted. The classification threshold may be a dimension, and may be predetermined. For example, the classification threshold may be a predetermined length, width, or height, of a known feature.

[0108]

[0083] The method also comprises estimating the velocity of the identified feature based on the differential vector field and the dimension associated with the identified region.

[0109]

[0084] Estimating the velocity of the identified feature comprises calculating a relative transverse speed of the identified feature using:

[0110]

[0111]

[0086] where ψregionis an uncalibrated relative transverse speed of the feature, vregion ⊥is the expected relative velocity of the feature, dregionis the estimated distance to the region, and dexpectedis an expected distance to the region. The term uncalibrated means its transverse speed without taking into account a distance between the feature and the camera. Thus the term calibrated may be taken to mean the relative transverse speed taking into account a distance between the feature and the camera. In this way, the expected relative velocity of the feature can take into account its distance to the camera, such that, for example, a fly close to the camera and a fighter jet far from the camera, which appear to be moving at the same speed when uncalibrated, can be calibrated such that their expected transverse velocity is more accurate.

[0112]

[0087] Estimating the velocity of the identified feature further comprises: calculating the uncalibrated relative transverse speed of the feature using:

[0113]

[0114]

[0089] where ψregion⊥is the uncalibrated relative transverse speed of the feature, Vexpectedis the expected vector field, and Vmeasuredis the measured, or actual, vector field.

[0115]

[0090] Estimating the velocity of the identified feature comprises calculating a relative longitudinal speed of the identified region using:

[0116]

[0117]

[0092] where ζregionis a vector field divergence ratio of the region, vexpectedis an expected relative longitudinal speed of the feature, vregionis an estimated relative longitudinal speed of the feature, dregionis the estimated distance to the region, and dexpectedis an expected distance to the region. The expected relative longitudinal speed of the feature is equal to the vehicle speed because the expected region is part of the environment not including an feature, and thus has equal magnitude, opposite direction, velocity to the vehicle. The vehicle speed is known from various means and sensors, including odometry.

[0118]

[0093] calculating the vector field ratio uses:

[0119]

[0120]

[0095] where ζregionis the vector field divergence ratio, Vexpectedis the expected vector field, Vmeasuredis the actual vector field.

[0121]

[0096] Other features of this aspect may be consistent with the features of the other aspects, including updating the expected vector field periodically, updating the expected vector field at a rate synchronised with updating the actual vector field, etc.

[0122]

[0097] The various aspects may be summarised as follows.

[0123]

[0098] With reference to Figure 7, there is provided a computer-implemented method of identifying a static feature. The method comprises: generating 100 an expected vector field representing a movement of an environment in front of a vehicle, relative to the vehicle; capturing 102 a plurality of image frames of the environment as the vehicle moves into the environment; calculating 104 an actual vector field of the environment from the plurality of images; identifying 106 a region in the actual vector field having vectors aligned with vectors of a corresponding region in the expected vector field, the vectors of the region of the actual vector field having substantially the same magnitude scaling factor; and classifying 108 the identified region as a static feature.

[0124]

[0099] With reference to Figure 8, there is provided a computer-implemented method of identifying a moving feature. The computer-implemented method comprises: generating 200 an expected vector field representing the anticipated relative movement of environment in front of a vehicle; capturing 202 a plurality of image frames of the environment as the vehicle moves into the environment; calculating 204 an actual vector field of the environment from the plurality of images; identifying 206 a region in the actualvector field that is not aligned with the corresponding region in the expected vector field; and classifying 208 the identified region as a moving feature.

[0125]

[0100] With reference to Figure 9, there is provided a computer-implemented method of computer-implemented method of extracting a static feature from an image. The computer-implemented method comprising: identifying 300 a static feature or a dynamic feature using the computer-implemented method summarised above; generating 302 a semantic segmentation map to segment the static feature from the image frame; and extracting 304 the segmented static feature from the image.

[0126]

[0101] With reference to Figure 10, there is provided a computer-implemented method of estimating a velocity of a moving feature. The computer-implemented method comprises: receiving 400 a plurality of images from a camera mounted to an entity moving through an environment; estimating 402 an expected vector field representing an expected velocity of an environment relative the entity; calculating 404 an actual vector field from the plurality of images, the actual vector field representing an actual velocity of the environment relative to the entity; 406 identifying a region in the actual vector field that is not aligned with a corresponding region in the expected vector field as an feature; identifying 408 a differential vector field for the identified region and a dimension associated with the identified region; and estimating 410 the velocity of the identified feature based on the differential vector field and the dimension associated with the identified region.

[0127]

[0102] With reference to Figure 11, according to an aspect of the present disclosure, there is provided a computer-implemented method of constructing a three-dimensional point cloud. The method comprises: identifying 500 an feature using any of the foregoing methods; identifying 502 a position of the static feature; and constructing 504 the three-dimensional point cloud including a plurality of points at the position of the feature. The feature may be the static feature or the moving feature.

[0128]

[0103] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.

[0129]

[0104] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude aplurality. A single processor or other unit may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.

[0130] Elements of the present disclosure may be understood with reference to the following clauses.

[0131] Clause 1. A computer-implemented method of identifying a moving feature, the computer-implemented method comprising: generating an expected vector field representing the anticipated relative movement of environment in front of a vehicle; capturing a plurality of image frames of the environment as the vehicle moves into the environment; calculating an actual vector field of the environment from the plurality of images; identifying a region in the actual vector field that is not aligned with the corresponding region in the expected vector field; and classifying the identified region as a moving feature.

[0132] Clause 2. The computer-implemented method of Clause 1, further comprising: updating the expected vector field periodically.

[0133] Clause 3. The computer-implemented method of Clause 2, further comprising updating the expected vector field at a rate synchronised with updating the actual vector field.

[0134] Clause 4. The computer-implemented method of any preceding clause, wherein the expected vector field and / or the actual vector field includes a direction and a magnitude for respective changes in positions of respective points in the environment over a time between image frames.

[0135] Clause 5. The computer-implemented method of any preceding clause, wherein generating an expected vector field comprises: generating the expected vector field based on a dynamic parameter of the vehicle and a camera property.

[0136] Clause 6. The computer-implemented method of Clause 5, wherein the dynamic parameter of the vehicle is one of a speed of the vehicle, an acceleration of the vehicle, a heading of the vehicle, and a turn radius of the vehicle.

[0137] Clause 7. The computer-implemented method of Clause 5 or Clause 6, wherein the camera property includes at least one of a frame rate and a lens.

[0138] Clause 8. The computer-implemented method of any preceding clause, wherein generating an actual vector field comprises: identifying points in an image frame of the plurality of image frames; identifying the points in a next image frame of the plurality ofimage frames; and calculating a change in position of the points between the image frame and the next image frame.

[0139] Clause 9. The computer-implemented method of Clause 8, wherein the points include points selected from a list of points including: an edge, a corner, and an feature feature and a colour

[0140] Clause 10. The computer-implemented method of any preceding clause, wherein the image frames are hyperspectral image frames.

[0141] Clause 11. The computer-implemented method of Clause 10, wherein calculating the actual vector field of the environment from the plurality of hyperspectral images comprises: obtaining a hyperspectral curve from each pixel of each hyperspectral image frame; grouping contiguous pixels together that have substantially the same hyperspectral signature to form a plurality of hyperspectral regions; monitoring changes in the hyperspectral regions between the hyperspectral image frames; and calculating the actual vector field based on the monitored changes.

[0142] Clause 12. The computer-implemented method of Clause 11, wherein monitoring changes in the hyperspectral regions comprises: identifying a point on each hyperspectral region; and monitoring changes in position of the point between hyperspectral image frames.

[0143] Clause 13. The computer-implemented method of Clause 12, wherein the identified point is one of: a point of an edge, a corner, an inflection point, a centre, and a centroid.

[0144] Clause 14. The computer-implemented method of any preceding clause, further comprising: identifying sudden changes in differences in the identified region.

[0145] Clause 15. A computer-implemented method of extracting a moving feature from an image, the computer-implemented method comprising: identifying a moving feature using the computer-implemented method according to any preceding clause; generating a semantic segmentation map to segment the moving feature from the image frame; and extracting the segmented moving feature from the image.

[0146] Clause 16. A computer-implemented method of constructing a three-dimensional point cloud, the computer-implemented method comprising: identifying a moving feature using the computer-implemented method according to any of Clauses 1 to 14; identifying a position of the moving feature; and constructing the three-dimensional point cloud including a plurality of points at the position of the moving feature

[0147] Clause 17. A computer-implemented method of estimating a velocity of a moving feature, the computer-implemented method comprising: receiving a plurality of images from a camera mounted to an entity moving through an environment; estimating an expectedvector field for pixel motion in the plurality of images, the expected vector field representing an expected velocity of the environment relative to the entity; calculating an actual vector field from the plurality of images, the actual vector field representing an actual velocity of the environment relative to the entity; identifying a region in the actual vector field that is not aligned with a corresponding region in the expected vector field as a moving feature; identifying a differential vector field for the identified region and a dimension associated with the identified region; and estimating the velocity of the identified feature based on the differential vector field and the dimension associated with the identified region.

[0148] Clause 18. The computer-implemented method of Clause 17, further comprising: estimating a divergence in the differential vector field, wherein estimating the velocity of the identified feature based on the differential vector field comprises: estimating the velocity of the identified feature based on the divergence in the differential vector field. Clause 19. The computer-implemented method of Clause 17 or Clause 18, further comprising: classifying the identified feature using a classification algorithm.

[0149] Clause 20. The computer-implemented method of Clause 19, wherein the dimension associated with the region is an estimated distance to the identified region, wherein estimating the distance to the identified region uses:

[0150]

[0151] where dregionis the estimated distance to the region, Lregionis a dimension of the feature in the plurality of images, Lobjectis a dimension of the feature obtained using known information about the classified feature, and β is a camera specific scaling factor.

[0152] Clause 21. The computer-implemented method of any of Clauses 17 to 20, further comprising: discounting the region if the region has a smaller size than a classification threshold

[0153] Clause 22. The computer-implemented method of any of Clauses 17 to 21, wherein calculating a relative transverse speed of the identified feature uses:

[0154]

[0155] where ψregionis an uncalibrated relative transverse speed of the feature, vregion ⊥is the expected relative transverse velocity of the feature, dregionis the estimated distance to the region, and dexpectedis an expected distance to the region.

[0156] Clause 23. The computer-implemented method of Clause 22, further comprising: calculating the uncalibrated relative transverse speed of the feature using:

[0157]

[0158] where ψregion⊥is the uncalibrated relative transverse speed of the feature, Vexpectedis the expected vector field, and Vmeasuredis the actual vector field

[0159] Clause 24. The computer-implemented method of any of Claims 17 to 23, wherein calculating a relative longitudinal speed of the identified region uses:

[0160] „ >. 7. Region

[0161] "regionll ~ expected region

[0162] u>

[0163] expected

[0164] where ζregionis a vector field divergence ratio of the region, vexpectedis an expected relative longitudinal speed of the feature, vregionis an estimated relative longitudinal speed of the feature, and dregionis the estimated distance to the region, and dexpectedis an expected distance to the region.

[0165] Clause 25. The computer-implemented method of Clause 24, wherein calculating the vector field ratio uses:

[0166] ' measured

[0167] tregion ~

[0168] V’V expected

[0169] where ζregionis the vector field divergence ratio, Vexpectedis the expected vector field, Vmeasuredis the actual vector field.

[0170] Clause 26. The computer-implemented method of any of Clauses 17 to 25, further comprising:

[0171] updating the expected vector field periodically.

[0172] Clause 27. The computer-implemented method of Clause 26, further comprising updating the expected vector field at a rate synchronised with updating the actual vector field.

[0173] Clause 28. The computer-implemented method of any of Clauses 17 to 27, wherein the expected vector field and / or the actual vector field includes a direction and a magnitude for respective velocities of respective points in the environment over a time between image frames.

[0174] Clause 29. The computer-implemented method of any of Clauses 17 to 28, wherein generating an expected vector field comprises: generating the expected vector field based on a dynamic parameter of the vehicle and a camera property.

[0175] Clause 30. The computer-implemented method of Clause 29, wherein the dynamic parameter of the vehicle is one of a speed of the vehicle, an acceleration of the vehicle, a heading of the vehicle, and a turn radius of the vehicle, and optionally wherein the camera property includes at least one of a frame rate and a lens.

[0176] Clause 31. The computer-implemented method of any of Clauses 17 to 30, wherein generating an actual vector field comprises: identifying points in an image frame of the plurality of image frames; identifying the points in a next image frame of the plurality of image frames; and calculating a change in position of the points between the image frame and the next image frame.

[0177] Clause 32. computer-implemented method of Clause 31, wherein the points include points selected from a list of points including: an edge, a corner, and an object feature and a colour.

[0178] Clause 33. The computer-implemented method of any of Clauses 17 to 32, further comprising: identifying sudden changes in differences in the identified region.

[0179] Clause 34. A computer-implemented method of identifying a static feature, the computer-implemented method comprising: generating an expected vector field representing a movement of an environment in front of a vehicle, relative to the vehicle; capturing a plurality of image frames of the environment as the vehicle moves into the environment; calculating an actual vector field of the environment from the plurality of image frames; calculating a scalar field of vector scaling factors of the actual vector field with reference to the expected vector field; grouping regions of the scalar field having substantially the same scaling factor; identifying a region of the grouped regions having vectors substantially aligned between the actual and expected vector fields; and classifying the identified region as a static feature.

[0180] Clause 35. The computer-implemented method of Clause 34, further comprising: calculating a distance to the identified region based on an expected distance to the region and the vector scaling factor.

[0181] Clause 36. The computer-implemented method of Clause 35, wherein calculating the distance to the identified region based on the expected distance to the region and the scaling factor comprises using:

[0182] dregion. ~ dexpected. y,

[0183] where dregionis the actual distance to the region, dexpectedis the expected distance to the region, and y is the vector scaling factor.

[0184] Clause 37. The computer-implemented method of Clause 36, further comprising: calculating the vector scaling factor, y, using:

[0185]

[0186] wherein Vexpectedis the expected vector field, and Vmeasuredis the actual vector field..

[0187] Clause 38. The computer-implemented method of any of Clauses 34 to 37, further comprising: updating the expected vector field periodically.

[0188] Clause 39. The computer-implemented method of Clause 38, further comprising updating the expected vector field at a rate synchronised with updating the actual vector field.

[0189] Clause 40. The computer-implemented method of any of Clauses 34 to 39, wherein the expected vector field and / or the actual vector field includes a direction and a magnitude for respective changes in positions of respective points in the environment over a time between image frames.

[0190] Clause 41. The computer-implemented method of any of Clauses 34 to 40, wherein generating an expected vector field comprises: generating the expected vector field based on a dynamic parameter of the vehicle and a camera property.

[0191] Clause 42. The computer-implemented method of Clause 41, wherein the dynamic parameter of the vehicle is one of a speed of the vehicle, an acceleration of the vehicle, a heading of the vehicle, and a turn radius of the vehicle.

[0192] Clause 43. The computer-implemented method of Clause 41 or Clause 42, wherein the camera property includes at least one of a frame rate and a lens.

[0193] Clause 44. The computer-implemented method of any of Clauses 34 to 43, wherein generating an actual vector field comprises: identifying points in an image frame of the plurality of image frames; identifying the points in a next image frame of the plurality of image frames; and calculating a change in position of the points between the image frame and the next image frame.

[0194] Clause 45. The computer-implemented method of Clause 44, wherein the points include points selected from a list of points including: an edge, a corner, an object feature, and a colour.

[0195] Clause 46. The computer-implemented method of any of Clauses 34 to 45, further comprising: identifying sudden changes in differences in the identified region.

[0196] Clause 47. The computer-implemented method of any of Clauses 34 to 46, wherein the image frames are hyperspectral image frames.

[0197] Clause 48. The computer-implemented method of Clause 47, wherein calculating the actual vector field of the environment from the plurality of hyperspectral image frames comprises: obtaining a hyperspectral curve for each pixel of each hyperspectral image frame; grouping contiguous pixels together that have substantially the same hyperspectral signature to form a plurality of hyperspectral regions; monitoring changes in the hyperspectral regions between the hyperspectral image frames; and calculating the actual vector field based on the monitored changes.

[0198] Clause 49. The computer-implemented method of Clause 48, wherein monitoring changes in the hyperspectral region comprises: identifying a point on each hyperspectral region; and monitoring changes in position of the point between hyperspectral image frames.

[0199] Clause 50. The computer-implemented method of Clause 49, wherein the identified point is one of: a point of an edge, a corner, an inflection point, a centre, and a centroid.

[0200] Clause 51. A computer-implemented method of extracting a static feature from an image, the computer-implemented method comprising: identifying a static feature using the computer-implemented method according to any of Clauses 34 to 50; generating a semantic segmentation map to segment the static feature from the image frame; and extracting the segmented static feature from the image.

[0201] Clause 52. A computer-implemented method of constructing a three-dimensional point cloud, the computer-implemented method comprising: identifying a static feature using the computer-implemented method according to any of Clauses 34 to 51; identifying a position of the static feature; and constructing the three-dimensional point cloud including a plurality of points at the position of the static feature.

[0202] Clause 53. A transitory, or non-transitory, computer-readable medium having instructions stored thereon that when executed by at least one processor, causes the at least one processor to perform the computer-implemented method of any preceding clause.

[0203] Clause 54. A vehicle comprising: a camera for capturing images of an environment into which the vehicle is moving; at least one processor; and storage storing instructions that when executed by the at least one processor causes the at least one processor to perform the computer-implemented method of any of Clauses 1 to 52.

Claims

CLAIMS1. A computer-implemented method of estimating a velocity of a moving feature, the computer-implemented method comprising: receiving a plurality of images from a camera mounted to an entity moving through an environment; estimating an expected vector field for pixel motion in the plurality of images, the expected vector field representing an expected velocity of the environment relative to the entity; calculating an actual vector field from the plurality of images, the actual vector field representing an actual velocity of the environment relative to the entity; identifying a region in the actual vector field that is not aligned with a corresponding region in the expected vector field as a moving feature; identifying a differential vector field for the identified region and a dimension associated with the identified region; and estimating the velocity of the identified object based on the differential vector field and the dimension associated with the identified region.

2. The computer-implemented method of Claim 1, further comprising: estimating a divergence in the differential vector field, wherein estimating the velocity of the identified feature based on the differential vector field comprises: estimating the velocity of the identified feature based on the divergence in the differential vector field.

3. The computer-implemented method of Claim 1 or Claim 2, further comprising: classifying the identified feature using a classification algorithm.

4. The computer-implemented method of Claim 3, wherein the dimension associated with the region is an estimated distance to the identified region, wherein estimating the distance to the identified region uses: dregion= βL, wheredregionis the estimated distance to the region, Lregionis a dimension of the feature in the plurality of images, Lobjectis a dimension of the feature obtained using known information about the classified object, and β is a camera specific scaling factor.

5. The computer-implemented method of any of Claims 1 to 4, further comprising: discounting the region if the region has a smaller size than a classification threshold.

6. The computer-implemented method of any of Claims 1 to 5, wherein calculating a relative transverse speed of the identified object uses: vregion⊥= ψregion*d / d, where ψregionis an uncalibrated relative transverse speed of the“expectedobject, vregion⊥is the expected relative transverse velocity of the object, dregionis the estimated distance to the region, and dexpectedis an expected distance to the region.

7. The computer-implemented method of Claim 6, further comprising: calculating the uncalibrated relative transverse speed of the feature using: ψregion⊥=where ψregion⊥is the uncalibrated relative transverse speed of the object, Vexpectedis the expected vector field, and Vmeasuredis the actual vector field.

8. The computer-implemented method of any of Claims 1 to 7, wherein calculating a relative longitudinal speed of the identified region uses: vregion∥= vexpected· ζregion·d / d, where ζregionis a vector field divergence ratio of the“expectedregion, vexpectedis an expected relative longitudinal speed of the object, vregionis an estimated relative longitudinal speed of the object, and dregionis the estimated distance to the region, and dexpectedis an expected distance to the region.

9. The computer-implemented method of Claim 8, wherein calculating the vector field ratio uses: ( rreegaiioonn =vw'venxepaesuctreedd’ where ( ^rreegaiioonn is the vector field divergence ratio, Vexpectedis the expected vector field, Vmeasuredis the actual vector field.

10. The computer-implemented method of any of Claims 1 to 9, further comprising: updating the expected vector field periodically.

11. The computer-implemented method of Claim 10, further comprising updating the expected vector field at a rate synchronised with updating the actual vector field.

12. The computer-implemented method of any of Claims 1 to 11, wherein the expected vector field and / or the actual vector field includes a direction and a magnitude for respective velocities of respective points in the environment over a time between image frames.

13. The computer-implemented method of any of Claims 1 to 12, wherein generating an expected vector field comprises: generating the expected vector field based on a dynamic parameter of the vehicle and a camera property.

14. The computer-implemented method of Claim 13, wherein the dynamic parameter of the vehicle is one of a speed of the vehicle, an acceleration of the vehicle, a heading of the vehicle, and a turn radius of the vehicle, and optionally wherein the camera property includes at least one of a frame rate and a lens.

15. The computer-implemented method of any of Claims 1 to 14, wherein generating an actual vector field comprises: identifying points in an image frame of the plurality of image frames; identifying the points in a next image frame of the plurality of image frames; and calculating a change in position of the points between the image frame and the next image frame.

16. The computer-implemented method of Claim 15, wherein the points include points selected from a list of points including: an edge, a corner, and an object feature and a colour.

17. The computer-implemented method of any of Claims 1 to 16, further comprising: identifying sudden changes in differences in the identified region.