Environmental condition monitoring for a vehicle

Abstract

An environmental condition monitoring for a vehicle (100). A computer-implemented method 200 for monitoring an environmental condition of a vehicle. The method 200 comprises receiving from each of a plurality of audio input devices disposed about the vehicle, respective audio data 201-1, …, 201-r. For each respective audio data: a spectrogram is determined 210 in dependence on the audio data indicative of a frequency distribution of the audio data over a selected time window. Each determined spectrogram is input 220 into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle. Obtaining 230 from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; and outputting 240 an environmental classification signal indicative of the one or more values. The model may be a neural network and may use tensors as an input.

Description

TECHNICAL FIELD The present disclosure relates to environmental condition monitoring for a vehicle. Aspects of the invention relate to methods, to a control system, to an environmental condition monitoring system, to a vehicle, and to computer-readable instructions. BACKGROUND As vehicles implement increasing levels of autonomy, vehicle systems for monitoring the environment surrounding the vehicle become increasingly important. Such monitoring includes identifying hazards in the vicinity of the vehicle, such as pedestrians, road accidents, debris and the like which may require the vehicle to adapt its behaviour, such as to alert the user, avoid the hazard or brake. Such monitoring may also include identifying conditions of a road surface on which the vehicle is driving, to facilitate the adjustment of vehicle dynamics. Typical environmental monitoring technologies include radar, lidar and image sensors mounted about a vehicle. However, these modalities may not be robust, in particular to conditions which are occluded from view, for example an approaching hazard around a corner or a road surface which is difficult to visibly detect, such as ice. It is an aim of the present invention to address one or more of the disadvantages associated with the prior art. SUMMARY OF THE INVENTION Aspects and embodiments of the invention provide a method for environmental condition monitoring, a control system, an environmental condition monitoring system, a vehicle, a method of training a classification model and computer-readable instructions as claimed in the appended claims. According to an aspect of the present invention there is provided a computer-implemented method for monitoring an environmental condition of a vehicle, the method comprising: receiving, from each of a plurality of audio input devices disposed about the vehicle, respective audio data; determining input data in dependence on each respective audio data; inputting the input data into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle; obtaining, from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; and outputting an environmental classification signal indicative of the one or more values. Advantageously, taking audio from about the exterior of the vehicle as input to a classification model facilitates the identification of environmental conditions which may be obscured from visual identification but may be isolated from environmental noise, such as road conditions or objects obscured behind obstacles. According to another aspect of the present invention there is provided a computer-implemented method for monitoring an environmental condition of a vehicle, the method comprising: receiving, from each of a plurality of audio input devices disposed about the vehicle, respective audio data; for each respective audio data: determining, in dependence on the audio data, a spectrogram indicative of a frequency distribution of the audio data over a selected time window; inputting each determined spectrogram into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle; obtaining, from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; and outputting an environmental classification signal indicative of the one or more values. Advantageously, taking audio from about the exterior of the vehicle as input to a classification model facilitates the identification of environmental conditions which may be obscured from visual identification but may be isolated from environmental noise, such as road conditions or objects obscured behind obstacles. Furthermore, the use of spectrograms as input to the classification model enables the decomposition of audio features at each audio input device, which enables the spatial location of these features to be encoded in the temporal discrepancies between each spectrogram. The audio input devices disposed about an exterior of the vehicle or otherwise arranged to record audio external to the vehicle. Optionally, the method comprises obtaining a waveform corresponding to a selected time window of the audio data, and determining the spectrogram in dependence on the waveform. Optionally, determining each spectrogram comprises dividing the audio data into p sliding time windows, and applying a Fast Fourier Transform with q frequency bins to each of the p time windows to determine a spectrogram having dimensions p and q. The sliding time windows may be overlapping. Advantageously, this ensures spectral information is captured over the whole of the duration of the audio data. In some embodiments, the selected time window of the audio data (i.e. the entirety of spectrogram) has a duration of between 0.1 and 5 seconds, such as 3 seconds, 2 seconds or 0.5 seconds. Each sliding time window may have a length of between 10ms and 50ms, such as10ms, 20ms or 30ms. Each sliding time window may have an overlap of between 30% and 70% of the sliding time window length, such as 40% or 50%. The method may comprise receiving respective audio data from r audio input devices, combining each spectrogram to obtain an input tensor I of dimensions p, q and r, and inputting the input tensor I into the classification model. Optionally, combining comprises concatenating each spectrogram. Advantageously, this ensures distinct features within the spectrogram are spatially aligned and the location is encoded in slight temporal misalignment, enabling accurate location identification of occupancy types. The classification model may comprise an artificial neural network for classifying at least one environmental condition of a surrounding environment of the vehicle, and the method may comprise: determining an input tensor for the artificial neural network in dependence on each of the determined spectrograms; inputting the input tensor to input nodes of an input layer of the neural network; and mapping the input tensor through connecting nodes of the neural network to output nodes of an output layer to obtain each of the one or more values indicative of at least one environmental condition being present. Advantageously, an artificial neural network (ANN) may perform automatic feature extraction, in contrast to other machine learning classifiers. This facilitates the input of spectrograms as raw input to the model. The artificial neural network has been trained to identify the environmental condition using a set of training data, the set of training data comprising audio data from a plurality of audio input devices disposed about a training vehicle under a set of conditions having each of the at least one environmental condition. The training vehicle is equivalent to the vehicle, with a corresponding arrangement of audio input devices about the training vehicle. The artificial neural network may comprise a backbone sub-network having connecting nodes connecting the input layer to a feature map layer, the connections of the connecting nodes having weights. Optionally, the feature map is downsampled. Advantageously, this layer of the artificial neural network can perform automated feature extraction, using any backbone network suitable for image processing or computer vision. Optionally, the artificial neural network comprises a head sub-network and an attention pooling sub-network, the attention pooling sub-network comprising connecting nodes between the feature map layer and an input layer of the head sub-network, the connections of the connecting nodes having weights, wherein the input layer of the head sub-network is arranged to receive values indicative of at least one audio feature fa. Each environmental condition may be a road surface condition. The classification model may be for classifying the road surface condition of the vehicle as one of a discrete set of N road surface conditions i = 1,..., N. The discrete set of road surface conditions may comprise one or more of: a normal surface condition; a slippery surface condition; an off-road surface condition; a snow surface condition; an ice surface condition; and a rain surface condition. Advantageously, the method can facilitate the classification of the road surface even when the road surface is not readily visible or detectable through other modalities, such as vision or radar systems. Optionally, at least a subset of the one or more values Ri obtained from the classification model are indicative of a probability of the road surface condition being of type i. Optionally, the method comprises: receiving, from a visual classification system of the vehicle, visual data indicative of a further one or more values Vi, V2,...., Vn, wherein at least a subset of the further values Vi are indicative of a probability of the road surface condition being of type i according to the visual classification system; and determining, in dependence on the one or more values Ri and the further one or more values Vi, an overall classification of the road surface condition. Optionally, the environmental classification signal may be output to be indicative of the overall classification. The overall classification maybe determined by determining an overall probability of the road surface being of type / as a weighted sum of Ri and Vi Advantageously, utilising both audio and visual classification results in improved confidence compared to either modality individually. Further, decoupling the modalities leads to a higher classification confidence as the predictions are independent. Optionally, the method comprises determining, from the classification model, at least one audio feature fa; receiving, from the visual classification system of the vehicle, at least one visual feature fv, inputting, to input nodes of an input layer of a fusion neural network for predicting a confidence of each of the one or more values Ri and the further one or more values Vi, the one or more values Ri, the at least one audio feature fa, the further one or more values Viand the at least one visual feature fv; mapping the input through connecting nodes of the fusion neural network to output nodes of an output layer to obtain one or more weight values W; determining a fused classification probability for each of the road surface conditions i as a combination of the values Ri and Vi weighted according to the one or more weight values Wi; and outputting the environmental classification signal in dependence on the fused classification probabilities. The audio feature fa may be determined from the feature map layer of the artificial neural network. Advantageously, providing a fusion neural network facilitates the flexible adaption of weight values for combining the modalities depending on the particular circumstances, characterised by the audio and visual features and probabilities. In this way, the fusion neural network can predict the confidence of each modality in the present circumstance. By decoupling these modalities initially and then providing a separate fusion network at the end, this allows the training of the original neural network on a large single modality dataset, requiring only a relatively small cross-modality training set for the fusion. The at least one environmental condition may comprise a set of N classes i of localized hazard i = 1,..., N, and the classification model may be for classifying whether each class i of localized hazard is present in a vicinity of the vehicle. The set of N classes of localized hazard may comprise one or more of: a construction area; an emergency vehicle; a vehicle horn; a speeding vehicle; a braking vehicle; a pedestrian in a vehicle path; an animal in a vehicle path; a road accident; or falling debris. Advantageously, the method can detect localized hazards earlier than other techniques such as visual and radar monitoring, as characteristic audio features can be detected even when the hazard is occluded from view. This provides the vehicle with more time to respond to the hazard. The artificial neural network may comprise a set of N output layers, each output layer corresponding to a class i of localized hazard in the set, each output layer having output nodes for outputting output data comprising at least a first value Ci indicative of a probability of the class i of localized hazard being present in the vicinity of the vehicle and a second value Pi indicative of a predicted position of the class i of localized hazard with respect to the vehicle. The method may comprise mapping the input tensor I through the connecting nodes to the output nodes of each output layer Oi to obtain each of the first value Ci and the second value Pi. The predicted position may be in an angular coordinate system (e.g. radius, azimuth, elevation). Thus, a plurality of second values Pi may be defined to characterize the predicted position. The method may comprise determining a prediction for whether each class i of localized hazard is present in dependence on each first value Ci; and in dependence on the determined prediction for each class i, outputting the environmental classification signal to be indicative of the predicted position of the class i of localized hazard. The prediction may be binary (yes / no) in dependence on whether the probability Ci is above a threshold. The method may comprise outputting a control signal to control a vehicle trajectory or vehicle motion in dependence on the determined prediction. The control signal may be output in dependence on the predicted position, to avoid the localized hazard. The method may comprise outputting a control signal to a brake preparation system to prepare the vehicle to come to a stop in dependence on the determined prediction. Optionally, the artificial neural network comprises: a backbone sub-network having connecting nodes connecting the input layer to a feature map layer, the connections of the connecting nodes having weights; and a respective head sub-network for each class / of localized hazard in the set, i = 1, ..., N, and wherein each respective head sub-network comprises connecting nodes between the feature map layer and the output layer corresponding to the class i of localized hazard in the set, the connections of the connecting nodes having weights. Advantageously, this provides a multi-headed structure wherein each head network can function independently to classify each individual class of hazard i using a common set of automatically extracted features, enabling improved accuracy in classification for each specific class. Optionally, the artificial neural network further comprises a respective attention pooling sub-network for each class i of localized hazard in the set, i = 1,..., N, wherein each respective attention pooling sub-network comprises connecting nodes connecting the feature map layer to an input layer of the respective head sub-network, wherein the input layer of each respective head sub-network is indicative of a class specific audio feature fa, / . Advantageously, the network structure facilitates class specific attention pooling, facilitating the extraction of audio features specific to each discrete class of hazard for input into the relevant head sub-network. The artificial neural network may comprise a mean pooling sub-network connecting the feature map layer to a mean pooled layer, wherein the mean pooling sub-network acts to pool the feature map output over the frequency dimension q. Each attention pooling sub-network may then connect the mean pooled layer to the respective input layer of the respective head sub-network. Optionally, the method comprises: receiving, from a visual classification system of the vehicle, visual data comprising, for each class i, a first visual value Ci v indicative of a probability of the class i of localized hazard being present in the vicinity of the vehicle and a second visual value Pi,v indicative of a predicted position of the class i of localized hazard with respect to the vehicle; and determining, in dependence on the N output data and the visual data, a predicted class and location for one or more identified localized hazards. Advantageously, utilising both audio and visual classification results in improved confidence compared to either modality individually. Further, decoupling the modalities leads to a higher classification confidence as the predictions are independent. Optionally, the method comprises: receiving, from the visual classification system of the vehicle, at least one class specific visual feature fv,i associated with each of the N classes of localized hazard i= 1,..., N; weighting each class specific visual feature fv,>by the associated first visual value Ci v at its predicted position Pi,v in a spatial model; weighting each class specific audio feature fs,i by the associated first value Ci at its predicted position Pi in the spatial model; inputting, to an input layer of a fusion neural network for object localization and classification, the spatial model; mapping the input through connecting nodes of the fusion neural network to output nodes of an output layer to obtain a fusion output; determining the class and the location of the one or more identified localized hazards according to the fusion output; and outputting the environmental classification signal in dependence on the determined class and location. The spatial model may be an X-Y-Z birds-eye-view (BEV) grid of size F-X-Y-Z, where F is a maximum length of the class specific features. The fusion neural network may comprise a convolutional neural network (e.g. BEV encoder) and a 3D object detection head sub-network (e.g. CenterHead). Advantageously, by decoupling these modalities initially, this allows the training of the original neural network on a large single modality dataset, requiring only a relatively small cross-modality training set for the fusion. Furthermore, fusing the features in a common spatial grid and then providing a separate network to classify features within the common grid facilitates an improved accuracy of object fusion. The fusion neural network may comprise: an input layer having input nodes for receiving an input tensor I indicative of the spatial model, at least one output layer having output nodes for outputting an output vector, wherein at least a subset of the values of the output vector are indicative of a class and a location of one or more identified localized hazards; and one or more hidden layers having connecting nodes connecting the input layer to the at least one output layer. The fusion neural network may be trained to classify and locate hazards using a set of training data, the set of training data comprising audio data from a plurality of audio input devices disposed about a training vehicle and visual data from one or more imaging devices for imaging a field of view of the vehicle, the set of training data taken under a set of conditions having each class of localized hazard at a range of positions relative to the vehicle. According to another aspect there is provided a control system for controlling an environmental condition monitoring system for a vehicle, the control system comprising one or more controller, the control system configured to: receive, from each of a plurality of audio input devices disposed about the vehicle, respective audio data; for each respective audio data, determine, in dependence on the audio data, a spectrogram indicative of a frequency distribution of the waveform over a selected time window; input each determined spectrogram into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle; obtain, from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; and output an environmental classification signal indicative of the one or more values. The control system may be configured to perform any method disclosed herein. The control system may comprise one or more controllers collectively comprising at least one electronic processor having an electrical input for receiving an input signal; and at least one memory device electrically coupled to the at least one electronic processor and having instructions stored therein; and wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions thereon so as to: receive, from each of a plurality of audio input devices disposed about the vehicle, respective audio data; for each respective audio data, determine, in dependence on the audio data, a spectrogram indicative of a frequency distribution of the waveform over a selected time window; input 4 each determined spectrogram into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle; obtain, from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; and output an environmental classification signal indicative of the one or more values. The control system may be configured to perform any method disclosed herein. According to another aspect there is provided an environmental condition monitoring system for a vehicle comprising: any control system disclosed herein; and a plurality of audio input devices arranged to be disposed about the vehicle, wherein the plurality of audio input devices are arranged to communicate the respective audio data to the control system. According to another aspect there is provided a vehicle comprising any control system or any environmental condition monitoring system as disclosed herein. According to another aspect there is provided computer-implemented method of training a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle, wherein the method comprises: receiving a set of training data to train the classification model, the set of training data comprising audio data from a plurality of audio input devices disposed about a training vehicle under a set of conditions having each environmental condition present, the set of training data comprising input-output pairs, each pair comprising audio data from each of the audio input devices and corresponding classification data indicative of a classification of whether each of the at least one environmental condition i is present for the associated audio data; for each input-output pair: determining, in dependence on each audio data from each of the audio input devices, a respective spectrogram indicative of a frequency distribution of the audio data over a selected time window; and determining an input tensor I for the input-output pair in dependence on each of the determined spectrograms; inputting the input tensor I into the classification model; obtaining, from the classification model, a predicted set of values Pi indicative of a predicted probability of each of the at least one environmental condition i being present; determining an objective function characterising an error between the values of the predicted set of values Pj compared to the corresponding classification data for each of the one or more input-output pairs; and using an appropriate optimisation algorithm operating on the objective function, updating the classification model to seek to minimise the objective function. The audio input devices may be disposed about an exterior of the training vehicle or otherwise configured to record audio external to the training vehicle. If the classification model is artificial neural network, updating the classification model may comprise updating the weights between connecting nodes of the artificial neural network. According to another aspect there is provided computer readable instructions which, when executed by a computer, are arranged to perform any method disclosed herein. Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and / or features of any embodiment can be combined in any way and / or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and / or incorporate any feature of any other claim although not originally claimed in that manner. BRIEF DESCRIPTION OF THE DRAWINGS One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows a vehicle in accordance with an embodiment of the invention; Figure 2 shows a flow chart of a method according to an embodiment; Figure 3A shows a schematic of an environmental condition monitoring system according to an embodiment; Figure 3B shows an example arrangement of audio input devices according to an embodiment of the environmental condition monitoring system; Figure 4 shows a schematic of a control system according to an embodiment; Figure 5 illustrates the processing of audio data according to an embodiment; Figure 6 illustrates a classification model according to an embodiment; Figure 7 illustrates a mean pooling layer of a classification model according to an embodiment; Figure 8 illustrates an attention pooling sub-network of a classification model according to an embodiment; Figure 9 illustrates a classification system according to another embodiment; Figure 10 illustrates a fusion method according to an embodiment; Figure 11 illustrates a fusion method according to a further embodiment; Figure 12 illustrates a fusion method according to a further embodiment; and Figure 13 illustrates a training method in accordance with an embodiment. DETAILED DESCRIPTION As vehicles implement increasing levels of autonomy, vehicle systems for monitoring the environment surrounding the vehicle become increasingly important. Such monitoring includes identifying hazards in the vicinity of the vehicle, such as pedestrians, road accidents, debris and the like which may require the vehicle to adapt its behaviour, such as to alert the user, avoid the hazard or brake. Such monitoring may also include identifying conditions of a road surface on which the vehicle is driving, to facilitate the adjustment of vehicle dynamics such as traction control, brake preparation, and dynamic adaption of a suspension system. Typical environmental monitoring technologies include radar, lidar and image sensors mounted about a vehicle. However, these modalities may not be robust, in particular to conditions which are occluded from view, for example an approaching hazard around a corner or a road surface which is difficult to visibly detect, such as ice. According to the present invention, there is provided an improved system and method for monitoring an environmental condition of a vehicle such as the vehicle 100 illustrated in Figure 1. The vehicle 100 illustrated is an automobile, but the present invention may be implemented with any type of vehicle, including land vehicles, watercraft, aircraft or the like. Figure 2 illustrates a method 200 according to an embodiment of the present invention. The method 200 is a method of monitoring an environmental condition of a vehicle 100, such as the vehicle 100 illustrated in Figure 1. In particular, the method 200 is a method of classifying at least one environmental condition of a surrounding environment of the vehicle. The surrounding environment of the vehicle may be defined as a region within a given distance of the vehicle, and thus the surrounding environment may change over time if the vehicle is moving. The method 200 utilises audio data received from audio input devices disposed about the vehicle to classify the environmental condition of the vehicle. Thus, the surrounding environment may be defined in dependence on the sensitivity of the audio input devices, as the region from which originating sounds can be detected by the audio input devices. Beneficially, classifying the surrounding environment using audio data facilitates the identification of environmental conditions even when they are obscured from view of any camera or lidar system, as diffraction enables sound from each location to traverse obstacles. Thus, the method 200 may accurately identify conditions such as road traffic collisions, emergency vehicles and the like when they are occluded by an obstacle. The method 200 may be particularly beneficial when fused with other sensing modalities to provide an additional class of environmental information, as will be discussed later with reference to Figures 10 to 12. The method 200 may be performed, at least in part, by a control system 310 for an environmental condition monitoring system 300, as illustrated in Figure 3A. The environmental condition monitoring system 300 is for use in a vehicle, such as the vehicle 100, and may be installed at least partially within or about the vehicle 100. The environmental condition monitoring system 300 comprises the control system 310 and a plurality of r audio input devices 320-1, ... ,320-r arranged to be disposed about the vehicle. Each of the r audio input devices are arranged to record audio at a respective location about the vehicle and communicate respective audio data to the control system 310 for use in the method 200, as will be explained. The audio input devices 320-1, ..., 320-rare, in use, arranged to record audio external to the vehicle. For example, they may be installed at various locations about an exterior of the vehicle. An example arrangement for the audio input devices 320-1,..., 320-r about the vehicle 100 is illustrated in Figure 3B. According to the arrangement illustrated in Figure 3B, five audio input devices 320-1, ..., 320-5 are provided. The audio input devices 320-1, ..., 320-5 are disposed around a perimeter of the vehicle, at front and rear corners and at a center front location. In this way, the audio input devices surround a perimeter of a the vehicle which can provide good localisation for surrounding objects. Each audio input device may be a stereo input device, such as a stereo 5 microphone, to further improve sound localisation. As such, each illustrated audio input device 320-1,..., 320-5 may actually represent two (or more) audio input devices, in some embodiments. The provision of additional audio input devices can provide increased accuracy in localisation of environmental conditions, however it may be desired to keep the number of audio input devices to a minimum to reduce cost. The arrangement shown in Figure 3B has been found to provide a good 10 accuracy. However, alternative arrangements can be envisaged. In particular, the minimum requirement is that at least two audio input devices be provided, to enable sounds to be localised. Providing three or more audio input devices can provide additional benefits such as improved accuracy for triangulation. Returning to Figure 2, the method 200 comprises receiving respective audio data 201-1, ..., 201 -r from each of a plurality rof audio input devices 15     disposed about a vehicle. The r audio input devices may be the devices 320-1,..., 320-r of the system 300 shown in Figures 3A and 3B. Each audio data 201-n is indicative of audio recorded about the exterior of the vehicle by the respective audio input device. The method 200 may be performed on an ad hoc basis, however in some embodiments the method 200 maybe performed continuously or periodically. Thus, the audio data 201-1, 201-rmay each comprise a respective audio stream communicated continuously or at intervals from each audio input device. In this way, the surrounding environment of the vehicle can be continually monitored. 20 The method 200 then comprises a step 210 of determining, in dependence on each respective audio data, a respective spectrogram indicative of a frequency distribution of the audio data over a selected time window. The selected time window may be a predefined interval, such as a most recent interval of a predefined length. For example, the selected time window may define the most recent 0.5 to 5 seconds of audio data, such as the most recent 3 seconds, 2 seconds or 0.5 seconds. In this way, if the method 200 is performed periodically, the resultant environmental classification can 25 be continually updated based on the most recent audio received. An example of the step 210 according to one embodiment of the invention is illustrated in Figure 5. With reference to Figure 5, four audio input devices are implemented and so four pieces of audio data have been received, but it will be appreciated that the example may be extrapolated to other embodiments having different numbers of audio input devices. 30 A respective waveform 511,512, 513, 514 is obtained corresponding to the selected time window T of each piece of audio data. The waveform 511, 512, 513, 514 can be obtained by extracting the selected time window T from the audio data, for example by extracting the last 3 seconds of audio. In some embodiments, the selected time window T may correspond to the entirety of the audio data, for example if only audio data corresponding to the selected time window T is received. Thus, a separate step of extracting the respective waveforms 511, 512, 513, 514 may be omitted. 35 A respective spectrogram 521, 522, 523, 524 may then be determined in dependence on each waveform 511, 512, 513, 514. Any signal processing technique for converting audio from an amplitude to a frequency domain may be utilised to determine each spectrogram. For example, with reference the illustrated embodiment, a Fast Fourier Transform technique is employed. In other examples, alternative techniques such as Short-time Fourier Transform may be used. In this embodiment, each waveform is divided into p sliding time windows. Each of the p sliding time windows may have a 40 length of between 5ms and 100ms, for example 10ms, 20ms, 30ms or 40ms. The p sliding time windows may be arranged to be overlapping. For example, the windows may overlap by between 30% and 70% of the time window length, such as 40%, 50% or 60%. Overlapping the windows beneficially ensures spectral information is captured over the whole length of the selected time window T. Then, a Fast Fourier Transform having q frequency bins is applied to each of the p time windows. The result can be represented as a spectrogram having dimensions p and q, as illustrated in Figure 5. Returning to Figure 2, in step 220, each of the determined spectrograms are provided as input into a classification model. When respective audio data is obtained from r audio input devices as described with reference to Figure 5, each of the r spectrograms can be combined to obtain an input tensor I of dimensions p, q and r. For example, the r spectrograms may be combined by concatenating the spectrograms. This ensures distinct features within each spectrogram are spatially aligned in the input tensor I and the location is encoded in slight temporal misalignment, enabling accurate location identification of audio features. The input tensor / can therefore be provided as input to the classification model in step 220. The classification model is a model for classifying at least one environmental condition of a surrounding environment of the vehicle. In step 230, the classification model is configured to output one or more values indicative of a probability of the at least one environmental condition being present. The classification model is configured to map the spectrogram input to an output indicative of a classification probability for each of the at least one environmental condition. In some embodiments, the classification model is for single-class classification. In these embodiments, the model is constrained in that only one environmental condition can be present. This is the case, for example, for the classification of a road surface condition, as will be described in more detail with reference to Figure 6. For example, the classification model may be a model for classifying the road surface condition to be one of a discrete set of N road surface conditions 1= 1,..., N: i = 1 - a norma / surface condition; i = 2- a slippery surface condition; i=3 - an off-road surface condition; i=4 - a snow surface condition; i=5 - an ice surface condition; and i=6-a rain surface condition. In this case, the road surface can only have one of the N discrete types and so the classification may be bound by the condition that one and only one of the i conditions is present. In other embodiments, the classification model may be for multi-class classification. In these embodiments, each of the at least one environmental condition can be independently present. For example, the at least one environmental condition may comprise a set of i objects, or localized hazards, such as: i=1 - a construction area; i=2 - an emergency vehicle; 1=3 - a vehicle horn; 1=4 - a speeding vehicle; i=5- a braking vehicle; 1=6 - a pedestrian in a vehicle path; 1= 7 - an animal in a vehicle path; 1=8-a road accident; or i=9 - falling debris. An example embodiment using a multi-class classification model for localized hazard conditions will be described with reference to Figure 9. Any suitable classification model may be used depending on the set of environmental conditions, for example an artificial neural network (ANN). An ANN based classifier can be particularly beneficial as the ANN can be constructed to perform automatic extraction of audio features, as will be explained. Two example ANNs according to different embodiments will be described with reference to Figures 6 and 9. In other embodiments, audio features may be predefined for extraction from the audio data. The predefined audio features may include, for example, peak amplitude, Root Mean Square (RMS) value or power, however any audio feature may be extracted. The predefined audio features may then be provided as input to any traditional machine learning based classifier, such as a Support Vector Machine (SVM) or the like. In some examples, the classification model may comprise a plurality of classification models. For example, the classification model may comprise a plurality of one dimensional convolutional neural network (CNN) based models, each one dimensional CNN model being configured to extract features from the audio data using a different kernel length. The classification model may further comprise additional models, such as the ANN of Figures 6 or 9, or an SVM based model as discussed above. The output of each of the plurality of classification models may be advantageously combined in an analogous method to that described in Figures 10 to 12. Using multiple classification models and subsequently combining the results can provide a higher confidence than any individual model. In some embodiments, the classification model may additionally comprise a regression component for locating the at least one environmental condition. In such embodiments, the environmental condition(s) may be localized, and the classification model may be further configured to output an estimated location of the environmental condition with respect to the vehicle. Typically, the (or each) classification model is trained using any suitable supervised method to classify each of the environmental conditions using annotated training data, as will be described with reference to Figure 13. The annotated training data comprises audio data obtained under a set of conditions having each of the environmental conditions present. The amount of audio data required will depend on the characteristics of the environmental conditions. For example, highly distinctive environmental conditions (such as a car horn) may require less training data than environmental conditions having a greater range, or less distinctive characteristics (such as a pedestrian in the vicinity of the vehicle, for example). In some embodiments, at least 5 minutes, at least 10 minutes or at least 15 minutes of audio data for each environmental condition may be sufficient to train the classification model. For example at least 25 minutes, at least 35 minutes or at least 30 minutes may be sufficient. In other embodiments, the amount of training data required may differ to be less or more than these examples. I n general, the more training data that it is possible to obtain, the better the accuracy of the training. Within the audio data for each environmental condition, it is desirable to include a diverse set of conditions including at least the following: • Diversity in location: including audio data from driving on highways, urban areas, small towns, passing traffic lights, static, and transitioning between static and moving; • Diversity in background noise: including audio data in the presence of other vehicles, going over potholes, through a tunnel, through a city, past trains, having loud music within the cabin, diversity of road surface, diversity of weather; • Different vehicle occupancies: adult / child inside the vehicle speaking, singing, shouting, crying, whispering; • Diversity in location of environmental condition with respect to the vehicle (if environmental condition is localized). A method of using this training data to train the classification model according to one embodiment of the invention will be described in further detail with reference to Figure 13. Finally, the method 200 comprises outputting an environmental classification signal in step 240 indicative of the one or more values obtained from the classification model. In some embodiments, the environmental classification signal may correspond directly to the one or more values obtained from the model. For example, the environmental classification signal may be indicative of the likelihood of each of the environmental conditions being present. In other embodiments, a binary classification (yes / no) for each environmental condition may be determined in dependence on the one or more values, and the environmental classification signal may correspond to the outcome of the binary classification. If the classification model comprises a regression component as discussed, the environmental classification signal may further comprise an estimated location for each environmental condition classified as present. In some embodiments, the environmental classification signal may comprise a control signal to control one or more vehicle settings in dependence on the classification. Thus, the vehicle may be controlled appropriately depending on the environmental condition(s) classified. For example, if a localized hazard is classified and located with respect to the vehicle, the environmental classification signal may be output to a motion planning 9 system of the vehicle to control the motion planning system to update a vehicle trajectory to avoid the localized hazard. Because the hazard may be detected before it is visible by an image or radar monitoring system, this facilitates an improvement in motion planning in the face of unexpected hazards. In another example, the environmental classification signal may be for controlling an alert system of the vehicle to output an alert to the user regarding the environmental condition. In another example, the environmental classification signal may be output to a brake preparation system to prepare the vehicle to come to a stop. Turning now to Figure 4, there is illustrated a control system 310 for a vehicle. The control system may form part of the system 300 and comprises one or more controller 400. The control system is arranged to perform the method 200. According to some embodiments, at least one of the controllers 400 may be arranged to be installed in the vehicle. However in other embodiments, one or more of the controllers 400 may be remote to the vehicle and arranged to communicate with other vehicle systems via a remote connection, such as the Internet. The control system 310 is configured to receive the respective audio data 201-1, ..., 201-r from each of the r audio input devices 320-1,..., 320-r of the system 300 and determine the spectrograms 521,..., 524. The control system 310 is then configured to provide the spectrograms as input to the classification model and obtain from the classification model the one or more values indicative of a probability of the at least one environmental condition being present; and output the environmental classification signal. As discussed, the environmental classification signal may comprise at least one control signal to control one or more vehicle systems in dependence on the one or more values. The control system 310 as illustrated in Figure 4 comprises one controller 400, although it will be appreciated that this is merely illustrative. The controller 400 comprises processing means 410 and memory means 420. The processing means 410 may be one or more electronic processing device 410 which operably executes computer-readable instructions. The memory means 420 may be one or more memory device 420. The memory means 420 is electrically coupled to the processing means 410. The memory means 420 is configured to store instructions, and the processing means 410 is configured to access the memory means 130 and execute the instructions stored thereon. The controller 400 comprises an input means 430 and an output means 440. The input means 430 may comprise an electrical input 430 of the controller 400. The output means 440 may comprise an electrical output 440 of the controller 400. The input 430 is arranged to receive a respective audio signal 435 from each of the plurality r of audio input devices 320-1,..., 320-r. The audio signal 435 is an electrical signal which is indicative of the respective audio data 201 -1,..., 201 -r. The output 150 is arranged to output the environmental classification signal 445. The method 200 may be performed by the control system 400 illustrated in Figure 4. In particular, the memory 420 may comprise computer-readable instructions which, when executed by the processor 410, perform the method 200 according to an embodiment of the invention. According to some embodiments, the memory 420 may be arranged to store the classification model. However, according to other embodiments, the classification model may be stored remotely to the control system 310, for example on a remote server. In such embodiments, the control system 310 is configured to communicate the spectrograms to the remote location of the classification model, and obtain the one or more values from the remote location of the classification model. Turning now to Figures 6 to 9, according to some embodiments of the invention, the classification model comprises an artificial neural network (ANN). Figure 6 illustrates a first classification model 600 according to one embodiment in the form of an ANN 600, and Figure 9 illustrates a second classification model 900 according to another embodiment in the form of an ANN 900. Each of the ANN 600 and ANN 900 are artificial neural networks for classifying the at least one environmental condition. They each comprise an input layer having a plurality of input nodes for receiving the input tensor I, at least one output layer comprising output nodes for outputting the one or more values, and one or more hidden layers having connecting nodes connecting the input layer to the output layer(s), the connections of the connecting nodes having weights. The weights are determined during the supervised training of the ANN 600, 900, as will be explained with reference to Figure 13. Road surface condition classification With reference to Figure 6, the ANN 600 is a classification model for classifying a road surface condition of the vehicle as one of a discrete set of N road surface conditions i = 1,..., N. Thus, the ANN 600 is a single-class classification model. The ANN 600 comprises a number of bespoke features which tailor the ANN 600 to the problem of classifying road surface conditions, and as a result the ANN 600 provides an improved accuracy of classification in comparison to traditional classification techniques such as an off-the-shelf ANN. In one illustrative example, the discrete set of road surface conditions comprise a normal surface condition (i=1); a slippery surface condition (i=2); an off-road surface condition (i=3); a snow surface condition (i=4); an ice surface condition (i=5); and a rain surface condition (i=6). In other embodiments, the discrete set of road surface conditions may comprise different conditions, including more or fewer conditions. Globally, the ANN 600 comprises an input layer 605 having input nodes for receiving the input tensor I, and a single output layer 650. The output layer 650 comprises output nodes for outputting an output vector including one or more values Ri, wherein each of the one or more values Ri is indicative of a probability of the road surface condition being of type i. Thus, in the above example, the output vector comprises six values Ri, 1=1,..., N as N=6, and so the output layer comprises at least six nodes. The ANN 600 comprises one or more hidden layers having connecting nodes connecting the input layer 605 to the output layers 650, the connections of the connecting nodes having weights. The values of the input tensor I are provided as input to the input nodes of the input layer 605. When the spectrograms are determined as described with reference to Figure 5, the input tensor I has dimensions [r, q, p] wherein r is the number of audio input devices, q the number of frequency bins used during the FFT and p the number of sliding windows into which the selected time window is divided. According to an illustrative example with 4 audio input devices, 512 frequency bins and a 3s selected time window divided into 20ms sliding windows having 10ms overlap, the input tensor has dimensions [4,512,300], The ANN 600 comprises a backbone sub-network 610 having connecting nodes connecting the input layer 605 to a feature map layer 615. The purpose of the backbone sub-network is to extract at least one audio feature map from the spectrogram input. Because each audio signal is converted into a spectrogram, any neural network architecture suitable for computer vision can be employed as the backbone sub-network 610, such as ResNet (Deep Residual Learning for Image Recognition, Heetal, 10 Dec 2015, https: / / arxiv.org / pdf / 1512.03385.pdf), MobileNet or EfficientNet. Typical inputs to such architectures are images of size [channels x height x width], comparable to the input tensor [N,512,300], As the amount of data input to the ANN 600 is relatively small compared to typical computer vision applications, a relatively shallow architecture can be employed for the backbone subnetwork 610, for example a ResNet-34 having 34 layers. The audio feature map 615 is typically downsampled in comparison to the input tensor, and may have dimensions [s, q / d, p / d] wherein s is the number of feature map channels, and d is the level of downsampling. For example, in the illustrative example having an input tensor of dimensions [4,512,300], the feature map may have dimensions [64, 16, 9] and the feature map layer 615 thus comprises 64 x 16x9 nodes. This corresponds to a downsampling level d of 16, and 64 channels. However, in other embodiments the values of s and d may be adjusted. The values d = 16 and s = 64 enable minimal modifications to be made to a typical computer vision backbone architecture such as ResNet-34. These minimal modifications comprise removing the convolution and pooling layers from the backbone architecture and substituting a 1x1 convolution to obtain the 64 channels. These modifications enable application specific pooling to be applied in the ANN 600, as will be explained. The ANN 600 comprises two pooling sub-networks 620, 630. The first pooling sub-network is a mean pooling sub-network 620 which acts to pool the frequency dimension q / d of the feature map. With reference to Figure 7, the mean pooling sub-network 620 connects the feature map layer 615 to a mean pooled layer 710. The mean pooling sub-network acts to pool the feature map output over the frequency dimension q / d such that the output at the mean pooled layer 710 is a mean pooled tensor having dimensions [s, p / d]. Any kind of averaging can be applied over the frequency dimension, for example a mean may be implemented as in the illustrated example. The second pooling sub-network 630 provides attention pooling over the time dimension. With reference to Figure 8, the attention pooling sub-network 630 takes as input the values of the mean pooled tensor. The attention pooling sub-network 630 comprises at least one fully connected layer 810 which maps the mean pooled layer 710 to an attention score vector 820 having length p / d. Thus, a separate fully connected layer 810 is utilised to determine an attention score over the time dimension p / d. Such attention pooling has been found to be beneficial as a separate fully connected layer computes the relevance of the feature at a given time. As depicted in the illustrative example of Figure 8, the attention score vector 820 emphasizes the last two samples with a score of 0.4 and 0.5 as a highly relevant time period. A matrix product 830 is then applied to pool the mean pooled tensor over the time axis according to the attention score vector 820. The matrix product 830 defines a product of the mean pooled tensor [s, p / d] with the attention score vector [p / d, 1] to obtain a road surface specific audio feature vector fa having length s. In this way, a road surface specific audio feature vector fa is obtained defining audio features likely to be relevant to the classification of road surface. Returning to Figure 6, the ANN 600 comprises a head sub-network 640. The audio feature vector fa is provided as input to input nodes of the head sub-network 640. The head sub-network 640 thus comprises at least s input nodes, one for each channel of the road surface specific audio feature vector. The head sub-network comprises connecting nodes between the input nodes of the head sub-network 640 and output nodes of the output layer 650. The output layer comprises at least N output nodes, one for providing each of the one or more values Ri, i = 1,..., N indicating a likelihood of the road surface condition being of type i. Thus, the head sub-network 640 is trained to classify the road surface condition based on the extracted audio feature vector fa. In this way, the input to the head sub-network is tailored to include audio features determined as likely to relate to the type road surface, facilitating an improved classification output. Localized hazard classification With reference to Figure 9, the ANN 900 is a classification model for classifying localized hazards. The ANN 900 may be used as the classification model of method 200. The at least one environmental condition described with reference to Figure 2 comprises a set of N classes i of localized hazard i = 1, ..., N in the illustrated embodiment. The ANN 900 is a multi-class classification model adapted to classify whether each class i of localized hazard is present in a vicinity of the vehicle. The ANN 900 further comprises a regression component for predicting a location of each class of hazard relative to the vehicle, when the class is determined to be present. The ANN 900 comprises a number of bespoke features which tailor the ANN 900 to the problem of classifying, and localizing surrounding hazards, and as a result the ANN 900 provides an improved accuracy of classification in comparison to traditional classification techniques such as an off-the-shelf ANN. A localized hazard may be defined as any object in a vicinity of the vehicle which may be associated with a specific location with respect to the vehicle, for example a set of polar or cartesian co-ordinates with respect to the vehicle. The term “hazard” may be applied to any such object which is not expected to be present under ordinary circumstances, and which may necessitate an urgent vehicle control response. A hazard may also be referred to as a high alert object, or rare event, and may be detected relatively infrequently during normal operation of the vehicle. Typically, the set of localized hazards selected for use with the method 200 comprise objects having readily audible characteristics. The set of N classes of localized hazard comprise, in an illustrative example, nine classes including: a construction area (i=1); an emergency vehicle (i=2); a vehicle horn (i=3); a speeding vehicle (i=4); a braking vehicle (i=5); a pedestrian in a vehicle path (i=6); an animal in a vehicle path (i=7); a road accident (i=8); or falling debris (i=9). In other embodiments, more or fewer classes of localized hazard may be included. Globally, as with the ANN 600, the ANN 900 comprises an input layer 905 having input nodes for receiving the input tensor / . However, the ANN 900 comprises a plurality of N output layers 950 including a respective output layer 95 / corresponding to each of the / classes of localized hazard i=1,..., N. Each output layer 95 / comprises output nodes for outputting at least a first value Ci indicative of a probability of the class i of localized hazard being present in the vicinity of the vehicle, and a second value (or set of values) Pi indicative of a predicted position of the class i. The ANN 900 is identical to the ANN 600 with respect to the input layer 905, backbone sub-network 910 and the mean pooling sub-network 920. A detailed repetition of these features will be omitted, and reference is made to the description of Figure 6. For the ANN 900 the second pooling sub-network 930 still provides attention pooling over the time dimension, but is applied separately for each of the N classes of localized hazard. Thus, in contrast to the ANN 600, at this point the ANN 900 splits out into separate branches for each of the N classes, including a respective attention pooling sub-network 93 / for each discrete location i. Each attention pooling sub-network 93 / functions independently as illustrated with respect to the sub-network 630 of Figure 8, taking as input the values of the mean pooled tensor. As the attention pooling sub-network 93 / is applied separately for each class / , the ANN 900 can focus on different time spans within the audio data for different classes of localized hazard i. For example, a vehicle horn may be audible at a different time to an emergency vehicle. The attention pooling sub-network 93 / thus acts to identify time points in the audio data which appear to relate to the respective class i. The product of the attention pooling sub-network is thus a different class specific audio feature vector faj for each of the N classes, defining audio features likely to be relevant to the specific class i. The ANN 900 comprises a head portion 940 comprising a respective head sub-network 94 / for each class i. The class specific audio feature vector fa,i determined for the class i is provided as input to input nodes of the head sub-network 94 / . The head sub-network 94 / thus comprises at least s input nodes, one for each channel of the class specific audio feature vector. The head sub-network comprises connecting nodes between the input nodes of the head sub-network 94 / and output nodes of the output layer 95 / corresponding to the class i in the set of localized hazards. Each output layer 95 / comprises at least 2 output nodes, one for providing the first value Ci and another for providing the second value Pi. The second value Pi may comprise a plurality of values to characterize the location of the hazard. The first value Ci can be used in binary classification (yes / no) to characterise whether the hazard i is present, e.g. if the first value Ci is above a threshold. To localize the hazard i in three-dimensional space with respect to the vehicle, the second value Pi may comprise a set of three-dimensional co-ordinates, such as (x, y, z) co-ordinates or angular co-ordinates (e.g. radius, azimuth, elevation). An angular co-ordinate system centered on the vehicle can be more beneficial than a cartesian system, as the values better fit the nature of the problem and thus can be more accurately trained. Thus, each head sub-network 94 / comprises both a classification component (to obtain the first value Ci) and a regression component (to estimate a location of each class of hazard i determined to be present). Thus, each head sub-network 94 / is trained to classify and locate the presence of each class of hazard i based on the class specific audio feature vector faj. Overall, it can be seen that the ANN 900 comprises a multi-headed structure, wherein each head sub-network 94 / is tailored to identifying sounds extracted for a particular class from common audio input. This facilitates efficient multi-class classification. Returning to Figure 2, the environmental classification signal output in step 240 may include both a binary prediction (yes / no) for the presence of each class i of hazard, and a predicted position for each class which has been classified as present. The environmental classification signal may comprise a control signal for controlling one or more vehicle functions depending on the classified hazards. For example, the control signal may be for controlling or updating a vehicle trajectory to avoid one or more of the classified hazards. Multi-modal fusion In some circumstances, the use of audio data alone may not accurately classify an environmental condition of the vehicle at all times. For example, some environmental conditions may not be accurately identifiable through audio, if the condition is silent or not easily distinguished from background noise. According to some embodiments of the invention, the environmental monitoring may be improved by fusing the method 200 with output from an external environmental monitoring method using a different modality, such as from a visual or radar monitoring system of the vehicle, or both. Including multi-modality input to a single classification model can be problematic due to the tendency for one or more modalities to dominate the classification. Therefore, according to the present invention, each modality provides an independent classification, which may then subsequently be fused according to Figures 10 to 12. Multi-modality fusion according to embodiments of the present invention advantageously provides a higher classification confidence for the overall environmental condition than any individual modality, which is ensured by implementing single modality classification trained on large single modality data sets, and fusing the result as described. Figure 10 illustrates a flow chart of a method 1000 of multi-modality fusion according to an embodiment of the invention. The method 1000 may be performed in conjunction with the method 200. The method comprises receiving audio data 1001 comprising the one or more values determined from the classification model of the method 200. As discussed, the one or more values are each indicative of the probability of each of the one or more environmental conditions being present according to the audio environmental condition monitoring system 300. These one or more values may be referred to as audio values 0 / where i is the environmental condition. The method 1000 further comprises receiving, from a second monitoring system of the vehicle, second classification data 1002. The second monitoring system is configured to classify the probability of each of the one or more environmental conditions being present using a non-audio modality. In some embodiments as will be described, the second monitoring system is a visual monitoring system and the second classification data 1002 is visual data 1002. However, it will be appreciated that in other embodiments the second monitoring system may use a non-visual modality, such as radar, lidar or ultrasonic (USS). The visual monitoring system (not illustrated) comprises a plurality of image sensors configured to capture images of the surrounding environment of the vehicle and a control system, which may be the same control system 400 as used in the audio monitoring system 300, or may be a separate control system. The visual monitoring system is configured to output one or more values Vi indicative of a probability of the environmental condition i being present in dependence on the images captured by the image sensors. The visual monitoring system may employ a visual classification model, such as an ANN trained using images captured by the image sensors or equivalent image sensors. The visual data 1002 received comprises at least the one or more values Vi, which may be referred to as visual values. The method 1000 comprises a step 1010 of determining, in dependence on the audio values Oi and the visual values Vi a fused classification probability of each environmental condition being present. That is, step 1010 comprises determining a fused classification probability in dependence on the probabilities determined independently by the audio environmental condition monitoring system 300 and the visual monitoring system. In some embodiments, the fused classification probability may be determined as a weighted sum of the audio value and visual value for each environmental condition. Advantageously, utilising both audio and visual monitoring results in improved confidence compared to either modality individually. Further, decoupling the modalities leads to a higher classification confidence as the predictions are independent. Although Figure 10 illustrates only two modalities, the concept may be extended to fuse predictions from three or more modalities, for example from the audio environmental condition monitoring system, a visual monitoring system and a radar monitoring system. Respective predictions are obtained from each system, and weighted as described. The method 1000 may be performed as part of the method 200, as has been explained. Thus, the environmental classification signal of step 240 may be output in dependence on each fused classification probability determined in step 910. In this way the environmental classification signal will reflect an overall probability of each environmental condition being present, based on the fusion of the audio monitoring with a second modality, such as visual monitoring. How the values are weighted in step 1010 may be approached in various ways. In a naive system, the weights may be fixed based on a pre-defined ratio, such as a weighting of 0.7 for the visual values and a weighting of 0.3 for the audio values. These fixed weights may be defined based on relative average accuracy of each monitoring system. In other embodiments, a more sophisticated method may be employed to define the weights in the method 1000, as illustrated in Figures 11 and 12. Figure 11 illustrates a fusion method 1100 for use with the ANN 600, i.e, for road surface classification. In this embodiment, the audio values Oi correspond to the values Ri and are indicative of a probability of the road surface condition being of type i. Likewise, the visual values Vi are indicative of a probability of the road surface condition being of type i according to the visual monitoring system. In this embodiment, the received audio data 1001 comprises the one or more values Ri as well as the road surface specific audio feature fa. The received visual data 1002 comprises the further one or more values Vi as well as a road surface specific visual feature fv associated with road surface and extracted from a classification model used by the visual monitoring system. According to the illustrated embodiment of Figure 11, a fusion neural network 1110 may be implemented to predict the weights. In this way, the confidence of each modality can be predicted in real time depending on the specific output of each of the audio environmental condition monitoring system 300 and the visual monitoring system. The fusion neural network 1110 is trained to predict a confidence for each of the audio data 1001 and the visual data 1002 for classifying the road surface condition. The fusion neural network 1110 comprises an input layer having input nodes for receiving the values Ri, i = i n, the road surface specific audio feature fa, the values Mi, i= i,..., n and the road surface specific visual feature fv. The fusion neural network 1110 comprises an output layer having output nodes arranged to output one or more weight values for weighting the visual values V / and the audio values R / , At least one respective weight value W is output. In some embodiments, an audio weight value Wa to be applied to the audio values Ri and a visual weight value Wv to be applied to the visual values Vi are each output at the output nodes. However, in other embodiments a single weight value, for example Wa is output, and the weight value to be applied to the other modality can be inferred through a constraint, e.g. with the constraint that Wa + Wv = 1. Then, in step 1120 a fused classification probability Li of the road surface condition corresponding to each of the conditions i is determined in dependence on the one or more values Ri and the further one or more values Vi. The overall probability Li may be calculated as the weighted sum WaRi+WvVi = Li. The fusion neural network 1110 comprises one or more hidden layers having connecting nodes connecting the input layer to the output layer. The connections of the connecting nodes have weights which are adjusted during training of the fusion neural network. The fusion neural network 1110 has been trained to predict the confidence of each modality, i.e. the relative weights to be applied, using a set of cross-modality training data. The set of cross-modality training data comprises audio data from a plurality of audio input devices disposed about a training vehicle and visual data from one or more imaging devices disposed about the training vehicle, wherein the training vehicle is arranged equivalently to the vehicle 100 having both an audio environmental condition monitoring system 300 and a visual monitoring system. The cross-modality training data is captured under a set of conditions having each road surface condition i. By decoupling the audio and visual modalities initially and then providing a separate fusion network 1110, this allows the training of the original classification model (such as the ANN 600 and equivalent classification model for the visual system) separately on large single modality datasets. Only a relatively small cross-modality training set is required to train the fusion network 1110. Figure 12 illustrates a fusion method 1200 for use with the ANN 900, i.e., for localized hazard classification. The fusion method 1200 differs from the fusion method 1100 as for localized hazards, both classification probabilities and estimated locations are fused. The fusion of both probabilities and locations together requires a different, end-to-end approach for accurate fusion. In method 1200, audio data 1201 and visual data 1202 are received, as for method 1000. The audio data 1201 comprises each class specific audio feature fa,i, the associated first value Ci indicative of the probability of the class i being present and the second value(s) Pi indicative of the predicted position of the class / with respect to the vehicle. The visual data 1202 comprises a corresponding set of class specific visual features fv,i, an associated first visual value Ci,v indicative of a probability of the class / of localized hazard being present according to the visual monitoring system, and second visual value(s) Pi,v indicative of the predicted position of the class i of localized hazard according to the visual monitoring system. The fusion method 1200 comprises fusing the class specific audio features and visual features in a common spatial model 1210. For example, the common spatial model 1210 may be as described in BEVFusion: Multi-Sensor Fusion with Unified Bird’s Eye View Representation, Liu et al, 16 June 2022 https: / / arxiv.org / Ddf / 2205.13542.pdf. Each class specific audio feature fa,i is weighted by the associated first value Ci at its predicted position Pi in a Bird’s Eye View (BEV) grid of size sa-X-Y-Z, wherein sa is the maximum length of each class specific audio feature. For example, in the illustrated embodiment the BEV grid is of size 64-X-Y-Z. The grid is flattened along the Z dimension. Each class specific visual feature fv,i is weighed by the associated first visual value Ci,v and placed at its predicted position Pi,v in a BEV grid of size Sv-X-Y-Z, wherein Sv is the maximum length of each class specific visual feature. The grid is flattened along the Z dimension. The spatial model 1210 is constructed by concatenating the visual features with the audio features in each flattened BEV grid along the channel (s) dimension. The spatial model 1210 defines a joint audio-visual feature map of size (sa+sv)-X-Y. In step 1220, the spatial model 1210 is provided as input to an input layer of a fusion neural network for object localization and classification. The fusion neural network comprises an input layer having input nodes for receiving the (sa+sv)-X-Y spatial model, an output layer having output nodes for outputting a fusion output vector, wherein at least a subset of the values of the output vector are indicative of a class and a location of one or more identified localized hazards, and one or more hidden layers having connecting nodes connecting the input layer to the at least one output layer. The fusion neural network has been trained to classify and locate hazards using a set of training data. The set of training data comprises both audio data from a plurality of audio input devices disposed about a training vehicle and visual data from one or more imaging devices for imaging a field of view of the vehicle, the set of training data taken under a set of conditions having each class of localized hazard at a range of positions relative to the vehicle. Notably, the set of cross-modality training data required to accurately train the fusion neural network is relatively small compared to the initial single modality training data. This is because the spatial model input to the fusion neural network already includes extracted audio and visual features, locations and confidences as predicted by the single modality systems. The fusion neural network may comprise a first convolutional sub-network in the form of a Birds-Eye-View (BEV) encoder which processes the BEV features from all modalities and outputs a cross-modality feature map defining a fused collection of audio-visual features. The fusion neural network may further comprise a 3D object detection head sub-network which maps the cross-modality feature map to the values of the output vector indicative of a location of one or more identified localized hazards. An example head sub-network which may be implemented is CenterHead (Objects as Points, Zhou et al, 25 April 2019, https: / / arxiv.or q / pdf / i 904.07850.pdf’), although other architectures may be used. In step 1230, the input values from the spatial model are mapped through the connecting nodes of the fusion neural network to obtain the fusion output vector at the output layer. A position and class of one or more localized hazards extracted from the cross-modality feature map are then determined from the fusion output vector. In step 1240, the environmental classification signal is output in dependence on the class and location of each localized hazard determined in step 1230. Advantageously, fusing the features extracted from both audio and visual classification for input to a fusion network results in improved confidence compared to either modality individually. Further, decoupling the modalities leads to a higher classification confidence as the predictions are independent. Training of classification model With reference to Figure 13, there is illustrated a supervised method 1300 of training the classification model of the method 200. For example, the method 1300 may be used to train the ANN 600 or the ANN 900, however it will be appreciated that the training method may be adapted in any embodiment where an alternative classification model is used. The method comprises receiving a set of training data 1310. The training data 1310 comprises a set of input output pairs, each pair comprising audio data 1311 and corresponding classification data 1312 indicative of a classification of whether each of the at least one environmental condition i is present for the associated audio data. Optionally, when training a classification model with a regression component such as the ANN 900, the classification data 1312 also comprises a location of each present environmental condition i. The location component of the classification data 1312 may be captured using a modality such as radar or lidar for localization of the hazard. The audio data 1311 is captured from a plurality of audio input devices disposed about a training vehicle. In some embodiments, the training vehicle may be the same vehicle in which the environmental condition monitoring system 300 is installed during use. In other embodiments, the training vehicle may be a separate vehicle set up in the same way, with the same arrangement of audio input devices. In this way, the classification model may be centrally trained in a single training vehicle and deployed on a fleet of vehicles having the same arrangement of audio input devices. Each input-output pair may correspond to a different permutation of environmental conditions i being present. Overall the training data should include input-output pairs corresponding to every possible permutation of environmental conditions i. Further requirements for the training data, such as diversity of background noise, should be applied within each permutation and are as described earlier with reference to Figure 2. Optionally, when training a classification model with a regression component such as the ANN 900, the training data should include input-output pairs corresponding to every permutation of environmental conditions i at a diverse range of locations with respect to the vehicle. In step 1320, one input-output pair is selected from the set of training data 1300. An input tensor / for the classification model is determined from the audio data 1311 of the input-output pair. The input tensor I may be determined as described with reference to steps 210, 220 and Figure 5. That is, a respective spectrogram is determined indicative of a frequency distribution of the audio data 1311 from each of the audio input devices over a selected time window, and the input tensor I is determined in dependence on the determined spectrograms, e.g. by concatenating the spectrograms. The input tensor I should be determined the same way in method 1300 as in method 200. In step 1330, the input tensor I is input into the classification model, as described with reference to step 220. The classification model trained in method 1300 is the same classification model as used, once trained, in the method 200. In some embodiments, the classification model is the ANN 600 or the ANN 900 and so reference will be made to the ANN 600 and ANN 900 in the context of the method 1300. For the ANN 600, step 1330 comprises providing the input tensor I as input to the input nodes of the input layer 610. For the ANN 900, step 1330 comprises providing the input tensor I as input to the input nodes of the input layer 910. In step 1340, a predicted set of values Pi are obtained from the classification model. The predicted set of values Pi are indicative of a predicted probability of the environmental condition i being present for the audio data 1311. Step 1340 may be equivalent to obtaining the one or more values in step 230. With reference to the ANN 600, the predicted set of values Pi are obtained by mapping the input tensor I through the connecting nodes of the ANN 600 to the output layer 650, such that the values Pi are obtained from output nodes of the output layer 650 analogously to the values Ri. With reference to the ANN 600, the predicted set of values Pi are obtained by mapping the input tensor I through the connecting nodes of the ANN 900 to each output layer 95 / , such that the values Pi are obtained from output nodes of each output layer 95 / . The predicted set of values Pi therefore comprise a predicted set of first values PCi indicative of the probability of the class i of localized hazard being present and a predicted set of second values PPi indicative of a predicted position of the class i of localized hazard with respect to the vehicle. In step 1350, at least one objective function is determined to characterise an error between the values of the predicted set of values Pi compared to the corresponding classification data 1312 of the input-output pair. The objective function may be selected differently dependent on the type of 17 classification model trained. For example, for a single-class classification, such as the ANN 600, the objective function may be a Softmax cross entropy loss function or similar. For a multi-class classification, as in the values Ci of the ANN 900, the objective function may be any suitable loss function for multi-class classification, such as Binary Cross Entropy applied to each value Ci. For a regression, as in the values Pi of the ANN 900, the objective function may be a Mean Squared Error (MSE) or Mean Average Error (MAE). In some examples, as for the ANN 900, the classification model may implement both regression and classification. Therefore, in step 1350 a plurality of objective functions may be determined, one for each type of prediction. In step 1350, an appropriate optimisation algorithm is used to update the classification model to seek to minimise the objective function(s). When the classification model is the ANN 600 or ANN 900, updating the ANN 600 comprises updating the weights between connecting nodes of the ANN 600. Steps 1320 to 1350 are repeated for each of the input-output pairs, and in step 1350 the optimisation algorithm updates the classification model in light of the objective function(s) calculated for each of the input-output pairs. Thus, the model is iteratively trained. The more training data is utilised, the further the objective function may be minimised, and so ideally as many input-output pairs as possible should be collected. As discussed with reference to Figure 2, the amount of audio data required to train the classification model will vary depending on the characteristics of the environmental conditions involved. In an illustrated example for the road surface classification or localized hazard classification described, around 30 minutes for each permutation of environmental conditions may facilitate the generation of a sufficient number of input-output pairs to train the classification model, however this may vary depending on the characteristics of the environmental conditions. For example, for the ANN 600, the training data may comprise at least 10 minutes, 30 minutes, 20 minutes or 40 minutes of audio data for each of the N road surface conditions. Within each road surface condition, a diversity of background conditions should be present as discussed with reference to Figure 2. For the ANN 900, the training data may comprise at least 30, 40, 20 or 5 minutes of training data for each class of localized hazard i. Within each class, there should be a diversity of location of the hazard with respect to the vehicle and a diversity of background conditions. Furthermore, the training data should include audio data having the localized hazards present in combination for each different permutation. In summary, the present invention provides a novel and high accuracy method for classifying an environmental condition of a vehicle using audio data captured about the vehicle. Furthermore, the present invention provides a new method for fusing the audio classification with other modalities, such as visual or radar based classification, to further improve the confidence of the classification beyond any individual modality. It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.

Claims

1. A computer-implemented method for monitoring an environmental condition of a vehicle, the method comprising:receiving, from each of a plurality of audio input devices disposed about the vehicle, respective audio data;for each respective audio data, determining, in dependence on the audio data, a spectrogram indicative of a frequency distribution of the audio data over a selected time window;inputting each determined spectrogram into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle;obtaining, from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; andoutputting an environmental classification signal indicative of the one or more values.

2. The method of claim 1, wherein determining each spectrogram comprises:dividing the audio data into p sliding time windows,applying a Fast Fourier Transform with q frequency bins to each of the p time windows to determine a spectrogram having dimensions p and q and, optionally,receiving respective audio data from r audio input devices, combining each spectrogram to obtain an input tensor I of dimensions p, q and r, and inputting the input tensor I into the classification model.

3. The method of claim 1 or 2, wherein:the classification model comprises an artificial neural network for classifying at least one environmental condition of a surrounding environment of the vehicle, and the method comprises:determining an input tensor for the artificial neural network in dependence on each of the determined spectrograms;inputting the input tensor to input nodes of an input layer of the neural network;mapping the input tensor through connecting nodes of the neural network to output nodes of an output layer to obtain each of the one or more values indicative of at least one environmental condition being present.

4. The method of any preceding claim, whereineach environmental condition is a road surface condition, andthe classification model is for classifying the road surface condition of the vehicle as one of a discrete set of N road surface conditions i = 1,..., N, and, optionally,at least a subset of one or more values Ri obtained from the classification model are indicative of a probability of the road surface condition being of type i.

5. The method of claim 4, wherein the method comprises:receiving, from a visual classification system of the vehicle, visual data indicative of a further one or more values Vi, V2,...., Vn, wherein at least a subset of the further values Vi are indicative of a probability of the road surface condition being of type i according to the visual classification system; anddetermining, in dependence on the one or more values Ri and the further one or more values Vi, an overall classification of the road surface condition.

6. The method of claim 5, comprising:determining, from the classification model, at least one audio feature fa;receiving, from the visual classification system of the vehicle, at least one visual feature fv; inputting, to input nodes of an input layer of a fusion neural network for predicting a confidence of each of the one or more values Ri and the further one or more values Vi, the one or more values Ri, the at least one audio feature fa, the further one or more values Vi and the at least one visual feature fv;19mapping the input through connecting nodes of the fusion neural network to output nodes of an output layer to obtain one or more weight values Wi; anddetermining a fused classification probability for each of the road surface conditions i as a combination of the values Ri and Vi weighted according to the one or more weight values Wi; and             outputting the environmental classification signal in dependence on the fusedclassification probabilities.

7. The method of any of claims 1 to 3, whereinthe at least one environmental condition comprises a set of N classes i of localized hazard i = 1, ..., N, and wherein the classification model is for classifying whether each class i of localized hazard is present in a vicinity of the vehicle; and, optionally,the artificial neural network comprises a set of N output layers, each output layer corresponding to a class i of localized hazard in the set, each output layer having output nodes for outputting output data comprising at least a first value Ci indicative of a probability of the class i of localized hazard being present in the vicinity of the vehicle and a second value Pi indicative of a predicted position of the class i of localized hazard with respect to the vehicle, the method comprising mapping the input tensor I through the connecting nodes to the output nodes of each output layer to obtain each of the first value Ci and the second value Pi.

8. The method of claim 7, comprising:determining a prediction for whether each class i of localized hazard is present in dependence on each first value Ci; andin dependence on the determined prediction for each class i, outputting the environmental classification signal to be indicative of the predicted position of the class i of localized hazard; and, optionallyoutputting a control signal to control a vehicle trajectory in dependence on the determined prediction.

9. The method of claim 7 or 8, wherein the artificial neural network comprises:a backbone sub-network having connecting nodes connecting the input layer to a feature map layer, the connections of the connecting nodes having weights; anda respective head sub-network for each class i of localized hazard in the set, i = 1,..., N, and wherein each respective head sub-network comprises connecting nodes between the feature map layer and the output layer corresponding to the class i of localized hazard in the set, the connections of the connecting nodes having weights.

10. The method of claim 9, wherein the artificial neural network further comprises a respective attention pooling sub-network for each class / ’ of localized hazard in the set, 1= 1,..., N, wherein each respective attention pooling sub-network comprises connecting nodes connecting the feature map layer to an input layer of the respective head sub-network, wherein the input layer of each respective head sub-network is indicative of a class specific audio feature fa,i.

11. The method of any of claims 7 to 10, further comprising:receiving, from a visual classification system of the vehicle, visual data comprising, for each class / , a first visual value Ci,v indicative of a probability of the class i of localized hazard being present in the vicinity of the vehicle and a second visual value Pi,v indicative of a predicted position of the class i of localized hazard with respect to the vehicle; anddetermining, in dependence on the N output data and the visual data, a predicted class and location for one or more identified localized hazards; and, optionally:receiving, from the visual classification system of the vehicle, at least one class specific visual feature fvj associated with each of the N classes of localized hazard / = 1,..., N;weighting each class specific visual feature fv,i by the associated first visual value Ci,v at its predicted position Pi,v in a spatial model;weighting each class specific audio feature fs,i by the associated first value Ci at its predicted position Pi in the spatial model;inputting, to an input layer of a fusion neural network for object localization and classification, the spatial model;mapping the input through connecting nodes of the fusion neural network to output nodes of an output layer to obtain a fusion output;determining the class and the location of the one or more identified localized hazards according to the fusion output; andoutputting the environmental classification signal in dependence on the determined class and location.

12. A control system for controlling an environmental condition monitoring system for a vehicle, the control system comprising one or more controller, the control system configured to:receive, from each of a plurality of audio input devices disposed about the vehicle, respective audio data;for each respective audio data, determine, in dependence on the audio data, a spectrogram indicative of a frequency distribution of the waveform over a selected time window;input each determined spectrogram into a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle;obtain, from the classification model, one or more values indicative of a probability of the at least one environmental condition being present; andoutput an environmental classification signal indicative of the one or more values13. A vehicle comprising:the control system of claim 12 oran environmental condition monitoring system comprising the control system of claim 12 and a plurality of audio input devices arranged to be disposed about the vehicle, wherein the plurality of audio input devices are arranged to communicate the respective audio data to the control system.

14. A computer-implemented method of training a classification model for classifying at least one environmental condition of a surrounding environment of the vehicle, wherein the method comprises:receiving a set of training data to train the classification model, the set of training data comprising audio data from a plurality of audio input devices disposed about a training vehicle under a set of conditions having each environmental condition present, the set of training data comprising input-output pairs, each pair comprising audio data from each of the audio input devices and corresponding classification data indicative of a classification of whether each of the at least one environmental condition i is present for the associated audio data;for each input-output pair:determining, in dependence on each audio data from each of the audio input devices, a respective spectrogram indicative of a frequency distribution of the audio data over a selected time window;determining an input tensor I for the input-output pair in dependence on each of the determined spectrograms;inputting the input tensor I into the classification model;obtaining, from the classification model, a predicted set of values Pi indicative of a predicted probability of each of the at least one environmental condition i being present; anddetermining an objective function characterising an error between the values of the predicted set of values Pj compared to the corresponding classification data for each of the one or more input-output pairs;using an appropriate optimisation algorithm operating on the objective function, updating the classification model to seek to minimise the objective function.

15. Computer readable instructions which, when executed by a computer, are arranged to perform a method according to claims 1 to 11 or 14.

Images