Adaptive facial recognition for drivers

Abstract

An adaptive facial recognition system for a vehicle generates a target set of images of drivers in a vehicle 21. The system holds image reference records 30 for each driver. A primary neural network 2

Description

The present invention relates to an adaptive facial recognition system for drivers and further to an adaptive facial recognition method for drivers using the system. The invention has been developed with a particular emphasis on use by fleet operators and is described predominately herein in relation to such use. It should be appreciated however that the invention is not limited in this way. Fleet operators controlling large groups of vehicles are increasing turning to facial recognition systems to enable identification of their drivers. Facilitating the association of a driver with a particular vehicle and the ability to assign a particular driver to a known journey can be beneficial for many reasons, including, but not limited to, meeting fleet compliance, reducing operational risks and dealing with claims following incident occurrence. Facial recognition systems utilising computer vision are typically trained with appropriately annotated data in order to try to predict the identity of the driver. Prior art systems tend to suffer from several drawbacks. Firstly, the systems are typically trained with data derived from a specific camera system. Variations in the type of camera, camera lens, image quality, frame rate, bit rate, resolution and camera installation position can affect the ability of the system to predict. In other words, in order to be reliable, known facial recognition systems require images from the same type of camera system and the same camera settings for all vehicles in a fleet and for the cameras to be installed at substantially the same location and position. Additionally, the ambient lighting conditions affect the accuracy of the prediction, particularly where low light cameras or IR cameras are utilised. A potential solution to this is to train the prior art systems for each camera type, fitting scenario etc. for each driver. However, to do so would be a huge undertaking for a significant sized fleet, with a constantly changing set of drivers, and is not feasible from a practical perspective. Another drawback is that such systems cannot account for changes to the facial profile of a driver, for example the addition / removal of facial hair, the addition of glasses, sunglasses, headgear, piercings and other items that are not present in the trained data. The addition / removal of such features obscure and alter the image of the driver such that the system is unable to predict their identity. Additionally, existing systems do not take into account changes in the appearance of the driver over time due to ageing. To account for the inevitable ageing process the system must be re-trained manually and periodically in order to maintain the accuracy level of the prediction. It is a principal aim of the present invention to provide a facial recognition system and method which addresses at least some of the above-mentioned problems associated with identification of drivers and which requires little or no ongoing operator intervention. According to a first aspect of this invention, there is provided an adaptive facial recognition system for drivers comprising: an image capturing device configured to generate a target set of images of a driver in a vehicle; an image reference record comprising an image reference data set for each of a plurality of drivers; a primary neural network trained to identify a driver from the target set of images based upon comparison with the image reference data sets, the primary neural 31 03 25 network being configured to provide a positive recognition of the driver if any image in the target sets identifies the driver with a confidence above a first threshold; a computer vision system configured, where none of the images in the target set identify the driver with a confidence above the first threshold, to identify, from the image reference record, the driver who best resembles the images in the target set and to compare the target set of images with the reference data set for that driver to identify image differences; the computer vision system being further configured to generate an adapted image set by emulating the identified image differences on the image reference data set for that driver; a secondary neural network trained with the adapted image set to identify a driver from the target set of images, the secondary neural network being configured to provide a positive recognition of the driver and to add the target set of images to the reference data set for that driver, if any of images in the target set identify the driver with a confidence above a second threshold; and a user interface to display driver data. According to a second by closely related aspect of this invention, there is provided an adaptive facial recognition method for drivers comprising: generating an image reference record by obtaining an image reference data set for each of a plurality of drivers; obtaining a target set of images of a driver in a vehicle, from an image capturing device; using a primary neural network to identify a driver from the target set of images based upon comparison with the image reference data sets, whereby the primary neural network provides a positive recognition of the driver if any image in the target sets identifies the driver with a confidence above a first threshold; if none of the images in the target set identify the driver with a confidence above the first threshold identifying using a computer vision system, from the image reference record, the driver who best resembles the images in the target set and comparing the target set of images with the images in the reference data set for that driver to identify image differences; emulating the identified image differences on the image reference data set for that driver to create an adapted image set; using a secondary neural network trained with the adapted image set to identify a driver from the target set of images, whereby the secondary neural network provides a positive recognition of the driver if any of images in the target set identify the driver with a confidence above a second threshold; and if any image in the target set identifies the driver with a confidence above a second threshold, adding the target set of images to the reference data set for that driver. Existing methods of driver identification tend to rely solely upon a fixed image reference data set to identify the driver and if no recognition is made with an initial attempt the same method is simply repeated until operator intervention is initiated to manually identify the driver. This prior art process is unreliable and does not work for many fleets. In contrast, the present invention captures and updates programmatically facial profiles of a driver in order continually to adapt the reference data set. In this way, the system and method can account for changes over time, as well as for facial changes. Furthermore, the system ensures that it is not necessary for every camera within an organisation (or “fleet”) to be the same nor for the settings to be identical, in order for the system to recognise accurately. An important aspect of the present invention is that it uses artificial data, in the form of an adaptive image set, to decide whether the real data i.e., the target set of images, can be verified. This maintains image reference data credibility and ensures that the result is as reliable as possible. To facilitate adaptive facial recognition of a driver, the system and method requires an image reference data set for each driver during an initial “set-up” process. The image reference data set for each driver preferably comprises a plurality of images of that driver. The images for the image reference data set may be obtained manually for each driver. When obtaining images manually, there can be difficultly in capturing images which depict the driver naturally in a driving position i.e. from the same angles / distance as would be obtained by the image capturing device. Preferably, the acquisition of images for the image reference data set is initiated automatically by the image capturing device. This alleviates effort required in order to start the initial “setup” process. This may be particularly beneficial for large fleets with many drivers. In a particularly preferred arrangement the acquisition of images for the image reference data set is initiated automatically upon the occurrence of a trigger event. The system may comprise a database configured to receive the reference data set images of a driver automatically upon a trigger event occurring. The trigger event may be the vehicle moving from a stationary position. When driving, the most common point at which the driver turns their head to either left or right (or sequentially both) is when moving off from a stationary position. Capturing facial images at a range of positions provides a broad reference data set and thus serves to optimise the accuracy of the system. Preferably the initial images obtained for the reference data set are fixed, i.e., once these images have been obtained they remain in the reference data set unless manually reset by an operator. The image capturing device may comprise one or more camera or other video recording means. The image capturing device may contain local data storage, such as a hard drive or SD card. Preferably, the image capturing device comprises a mobile digital video recorder (MDVR) or dashcam. The frame rate, resolution and bit rate of the image capturing device as well as other settings may be appropriately selected during initial set up. Preferably the image capturing device is configured to record and store video of the vehicle internally, to obtain images of the driver, and externally, to obtain images and / or footage of the vehicle journey. The combination of forward and driver facing images allows observation of a vehicle journey along with identification of the driver for the whole journey. Preferably the target set of images may be retrieved from the image capturing device automatically upon initiation of a trigger event. Preferably a trigger event comprises movement of the vehicle from a stationary position. Additionally, or alternatively, a trigger event may be based upon journey activity, such as driver behaviour or vehicle incidents. Driver behaviour events may include the vehicle being too close to another vehicle (tailgating) or an object, speeding events or erratic driving. The image combination may advantageously enable identification of risks both in and out of the vehicle, including dangerous driver behaviour such as mobile phone use, or fatigue. The image reference data set may comprise the initial fixed reference data images (ground truth data) and dynamic target data images. The dynamic nature of the target data ensures that it supplements the original ground truth data rather than replacing it. Preferably there is a finite number of images within the image reference data set with the dynamic target data adapting as necessary. Preferably, the image reference data set comprises at least a finite number of dynamic target data images. This provides the optimum spread of learnt data across the various cameras and images captured by the overall system. Preferably the system is configured to control the images within the recognised target data to ensure that the image reference data set includes the broadest and optimum scope of images. To do this, when the limit of the finite number of images within the reference data set has been reached, the system must decide which images to replace, and effectively remove. In a preferred arrangement, the system includes a second computer vision system configured to identify the presence of multiple target data images in the full reference data set having the same identified image differences and, if identified to replace at least one of the multiple target data images with at least one of the new target data images. In effect, the second computer vision system looks for repetition of the identified image differences within the target data images of image reference data set. In choosing which image to replace, the second computer vision system may be configured to replace the oldest of the at least one of the multiple target data images. A bias to remove the oldest image where there is repetition allows the system to maintain the most recent record. Preferably, the second computer vision system is configured to compare the fixed reference data images against the new target data images and the dynamic target data images in the full reference data set to identify the images having the most differences and to update the reference data set to contain the target data images with the most differences. In this way, the system enables the dynamic target data images within the reference data set to always include as many of the identified differences as possible, in the smallest data set. An important aspect of the present invention is ensuring optimum accuracy operating in a closed loop fashion with automatic continuous facial profiling in order to minimise anomalies and inaccuracies. The second threshold may be higher than the first to ensure that system is as accurate as it can be. The thresholds are preferably pre-set according to the requirements of the system. To ensure that the system and method of the present invention are reliable, it is preferable that, during an initial learning process of the primary neural network, the target set of images undergoes an operator verification process. To achieve this the system may further comprise an interface configured to display the images from the target set of images to an operator. Where the method has enabled the recognition of the driver, the system may request verification as to whether the identity recognised is correct. The system may be configured to initiate operator verification until a certain level of correct verification has been achieved. In this way, only once the primary neural network starts to recognise with sufficient accuracy will the system start fully to automate the recognition process for that driver. The level of accuracy may vary depending upon a pre-defined level of correct verification. This process refines the identification learning of the primary neural network and identifies incorrect identifications where the reference data set needs to be expanded or adapted. The process of emulating the identified image differences on the image reference data set for that driver, in order to create an adapted image set, is preferably carried out by the computer vision system. The computer vision system preferably utilises computer vision methods to identify image differences. The computer vision methods may employ object recognition techniques. Preferably the computer vision system comprises a plurality of supplementary neural networks or subnetworks, each configured to carry out a comparative process to identify image differences and to carry out image manipulation on the image reference data set to emulate the identified differences. For example, if the computer vision system identifies that a difference appears to be a pair of glasses, the system manipulates the image reference data set to simulate the face in the image wearing a pair of glasses. The system of the present invention may further comprising a server configured to receive the target set of images of a driver automatically upon a trigger event occurring. The server preferably comprises the primary and secondary neural networks and the computer vision system. In a particularly preferred embodiment of the present invention, the server comprises a digital platform, hosted in the cloud. The cloud platform preferably pulls a pre-defined period of video from the image capturing device, and extracts images therefrom. For example, the platform may pull one second of video from the image capturing device and extract 15 frames (images) therefrom. Preferably, the cloud platform obtains the target set of images from the image capturing device automatically upon initiation of a trigger event, such as movement of the vehicle from a stationary position or journey activity, such as driver behaviour or vehicle incidents. The platform may pass the images to the primary neural network for further processing. The primary and / or secondary neural network may be a convolutional neural network. The primary neural network may be trained with data to recognise faces in general rather than specifically trained with data relating to a particular face. The primary network is a core feature of the system and preferably operates in a fixed way. The secondary neural network, unlike the primary neural network, is trained using different data for each driver, namely the adapted image set for the most similar driver, thereby operating in a dynamic fashion. The primary and / or secondary neural network may comprise a plurality of subnetworks together configured to carry out the identification process. The primary and secondary neural networks may comprise the same neural network, or the same plurality of subnetworks, trained and utilised in a different way for each. In effect, whilst the system requires a primary and a secondary neural network, it is conceivable that in software terms the same network is used for the two distinct purposes. The user interface may comprise a web portal to display driver data. Alternatively or additionally, the user interface may comprise a mobile device application. The user interface may allow an operator to view and / or manage features of the system. Preferably, the image reference data set is reviewed by an operator. The operator may use the image reference data set to generate a data record for a driver (referred to also as a driver credential). In a particularly preferred arrangement, the vehicle journey and / or timeline and events are displayed on the web portal and / or mobile device. Images of the driver and / or other features may also be displayed. The present invention facilitates the use of different cameras, different camera settings and set-up and driver facial changes and thus allows versatility whilst enabling accurate driver recognition. In order that it be better understood, but by way of example only the present invention will now be described with reference to the accompanying drawings in which: Figure 1 is a schematic representation of a prior art facial recognition system for drivers; Figure 2 is a schematic representation of an adaptive facial recognition system for drivers according to the present invention; Figure 3 illustrates a screenshot of a user interface of the adaptive facial recognition system of the present invention illustrating an operator verification stage; Figure 4 illustrates a screenshot of a user interface of the adaptive facial recognition system of the invention illustrating an image reference record; Figure 5 illustrates a screenshot of a user interface of the adaptive facial recognition system illustrating an adaptive image set according to the present invention; Figure 6 illustrates a reference data set of the facial recognition system for drivers in accordance with the present invention; and Figure 7 illustrates an example of the process of reference data set image allocation. Referring initially to Figure 1, there is shown a prior art facial recognition system for drivers 10. In its most basic form the prior art system 10 comprises a plurality of vehicles 11 (only one is illustrated), each vehicle 11 having the same image capturing device 12 mounted therein. The image capturing device 12 may be a dashcam or a mobile digital video recorder (MDVR) and will be mounted at a position suitable for capturing images of the driver within the vehicle 11. The system 10 includes a server 14 for receiving, over a wireless network 13, images captured by the image capturing device 12. The server 14 includes a neural network 15 which has been trained to recognise a driver. The server also includes a reference data set of images 16 for each of the drivers. A web-portal 17 allows the driver identifications to be viewed. In the prior art methods, as a first step, a reference data set of images is initially created for each driver. This is typically a manual process whereby specific images are obtained either by the driver or by an operator. The reference data sets for all drivers are stored in an image reference record. During operation of the system 10, the image capturing device 12 in each vehicle 11 captures images of the driver and these are transmitted over the wireless network 13 to the server 14. The neural network 15 uses the images from each of the reference data set of images, within the image reference record, to assess comparatively and predict the identity of a driver in a vehicle 11. The prior art method thus comprises two broad phases; the first step is obtaining a reference data set of images of each driver (collection phase) and the second step is using the trained neural network 15 to identify the driver based upon the reference data set 16 (assignment phase). The ability to assign a driver to a particular journey can be highly beneficial but the prior art systems tend to suffer from several limitations. Training of the neural network is based on generic data and not data specific to the driver. The data used to train is often derived from a specific camera system and / or camera setup, i.e. standard training data. Even in a scenario where standard training data is not used it is not feasible for all camera types and setup configurations to be utilised due to variability and availability of such systems, in addition to the quantity of data which would be required. The accuracy of predictions made by prior art neural networks are dependent largely upon the ability of the system to recognise a driver even where there are variations in the type of camera, camera lens, image quality, frame rate, bit rate, resolution and camera installation position. Often such variations result in either no predication or an incorrect predication. Furthermore, the predictions carried out by prior art systems do not tend to take into account varied ambient lighting conditions -some cameras operate using IR either permanently (in order to reduce variability of the images) or during the low light / night time use and the resultant images may not be predictively comparable as a result. To improve accuracy, the prior art systems tend to utilise the same type of image capturing device 12 with the same settings for all vehicles of the system. Another drawback of known systems is their inability to predict where there are changes in the images of the driver, which are not present in the reference data set of images 16. Such changes may include, but are not limited to: - changes in the facial profile of the driver which may be as a result of new facial hair, different hair style / colour or indeed the removal of facial / head hair; - changes to driver images due to glasses, sunglasses, hats, piercings and other garments; - changes to the driver facial profile due to ageing. To account for the inevitable ageing process the prior art system 10 must be re-trained manually and periodically in order to maintain the accuracy level of the prediction. The present invention, as shown in Figure 2 seeks to address at least some of the prior art drawbacks. Figure 2 shows an adaptive facial recognition system 20 in accordance with the present invention. The system 20 comprises a plurality of vehicles 21 (only one is illustrated), each vehicle having an image capturing device 22 mounted therein. The image capturing device 22 may be a dashcam or a mobile digital video recorder (MDVR) and is mounted in the vehicle 21 at a position suitable for capturing images of the driver sat in the driving seat. In contrast to the prior art systems, the present invention facilitates the use of a range of different image capturing devices 22 without adversely affecting performance of the system. The system 20 includes a server 24 for receiving, over a wireless network 23, images captured by the image capturing devices 22. The server 24 includes a primary neural network 25, which has been trained to identify a driver. The primary neural network 25 is trained with annotated data in order to try to predict the identity of the driver. The server 24 of the present invention also includes a computer vision system 29 which comprises a plurality of supplementary neural networks configured to carry out object recognition and to identify differences between images. The computer vision system 29 identifies image differences using a comparative process and manipulates the image reference data set to emulate any identified differences. A secondary neural network 29 provided on the server is trained to identify a driver based upon the manipulated images. The server 24 also includes an image reference record 30 comprising an image reference data set 26 for each of a plurality of drivers. A web-portal and / or mobile phone app 27 allows the driver identifications to be viewed. Operation of the system of the present invention will now be discussed in more detail, with reference to Figures 2 to 6. The reference data set of images 26 of each of a plurality of drivers is created during an initial “set up” process. The image reference data set 26 for each driver comprises a plurality of images of that driver. The number of images obtained for the reference data set is pre-determined. These initial images obtained for the image reference data set 26 are fixed and cannot be changed unless the system is manually reset by an operator. The images for the image reference data set 26 are obtained automatically by the image capturing device 22 within the vehicle 21 upon the occurrence of a trigger event, namely the vehicle moving from a stationary position. Each of the image reference data sets are stored together within the image reference record 30. During operation of the adaptive facial recognition system 20, the image capturing device 22 in each vehicle 21 records videos of the driver and these are stored locally on the local data storage (hard drive or SD card). The image capturing device 22 is configured also to obtain images and footage of the vehicle journey. The server 24 comprises a digital platform, hosted in the cloud and this is configured to pull a pre-defined period of video over the wireless network 23 from the image capturing device 22, and to extract a “target set” of images 32 therefrom. The cloud platform 24 obtains the target set of images 32 from the image capturing device 22 automatically upon initiation of a trigger event. The trigger events may include movement of the vehicle 21 from a stationary position as well as journey-based activities, such as driver behaviour or vehicle incidents. Driver behaviour events include (but are not limited to) the vehicle being too close to another vehicle (tailgating) or too close to an object, person or animal, speeding events or erratic driving. The combination of driver and journey images may advantageously enable identification of risk both in and out of the vehicle 21, including dangerous driver behaviour such as mobile phone use, or fatigue. The cloud platform 24 passes the images to the primary neural network 25 for further processing. The primary neural network 25 is trained to identify a driver from the target set of images 32 based upon comparison with the image reference data sets 26 within the image reference record 30. The primary neural network 25 is configured to provide a positive recognition of the driver if any image in the target sets 32 identifies the driver with a confidence above a first threshold. The method of the present invention requires an initial learning process of the primary neural network 25. As best seen in Figure 3, initially, the target set of images 32 undergoes an operator verification process. During the learning stage the target set images 32 are displayed to an operator on the web-portal or mobile phone app 27 and if the primary neural network 25 has resulted in the recognition of the driver, the system 20 will request verification as to whether the identity recognised is correct. The operator will be asked to confirm the driver identity by selecting one of the buttons labelled “No” 33, “Yes” 34 or “Unknown Driver” 35. The system 20 is configured to initiate operator verification until a certain level of correct verification has been achieved. In this way, only once the primary neural network starts to recognise with sufficient accuracy will the system start fully to automate the recognition process for that driver. The level of accuracy may vary depending upon a pre-defined level of correct verification. The web-portal 27 allows verification to be carried out at the convenience of the operator, with a list of verification states 36 being displayed on the portal 27. This process refines the identification learning of the primary neural network 25 and determines incorrect identifications where the reference data set 26 needs to be expanded or adapted, as discussed below. If the primary neural network 25 does not identify the driver from the images in the target set with sufficient confidence (i.e., confidence above a first pre-set threshold), the computer vision system 28 identifies, from the image reference record 30, the driver who resembles the images in the target set the most (i.e., the most likely candidate). The computer vision system 28 then compares the target set of images 32 with the reference data set 26 for that driver to identify image differences. An adapted image set 38 is generated by the computer vision system 28 by emulating the identified image differences on the image reference data set 26 for that most likely driver. The server 24 includes a library of objects for potential differences, for example glasses, sunglasses, hats, facial hair etc. New objects may be added to the library by creating a three-dimensional model using a standard three-dimensional computer graphics software tool. The 3D model can then be exported as an object and one of the supplementary neural networks of the computer vision system 28 used to map the custom object model to a drivers face. The emulation of the differences on the images reference data set 26 involves the process of pose estimation. The computer vision system 28 comprises the following sequential steps: 1. Two dimensional coordinates are obtained of a few points on the face in order to obtain these face features as landmarks. The points include: corners of the eyes, the tip of the nose, comers of the mouth etc. This is achieved using the primary neural network, facial recognition system 25. 2. The three-dimensional locations of the same points are then determined. A generic three-dimensional model will suffice or one may simply obtain the three-dimensional locations of a few points in an arbitrary reference frame. 3. The intrinsic parameters of the camera are determined so that differences relating to such features may be emulated. 4. Using an appropriate Perspective-n-Point (PnP) algorithm with Random sample consensus (RANSAC) the translation (T) and rotation (R) matrix can be solved from the camera two-dimensional points of the face features to the three-dimensional points of face features. 5. With the RT matrix any object can be added and by multiplying it by RT the objects can be translated onto the face in three-dimensional real time. The system may apply the above steps to video footage of a driver (i.e. to the image reference data set 26) which will allow feature differences, such as glasses, to be applied to every image frame. Those adapted emulated images are then saved as an adapted image set 38, as best seen in Figure 5. The secondary neural network 29 is trained to identify a driver from the target set of images 32 based upon the adapted image set 38. The secondary neural network 29 is thus trained using different data for each driver. The secondary neural network 29 thus operates in a dynamic, forever changing, fashion. The secondary neural network 29 is configured to provide a positive recognition of the driver if any of images in the target set 32 identify the driver with a confidence above a second threshold. If the secondary neural network 29 identifies the driver in any image 39 in the target set 32 with a confidence above the second pre-set threshold, it adds the target set of images 32 to the dynamic recognised target data of the reference data set 26 for that driver. An example of a image reference data set 26 can be seen in Figure 6 and this comprises a plurality of images. The image reference data set 26 includes a series of fixed images 40 obtained during set up of the system and a finite number of dynamic target data images 41, in this case twenty. This provides the optimum spread of learnt data across the various cameras and images captured by the overall system. The system is configured to control the Images within the dynamic target data 41 to ensure that the image reference data set 26 includes the broadest and optimum scope of images. The system operates in a similar way to the process of developing the adapted image set. The new target images 46 have known "differences" as compared to the current images within the image reference data set 26, so at this point the distinctions are known. For example, the system has already identified if images are from a camera positioned further away or at a lower resolution than the fixed images of the driver obtained during setup of the system. If there is space in the image reference set these new target images 46 are simply added to the reference data set. When the limit of the finite number of images within the reference data set 26 has been reached, the system must decide which images to replace, and effectively remove. A second computer vision system 42 controls the allocation of images into the image reference data set 26. The second computer vision system 42 identifies the presence of multiple target data images in the full reference data set having the same identified image differences 45 and, if identified to replace at least one of the multiple target data images 45 with at least one of the new target data images 46. In effect, the second computer vision system 42 looks for repetition of the identified image differences within the target data images 41 of image reference data set 26. In choosing which image to replace, where there is more than one option, the second computer vision system 42 is configured to replace the oldest of the multiple target data images 45. A bias to remove the oldest image where there is repetition allows the system to maintain the most recent record. Repetition of image differences is the first step in the process of reference data set image allocation. If there is no repetition, i.e. no multiple target data images having the same identified image differences, the second computer vision system compares the fixed reference data images 40 against the new target data images 46 and the dynamic target data images 41 in the full reference data set 26 to identify the images having the most differences and to update the reference data set to contain the target data images with the most differences. Referring to Figure 7, the image having glasses and a hat 46 includes more differences, as compared to the fixed reference data images 40 (only one illustrated in Figure 7), than the current dynamic target data images 41 (only one illustrated in Figure 7) and so this new image 46 would replace the oldest current image within the current dynamic target data images with the least differences as compared to the fixed reference data images 40. The system enables the dynamic target data images 41 within the reference data set 26 to always include as many of the identified differences as possible, in the smallest data set. This enables the system and method of the present invention to account for changes over time, as well as for facial changes. Furthermore, the system and method ensures that it is not necessary for every camera within an organisation (or “fleet”) to be the same nor for the settings to be identical, in order for the system to recognise accurately. Importantly, it is the target set of images 32 and not the adapted image set which is controllably added to the reference data set 26. The method of the present invention utilises artificial data, in the form of the adaptive image set 38, to decide whether the real data i.e., the target set of images 32, can be verified. This enables the credibility of the image reference data 26 to be maintained and ensures that the result is as reliable as possible. By emulating the identified differences on the reference data set 26, the adapted image set 38 enables more detailed and accurate recognition of drivers. Variations in the type of camera, camera lens, image quality, frame rate, bit rate, resolution and camera installation position, as well as lighting conditions can be identified and accounted for in order to avoid adversely affecting the ability of the system to predict. The system and method can account for changes to the facial profile of a driver, for example the addition / removal of facial hair, the addition of glasses, sunglasses, headgear, piercings and other items that are not present in the image reference data set 26. Additionally, the system and method takes into account changes in the appearance of the driver over time due to ageing. In contrast to the prior art method, the present invention comprises four broad phases; the first step is obtaining a reference data set of images 26 of each driver (collection phase). The second step requires verification of the predictions made by the primary neural network 25 in order to refine the identification learning, where these might come across multiple cameras and devices 22 (verification phase). This phase allows the system 20 to identify any incorrect identifications where the reference data set 26 needs to be expanded or adapted. In the third step the system 20 predicts the identity of the drivers and no further user input is requested. In this stage the system 20 is operating fully and identifying drivers and associating, in the platform 24, the data for the journeys and video with that driver (assignment phase). Finally, in the fourth step the computer vision system 28 is monitoring the primary neural network 25 predictions and cross referencing this with the image reference data set 26 in order to identify any detections where there are identifications made, in the presence of aspects known to alter the detection ability of the primary neural network 25, i.e. this person now has glasses on, or is being recorded in a wider angle lens etc. Where these detections are made, in the presence of these aspects being changed / altered in the environment, the data 32 is automatically formatted (into the necessary files) added to the reference data set 26 for the given driver. The system 20 then repeats the detections (for unknown drivers) in order to make further detections. This iterative process leads to an adaption of the reference data set 26 to include the changes, without user intervention, and make iterative passes at identification of unknown drivers (adaption phase). Doing this continually provides the ability to limit the changes (they are likely small and occur over time) so increases the ability of the system to automatically adapt. This system 20 also negates the need for the end users to “train” the system for each camera type, fitting scenario etc. for each driver as that would be a huge undertaking for a significant sized fleet with a constantly changing set of drivers. 31 03 25 25

Claims

1. An adaptive facial recognition system for drivers comprising:- an image capturing device configured to generate a target set of images of a 5 driver in a vehicle;- an image reference record comprising an image reference data set for each of a plurality of drivers;- a primary neural network trained to identify a driver from the target set of images based upon comparison with the image reference data sets, the primary neural10 network being configured to provide a positive recognition of the driver if anyimage in the target sets identifies the driver with a confidence above a first threshold;- a computer vision system configured, where none of the images in the target set identify the driver with a confidence above the first threshold, to identify, 15 from the image reference record, the driver who best resembles the images inthe target set and to compare the target set of images with the reference data set for that driver to identify image differences; the computer vision system being further configured to generate an adapted image set by emulating the identified image differences on the image reference data set for that driver;20 - a secondary neural network trained to identify a driver from the target set ofimages based upon the adapted image set, the secondary neural network being configured to provide a positive recognition of the driver and to add the target set of images to the reference data set for that driver, if any of images in the target set identify the driver with a confidence above a second threshold; and- a user interface to display driver data.31 03 252. An adaptive facial recognition system for drivers as claimed in claim 1, further comprising a server configured to receive the target set of images of a driver automatically upon a trigger event occurring.

53. An adaptive facial recognition system for drivers as claimed in claim 1 or claim 2, further comprising an interface configured to display the target set of images to an operator.10 4. An adaptive facial recognition system for drivers as claimed in any of claims 1 to 3, wherein the image capturing device comprises a mobile digital video recorder.

5. An adaptive facial recognition system for drivers as claimed in any of the preceding claims, wherein the image reference data set comprises a plurality of fixed images 15 of the driver, acquired automatically upon initiation of a trigger event.

6. An adaptive facial recognition system for drivers as claimed in any of the preceding claims, wherein the image reference data set comprises fixed reference data images and dynamic target data images.

207. An adaptive facial recognition system as claimed in claim 6, wherein the image reference data set comprises a finite number of dynamic target data images.31 03 258. An adaptive facial recognition system for drivers as claimed in claim 7, further comprising a second computer vision system for controlling the addition of new target data images to a full reference data set.

9. An adaptive facial recognition system as claimed in claim 8, wherein the second computer vision system is configured to identify the presence of multiple target data images in the full reference data set having the same identified image differences and, if identified, to replace at least one of the multiple target data images with at least one of the new target data images.

10. An adaptive facial recognition system as claimed in claim 9, wherein the second computer vision system is configured to replace the oldest of the at least one of the multiple target data images.

11. An adaptive facial recognition system as claimed in any of claims 8 to 10, wherein the second computer vision system is configured to compare the fixed reference data images against the new target data images and the dynamic target data images in the full reference data set to identify the images having the most differences and to update the reference data set to contain the target data images with the most differences.

12. An adaptive facial recognition system as claimed in any of the preceding claims, wherein the image reference record further comprises driver data.31 03 252513. An adaptive facial recognition system as claimed in any of the preceding claims, wherein the image capturing device is configured to obtain images simultaneous of the driver and the vehicle journey.

514. An adaptive facial recognition method for drivers comprising:- generating an image reference record by obtaining an image reference data set for each of a plurality of drivers;- obtaining a target set of images of a driver in a vehicle, from an image capturing10 device;- using a primary neural network to identify a driver from the target set of images based upon comparison with the image reference data sets, whereby the primary neural network provides a positive recognition of the driver if any image in the target sets identifies the driver with a confidence above a first threshold;15 - if none of the images in the target set identify the driver with a confidence abovethe first threshold identifying using a computer vision system, from the image reference record, the driver who best resembles the images in the target set and comparing the target set of images with the images in the reference data set for that driver to identify image differences;20 - emulating the identified image differences on the image reference data set forthat driver to create an adapted image set;- using a secondary neural network trained with the adapted image set to identify a driver from the target set of images, whereby the secondary neural network provides a positive recognition of the driver if any of images in the target set identify the driver with a confidence above a second threshold; and31 03 25- if any image in the target set identifies the driver with a confidence above a second threshold, adding the target set of images to the reference data set for that driver.5 15. An adaptive facial recognition method for drivers as claimed in claim 14, whereinthe image reference data set comprises a plurality of fixed images of the driver, whereby the acquisition of the fixed images is initiated automatically by a trigger event.10 16. An adaptive facial recognition method for drivers as claimed in claim 14 or claim15, wherein the target set of images are obtained from the image capturing device automatically upon initiation of a trigger event.

17. An adaptive facial recognition method for drivers as claimed in claim 15 or claim15 16, wherein a trigger event comprises movement of the vehicle from a stationaryposition.

18. An adaptive facial recognition method for drivers as claimed in any of claims 14 to 17 wherein the target set of images are dynamically added to the reference data 20 set.

19. An adaptive facial recognition method for drivers as claimed in claim 18, wherein the number of dynamic target set images in the reference data set is limited.31 03 25-2.(-20. An adaptive facial recognition method for drivers as claimed in claim 19 wherein the addition of a new target set of images to a full reference data set is controlled by a second computer vision system.

21. An adaptive facial recognition system as claimed in claim 20, wherein the second computer vision system identifies the presence of multiple images in the dynamic target set of images of the full reference data set, having the same identified image differences and, if identified, replaces at least one of the multiple images with at least one of the images from the new target set of images.

22. An adaptive facial recognition system as claimed in claim 21, wherein, the second computer vision system replaces the oldest of the at least one of the multiple images.

23. An adaptive facial recognition system as claimed in any of claims 20 to 22 wherein the second computer vision system compares the fixed reference data images against the new target data images and the dynamic target data images in the full reference data set to identify the images having the most differences and updates the reference data set to contain the target data images with the most differences.

24. An adaptive facial recognition method for drivers as claimed in any of claims 14 to 23, whereby a positive recognition of the driver by the primary neural network is authenticated by an operator verification procedure during the learning process of the primary neural network.31 03 2525. An adaptive facial recognition method for drivers as claimed in any of claims 14 to 24 wherein the computer vision system utilises computer vision methods to identify image differences.5 26. An adaptive facial recognition method for drivers as claimed in claim 25, whereinthe computer vision methods employed include object recognition.

27. An adaptive facial recognition method for drivers as claimed in any of claims 14 to 26, wherein images of the driver and the vehicle journey are obtained 10 simultaneously by the image capturing device.

28. An adaptive facial recognition method for drivers as claimed in claim 27, wherein the target set of images are obtained from the image capturing device automatically upon initiation of a trigger event.1529. An adaptive facial recognition method for drivers as claimed in claim 28, wherein a trigger event comprises one or more of: movement of the vehicle from a stationary position, driver behaviour and vehicle incidents.

Images