SAFETY CARD TRACKING AND ADVISORY SYSTEM FOR A MOTOR VEHICLE

Abstract

A security token for accessing a motor vehicle, such as an NFC access card or an NFC-enabled mobile phone, is monitored to help prevent it from being left behind in the vehicle when the user exits the vehicle. An NFC reader in the passenger compartment communicates with the token to authorize access to predetermined functions of the vehicle. An image sensor captures images in the passenger compartment after the security token is presented to the NFC reader. A location of the token is tracked in the captured images. One or more predetermined carrier locations are identified that are associated with the user and can remain with the user when the user exits the motor vehicle (e.g., a pocket or purse).A message to the user is generated as a reminder if it has not been detected that the security token has been returned to an identified predetermined carrier location.

Description

CROSS REFERENCE TO RELATED APPLICATIONSNot true.EXPLANATION OF FEDERALLY SPONSORED RESEARCHNot true.BACKGROUND OF THE INVENTIONThe present invention relates generally to motor vehicle security systems that utilize access cards, and more particularly to a system for tracking a physical location of an access card to prevent an access card from being accidentally left upon leaving a vehicle.Mechanical keys conventionally used for unlocking access doors to motor vehicles and for unlocking a steering wheel and starting a drive unit (e.g. an internal combustion engine or a drive train of an electric vehicle) have been gradually replaced by passive and active electronic devices. Active devices include fobs and smart phones (e.g., phone-as-a-key) with stand-alone power supplies. Passive devices include transponders such as near-field communication (NFC) access cards.These electronic devices may wirelessly identify a user to gain access to the vehicle (e.g., door unlock), activate the vehicle (e.g., ignition of an internal combustion engine or activating a powertrain of an electric vehicle), and perform other functions.NFC devices are advantageous because their very short transmission range makes it difficult for third parties to intercept and copy security data. NFC hardware is included in many smartphones, so that a suitable app installed on the smartphone may be configured to operate as a smart key for the user. Additionally, dedicated NFC-based access cards that are small and thin (e.g., the size of a credit card) are being popular. An NFC access card may be configured to receive its power from the vehicle during a read operation or may include a stand-alone power source. Digital key protocols have been developed to maintain vehicle security while enabling vehicle access sharing and management. For example, a digital key standard has been published by the Car Connectivity Consortium.In some embodiments, an NFC-based security system includes a portable NFC access card carried by the user and a fixed NFC unit mounted on the vehicle (e.g., an NFC card reader). For example, an NFC reader for obtaining access to the vehicle (e.g., for unlocking a door) may be mounted behind a B-pillar applique (e.g., on the vertical pillar located between the windows of a front door and a rear door on a side of the vehicle). Once a user (e.g., driver) has gained access to the passenger cabin of a vehicle, the NFC access card may be used as a key to access secure functions such as starting the vehicle and distinguishing an authorized user from another (e.g., to take user-defined settings such as the seat position). Accordingly, another NFC reader may be provided within the passenger cabin, for example, on an instrument panel or center console.One or more display panels are typically mounted as part of a human machine interface (HMI) on the instrument panel or dashboard of a vehicle. A touch screen display panel typically creates "soft keys" of the menu for collecting user input. Display panels have also become available in which wireless antennas (including NFC antennas) have been integrated into the surface of the display. The antenna can remain hidden and has a negligible influence on the display surface itself. Because the read operation would only briefly obscure the display panel, the inclusion of an NFC antenna for an NFC reader in the display panel provides the functionality of an NFC key without requiring the space on any interior surfaces of the vehicle to be occupied and allows a common location across product lines of a vehicle manufacturer.To be used when starting a vehicle, the NFC access card must be handled by the user to place it near the NFC reader (e.g., parallel to and within about 4 centimeters of the reader surface). Thus, the access card is normally removed from a user's pocket, wallet or wallet to present it to the NFC reader. If the access card is not immediately moved into the pocket, wallet or wallet, there is the possibility that the access card is forgotten in the vehicle when the user arrives at a destination and leaves the vehicle. Leaving an access card in the vehicle poses safety concerns. Additionally, if the vehicle does not have an outside keypad or if the driver or user does not know the keypad code, it may be unlocked from the vehicle if he relies solely on the NFC access key and he has been left in the vehicle. Moreover, in some countries, there are regulations to leave access keys in an automobile, and damage from vehicle theft may not cover the insurance if access keys are left in the vehicle. The same concerns may arise when an NFC-enabled mobile phone is left therein. In particular, a mobile phone may not include the full capability of a phone-as-key, but instead only has a simple NFC-based access and start function, which may make it more likely that the NFC phone will also be forgotten when leaving the vehicle. As used herein, references to a security token or an NFC access card are intended to include any NFC access device (NFC card, NFC bracelet, NFC phone, etc.).SUMMARY OF THE INVENTIONIn one aspect of the invention, a motor vehicle having a passenger compartment includes a near-field communication (NFC) reader defining a touch point in the passenger compartment configured to communicate with a token carried by a user of the vehicle to authorize access to predetermined functions of the motor vehicle when the token is presented to the touch point. An image sensor is configured to capture images within the passenger compartment. A tracking controller coupled to the image sensor and the NFC reader is configured to track a location of the token in the captured images after being presented to the touch point. The tracking controller is configured to identify one or more predetermined carrier locations associated with the user that may remain with the user when the user exits the motor vehicle. A user interface coupled to the tracking controller generates a message to the user as a reminder when the tracking controller determines that the token has not been detected as being returned to an identified predetermined carrier location after the token is presented to the touch point.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a schematic illustration of a user with an NFC access card that enables access to and use of a vehicle based on NFC card readers on the vehicle. FIG. 2 is a schematic diagram showing a vehicle passenger cabin including NFC card readers. FIG. 3 is a schematic diagram showing tracking regions within a passenger cabin for monitoring placement of an NFC access card after use to start the vehicle. FIG. 4 is a block diagram showing one embodiment of a vehicle system for tracking the proper preservation of an NFC access card following its use. FIG. 5 is a front view of an NFC access card according to an embodiment of the invention. FIG. 6 is a side view of the access card of FIG. 5. FIG. 7 is a flow diagram showing a preferred method of the invention for tracking an NFC access card within a passenger cabin.DETAILED DESCRIPTION OF PREFERRED EMBODIMENTSA monitoring for mispositioning of an access token (e.g., NFC access card) may be initiated in response to a vehicle controller detecting that the token was used to start the vehicle. At the time the token is presented to an internal reader, it can be easily recognized on images captured in the passenger cabin of the vehicle and then tracked while the user is removing the token. An NFC card can be readily visible on captured images, as people typically hold it at an edge so that the NFC card lies substantially parallel to the reader surface. For tracking purposes, camera data may be analyzed using artificial intelligence (AI) and / or machine learning (ML) tools to follow the NFC card (while being held in the user's hand) to encourage the card to be reinserted into a hand bag or bag and not to be placed on a center console, cup holder, or storage bin where it could be left in the vehicle as the user later exits the vehicle.In some embodiments, if the map is not immediately re-entered into a pocket or hand-held pocket, an immediate reminder may be generated to notify the user (e.g., via messages such as audio playback via internal or external speakers, a text field on a touch screen display, or similar messages sent to the user's smartphone). In some embodiments, generating a reminder message may be moved until the driver door opens if the user places the NFC card in an improper location (e.g., a cup holder without an NFC reader). In some embodiments, the user may be given an option to discard a reminder message and enable adaptation or reconfiguration of a vehicle security system to prevent unauthorized use of a security token left in the vehicle. For example, the token's access rights may be invalidated. The call back may be temporary until the user restores the token to a smartphone using a security caplet, for example. In some embodiments, there may be a fleet manager that controls security for a fleet of vehicles, and the fleet manager may be automatically notified when a security token is left in a vehicle. The fleet manager may determine whether or not access rights for the abandoned token have been cancelled.The visibility of the security token (e.g., the NFC card) can be further improved by applying a long persistence phosphor and illuminating a blue LED integrated into the reader to excite the phosphor when the token is presented to the NFC reader. The blue LED may be mounted on a surface of the NFC reader, for example. This immediately causes the card to illuminate, making it easier for the camera to track it even in the dark. Alternatively, tracking may be improved using a photo-reflective material distributed in a predetermined pattern on the token and configured to reflect infrared light. By continuously activating an infrared light source during tracking, the position of the token can be determined more reliably.Referring to FIG. 1, a vehicle 10 is accessible to a user 11 by interaction between a vehicle interface 12 and a mobile security token 13 (e.g., an NFC card or a wireless transponder or fob). The interface 12 may be an NFC reader that includes an NFC antenna to perform near field communication between a vehicle security controller (not shown) and the security token 13 to unlock the vehicle 10. Once access to the interior passenger cabin of the vehicle 10 is available, the security token 13 may also be presented to an interior NFC reader 15 to access additional functions such as starting the vehicle 10.In FIG. 2, a passenger compartment 15 of the vehicle is shown having an instrument panel 16 to which the NFC reader 14 is mounted to allow easy access for placing a security token 13 in close proximity to the NFC reader 14. The passenger compartment 15 also includes a center console 17 with cup holders 18. the console 17 may include a storage tray 20, where the tray 20 and the cup holders 18 represent examples of impermissible storage locations where the security token 13 could be placed after being presented to the NFC reader 14. The instrument panel 16 may further include a storage bin 21 (i.e., glove box) that represents another impermissible storage location where the security token 13 could be placed and then inadvertently left in the passenger compartment 15 when the user exits the vehicle. The NFC reader 14 has a touch point 22 located near an NFC antenna for communicating with the token 13. The NFC reader 14 may further include an LED 23 for generating light according to a wavelength for exciting a photoluminescent material on the security token 13, or an LED 24 configured to generate infrared light for interacting with a photoluminescent material on the token 13, as described below. The dashboard 16 may further include a human-machine interface 25, which may include a display panel and audio transducers known in the art.In some embodiments of the invention, different regions within the passenger compartment may be defined according to whether particular locations represent an appropriate repository location or an impermissible repository location for the security token. As shown in FIG. 3, the tracking of the security token 13 begins when it is in an origin region 30 surrounding the NFC reader 14. When the user removes the token 13 from the NFC reader 14, it enters an open region 31 that includes intermediate locations where the token cannot be stored (e.g., suspended without a bearing surface in the passenger compartment). One or more appropriate storage regions 32 may be defined according to static or dynamic elements in the passenger compartment that are associated with the user and that may be expected to leave the vehicle with the user. The proper storage location 32 may include the pocket of a user's garment or a worn container such as a hand bag, a backpack, or a briefcase. A region 33 represents an improper location, such as a cup holder. In some cases, an improper storage region 34, such as the area of the passenger seat, may surround an appropriate storage location 35, which may be, for example, a hand bag placed on the seat.FIG. 4 shows a vehicle system of the invention in more detail. The vehicle 10 includes a main controller 40, which may be a programmable microprocessor configured in accordance with the present invention. The controller 40 is coupled to the NFC reader 14 to communicate with the security token 13 held by a user 11 while within the vehicle 10. The user 11 has a pocket 41 in a worn garment that represents one of a plurality of predetermined wearer locations where the user would be expected to place the security token 13 such that the security token 13 would remain with the user when the user exits the vehicle 10. The predetermined carrier locations may further include a carried storage device such as a hand bag, a briefcase, a backpack, or a bag in a garment that is not currently being worn, but that has arrived with the user in the vehicle and is expected to be removed (e.g., a jacket) upon exiting the vehicle. The classification and recognition of such predetermined carrier locations may be handled using artificial intelligence or machine learning tools. The selection and configuration of such tools is within the skill of those of ordinary skill in the art.The controller 40 is connected to the LED 23 associated with the NFC reader 14 and to an image sensor or camera 42 that may also be associated with (e.g., integrated with) the NFC reader 14. The camera 42 may be, for example, a dedicated camera for tracking the security token 13. Additionally or alternatively, the controller 40 may be coupled to a camera 43 mounted in the passenger compartment that may be present to perform other functions, such as driver eye tracking, fatigue detection, or occupant detection. When the camera 43 is operated to perform gaze detection, it may use infrared illumination provided by an infrared LED 44 also coupled to the controller 40. Images captured by the camera 43 may also be used to track the security token 13 and compare locations of the token 13 to potential carrier / storage locations provided by a user's clothing or equipment (suitable locations remaining with the user when the user exits the vehicle 10) and provided by vehicle surfaces (unsuitable locations).The controller 40 is coupled to a human machine interface 35 that may include a touch screen display panel, speakers, and / or microphones for exchanging messages with the user 11. The controller 40 is further coupled to a wireless transceiver 46 having an antenna 47 configured to interact with a smartphone 50 worn by the user 11, thereby providing an alternative channel for generating messages to the user 11 using the display screen or speakers of the smartphone 50. As noted above, in some embodiments, the smartphone 50 may also include the security token 13 used as the key for the vehicle. In these situations, an alternative channel may be used to generate messages to the user 11, such as an in-vehicle or external speaker, an internal or external display screen, and / or an alternative smart phone detected by the wireless transceiver 46.Preferably, the transceiver 46 may be a BUN transceiver module capable of interacting via Bluetooth® communication, ultra wideband _ UWB (ultra-wide band) communication, and NFC communication. In particular, Bluetooth® may be used to connect the controller 40 to a Applet in the smartphone 50 when generating messages for the user 11. The transceiver 40 may also include a cellular data connection with a cellular system 51 to provide communication between the controller 40 and a fleet manager 52 over a data network 53.The controller 40 is coupled to a card status memory 55, which can store data corresponding to a set of uniquely identifiable NFC cards and the status of the corresponding access rights. A door switch 56 is coupled to the controller 40 to provide a signal when the user 11 opens a driver door in preparation for getting out of the vehicle 10. A driver seat sensor (not shown) may also be used in some embodiments to detect whether the door may be open but the driver remains seated. Accordingly, the message could be delayed or suspended until the driver's intention to exit the vehicle is verified.Changes to the security token 13 may be made to facilitate tracking based on camera images. As shown in FIGS. 5 and 6, the token 13 may be comprised of an NFC access card having an upper substrate 60 and a lower substrate 61. Each may be formed from a thin sheet of plastic. The plastic can be at least partially transparent. As is known, an NFC controller 62 and an NFC antenna 63 may be disposed between the substrates 60 and 61. A luminescent material 64 may be coated or impregnated into one or both of the substrates 60 and 61. A long persistence phosphor formulation is preferably used (e.g., CaAl 2 O 4: Eu 2+, Nd 3+), such that illumination of the excited phosphor may aid in tracking the NFC card 13 for at least several seconds or minutes. Light transmissive portions of the substrates 60 and 61 may be disposed adjacent to the luminescent material 64 to facilitate excitation by an external light source (e.g., the blue LED 23 in FIG. 4 ) and to allow subsequent light emission to be detected from each side of the NFC access card.The token 13 may be provided with a photo-reflective material 65, either with or without luminescent material 64, distributed in a predetermined pattern and configured to reflect incident light. The incident light may be generated by an LED using a predetermined spectrum of light, such as infrared light. In some embodiments, an infrared LED is utilized that is present in a vehicle within a gaze tracking system that is intended to monitor a direction of gaze of the driver for purposes of providing driver assistance and driver assessment (as such systems are typically based on the use of infrared light). Thus, the image sensor used to track the security token may be an eye tracking camera and the stimulating light source may be an infrared generating LED integrated with the eye tracking camera. Alternatively, a separate dedicated infrared generating LED source may be installed to be directed to the location(s) of the NFC reader.In an embodiment where an NFC phone is used as a security token, a photoluminescent or photo-reflective construction as discussed above may be similarly applied to the device. For example, a luminescent material or a photo-reflective material may be integrated into or coated on a telephone back plate or a telephone shell. Alternatively, a film could be applied to the back of the NFC phone for bonding.Figure 7 shows a preferred method of the invention. In step 70, a check is made to determine whether the NFC card has been used to access the vehicle functions (e.g., accessory mode or vehicle start) via an internal NFC reader. When it is detected that the NFC card has been used in this way, the supporting illumination is activated in step 71. The supplemental illumination may include one or more pulses of an excitation light source, such as a blue LED used when an NFC access card carries a photoluminescent material, and / or a continuous activation of an infrared light source when a reflected infrared trace is to be deployed. In step 72, the NFC card is still in close proximity to the touch point of the NFC reader, and then a stream of images captured by an image sensor is continuously monitored to track movement of the NFC access card.In step 73, a check is made to determine whether the card has been placed in a common carrier (i.e., a storage location), such as a predetermined pocket in a driver's garment, or other predetermined carrier locations, such as a hand bag or a backpack. If the NFC access card has been placed in this way, the method is concluded in step 74. Otherwise, tracking of the location of the NFC access card continues and a check is made in step 75 to determine if the access card was placed improperly. Improper locations may include a cup holder, console tray, or other vehicle surface that does not have NFC readers, which could allow the user to exit the vehicle without fear of carrying the NFC access card. If step 75 determines that the card is not properly placed, a message is generated in step 77 as a reminder that the token has not been detected as being returned to one of the appropriate carrier locations that may have been identified by the controller (e.g., a bag or hand bag). Such a message may include a text display on a display panel, an audio message via a speaker, or a transmission to a mobile wireless device or smartphone of the user.If it has not been detected in step 75 that the card has been placed in an appropriate location, a check is made in step 76 to determine whether an excessive amount of time has elapsed while waiting for the NFC access card to be placed elsewhere. One purpose of waiting for a predetermined amount of time is to avoid or remedy errors in tracking. If the predetermined time has not yet elapsed, return is made to step 73. Otherwise, the user interface may be activated in step 77 to generate a corresponding message to the user.In some embodiments, a user may be given an option to discard a reminder message and prevent the generation of further reminders. For example, a "discard" button may be provided on a touch screen display. In step 78, a check is made to determine whether the user has pressed the discard button. If not, further follow messages may be made (e.g., at the time of leaving the vehicle). Accordingly, a check is made in step 80 to determine whether the driver door opens indicating that the user is preparing to exit. Once the door is detected as open, the reminder message may be repeated at step 81.If the user selects to discard the message in step 78, a check may be made in step 82 to determine if there is a fleet manager that may have primary authority over the vehicle with the authority to decide whether further action should be taken on a abandoned NFC access card. If there is no fleet manager, the method may end in step 83. Otherwise, in step 84, a message may be sent to the fleet manager informing that the fleet vehicle driver leaves an access card in an unsupervised vehicle. In response to a message sent in step 84, a fleet manager may determine that the access card should be deactivated and become unable to operate the vehicle until further corrective actions are taken by the user or fleet manager. The fleet manager may initiate a message to the controller to update a status of the NFC access card and render it inactive.According to an embodiment of the present invention, the token consists of a mobile phone capable of near-field communication (NFC).According to an embodiment of the present invention, the mobile phone comprises a substrate or an add-on cover or an add-on label carrying a photoluminescent material or a photo-reflective material.In one aspect of the invention, the NFC card or cellular phone comprises a substrate carrying a photo-reflective material distributed in a predetermined pattern and configured to reflect infrared light, and wherein the method further comprises the step of: activating an infrared light source to improve tracking of the location of the NFC card or cellular phone.In one aspect of the invention, the method includes: identifying one or more improper storage locations of the motor vehicle; wherein the message is generated if the token is tracked to the one or more improper storage locations.In one aspect of the invention, the improper storage locations are comprised of a storage container, a cup holder, and a console tray.

Claims

A motor vehicle having a passenger compartment, the motor vehicle comprising: a near-field communication (NFC) reader defining a touch point in the passenger compartment configured to communicate with a token carried by a user of the motor vehicle to authorize access to predetermined functions of the motor vehicle when the token is presented to the touch point; an image sensor configured to capture images within the passenger compartment; a tracking controller coupled to the image sensor and the NFC reader, the tracking controller configured to track a location of the token in the captured images after being presented to the touch point and configured to identify one or more predetermined carrier locations associated with the user that may remain with the user when the user exits the motor vehicle; and a user interface coupled to the tracking controller and configured to generate a message to the user as a reminder when the tracking controller determines that the token has not been detected as being returned to an identified predetermined carrier location after the token has been presented to the touch point.The motor vehicle of claim 1, wherein the one or more predetermined wearer locations identified by the tracking controller consist of a pocket of a user's garment and a carry pocket of the user.The motor vehicle of claim 1, wherein the token consists of a near-field communication (NFC) card.The motor vehicle of claim 3, wherein the NFC card comprises a substrate carrying a photoluminescent material, and wherein the motor vehicle further comprises: a stimulating light source activated by the tracking controller to improve tracking of the location of the NFC card.The motor vehicle of claim 4, wherein the photoluminescent material is comprised of a phosphor and wherein the exciting light source is comprised of a blue light LED near the touch point that is activated when the NFC card is presented to the touch point.The motor vehicle of claim 3, wherein the NFC card comprises a substrate carrying a photo-reflective material distributed in a predetermined pattern and configured to reflect infrared light, and wherein the motor vehicle further comprises: a stimulating light source activated by the tracking controller to enhance tracking of the location of the NFC card, wherein the stimulating light source is configured to emit infrared light.The motor vehicle of claim 6, wherein the image sensor consists of an eye tracking camera and wherein the stimulating light source is integrated into the eye tracking camera.The motor vehicle of claim 1, wherein the tracking controller is further configured to identify one or more improper storage locations of the motor vehicle and is configured to trigger the user interface to generate the message if the token is tracked to the one or more improper storage locations.The motor vehicle of claim 8, wherein the improper storage locations consist of a storage container, a cup holder, and a console tray.The motor vehicle of claim 1, wherein the tracking controller is further configured to detect a user action indicating that the user exits the motor vehicle and is configured to instruct the user interface to generate the message when the user exits.A method of monitoring a security token used in a motor vehicle, wherein a near-field communication (NFC) reader defines a touch point in a passenger compartment configured to communicate with the security token carried by a user of the motor vehicle to authorize access to predetermined functions of the motor vehicle, the method comprising the steps of: capturing images within the passenger compartment using an image sensor; detecting presentation of the security token to the NFC reader; tracking a location of the token in the captured images after being presented to the touch point; identifying one or more predetermined carrier locations associated with the user that may remain with the user when the user leaves the motor vehicle; and generating a message to the user as a reminder if it has not been detected that the token has been returned to an identified predetermined carrier location after the token has been presented to the touch point.The method of claim 11, wherein the one or more predetermined wearer locations consist of a pocket of a user's garment and a carry pocket of the user.The method of claim 11, wherein the token consists of a near-field communication (NFC) card or an NFC-enabled mobile phone.The method of claim 13, wherein the NFC card or mobile phone capable of NFC comprises a substrate carrying a photoluminescent material, and wherein the method further comprises the steps of: activating a light source to stimulate the photoluminescent material to improve tracking of the location of the NFC card or mobile phone capable of NFC.The method of claim 14, wherein the photoluminescent material is comprised of a phosphor, and wherein the light source is comprised of a blue light LED near the touch point that is activated when the NFC card or the NFC-enabled mobile phone is presented to the touch point.

Images