Secure card tracking and advisory system for motor vehicles

Abstract

The present disclosure provides a security card tracking and advisory system for a motor vehicle. A security token for accessing a motor vehicle, such as an NFC access card or an NFC-enabled mobile phone, is monitored to help prevent it from being left in the vehicle when a user leaves the vehicle. An NFC reader in the passenger compartment communicates with the token to authorize access to a predetermined function of the vehicle. After the security token is presented to the NFC reader, an image sensor captures an image within the passenger compartment using an image sensor. A location of the token is tracked in the captured image. One or more predetermined vehicle locations are identified that are associated with the user and that can be held together (e.g., a pocket or wallet) with the user when the user leaves the vehicle. When the security token has not been detected to have returned to the identified predetermined vehicle location, a message is generated to the user as a reminder.

Description

[0001] Cross - Reference to Related Applications

[0002] None.

[0003] Statement Regarding Federally Sponsored Research

[0004] None. Background Art

[0005] The present invention generally relates to motor vehicle security systems that utilize access cards, and more particularly, to a system for tracking the physical location of an access card to prevent inadvertently leaving the access card in the vehicle when exiting.

[0006] Traditionally, the mechanical keys used to unlock the access doors of motor vehicles and to unlock the steering wheel and start the propulsion unit (e.g., an internal combustion engine or an electric vehicle powertrain) have been gradually replaced by passive and active electronic devices. Active devices include key fobs and smart phones (e.g., phone - as - a - key) having an independent power source. Passive devices include transponders, such as near - field communication (NFC) access cards.

[0007] These electronic devices can wirelessly authenticate a user to obtain access to the vehicle (e.g., unlock the vehicle doors), start the vehicle (e.g., ignite a combustion engine or start an electric vehicle powertrain), and perform other functions.

[0008] NFC devices are advantageous because their very short transmission range makes it difficult for third parties to intercept and copy security data. Many smart phones include NFC hardware, such that a suitable application installed on the smart phone can be configured to operate as the user's smart key. Additionally, dedicated NFC - based access cards that are small and thin (e.g., credit - card sized) are becoming popular. NFC access cards can be configured to obtain their power from the vehicle during a read operation, or can include an independent power source. Digital key protocols have been developed to maintain vehicle security while enabling the sharing and management of vehicle access. For example, the Car Connectivity Consortium has released digital key standards.

[0009] In some embodiments, an NFC-based security system includes a portable NFC access card carried by a user and a fixed NFC unit (i.e., an NFC reader) installed in a vehicle. For example, an NFC reader for obtaining access to a vehicle (e.g., for unlocking a vehicle door) may be installed behind a B-pillar trim (e.g., on a vertical pillar between the windows of a front door and a rear door on one side of the vehicle). Once a user (e.g., a driver) has obtained access to the passenger compartment of the vehicle, the NFC access card can be used as a key for accessing security functions (such as starting the vehicle) and for differentiating one authorized user from another (e.g., for adopting customized settings, such as seat position). Thus, for example, another NFC reader may be provided on a dashboard or a center console within the passenger compartment.

[0010] One or more display panels are typically installed on a dashboard or an instrument panel of a vehicle as part of a human-machine interface (HMI). Touchscreen display panels typically create "soft" menu buttons for collecting user input. Display panels are also becoming available where wireless antennas (including NFC antennas) have been integrated into the surface of the display. The antennas can remain hidden and have a negligible impact on the display surface itself. Since the read operation will only briefly obscure the display panel, including an NFC antenna for an NFC reader in the display panel provides the functionality of an NFC key without having to monopolize space on any interior surface of the vehicle and allows for a common location across the product lines of vehicle manufacturers.

[0011] For use when starting a vehicle, an NFC access card must be handled by a user to place it near an NFC reader (e.g., parallel to the reader surface and within about 4 centimeters of the reader surface). Thus, the access card is typically removed from the user's pocket, wallet, or purse to present it to the NFC reader. If the access card is not immediately returned to the pocket, wallet, or purse, the access card may be left in the vehicle when the user reaches their destination and exits the vehicle. Leaving the access card in the vehicle creates a security issue. Additionally, if the vehicle does not have an exterior door keypad or if the driver or user does not know the keypad code, the driver or user may be locked out of the vehicle if they rely solely on the NFC access key and the NFC access key has been left inside the vehicle. Further, some countries have regulations regarding leaving access keys in a car, and insurance may not cover losses due to vehicle theft if the access key is left in the vehicle. The same problem can occur when an NFC-enabled mobile phone is left inside. Specifically, the mobile phone may not include all the capabilities of a phone-as-a-key but only have simple NFC-based access and start functions, which may also make it more likely to forget the NFC phone when leaving the vehicle. As used herein, reference to a security token or NFC access card is intended to include any NFC access device (NFC card, NFC bracelet, NFC phone, etc.). Summary of the Invention

[0012] In one aspect of the invention, a motor vehicle having a passenger compartment includes a near field communication (NFC) reader that defines a touch point in the passenger compartment, the touch point being configured to communicate with a token carried by a user of the vehicle to authorize access to a predetermined function of the motor vehicle when the token is presented to the touch point. An image sensor is configured to capture an image inside the passenger compartment. A tracking controller coupled to the image sensor and the NFC reader is configured to track the position of the token in the captured image after the token has been presented to the touch point. The tracking controller is configured to identify one or more predetermined vehicle locations associated with the user that may remain with the user when the user exits the motor vehicle. When the tracking controller determines after the token has been presented to the touch point that the token has not been detected as having returned to the identified predetermined vehicle location, a user interface coupled to the tracking controller generates a message as a reminder to the user. Brief Description of the Drawings

[0013] Figure 1 is a schematic illustration of a user with an NFC access card that enables access to and use of a vehicle based on an NFC reader on the vehicle.

[0014] Figure 2 It is a schematic diagram showing a vehicle passenger compartment including an NFC reader.

[0015] Figure 3 It is a schematic diagram showing a tracking area inside the passenger compartment, which is used to monitor the placement of an NFC access card after starting the vehicle.

[0016] Figure 4 It is a block diagram showing an embodiment of a vehicle system for tracking the proper storage after using an NFC access card.

[0017] Figure 5 It is a front view of an NFC access card according to an embodiment of the present invention.

[0018] Figure 6 It is Figure 5 a side view of the access card.

[0019] Figure 7 It is a flowchart showing a preferred method for tracking an NFC access card inside the passenger compartment according to the present invention. Detailed Description

[0020] In response to the vehicle controller detecting that an access token has been used to start the vehicle, monitoring for misplacement of the access token (e.g., an NFC access card) can be initiated. When presenting the token to the internal reader, the token can be easily detected on an image captured in the vehicle passenger compartment, and then tracked when the user puts away the token. The NFC card can be easily seen in the captured image because people usually hold the NFC card by the edge to keep the NFC card substantially parallel to the reader surface. For tracking purposes, artificial intelligence (AI) and / or machine learning (ML) tools can be used to analyze the camera data to follow the NFC card (when held in the user's hand) to prompt the card to be put back into the wallet or pocket rather than stored on the center console, cup holder, or storage container, where the card may be left in the vehicle when the user later leaves the vehicle.

[0021] In some embodiments, when the card is not immediately returned to a pocket or wallet, an immediate reminder can then be generated to notify the user (e.g., via a message, such as audio playback via an internal or external speaker, a text box on a touchscreen display panel, or a similar message sent to the user's smart phone). In some embodiments, when the user places the NFC card in an inappropriate location (e.g., a cup holder without an NFC reader), generation of the reminder message can be postponed until the driver's door is opened. In some embodiments, the user can be provided with the option to dismiss the reminder message and initiate an adjustment or reconfiguration of the vehicle security system to prevent unauthorized use of the security token left in the vehicle. For example, access to the token can be revoked. For example, the revocation can be temporary until the user restores the token using a security applet on the smart phone. In some embodiments, there can be a fleet manager who controls the security of a fleet of vehicles and can be automatically notified when a security token is left in a vehicle. The fleet manager can determine whether to revoke access to the abandoned token.

[0022] When the token is presented to the NFC reader, the visibility of the security token (e.g., NFC card) can be further enhanced by applying a long - afterglow phosphor and illuminating a blue LED integrated with the reader to excite the phosphor. For example, the blue LED can be mounted on the surface of the NFC reader. This will immediately cause the card to glow, making it easier for a camera to track even in the dark. Alternatively, light - reflecting materials distributed in a predetermined pattern on the token can be used to enhance tracking, and the light - reflecting materials are configured to reflect infrared light. By continuously activating an infrared light source during tracking, the position of the token can be more reliably determined.

[0023] Reference Figure 1 , user 11 can access vehicle 10 through the interaction between vehicle interface 12 and mobile security token 13 (e.g., NFC card or wireless transponder or tag). Interface 12 can be an NFC reader that includes an NFC antenna to perform near - field communication between a vehicle security controller (not shown) and security token 13 to unlock vehicle 10. Once access to the interior passenger compartment of vehicle 10 is available, security token 13 can also be presented to internal NFC reader 15 to access additional functions, such as starting vehicle 10.

[0024] In Figure 2In this case, the passenger compartment 15 of the vehicle is shown as having a dashboard 16, on which an NFC reader 14 is mounted to support easy access for placing the security token 13 adjacent to the NFC reader 14. The passenger compartment 15 also includes a center console 17 having cup holders 18. The console 17 may include a storage shelf 20, where the shelf 20 and the cup holders 18 represent examples of inappropriate storage locations where the security token 13 may be placed after being presented to the NFC reader 14. The dashboard 16 may also include a storage bin 21 (i.e., glove box), which represents another inappropriate storage location where the security token 13 may be placed and then inadvertently left in the passenger compartment 15 when the user exits the vehicle. The NFC reader 14 has a touch point 22 that is close to the NFC antenna for communicating with the token 13. The NFC reader 14 may also include: an LED 23 for generating light according to the wavelength for exciting the photoluminescent material on the security token 13; or an LED 24 that is configured to generate infrared light for interacting with the light-reflective material on the token 13, as described below. The dashboard 16 may also include a human-machine interface 25, which may include a display panel and an audio transducer known in the art.

[0025] In some embodiments of the present invention, various regions within the passenger compartment may be defined based on whether a particular location represents an appropriate or inappropriate storage location for the security token. As Figure 3 shown, when the security token 13 resides in the starting region 30 around the NFC reader 14, tracking of the security token begins. When the user removes the token 13 from the NFC reader 14, it enters the open region 31, which includes intermediate positions where the token cannot be stored (e.g., hanging in the passenger compartment without any support surface). One or more appropriate storage regions 32 may be defined based on static or dynamic elements within the passenger compartment that are associated with the user and can be expected to leave the vehicle with the user. Appropriate storage locations 32 may include the pockets of the user's clothing or carried containers such as wallets, backpacks, or briefcases. Region 33 represents an inappropriate location such as a cup holder. In some instances, an inappropriate storage region 34 (such as the surface of the passenger seat) may surround an appropriate storage location 35, which may be, for example, a wallet placed on the seat.

[0026] Figure 4More particularly, the vehicle system of the present invention is shown. Vehicle 10 includes a main controller 40, which may be composed of a programmable microprocessor configured according to the present invention. The controller 40 is coupled to an NFC reader 14 for communicating with a security token 13 held by a user 11 when the user is inside the vehicle 10. The user 11 has a pocket 41 in the clothing being worn, which represents one of a plurality of predetermined vehicle locations where the user would expect to place the security token 13 such that when the user leaves the vehicle 10, the security token 13 will stay with the user. The predetermined vehicle locations may also include carried storage devices such as wallets, briefcases, backpacks or pockets in clothing that are not currently being worn but are brought into the vehicle with the user and are expected to be removed when leaving (e.g., a jacket). The classification and identification of such predetermined vehicle locations can be handled using artificial intelligence or machine learning tools. The selection and configuration of such tools are within the scope of those of ordinary skill in the art.

[0027] The controller 40 is connected to an LED 23 associated with the NFC reader 14 and is connected to an image sensor or camera 42 that may also be associated (e.g., integrated) with the NFC reader 14. For example, the camera 42 may be a dedicated camera for tracking the security token 13. Additionally or alternatively, the controller 40 may be coupled to a camera 43 installed in the passenger compartment, which may be present for performing other functions such as driver eye tracking, drowsiness detection or occupant detection. When the camera 43 operates to perform eye gaze detection, it may utilize infrared illumination provided by an infrared LED 44 that is also coupled to the controller 40. Images captured by the camera 43 may also be used to track the security token 13 and to compare the position of the token 13 with potential vehicle / storage locations provided by the user's clothing or equipment and by the vehicle surface (inappropriate locations) with the appropriate locations that stay with the user when the user leaves the vehicle 10.

[0028] The controller 40 is coupled to a human-machine interface 35, which may include a touchscreen display panel, a speaker and / or a microphone for exchanging messages with the user 11. The controller 40 is also coupled to a wireless transceiver 46 having an antenna 47, which is configured to interact with a smart phone 50 carried by the user 11, thereby providing an alternative channel for generating messages to the user 11 using the display screen or speaker of the smart phone 50. As previously mentioned, in some embodiments, the smart phone 50 may also include a security token 13 that serves as the key to the vehicle. In these cases, an alternative channel for generating messages to the user 11 may be utilized, such as vehicle internal or external speakers, internal or external display screens and / or alternative smart phones detected by the wireless transceiver 46.

[0029] Preferably, the transceiver 46 may be capable of via A BUN transceiver module that interacts with communication, ultra-wideband (UWB) communication, and NFC communication. When generating a message for user 11, It can be particularly used to link the controller 40 to an applet in the smart phone 50. The transceiver 40 can also include a cellular data link to the cellular system 51 to provide communication between the controller 40 and the fleet manager 52 through the data network 53.

[0030] The controller 40 is coupled to a card status memory 55, which can store data corresponding to a set of uniquely identifiable NFC cards and the status of corresponding access permissions. The door switch 56 is coupled to the controller 40 to provide a signal when the user 11 opens the driver's door to prepare to leave the vehicle 10. In some embodiments, a driver seat sensor (not shown) can also be used to detect whether the door can be opened but the driver remains seated. Thus, the message can be delayed or suspended until the intention of the driver to leave the vehicle is verified.

[0031] The security token 13 can be modified to facilitate camera image-based tracking. As Figure 5 and Figure 6 shown, the token 13 can consist of an NFC access card having an upper substrate 60 and a lower substrate 61. Each substrate can be formed of a plastic sheet. The plastic can be at least partially translucent. As is well known, the NFC controller 62 and the NFC antenna 63 can be sandwiched between the substrate 60 and the substrate 61. The luminescent material 64 can be coated or impregnated on one or both of the substrates 60 and 61. A long afterglow phosphor formulation (e.g., CaAl 2 O 4 :Eu 2+ ,Nd 3+ ) is preferably used so that the glow from the excited phosphor can help track the NFC card 13 for at least a few seconds or minutes. The translucent portions of the substrates 60 and 61 can be arranged adjacent to the luminescent material 64 to facilitate excitation by an external light source (e.g., Figure 4 the blue LED 23 in

[0032] With or without the luminescent material 64, the token 13 can be provided with a light-reflective material 65 that is distributed in a predetermined pattern and configured to reflect incident light. The incident light can be generated by an LED using a predetermined spectrum, such as infrared light. In some embodiments, an infrared LED is utilized that is present within a gaze tracking system in a vehicle, the gaze tracking system being intended to monitor a driver's gaze direction for driver assistance and driver assessment (since such systems typically operate based on the use of infrared light). Thus, the image sensor for tracking the security token can consist of an eye tracking camera, and the stimulation light source can be an infrared-generating LED integrated with the eye tracking camera. Alternatively, a separate dedicated infrared-generating LED source can be installed such that it is directed at the NFC reader location.

[0033] In embodiments where an NFC phone is used as a security token, the photoluminescent or light-reflective configurations discussed above can be similarly applied to the device. For example, a luminescent material or a light-reflective material can be integrated into or coated on the phone's back panel or phone housing. Alternatively, an adhesive sheet can be applied to the back of the NFC phone.

[0034] Figure 7 A preferred method of the present invention is shown. In step 70, a check is performed to determine whether an NFC card has been used to access vehicle functions (e.g., accessory mode or vehicle start) via an internal NFC reader. After detecting that the NFC card has been used in this manner, auxiliary lighting is activated in step 71. The auxiliary lighting can include one or more pulses of the stimulation light source, such as a blue LED used when the NFC access card carries a photoluminescent material, and / or continuous activation of an infrared light source when reflective infrared tracking is to be employed. In step 72, the NFC card is positioned while still adjacent to the touch point of the NFC reader, and then the image stream captured by the image sensor is continuously monitored to track the movement of the NFC access card.

[0035] In step 73, a check is performed to determine whether the card has been placed in a conventional vehicle (i.e., a storage location), such as a predetermined pocket in the driver's clothing or other predetermined vehicle locations, such as a wallet or backpack. If the NFC access card has been placed as such, the method is completed at step 74. Otherwise, the location of the NFC access card continues to be tracked, and in step 75, a check is performed to determine whether the access card is placed inappropriately. Inappropriate locations may include cup holders, console shelves, or other vehicle surfaces without an NFC reader that may enable the user to leave the vehicle without remembering to carry the NFC access card with them. If step 75 determines that the card is placed inappropriately, a message is generated in step 77 as a reminder that the token has not been detected as having returned to any of the appropriate vehicle locations (e.g., pocket or wallet) that may have been recognized by the controller. Such a message may include a text display on a display panel, an audio message on a speaker, or a transmission to the user's mobile wireless device or smartphone.

[0036] If it has not been detected in step 75 that the card has been placed in an inappropriate location, a check is performed in step 76 to determine whether too much time has elapsed while waiting for the NFC access card to be placed somewhere. The purpose of waiting for a predetermined amount of time is to avoid tracking errors or recover from tracking errors. If the predetermined time has not expired, return to step 73. Otherwise, a user interface may be initiated in step 77 to generate a corresponding message to the user.

[0037] In some embodiments, an option may be provided to the user to dismiss the reminder message and prevent further reminders from being generated. For example, a "dismiss" button may be provided on a touchscreen display. A check is performed in step 78 to determine whether the user has pressed the dismiss button. If not, further subsequent messages may be generated (e.g., when leaving the vehicle). Thus, a check is performed in step 80 to determine whether the driver's door is opened as an indication that the user is ready to leave. Once the door opening is detected, the reminder message may be repeated in step 81.

[0038] If the user does indeed select to dismiss the message in step 78, a check can be performed in step 82 to determine whether there is a fleet manager who can have primary authority over the vehicle and is empowered to decide whether further action should be taken with respect to the discarded NFC access card. If there is no fleet manager, the method can end at step 83. Otherwise, a message can be sent to the fleet manager in step 84 notifying the fleet that the driver of the vehicle has left the access card in the unattended vehicle. In response to the message sent in step 84, the fleet manager can determine that the access card should be deactivated and made inoperable for operating the vehicle until further corrective action is taken by the user or the fleet manager. The fleet manager can initiate a message to the controller to update the status of the NFC access card and render it invalid.

[0039] According to an embodiment of the present invention, the token consists of a mobile phone having a near field communication (NFC) function.

[0040] According to an embodiment of the present invention, the mobile phone includes a substrate or an additional housing or an additional label carrying a photoluminescent material or a light-reflective material.

[0041] In one aspect of the present invention, the NFC card or the mobile phone having NFC function includes a substrate carrying a light-reflective material, the light-reflective material being distributed in a predetermined pattern and configured to reflect infrared light, and wherein the method further comprises the step of activating an infrared light source to enhance the tracking of the position of the NFC card or the mobile phone having NFC function.

[0042] In one aspect of the present invention, the method includes: identifying one or more inappropriate storage locations of the motor vehicle; wherein if the token is tracked to the one or more inappropriate storage locations, the message is generated.

[0043] In one aspect of the present invention, the inappropriate storage locations consist of a storage bin, a cup holder, and a console shelf.

Claims

1. A motor vehicle having a passenger compartment, the motor vehicle comprising: a near field communication (NFC) reader defining a touch point in the passenger compartment, the touch point configured to communicate with a token carried by a user of the motor vehicle to authorize access to a predetermined function of the motor vehicle when the token is presented to the touch point; an image sensor configured to capture an image of the interior of the passenger compartment; a tracking controller coupled to the image sensor and the NFC reader, wherein the tracking controller is configured to track a position of the token in the captured image after being presented to the touch point and to identify one or more predetermined vehicle positions associated with the user that can remain with the user when the user exits the motor vehicle; as well as A user interface is coupled to the tracking controller and configured to generate a message as a reminder to the user when the tracking controller determines that the token has not been detected as having returned to the identified predetermined vehicle position after the token is presented to the touch point.

2. The motor vehicle of claim 1, wherein the one or more predetermined vehicle locations identified by the tracking controller consist of pockets of the user's clothing and the user's handbag.

3. The motor vehicle of claim 1, wherein the token is comprised of a Near Field Communication (NFC) card.

4. The motor vehicle of claim 3, wherein the NFC card comprises a substrate carrying a photoluminescent material, and wherein the motor vehicle further comprises: A stimulation light source is activated by the tracking controller to enhance the tracking of the position of the NFC card.

5. The motor vehicle of claim 4, wherein the photoluminescent material is comprised of a phosphor, and wherein the stimulus light source is comprised of a blue LED proximate to the touch point, the blue LED being activated when the NFC card is presented to the touch point.

6. The motor vehicle of claim 3, wherein the NFC card includes a substrate carrying a light reflective material, the light reflective material being distributed in a predetermined pattern and configured to reflect infrared light, and wherein the motor vehicle further includes: A stimulation light source is activated by the tracking controller to enhance the tracking of the position of the NFC card, wherein the stimulation light source is configured to emit infrared light.

7. The motor vehicle of claim 6, wherein the image sensor consists of an eye-tracking camera, and wherein the stimulus light source is integrated with the eye-tracking camera.

8. A motor vehicle as described in claim 1, wherein the tracking controller is further configured to identify one or more inappropriate storage locations of the motor vehicle and is configured to trigger the user interface to generate the message if the token is tracked to the one or more inappropriate storage locations.

9. The motor vehicle of claim 8, wherein the inappropriate storage location is comprised of a storage bin, a cup holder, and a console shelf.

10. The motor vehicle of claim 1, wherein the tracking controller is further configured to detect an action of the user indicating that the user is leaving the motor vehicle, and is configured to direct the user interface to generate the message when the user is leaving.

11. A method of monitoring a security token used in a motor vehicle, wherein a vehicle near field communication (NFC) reader defines a touch point in the passenger compartment, the touch point being configured to communicate with the security token carried by a user of the motor vehicle to authorize access to a predetermined function of the motor vehicle, the method comprising the steps of: capturing an image of the interior of the passenger compartment using an image sensor; detecting presentation of the security token to the NFC reader; tracking a position of the token in the captured image after the token is presented to the touch point; identifying one or more predetermined vehicle locations associated with the user, the one or more predetermined vehicle locations being capable of remaining with the user when the user exits the motor vehicle; as well as When the token has not been detected as having returned to the identified predetermined carrier position after the token is presented to the touch point, a message is generated to the user as a reminder.

12. The method of claim 11, wherein the one or more predetermined carrier locations consist of pockets of the user's clothing and a handbag of the user.

13. The method of claim 11, wherein the token consists of a Near Field Communication (NFC) card or a mobile phone with NFC functionality.

14. The method of claim 13, wherein the NFC card or the mobile phone with NFC function comprises a substrate carrying a photoluminescent material, and wherein the method further comprises the following steps: A light source is activated to excite the photoluminescent material to enhance tracking of the location of the NFC card or the NFC-enabled mobile phone.

15. The method of claim 14, wherein the photoluminescent material is comprised of a phosphor, and wherein the light source is comprised of a blue LED proximate to the touch point, the blue LED being activated when the NFC card or the NFC-enabled mobile phone is presented to the touch point.

Images