System and method for unlocking a vehicle based on visual domain data and wireless domain data

Abstract

A method is provided for unlocking a vehicle with a handheld device by a user based on visual domain and radio frequency (RF) domain technologies. The method includes activating onboard equipment of the vehicle when the user's handheld device is within a threshold distance from the vehicle. The method further includes comparing the user's visual domain position based on visual domain data with the user's RF domain position based on RF domain data using a first equation and a second equation. The method further includes confirming the user when the first and second equations are true, and unlocking the vehicle after confirming the user.

Description

Technical Field

[0001] This disclosure relates to remotely unlocking vehicles, and more specifically to systems and methods for unlocking vehicles based on visual domain data and wireless domain data. Background Technology

[0002] The current systems and methods for unlocking vehicles are sufficient. However, one challenge is to more easily detect the user of a vehicle when they approach it. Another challenge is to more securely verify the user's identity for vehicles associated with that user. Summary of the Invention

[0003] Therefore, while current systems and methods for unlocking vehicles achieve their intended purpose, a new system and method are needed to more easily detect users and more securely verify their identity to unlock vehicles remotely.

[0004] According to one aspect of this disclosure, a method is provided for unlocking a vehicle with a handheld device by a user based on visual and radio frequency (RF) domain technologies. The method includes activating the vehicle's onboard equipment when the user's handheld device is within a threshold distance from the vehicle. In this aspect, the method further includes unlocking a vehicle via a first equation. Second Equation Compare the visual domain location of a user based on visual domain data with the RF domain location of a user based on RF domain data.

[0005] In this regard, It is the radial distance between the user and the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle relative to the RF domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is the distance consistency threshold used to confirm the distance consistency between visual domain data and RF domain data, and It is the angle consistency threshold for confirming the angular consistency between visual domain data and RF domain data.

[0006] Furthermore, the method also includes confirming the user when the first and second equations are true and unlocking the vehicle after confirming the user.

[0007] According to one example, the activation step includes monitoring the vehicle's location and the user's handheld device's location based on an intentional detection module. The activation step also includes detecting whether the handheld device is within a threshold distance of the vehicle.

[0008] In another example, the comparison step includes verifying the user's facial identity based on the user's visual domain data and estimating the user's visual domain position based on the user's visual domain data. The comparison step further includes determining the vehicle's RF domain position based on the vehicle's RF domain data. In this example, the comparison step further includes determining the user's RF domain position based on the handheld device's RF domain data.

[0009] In yet another example, the comparison step further includes detecting a non-user with a non-user device based on non-user visual domain data and non-user device RF domain data to define the non-user's visual domain location and RF domain location. The comparison step also includes using a third-party program... Fourth equation Fifth equation and the sixth equation Compare the user's visual field position and RF field position with the non-user's visual field position and RF field position.

[0010] In this embodiment, It is the non-user-to-vehicle radial distance relative to the vehicle's coordinate system, based on visual domain data. It is the non-user-to-vehicle radial distance relative to the vehicle's coordinate system, based on wireless domain data. It is a third-party positional angle from the user to the vehicle relative to the coordinate system, based on visual domain data. It is the fourth azimuth angle from the non-user to the vehicle relative to the coordinate system, θ, based on wireless domain data. d It is a threshold for inter-target distance differences used to identify the distance differences between users and non-users around a vehicle. It is the threshold for inter-target angular difference used to identify the angular differences between users and non-users around a vehicle.

[0011] In this example, the comparison step further includes the step of verifying the user's identity when the third, fourth, fifth, and sixth equations are true.

[0012] In another example, the method also includes performing a series of actions inside the vehicle based on the user's personalized settings after the vehicle is unlocked.

[0013] In yet another example, the method further includes detecting the user's facial identity and visual domain location based on the user's visual domain data when the user is in the vehicle. In this example, the method further includes verifying the user's facial identity based on the user's visual domain data when the user is in the vehicle.

[0014] In another example, the method further includes detecting a companion user with a companion user device based on the companion user's visual domain data and the companion user's device's RF domain data, to define the companion user's visual domain location and RF domain location. In this example, the method further includes performing a series of actions within the vehicle based on the companion user's personalized settings after unlocking the vehicle.

[0015] In another example, the step of verifying a user's facial identity includes evaluating the image data quality of the user's facial identity based on the user's visual domain data, and determining a confidence threshold based on the image quality of the user's facial identity. The verification step further includes processing the user's visual domain data with a facial recognition module to define a confidence score for the user's facial identity. Additionally, the verification step includes confirming the user's facial identity with an authentication module when the confidence score is greater than the confidence threshold.

[0016] According to another aspect of this disclosure, a system is provided for unlocking a vehicle with a handheld device for a user based on visual domain and radio frequency (RF) domain technologies. The system includes in-vehicle equipment and a cloud server. The in-vehicle equipment includes an electronic control unit (ECU) and an external camera disposed within the vehicle. In this respect, the external camera is disposed on the vehicle and communicates with the ECU. Moreover, the external camera is arranged to detect the user's facial identity and visual domain location based on the user's visual domain data.

[0017] In this regard, the in-vehicle equipment includes a wireless positioning transmitter installed in the vehicle and communicating with the ECU. The wireless positioning transmitter is arranged to detect the vehicle's RF domain location based on the vehicle's RF domain data. Furthermore, the wireless positioning transmitter is arranged to detect the user's RF domain location based on the RF domain data of a handheld device.

[0018] In this respect, the cloud server is located away from the vehicle setup and communicates with the ECU and handheld device. The cloud server is positioned to communicate via the first equation. Second Equation Compare the user's visual domain location with the user's RF domain location.

[0019] In this regard, It is the radial distance between the user and the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle based on wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is the distance consistency threshold used to confirm the distance consistency between visual domain data and RF domain data, and This is the angular consistency threshold for confirming the angular consistency between visual domain data and RF domain data.

[0020] The cloud server is configured to verify the user when the first and second equations are true. Furthermore, the cloud server is configured to unlock the vehicle upon user verification. Additionally, the ECU is configured to activate the vehicle's external cameras and wireless positioning transmitter when the user's handheld device is within a threshold distance of the vehicle.

[0021] In one embodiment, the ECU is configured to monitor the vehicle's location. Furthermore, a cloud server is configured to monitor the location of the user's handheld device based on an intentional detection system. Additionally, the cloud server is configured to detect whether the handheld device is within a threshold distance of the vehicle.

[0022] In another embodiment, the cloud server is configured to verify the user's facial identity based on the user's visual domain data. Furthermore, the cloud server is configured to estimate the user's visual domain location based on the user's visual domain data. Additionally, the cloud server is configured to determine the vehicle's RF domain location based on the vehicle's RF domain data. Moreover, the cloud server is configured to determine the user's RF domain location based on the handheld device's RF domain data.

[0023] In yet another embodiment, the cloud server is configured to detect a non-user with a non-user device based on non-user visual domain data and non-user device RF domain data, in order to define the non-user's visual domain location and RF domain location. Furthermore, the cloud server is configured to communicate via a third-party program. Fourth equation Fifth equation and the sixth equation Compare the user's visual field position and RF field position with the non-user's visual field position and RF field position.

[0024] In this embodiment, It is the non-user-to-vehicle radial distance relative to the vehicle's coordinate system, based on visual domain data. It is the non-user-to-vehicle radial distance relative to the vehicle's coordinate system, based on wireless domain data. It is a third-party positional angle from the user to the vehicle relative to the coordinate system, based on visual domain data. It is the fourth azimuth angle from the non-user to the vehicle relative to the coordinate system, θ, based on wireless domain data. d It is a threshold for inter-target distance differences used to identify the distance differences between users and non-users around a vehicle. This is the target angle difference threshold used to identify the angle differences between users and non-users around the vehicle. Furthermore, the cloud server is configured to continue verifying the user's identity as long as the third, fourth, fifth, and sixth equations are true.

[0025] In yet another embodiment, the vehicle's onboard equipment further includes an internal camera located within the vehicle and communicating with the ECU. Moreover, the internal camera is configured to detect the user's facial identity and visual field location based on the user's visual field data when the user is in the vehicle. Additionally, a cloud server is configured to verify the user's facial identity based on the user's visual field data when the user is in the vehicle.

[0026] In another embodiment, the cloud server is configured to perform a series of actions within the vehicle based on the user's personalized settings after the vehicle is unlocked.

[0027] In yet another embodiment, the in-vehicle device further includes an internal camera disposed within the vehicle and communicating with the ECU. The internal camera is arranged to detect a companion user with a companion user device based on the companion user's visual domain data and the companion user device's RF domain data, to define the companion user's visual domain location and RF domain location. In this embodiment, a cloud server is arranged to perform a series of actions within the vehicle based on the companion user's personalized settings after the vehicle is unlocked.

[0028] According to another aspect of this disclosure, a method is provided for unlocking a vehicle with a handheld device belonging to a user based on visual and radio frequency (RF) domain technologies. The method includes monitoring the location of the vehicle and the user's handheld device using an intentional detection system and detecting whether the handheld device is within a threshold distance from the vehicle. In this aspect, the method further includes activating the vehicle's onboard equipment when the handheld device is within the threshold distance from the vehicle and verifying the user's facial identity based on the user's visual domain data.

[0029] In this regard, the method further includes estimating the user's visual domain position based on the user's visual domain data, determining the vehicle's RF domain position based on the vehicle's RF domain data, and determining the user's RF domain position based on the handheld device's RF domain data.

[0030] Moreover, in this respect, the method also includes a first comparison. Second comparison Compare the user's visual domain location with the user's RF domain location. In this regard, It is the radial distance between the user and the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle relative to the wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold that helps confirm the distance consistency between visual domain data and RF domain data, and It is an angle consistency threshold that helps confirm the angle consistency between visual domain data and RF domain data.

[0031] In addition, the method further includes confirming the user when the first and second comparisons are true, and unlocking the vehicle after confirming the user.

[0032] In another example of this disclosure, the comparison step further includes detecting a non-user with a non-user device based on non-user visual domain data and non-user device RF domain data to define the non-user's visual domain location and RF domain location. In this example, the comparison step includes using a third-party program... Fourth equation Fifth equation and the sixth equation Compare the user's visual field position and RF field position with the non-user's visual field position and RF field position.

[0033] In this embodiment, It is the non-user-to-vehicle radial distance relative to the vehicle's coordinate system, based on visual domain data. It is the non-user-to-vehicle radial distance relative to the vehicle's coordinate system, based on wireless domain data. It is a third-party positional angle from the user to the vehicle relative to the coordinate system, based on visual domain data. It is the fourth azimuth angle from the non-user to the vehicle relative to the coordinate system, θ, based on wireless domain data. d It is a threshold for inter-target distance differences used to identify the distance differences between users and non-users around a vehicle. It is the threshold for inter-target angular difference used to identify the angular differences between users and non-users around a vehicle.

[0034] In this example, the comparison step further includes the step of verifying the user's identity when the third, fourth, fifth, and sixth equations are true.

[0035] In another example, the method also includes performing a series of actions inside the vehicle based on the user's personalized settings after the vehicle is unlocked.

[0036] In yet another example, the method further includes detecting a companion user with a companion user device based on the companion user's visual domain data and the companion user's device's RF domain data, to define the companion user's visual domain location and RF domain location. Additionally, the method further includes performing a series of actions within the vehicle based on the companion user's personalized settings after unlocking the vehicle.

[0037] In yet another example of this disclosure, the step of verifying a user's facial identity includes evaluating the image data quality of the user's facial identity based on the user's visual domain data, and determining a confidence threshold based on the image quality of the user's facial identity. Furthermore, the verification step includes processing the user's visual domain data with a facial recognition module to define a confidence score for the user's facial identity. Additionally, the verification step includes verifying the user's facial identity with an authentication module when the confidence score is greater than the confidence threshold.

[0038] The present invention has the following solutions.

[0039] Solution 1. A method for unlocking a vehicle with a handheld device based on visual domain and radio frequency (RF) domain technologies, the method comprising:

[0040] When the user's handheld device is within a threshold distance from the vehicle, the vehicle's onboard equipment is activated; via the first equation

[0041]

[0042] Second Equation

[0043]

[0044] The user's visual domain position based on visual domain data is compared with the user's RF domain position based on RF domain data.

[0045] in It is the radial distance of the user from the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle relative to the RF domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold used to confirm the distance consistency between the visual domain data and the RF domain data, and It is the angle consistency threshold that confirms the angle consistency between the visual domain data and the RF domain data;

[0046] The user is confirmed when the first and second equations are true; and

[0047] The vehicle is unlocked after the user is confirmed.

[0048] Option 2. The method as described in Option 1, wherein the activation step includes:

[0049] Based on the intentional detection module, the location of the vehicle and the location of the user's handheld device are monitored; and

[0050] Detect whether the handheld device is within the threshold distance from the vehicle.

[0051] Option 3. The method as described in Option 1, wherein the comparison step includes:

[0052] Verify the user's facial identity based on the user's visual domain data;

[0053] Estimate the user's visual domain position based on the user's visual domain data;

[0054] The RF domain location of the vehicle is determined based on the vehicle's RF domain data; and

[0055] The user's RF domain location is determined based on the RF domain data of the handheld device.

[0056] Option 4. The method as described in Option 3, wherein the comparison step further includes:

[0057] Detect non-users with non-user devices based on non-user visual domain data and non-user device RF domain data to define the visual domain location and RF domain location of the non-users;

[0058] Through a third-party process

[0059]

[0060] Fourth equation

[0061]

[0062] Fifth equation

[0063]

[0064] Sixth equation

[0065]

[0066] The visual domain position and RF domain position of the user are compared with the visual domain position and RF domain position of the non-user, wherein It is the radial distance from the non-user to the vehicle relative to the coordinate system, based on visual domain data. It is the radial distance from the non-user to the vehicle relative to the coordinate system, based on wireless domain data. It is the azimuth angle from the non-user to the vehicle relative to the coordinate system, based on visual domain data. It is the azimuth angle θ from the non-user to the vehicle relative to the coordinate system, based on wireless domain data. dIt is a target distance difference threshold used to identify the distance difference between the user and the non-user around the vehicle. It is a threshold for inter-target angular difference used to identify the angular differences between the user and non-user around the vehicle; and

[0067] If the third, fourth, fifth, and sixth equations are true, the user's steps are then confirmed.

[0068] Option 5. The method as described in Option 1, further comprising performing a series of actions within the vehicle based on the user's personalized settings after the vehicle is unlocked.

[0069] Option 6. The method as described in Option 1, further comprising:

[0070] When the user is in the vehicle, the user's facial identity and visual domain location are detected based on the user's visual domain data; and

[0071] When the user is in the vehicle, the user's facial identity is verified based on the user's visual domain data.

[0072] Option 7. The method as described in Option 1, further comprising:

[0073] Detecting companion users with companion user devices based on their visual domain data and RF domain data to define the visual domain location and RF domain location of the companion users; and

[0074] After unlocking the vehicle, a series of actions are performed inside the vehicle based on the companion user's personalized settings.

[0075] Option 8. The method as described in Option 3, wherein verifying the user's facial identity includes:

[0076] The image data quality for the user's facial identity is evaluated based on the user's visual domain data.

[0077] A confidence threshold is determined based on the image quality of the user's facial identity.

[0078] The facial recognition module processes the user's visual domain data to define a confidence score for the user's facial identity; and

[0079] When the confidence score is greater than the confidence threshold, the authentication module verifies the user's facial identity.

[0080] Option 9. A system for unlocking a vehicle with a handheld device based on visual and radio frequency (RF) domain technologies, the system comprising:

[0081] The vehicle's on-board equipment, the on-board equipment including:

[0082] Electronic control unit (ECU) installed in the vehicle;

[0083] An external camera mounted on the vehicle and communicating with the ECU, the external camera being arranged to detect the user's facial identity and visual field location based on the user's visual field data; and

[0084] A wireless positioning transmitter installed on the vehicle and communicating with the ECU, the wireless positioning transmitter being arranged to detect the RF domain location of the vehicle based on the vehicle's RF domain data, and the wireless positioning transmitter being arranged to detect the user's RF domain location based on the RF domain data of the handheld device; and

[0085] A cloud server located away from the vehicle and communicating with the ECU and the handheld device, the cloud server being arranged via a first equation

[0086]

[0087] Second Equation

[0088]

[0089] Compare the user's visual domain position with the user's RF domain position.

[0090] in It is the radial distance of the user from the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle based on wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold used to confirm the distance consistency between the visual domain data and the RF domain data, and This is an angle consistency threshold that confirms the angular consistency between the visual domain data and the RF domain data, wherein the cloud server is configured to confirm the user when the first and second equations are true.

[0091] The cloud server is configured to unlock the vehicle when the user is confirmed.

[0092] The ECU is configured to activate the vehicle's external camera and wireless positioning transmitter when the user's handheld device is within a threshold distance from the vehicle.

[0093] Option 10. The system of Option 9, wherein the ECU is arranged to monitor the location of the vehicle, wherein the cloud server is arranged to monitor the location of a user's handheld device based on an intentional detection system, and wherein the cloud server is arranged to detect whether the handheld device is within the threshold distance from the vehicle.

[0094] Solution 11. The system of Solution 10, wherein the cloud server is configured to verify the user's facial identity based on the user's visual domain data, wherein the cloud server is configured to estimate the user's visual domain position based on the user's visual domain data, wherein the cloud server is configured to determine the vehicle's RF domain position based on the vehicle's RF domain data, and wherein the cloud server is configured to determine the user's RF domain position based on the handheld device's RF domain data.

[0095] Option 12. The system as described in Option 11, wherein the cloud server is configured to detect a non-user with a non-user device based on non-user visual domain data and non-user device RF domain data, to define the visual domain location and RF domain location of the non-user, wherein the cloud server is configured to use a third-party program

[0096]

[0097] Fourth equation

[0098]

[0099] Fifth equation

[0100]

[0101] Sixth equation

[0102]

[0103] The visual domain position and RF domain position of the user are compared with the visual domain position and RF domain position of the non-user, wherein It is the radial distance from the user to the vehicle based on the vehicle's coordinate system, using visual domain data. It is the non-user radial distance θ based on the coordinate system. d It is a target distance difference threshold used to identify the distance difference between the user and the non-user around the vehicle. It is a threshold for inter-target angular difference used to identify the angular differences between users and non-users around the vehicle.

[0104] The cloud server is configured to continue verifying the user when the third, fourth, fifth, and sixth equations are true.

[0105] Option 13. The system of Option 9, wherein the on-board equipment of the vehicle further includes:

[0106] An internal camera installed in the vehicle and communicating with the ECU is configured to detect the user's facial identity and visual domain location based on the user's visual domain data when the user is in the vehicle, wherein the cloud server is configured to verify the user's facial identity based on the user's visual domain data when the user is in the vehicle.

[0107] Option 14. The system as described in Option 9, wherein the cloud server is configured to perform a series of actions within the vehicle based on the user's personalized settings after the vehicle is unlocked.

[0108] Option 15. The system as described in Option 9, wherein the on-board equipment further includes:

[0109] An internal camera installed in the vehicle and communicating with the ECU is arranged to detect a companion user with a companion user device based on the companion user's visual domain data and the companion user device's RF domain data, to define the companion user's visual domain position and RF domain position; and

[0110] The cloud server is configured to perform a series of actions within the vehicle based on the companion user's personalized settings after the vehicle is unlocked.

[0111] Solution 16. A method for unlocking a vehicle with a handheld device based on visual domain and radio frequency (RF) domain technologies, the method comprising:

[0112] The system monitors the location of the vehicle and the location of the user's handheld device based on intentional detection.

[0113] Detect whether the handheld device is within a threshold distance from the vehicle;

[0114] When the handheld device is within the threshold distance from the vehicle, the vehicle's on-board equipment is activated;

[0115] Verify the user's facial identity based on the user's visual domain data;

[0116] Estimate the user's visual domain location based on the user's visual domain data;

[0117] The RF domain location of the vehicle is determined based on the vehicle's RF domain data;

[0118] The user's RF domain location is determined based on the RF domain data of the handheld device;

[0119] Through the first comparison

[0120]

[0121] Second comparison

[0122]

[0123] Compare the user's visual domain position with the user's RF domain position.

[0124] in It is the radial distance of the user from the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle relative to the wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold that helps confirm the distance consistency between the visual domain data and the RF domain data, and It is an angle consistency threshold that helps confirm the angle consistency between the visual domain data and the RF domain data;

[0125] The user is confirmed when the first and second comparisons are true; and

[0126] The vehicle is unlocked after the user is confirmed.

[0127] Option 17. The method as described in Option 16, wherein the comparison step further includes:

[0128] Detect non-users with non-user devices based on non-user visual domain data and non-user device RF domain data to define the visual domain location and RF domain location of the non-users;

[0129] Through a third-party process

[0130]

[0131] Fourth equation

[0132]

[0133] Fifth equation

[0134]

[0135] Sixth equation

[0136]

[0137] The visual domain position and RF domain position of the user are compared with the visual domain position and RF domain position of the non-user, wherein It is the radial distance from the user to the vehicle based on the vehicle's coordinate system, using visual domain data. It is the non-user radial distance θ based on the coordinate system. d It is a target distance difference threshold used to identify the distance difference between the user and the non-user around the vehicle. It is a threshold for inter-target angular difference used to identify the angular differences between the user and non-user around the vehicle; and

[0138] If the third, fourth, fifth, and sixth equations are true, continue with the user's confirmation process.

[0139] Option 18. The method as described in Option 16, further comprising performing a series of actions within the vehicle based on the user's personalized settings after the vehicle is unlocked.

[0140] Option 19. The method as described in Option 16, further comprising:

[0141] Detecting companion users with companion user devices based on their visual domain data and RF domain data to define the visual domain location and RF domain location of the companion users; and

[0142] After unlocking the vehicle, a series of actions are performed inside the vehicle based on the companion user's personalized settings.

[0143] Option 20. The method as described in Option 16, wherein verifying the user's facial identity includes:

[0144] The image data quality for the user's facial identity is evaluated based on the user's visual domain data.

[0145] A confidence threshold is determined based on the image quality of the user's facial identity.

[0146] The facial recognition module processes the user's visual domain data to define a confidence score for the user's facial identity; and

[0147] When the confidence score is greater than the confidence threshold, the authentication module verifies the user's facial identity.

[0148] Further applications will become apparent from the description provided herein. It should be understood that the descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0149] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0150] Figure 1 This is a schematic diagram of a system for unlocking a vehicle according to an embodiment of the present disclosure.

[0151] Figure 2 Is to implement Figure 1 A schematic diagram of the vehicle's system components.

[0152] Figure 3 This is based on an example of comparing a user's visual domain location and RF domain location. Figure 2 A graphical view of the local coordinate system of the vehicle.

[0153] Figure 4 This is an example of the use of this disclosure. Figure 1 The flowchart shows the system's method for unlocking vehicles.

[0154] Figure 5 This is based on the usage of another example. Figure 1 The flowchart shows the system's method for unlocking vehicles. Detailed Implementation

[0155] The following description is exemplary in nature and is not intended to limit this disclosure, application, or use.

[0156] This disclosure provides a system and method for securely unlocking a vehicle for a user with a handheld device based on visual and wireless (e.g., radio frequency) domain technologies. The system and method more securely and accurately identify the vehicle user when the user is within a threshold distance of the vehicle. The system and method compare the user's visual domain data with the wireless domain data of the user's handheld device to more securely identify the vehicle user. Upon user confirmation, the system and method unlock the vehicle and activate the user's personalized settings within the vehicle.

[0157] Figure 1 and 2This describes a system 10 for unlocking a vehicle 12 belonging to a user 14 with a handheld device 15 using visual and wireless (e.g., radio frequency) domain technologies. As shown, system 10 includes an in-vehicle device 16 for the vehicle 12 and a cloud server 18. The in-vehicle device 16 includes an electronic control unit (ECU) 20 disposed in the vehicle 12, a wireless positioning transmitter 22 disposed in or on the vehicle 12, and external cameras 24, 25 disposed outside the vehicle 12. Preferably, the cloud server 18 communicates with the in-vehicle device 16 via the ECU 20. It should be understood that the cloud server 18 can transmit data signals 26 to and receive data signals 26 from the ECU 20.

[0158] As shown in the figure, each of the external cameras 24 and 25 and the wireless positioning transmitter 22 communicates bidirectionally with the ECU 20. That is, the external cameras 24 and 25 can transmit data signals 28 and 29 to and receive data signals 28 and 29 from the ECU 20, respectively. As described in more detail below, the external camera 24 is arranged to detect the facial identity and visual field location of the user 14 based on the user 14's visual field data. Furthermore, the wireless positioning transmitter 22 can also transmit data signal 30 to and receive data signal 30 from the ECU 20, as described below. Additionally, the cloud server 18 can transmit data signal 32 to and receive data signal 32 from the handheld device 15. Moreover, the wireless positioning transmitter 22 can transmit data signal 34 to and receive data signal 34 from the handheld device 15, as described in more detail below. It should be understood that any suitable cloud server, ECU, external camera, and wireless positioning camera can be used to achieve their respective functions as provided herein without departing from the spirit or scope of this disclosure.

[0159] As described above, the vehicle-mounted device 16 includes a wireless positioning transmitter 22 disposed on the vehicle 12 and communicating with the ECU 20. The wireless positioning transmitter 22 is arranged to receive and transmit radio frequency (RF) domain location data of the vehicle 12, such that the RF domain location of the vehicle 12 can be detected as needed via the wireless positioning transmitter 22, the ECU 20, or the cloud server 18. Furthermore, the wireless positioning transmitter 22 is arranged to receive and transmit RF domain location data of the handheld device 15, such that the RF domain location of the user 14 can be detected as needed via the wireless positioning transmitter 22, the ECU 20, or the cloud server 18.

[0160] As shown in the figure, the cloud server 18 is located remotely from the vehicle 12. The cloud server communicates with the ECU 20 and the user's handheld device 15. The handheld device 15 can be a smartphone, such as an Apple iPhone or a key fob.

[0161] In one embodiment, when user 14 moves away from vehicle 12, the system monitors the proximity of user 14 to vehicle 12. When the proximity of user 14 to vehicle 12 is within a predetermined distance, the in-vehicle device 16 is activated for use. For example, when handheld device 15 is within a threshold distance from vehicle 12, ECU 20 is arranged to receive a signal from cloud server 18 to activate in-vehicle device 16. To activate in-vehicle device 16, ECU 20 is arranged to monitor the position of vehicle 12 based on an intention detection system (e.g., GPS) and accordingly send the GPS data signal of vehicle 12 to cloud server 18. Additionally, cloud server 18 is arranged to monitor the position of user 14's handheld device 15 based on an intention detection system (e.g., GPS). Preferably, cloud server 18 can receive GPS data signals from handheld device 15 (e.g., via a telephone relative positioning and motion / intention detection module) to monitor the position of handheld device 15. Using the GPS-based position of vehicle 12 and the position of handheld device 15, cloud server 18 is arranged to detect whether handheld device 15 is within a threshold distance from vehicle 12. When the handheld device 12 is within a threshold distance from the vehicle 12, the cloud server sends a signal to the ECU 20 to activate the on-board device 16 for operation.

[0162] refer to Figure 1-2 The cloud server 18 is configured to compare the visual domain position of user 14 with the RF domain position of user 14. That is, the in-vehicle device 16 obtains the visual domain position data and RF domain position data of user 14 and sends them to the cloud server 18 for data comparison. Therefore, upon activation, the external camera 24 senses user 14 and captures an image of user 14 for their visual domain position. Based on the image of user 14, the user's visual domain data is sent to the ECU 20, which then transmits the user's visual domain data to the cloud server 18. Furthermore, upon activation, the wireless positioning transmitter 22 senses the handheld device 15 and receives RF signals from the handheld device 15 for the RF domain position of user 14. Based on the RF data from the handheld device 15, the device's RF domain data is sent to the ECU 20, which then transmits the device's RF domain data to the cloud server 18.

[0163] Using visual domain data and RF domain data from ECU 20, cloud server 18 is deployed to verify the facial identity of user 14 based on the visual domain data of user 14. In one verification example, cloud server 18 is deployed to evaluate the image data quality of user 14's facial identity based on the visual domain data of user 14. Cloud server 18 is deployed to determine a confidence threshold based on the image quality of user 14's facial identity. If the image quality is relatively low, the confidence threshold can be lowered. Conversely, if the image quality is relatively high, the confidence threshold can be increased. Through a facial recognition module, cloud server 18 is deployed to process user 14's visual domain data to define a confidence score for user 14's facial identity. Through an authentication module, cloud server 18 is deployed to confirm (verify) user 14's facial identity when the confidence score is greater than the confidence threshold.

[0164] During facial authentication, cloud server 18 is configured to estimate the visual domain position of user 14 based on visual domain data from ECU 20. Additionally, cloud server 18 is configured to determine the RF domain position of vehicle 12 based on RF domain data from vehicle 12. Furthermore, cloud server 18 is configured to determine the RF domain position of user 14 based on RF domain data from handheld device 15.

[0165] In this embodiment, the cloud server 18 is arranged to compare a first distance with a second distance. The first distance is the distance between user 14 and vehicle 12 based on the visual domain data of user 14. The second distance is the distance between user 14 and vehicle 12 based on RF domain data. Furthermore, the vehicle's local coordinate system can be used to describe the relationship between vehicle 12 and user 14. It should be understood that the vehicle's local coordinate system can be a Cartesian coordinate system, a spherical coordinate system, or any other suitable coordinate system without departing from the spirit or scope of this disclosure.

[0166] For example, relative to Figure 3 The coordinate system 40 shown is used to arrange the cloud server 18 according to the first equation. Second Equation Compare the visual domain position of user 14 with the RF domain position of user 14.

[0167] In this example, the item It is based on the visual domain data in coordinate system 40 relative to vehicle 12. Figure 3 The radial distance between user 14 and vehicle 12. Additionally, item It is based on wireless domain data and is the radial distance between user 14 and vehicle 12 relative to coordinate system 40. Furthermore, the item... It is calculated relative to visual domain data. Figure 3 The first azimuth angle of the coordinate system at 40°. Additionally, the item... It is the second azimuth angle relative to the coordinate system 40, calculated based on wireless domain data. Furthermore, the term ε... d It is the distance consistency threshold for confirming the distance consistency between visual domain data and RF domain data, and the item It confirms the visual domain data and RF domain data relative to Figure 3 The coordinate system shown is 40, and the angle consistency threshold is used for angle consistency.

[0168] Furthermore, when the first and second equations are true, the cloud server 18 is configured to verify user 14. Additionally, the cloud server 18 is configured to unlock vehicle 12 upon verification of user 14. For example, the cloud server 18 can send a command signal to the ECU 20 to unlock the doors of vehicle 12.

[0169] For security purposes, cloud server 18 can be configured to detect non-users within a threshold distance relative to user 14, and if a non-user is relatively close to the user, then suspend any steps of the system. Such features can help prevent potential security vulnerabilities caused by malicious non-users. Therefore, cloud server can be configured to check the distance between user and non-user. Furthermore, if a non-user is detected within a minimum distance from the user, cloud server can be configured to suspend any steps of unlocking the vehicle. If necessary, cloud server can also be configured to contact an authoritative body.

[0170] For example, cloud server 18 can be configured to detect non-user 42 (not shown) based on visual domain data of non-user 42 and RF domain data of non-user device. Figure 3 The non-user's visual domain position and RF domain position are defined separately. In one example, one of the external cameras 24 and 25 can sense the non-user 42 and capture an image of the non-user 42 for its visual domain position. Based on the image of the non-user 42, the non-user visual domain data signal is sent to the ECU 20, which then transmits the non-user visual domain data to the cloud server 18.

[0171] Furthermore, the wireless positioning transmitter 22 can sense the handheld device of the non-user 42 and can be configured to receive RF data for the non-user's RF domain location from the non-user's handheld device (via the positioning module or motion / intent detection module). Based on the RF data from the non-user's handheld device, the non-user RF domain data signal is sent to the ECU 20, which then transmits the non-user RF domain data to the cloud server 18.

[0172] In this example, cloud server 18 is configured to use a third-party program. Fourth equation Fifth equation and the sixth equation Compare the visual field position and RF field position of user 14 with the visual field position and RF field position of non-user 42.

[0173] In this embodiment, It is based on the coordinate system relative to vehicle 12, using visual domain data. Figure 3 The radial distance from non-user 42 to vehicle 12. It is a coordinate system relative to vehicle 12 based on wireless domain data. Figure 3 The radial distance from non-user 42 to vehicle 12. It is based on visual domain data relative to the coordinate system ( Figure 3 The third-party azimuth angle from user 42 to vehicle 12, It is based on wireless domain data relative to a coordinate system. Figure 3 The fourth azimuth angle from user 42 to vehicle 12, θ d It is the target distance difference threshold for identifying the distance difference between users 14 and non-users 42 around vehicle 12, and It is the threshold for inter-target angular difference that identifies the angular differences between users 14 and non-users 42 around vehicle 12.

[0174] Furthermore, when equations three, four, five, and six are true, cloud server 18 is configured to continue verifying user 14. However, if any of equations three, four, five, or six is ​​false, then it can be determined that non-user 42 is within a minimum distance relative to user 14. Therefore, any step of unlocking vehicle 12 would be unsafe or unreliable. Therefore, all steps of unlocking the vehicle can be temporarily suspended.

[0175] In another embodiment, the user's facial identity can be repeatedly verified while the user 14 is in the vehicle 12. Such verification can help add security and authorization, especially when the user may frequently leave the vehicle temporarily. For example, when the vehicle is used in the transportation and delivery industry, facial authentication can be expected to be used to enhance vehicle security and authorization.

[0176] As an example and as Figure 2As shown, the vehicle-mounted equipment 16 of vehicle 12 further includes an internal camera 36 disposed within vehicle 12 and communicating with ECU 20. Furthermore, the internal camera 36 is arranged to detect the facial identity and visual domain position of user 14 within vehicle 12 based on visual domain data. In one example, the internal camera 36 can sense user 14 and capture an image of user 14's visual domain position while in vehicle 12. Based on the image of user 14, the user's visual domain data signal is sent to ECU 20, which then transmits the visual domain data to cloud server 18. Preferably, cloud server 18 is arranged to verify the facial identity of user 14 based on the user's visual domain data when user 14 is in vehicle 12.

[0177] When vehicle 12 is unlocked, cloud server 18 is configured to perform a series of actions within vehicle 12 based on user 14's personalized settings. For example, the user may have preset personalized settings stored in ECU 20, and such personalized settings may be activated within vehicle 12. Such personalized settings may include, but are not limited to, temperature control, audio, stereo, seat, steering wheel, and driving mode settings.

[0178] For convenience and additional safety purposes, the system can be configured to detect and verify companion users while they are in the vehicle. During detection and verification, system 10 can be configured to perform a series of actions on vehicle 12 based on the companion user's personalized settings. For example, the companion user may have preset personalized settings stored in ECU 20, and these personalized settings can be activated within vehicle 12. Such personalized settings may include, but are not limited to, temperature control, audio, stereo, seat, steering wheel, and driving mode settings.

[0179] As an example, an internal camera can be arranged to detect a companion user with a companion user device based on the visual domain data and RF domain data of the companion user device, thereby defining the companion user's visual domain position and RF domain position. In this example, the internal camera is arranged to detect the companion user's facial identity and visual domain position based on the companion user's visual domain data in vehicle 12. Additionally, when in vehicle 12, the internal camera can sense the companion user and capture an image of the companion user for their visual domain position. Based on the image of the companion user, the companion user's visual domain data signal is sent to ECU 20, which then transmits the companion user's visual domain data to cloud server 18. Preferably, cloud server 18 is arranged to verify the companion user's facial identity based on the companion user's visual domain data in vehicle 12.

[0180] During companion user verification, the cloud server 18 is configured to perform a series of actions within the vehicle 12 based on the companion user's personalized settings. As described above, the companion user can have preset personalized settings stored in the ECU 20, and these personalized settings can be activated within the vehicle 12. Such personalized settings may include, but are not limited to, temperature control, audio, stereo, seat, steering wheel, and driving mode settings.

[0181] It should be understood that at least one of the cloud server 18 and the vehicle-mounted device 16, such as ECU 20, may include various algorithms and modules with algorithms to achieve the tasks provided herein. For example, the cloud server and the vehicle-mounted device may include a user / vehicle registration module to verify users and their vehicles. Furthermore, the cloud server and the vehicle-mounted device may include a resource discovery module to monitor and record the location of the user's vehicle and handheld device 15. Additionally, the cloud server and the vehicle-mounted device may include an image processing module (including facial recognition and localization) to identify and locate users based on visual domain data. Moreover, the cloud server and the vehicle-mounted device may include a relative positioning sensor fusion module to compare the visual domain location of any of the user, non-user, and companion users with their RF domain location. Additionally, the cloud server and the vehicle-mounted device may include a user identification and personalization module to store user identification and their personalized settings.

[0182] Furthermore, the cloud server and in-vehicle device may include a WiFi / UWB positioning module to detect and transmit wireless domain data from any of the vehicle, user device, non-user device, and companion user device. Additionally, the cloud server and in-vehicle device may include a quality assessment module to evaluate the image quality of any of the user, non-user, and companion users. Furthermore, the cloud server and in-vehicle device may include an authentication module to detect and verify the face of any of the user, non-user, and companion users based on visual domain data. Moreover, the cloud server and in-vehicle device may include a face recognition and face localization module to identify and locate any of the user, non-user, and companion users based on visual domain data. Other modules and algorithms may be used without departing from the scope or spirit of this disclosure.

[0183] It should be understood that the handheld devices of the user, non-user, and companion users may include various algorithms and modules with algorithms to achieve the tasks provided herein. For example, the handheld device may include a vehicle / phone relative positioning module for locating the vehicle and the handheld device. Moreover, the handheld device may include a motion / intention detection module for detecting and sensing the movement of the vehicle or device.

[0184] Figure 4 A method 110 is described, according to an example of this disclosure, for unlocking a vehicle with a handheld device based on visual and radio frequency (RF) domain technologies. Preferably, the method is implemented... Figure 1-3System 10 thus implements method 110. For example... Figure 4 As shown and referenced Figure 1-3 Method 110 includes step 112 of activating the in-vehicle equipment 16 of the vehicle 12 when the user 14's handheld device 15 is within a threshold distance from the vehicle 12. In this respect, method 110 further includes activating the in-vehicle equipment 16 of the vehicle 12 via a first equation. Second Equation Step 114: Comparing the visual domain position of user 14 based on visual domain data with the RF domain position of user 14 based on RF domain data.

[0185] In this regard, It is the radial distance between user 14 and vehicle 12 based on the visual domain data and the coordinate system 40 of vehicle 12. It is the radial distance between user 14 and vehicle 12 relative to the RF domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is the distance consistency threshold used to confirm the distance consistency between visual domain data and RF domain data, and It is the angle consistency threshold for confirming the angular consistency between visual domain data and RF domain data.

[0186] Furthermore, method 110 also includes step 116 of confirming user 14 when the first and second equations are true, and step 118 of unlocking vehicle 12 after confirming user 14.

[0187] Figure 5 A method 210 is described, according to another example of this disclosure, for unlocking a vehicle 12 with a handheld device 15 belonging to a user 14 using visual and radio frequency (RF) domain technologies. Preferably, the method is implemented... Figure 1-3 System 10 implements method 210. Method 210 includes steps 212 of intentionally detecting the location of vehicle 12 and the location of handheld device 15 of user 14, and step 214 of detecting whether handheld device 15 is within a threshold distance from vehicle 12. In this regard, method 210 further includes steps 216 of activating in-vehicle equipment 16 of vehicle 12 when handheld device 15 is within the threshold distance from vehicle 12, and step 218 of verifying the facial identity of user 14 based on visual domain data of user 14.

[0188] In this regard, method 210 further includes a step 220 of estimating the visual domain position of user 14 based on visual domain data of user 14, a step 222 of determining the RF domain position of vehicle 12 based on RF domain data of vehicle 12, and a step 224 of determining the RF domain position of user 14 based on RF domain data of handheld device 15.

[0189] Moreover, in this respect, method 210 further includes using the first equation Second Equation Step 226 involves comparing the visual domain position of user 14 with the RF domain position of user 14. In this regard, It is the radial distance between user 14 and vehicle 12 based on the visual domain data and the coordinate system 40 of vehicle 12. It is the radial distance between user 14 and vehicle 12 relative to the wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold that helps confirm the distance consistency between visual domain data and RF domain data, and It is an angle consistency threshold that helps confirm the angle consistency between visual domain data and RF domain data.

[0190] In addition, method 210 further includes step 228 of confirming user 14 when the first and second comparisons are true, and step 230 of unlocking vehicle 12 after confirming user 14.

[0191] The description in this disclosure is exemplary in nature only, and changes that do not depart from the spirit and scope of this disclosure are intended to remain within its scope. Such changes should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A method for unlocking a vehicle with a handheld device based on visual domain and radio frequency domain technologies, the method comprising: When the user's handheld device is within a threshold distance from the vehicle, the vehicle's in-vehicle equipment is activated; Through the first equation Second Equation The user's visual domain position based on visual domain data is compared with the user's radio frequency domain position based on radio frequency domain data. in It is the radial distance of the user from the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle relative to the radio frequency domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold that confirms the distance consistency between the visual domain data and the radio frequency domain data, and It is the angle consistency threshold that confirms the angle consistency between the visual domain data and the radio frequency domain data; The user is confirmed when the first and second equations are true. and The vehicle is unlocked after the user is confirmed.

2. The method of claim 1, wherein the activation step comprises: The intentional detection module monitors the location of the vehicle and the location of the user's handheld device. and Detect whether the handheld device is within the threshold distance from the vehicle.

3. The method of claim 1, wherein the comparison step comprises: Verify the user's facial identity based on the user's visual domain data; Estimate the user's visual domain position based on the user's visual domain data; The radio frequency domain location of the vehicle is determined based on the vehicle's radio frequency domain data. and The user's radio frequency domain location is determined based on the radio frequency domain data of the handheld device.

4. The method of claim 3, wherein the comparison step further comprises: Detecting non-users with non-user devices based on non-user visual domain data and non-user device radio frequency domain data, in order to define the visual domain location and radio frequency domain location of the non-users; Through a third-party process Fourth equation Fifth equation Sixth equation The visual domain position and radio frequency domain position of the user are compared with the visual domain position and radio frequency domain position of the non-user. in It is the radial distance from the non-user to the vehicle relative to the coordinate system, based on visual domain data. It is the radial distance from the non-user to the vehicle relative to the coordinate system, based on wireless domain data. It is the azimuth angle from the non-user to the vehicle relative to the coordinate system, based on visual domain data. It is the azimuth angle θ from the non-user to the vehicle relative to the coordinate system, based on wireless domain data. d It is a target distance difference threshold used to identify the distance difference between the user and the non-user around the vehicle. It is a threshold for inter-target angular difference used to identify the angular differences between the user and non-user around the vehicle; and If the third, fourth, fifth, and sixth equations are true, the user's steps are then confirmed.

5. The method of claim 1, further comprising performing a series of actions within the vehicle based on the user's personalized settings after the vehicle is unlocked.

6. The method of claim 3, further comprising: When the user is in the vehicle, the user's facial identity and visual domain location are detected based on the user's visual domain data; and When the user is in the vehicle, the user's facial identity is verified based on the user's visual domain data.

7. The method of claim 1, further comprising: Based on the visual domain data of the companion user and the radio frequency domain data of the companion user device, a companion user with a companion user device is detected to define the visual domain position and radio frequency domain position of the companion user. and After unlocking the vehicle, a series of actions are performed inside the vehicle based on the companion user's personalized settings.

8. The method of claim 3, wherein verifying the user's facial identity comprises: The image data quality for the user's facial identity is evaluated based on the user's visual domain data. A confidence threshold is determined based on the image quality of the user's facial identity. The user's visual domain data is processed using a facial recognition module to define a confidence score for the user's facial identity; and When the confidence score is greater than the confidence threshold, the authentication module verifies the user's facial identity.

9. A system for unlocking a vehicle with a handheld device based on visual and radio frequency domain technologies, the system comprising: The vehicle's on-board equipment, the on-board equipment including: The electronic control unit installed in the vehicle; An external camera mounted on the vehicle and communicating with the electronic control unit, the external camera being arranged to detect the user's facial identity and visual field location based on the user's visual field data; and A wireless positioning transmitter installed on the vehicle and communicating with the electronic control unit, the wireless positioning transmitter being arranged to detect the vehicle's radio frequency domain location based on the vehicle's radio frequency domain data, and the wireless positioning transmitter being arranged to detect the user's radio frequency domain location based on the radio frequency domain data of the handheld device; and A cloud server located away from the vehicle and communicating with the electronic control unit and the handheld device, the cloud server being arranged via a first equation Second Equation Compare the user's visual domain position with the user's radio frequency domain position. in It is the radial distance of the user from the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle based on wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d It is a distance consistency threshold that confirms the distance consistency between the visual domain data and the radio frequency domain data, and This is the angle consistency threshold for confirming the angular consistency between the visual domain data and the radio frequency domain data. The cloud server is configured to confirm the user when the first and second equations are true. The cloud server is configured to unlock the vehicle when the user is confirmed. The electronic control unit is configured to activate the vehicle's external camera and wireless positioning transmitter when the user's handheld device is within a threshold distance from the vehicle.

10. The system of claim 9, wherein the electronic control unit is arranged to monitor the location of the vehicle, wherein the cloud server is arranged to monitor the location of a user's handheld device based on an intentional detection system, and wherein the cloud server is arranged to detect whether the handheld device is within the threshold distance from the vehicle.

11. The system of claim 10, wherein the cloud server is configured to verify the user's facial identity based on the user's visual domain data, wherein the cloud server is configured to estimate the user's visual domain position based on the user's visual domain data, wherein the cloud server is configured to determine the vehicle's radio frequency domain position based on the vehicle's radio frequency domain data, and wherein the cloud server is configured to determine the user's radio frequency domain position based on the handheld device's radio frequency domain data.

12. The system of claim 11, wherein the cloud server is configured to detect a non-user with a non-user device based on non-user visual domain data and non-user device radio frequency domain data, to define the non-user's visual domain location and radio frequency domain location. The cloud server mentioned above is configured to operate via a third-party program. Fourth equation Fifth equation Sixth equation The visual domain position and radio frequency domain position of the user are compared with the visual domain position and radio frequency domain position of the non-user. in It is the radial distance from the user to the vehicle based on the vehicle's coordinate system, using visual domain data. It is the non-user radial distance θ based on the coordinate system. d It is a target distance difference threshold used to identify the distance difference between the user and the non-user around the vehicle. It is a threshold for inter-target angular difference used to identify the angular differences between users and non-users around the vehicle. The cloud server is configured to continue verifying the user when the third, fourth, fifth, and sixth equations are true.

13. The system of claim 9, wherein the on-board equipment of the vehicle further comprises: An internal camera installed in the vehicle and communicating with the electronic control unit is configured to detect the user's facial identity and visual domain location based on the user's visual domain data when the user is in the vehicle. The cloud server is configured to verify the user's facial identity based on the user's visual domain data when the user is in the vehicle.

14. The system of claim 9, wherein the cloud server is configured to perform a series of actions within the vehicle based on the user's personalized settings after the vehicle is unlocked.

15. The system of claim 9, wherein the vehicle-mounted device further comprises: An internal camera installed in the vehicle and communicating with the electronic control unit is arranged to detect a companion user with a companion user device based on the companion user's visual domain data and the companion user device's radio frequency domain data, in order to define the companion user's visual domain position and radio frequency domain position. and The cloud server is configured to perform a series of actions within the vehicle based on the companion user's personalized settings after the vehicle is unlocked.

16. A method for unlocking a vehicle with a handheld device based on visual domain and radio frequency domain technologies, the method comprising: The system monitors the location of the vehicle and the location of the user's handheld device based on intentional detection. Detect whether the handheld device is within a threshold distance from the vehicle; When the handheld device is within the threshold distance from the vehicle, the vehicle's on-board equipment is activated; Verify the user's facial identity based on the user's visual domain data; Estimate the user's visual domain location based on the user's visual domain data; The radio frequency domain location of the vehicle is determined based on the vehicle's radio frequency domain data. The user's radio frequency domain location is determined based on the radio frequency domain data of the handheld device; Through the first comparison Second comparison Compare the user's visual domain position with the user's radio frequency domain position. in It is the radial distance of the user from the vehicle relative to the vehicle's coordinate system, based on visual domain data. It is the radial distance between the user and the vehicle relative to the wireless domain data. It is the first azimuth angle calculated based on visual domain data. It is the second azimuth angle calculated based on wireless domain data, ε d This is a distance consistency threshold that helps confirm the distance consistency between the visual domain data and the radio frequency domain data, and It is an angle consistency threshold that helps confirm the angle consistency between the visual domain data and the radio frequency domain data; The user is confirmed when the first and second comparisons are true. and The vehicle is unlocked after the user is confirmed.

17. The method of claim 16, wherein the comparison step further comprises: Detecting non-users with non-user devices based on non-user visual domain data and non-user device radio frequency domain data, in order to define the visual domain location and radio frequency domain location of the non-users; Through a third-party process Fourth equation Fifth equation Sixth equation The visual domain position and radio frequency domain position of the user are compared with the visual domain position and radio frequency domain position of the non-user. in It is the radial distance from the user to the vehicle based on the vehicle's coordinate system, using visual domain data. It is the non-user radial distance θ based on the coordinate system. d It is a target distance difference threshold used to identify the distance difference between the user and the non-user around the vehicle. It is a threshold for inter-target angular difference used to identify the angular differences between the user and non-user around the vehicle; and If the third, fourth, fifth, and sixth equations are true, continue with the user's confirmation process.

18. The method of claim 16, further comprising performing a series of actions within the vehicle based on the user's personalized settings after unlocking the vehicle.

19. The method of claim 16, further comprising: Based on the visual domain data of the companion user and the radio frequency domain data of the companion user device, a companion user with a companion user device is detected to define the visual domain position and radio frequency domain position of the companion user. and After unlocking the vehicle, a series of actions are performed inside the vehicle based on the companion user's personalized settings.

20. The method of claim 16, wherein verifying the user's facial identity comprises: The image data quality for the user's facial identity is evaluated based on the user's visual domain data. A confidence threshold is determined based on the image quality of the user's facial identity. The user's visual domain data is processed using a facial recognition module to define a confidence score for the user's facial identity; and When the confidence score is greater than the confidence threshold, the authentication module verifies the user's facial identity.

Images