Method and system for controlling vehicle status notification to a user

By determining whether the user is in the vehicle and within the line of sight, combining image capture and facial recognition technology to control the sending of vehicle status notifications, the problem of unnecessary notifications in the prior art is solved and the user experience is improved.

CN118974785APending Publication Date: 2024-11-15ZHEJIANG GEELY HLDG GRP CO LTD +1
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202380029959.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-28
Filing Date
2023-03-14
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing smart vehicles frequently send vehicle status notifications when the user does not request it, resulting in unnecessary interference and trouble.

Method used

By determining whether the user is within the vehicle and within the threshold range from the vehicle, combining image capture and facial recognition technology, it is determined whether the user needs to receive vehicle status notifications, and only sends notifications when the user is within the line of sight and is focused.

Benefits of technology

It effectively reduces unnecessary vehicle status notifications, improves user experience, and avoids the situation where the vehicle is considered "stupid".

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118974785A_ABST
    Figure CN118974785A_ABST
Patent Text Reader

Abstract

A method for controlling vehicle status notification to a user includes: determining whether the user is within a vehicle; and if the user is not within the vehicle, determining whether the user is within a predetermined threshold distance from the vehicle; wherein, if the user is within the predetermined threshold distance from the vehicle, vehicle state notification to the user is disabled; and if the user is not within the predetermined threshold distance from the vehicle, enabling vehicle status notification to the user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method and system for controlling vehicle status notifications to a user. Background Art

[0002] Today's smart vehicles or intelligent vehicles are arranged to communicate with users to provide vehicle information. For example, users can receive notifications containing updated vehicle information, including driving data (range, driving distance, fuel economy, etc.), vehicle status (locked / unlocked, doors / windows closed / opened, lights on / off, heating on / off, charging on / off, etc.), location. This information may be useful to users in deciding whether they need or want to take any action or action.

[0003] However, in some cases, such notifications may be unnecessary or intrusive if the owner already knows the vehicle status or has not requested the information. For example, when the user intentionally leaves the vehicle unlocked, such as when the user is standing near the vehicle. In this case, repeated notifications may be perceived as annoying and the vehicle may appear "dumb".

[0004] Therefore, there is a need to provide an improved method and system for controlling vehicle status notifications to a user. Summary of the invention

[0005] An object of the present disclosure is to provide an intelligent method and system that intercepts unnecessary and redundant notifications of vehicle status.

[0006] This object is achieved in a first aspect by a method for controlling vehicle status notifications to a user, the method comprising: determining whether the user is in the vehicle; and if the user is not in the vehicle, determining whether the user is within a predetermined threshold distance from the vehicle; wherein, if the user is within the predetermined threshold distance from the vehicle, the vehicle status notification to the user is deactivated; and if the user is not within the predetermined threshold distance from the vehicle, the vehicle status notification to the user is activated.

[0007] By determining whether the user does not need or want the vehicle status notification based on the proximity between the user and the vehicle, it is possible to control whether the notification should be sent.

[0008] In one embodiment, the method further includes: if the user is not in the vehicle, capturing at least one image of the environment around the vehicle; determining whether the user is present in the at least one image; wherein if the user is present in the at least one image, disabling the vehicle status notification for the user; and if the user is not present in the at least one image, enabling the vehicle status notification for the user. By capturing an image of the environment around the vehicle, it is possible to further determine whether the user is within the line of sight of the vehicle, that is, as an additional condition for disabling the vehicle status notification, the user is both close enough to the vehicle and can clearly see the vehicle.

[0009] In one embodiment, the method further includes: if the user is present in the at least one image, determining whether the user is facing the vehicle within a predetermined time period; and if the user is facing the vehicle within the predetermined time period, disabling the vehicle status notification for the user; and if the user is not facing the vehicle within the predetermined time period, enabling the vehicle status notification for the user. By checking whether the user is facing the vehicle, the following situation can be further determined: the vehicle is not only near the user and visible to the user, but the user is also in a position where the vehicle can be actually seen and any activities near the vehicle can be observed. This provides an additional (extra) condition for disabling the vehicle status notification.

[0010] In one embodiment, facial recognition is used to determine whether the user is present in the at least one image and / or whether the user is facing the vehicle. Facial recognition provides a simple and effective method of determining whether the user is facing the vehicle.

[0011] In one embodiment, whether the user is present in the at least one image is determined by correlating the position of the person in the at least one image with the position of a key associated with the vehicle. In the event that more than one person is present in the captured image, it may be determined which person is the user of the vehicle by correlating the position of the key with the position of the person in the image.

[0012] In one embodiment, the at least one image is captured using at least one imaging device disposed within the vehicle. The onboard imaging device provides images from a known perspective in a well-defined coordinate system, thereby facilitating determination of the distance to and / or position of a person in the captured image.

[0013] In one embodiment, whether the user is in the vehicle is determined by a pressure sensor disposed in at least one seat in the vehicle, at least one interior imaging device in the vehicle, and / or at least one sensor or antenna disposed in the vehicle and configured to detect whether a key associated with the vehicle is present and / or whether a door has been opened. Thus, several ways of determining whether the user is still in the vehicle are provided, which determines whether the next step of the method should be triggered, thereby saving unnecessary data collection and processing.

[0014] In one embodiment, determining whether the user is within a threshold distance from the vehicle includes: measuring the distance between the vehicle and a key associated with the vehicle based on a sensor / antenna disposed within the vehicle; determining whether the user is present in at least one image of the vehicle's surroundings; determining wireless signal communication between the vehicle and the key associated with the vehicle; and / or geo-locating the vehicle and the key associated with the vehicle. Thus, several ways of determining whether the user is within a threshold distance from the vehicle are provided. Geo-locating can include using GNSS positioning, mobile device tracking, and / or Wi-Fi positioning.

[0015] In one embodiment, the vehicle status includes door lock status, lighting system status and / or heating system status. Therefore, any suitable information related to the status of the vehicle that may be useful to the user in deciding whether any action is needed or desired can be notified to the user.

[0016] In one embodiment, the vehicle status notification to the user includes sending a signal to at least one mobile device associated with the vehicle. The mobile device can be a vehicle key, or any mobile device selected by the user and capable of receiving and displaying the vehicle status notification via wireless signal communication.

[0017] In a second aspect of the present disclosure, a system is provided for controlling vehicle status notification to the user, comprising: a processing circuit; and a memory, wherein the memory contains instructions executable by the processing circuit; wherein the system is arranged to communicate with at least one sensor, antenna and / or imaging device within the vehicle to receive data; wherein the system is configured to execute the method according to the first aspect.

[0018] In a third aspect, a vehicle comprising a system according to the second aspect is provided.

[0019] In a fourth aspect, there is provided a computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to the first aspect.

[0020] In a fifth aspect, there is provided a computer readable data carrier having stored thereon a computer program according to the fourth aspect.

[0021] Other features and advantages of the present disclosure will become apparent when studying the attached claims and the following description. Those skilled in the art realize that different features of the present disclosure may be combined to create other embodiments than those described below, without departing from the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings, in which:

[0023] Figure 1 Schematic diagrams showing a vehicle in different scenarios and users at different positions relative to the vehicle according to the present disclosure.

[0024] Figure 2 is a flow chart illustrating a method for controlling vehicle status notification to a user according to the present disclosure.

[0025] Figure 3 is a schematic diagram of a system for controlling vehicle status notifications to a user according to the present disclosure. DETAILED DESCRIPTION

[0026] In the following, a detailed description of a method and system for controlling vehicle status notification to a user according to the present disclosure is shown. In the accompanying drawings, the same reference numerals represent the same or corresponding elements in multiple drawings. It should be understood that these drawings are only for illustration and are not intended to limit the scope of the present disclosure in any way.

[0027] Reference now Figure 1 , a schematic diagram of a vehicle 5 and users at different positions 1-4 around the vehicle 5 is shown. The different positions 1-4 represent different scenarios that are relevant to deciding whether a vehicle status notification should be sent to the user, for example by means of wireless signal communication to a mobile device (such as a smart phone) associated with the vehicle 5. The vehicle may include an imaging device 30 that is arranged to capture images of the vehicle's surroundings. Other devices suitable for performing the method may be arranged in the vehicle, such as sensors, antennas and other imaging devices that will be discussed in more detail below.

[0028] At position 1, the user is several meters away from the vehicle 5, but within a predetermined threshold distance represented by a circle centered on the vehicle 5. The user is within the line of sight of the vehicle 5, i.e. there are no obstacles between the user and the vehicle 5 that block the view. This represents a first scenario in which it is believed that the user can see the full picture of the vehicle 5 and can observe any suspicious activity, i.e. someone trying to steal items from the vehicle 5. In this case, the user may not need or want notifications of the status of the vehicle 5, such as the vehicle 5 being unlocked or the lights or heating being turned on.

[0029] At position 2, the user is slightly further from the vehicle 5 than at position 1, but still within the circle representing the predetermined threshold distance. However, unlike at position 1, there is an obstacle between the user and the vehicle 5, which is shown here as a fence 6. This represents a second scenario in which the user, although close to the vehicle 5, is out of sight and the user is not considered to be able to see a complete overview of the vehicle 5. In this case, the user will therefore wish to receive vehicle status notifications and wish to be able to react when learning that the vehicle 5 is unlocked or the lights or heating are turned on.

[0030] At position 3, similar to position 1, the user is located a few meters away from the vehicle 5 and within the line of sight of the vehicle 5. However, the user has his / her back to the vehicle 5, so that his / her attention is not focused on the vehicle 5. This represents a third scenario in which the user, although close to the vehicle 5 and within the line of sight, is not considered to be able to see a complete overview of the vehicle 5. In this case, the user will therefore wish to receive vehicle status notifications and wish to be able to react when learning that the vehicle 5 is unlocked or the lights or heating are turned on.

[0031] At position 4, the user is located far from the vehicle 5, outside the circle representing the predetermined threshold distance. This represents a fourth scenario in which it is considered that the user does not have a full view of the vehicle 5. In this case, the user will therefore wish to receive vehicle status notifications and wish to be able to react when learning that the vehicle 5 is unlocked or the lights or heating are turned on.

[0032] Reference now Figure 2 , a flowchart showing the steps of a method according to the present disclosure is shown. The purpose of the method is to determine the above Figure 1 The four scenarios described above are described, and the vehicle status notification to the user is controlled accordingly. In the first step S1, it is determined whether the user is in the vehicle 5 or has walked out of the vehicle 5. In the former case, there is no need to send a notification of the vehicle status, because it is considered that the user in the vehicle 5 knows the vehicle 5 status itself.

[0033] The present disclosure covers several ways to determine whether a user is in the vehicle 5. For example, a pressure sensor may be arranged in one or more seats of the vehicle 5 to sense the weight of the person occupying the seat. The reading from the pressure sensor provides an indication that the user is still in the vehicle 5, and the pressure sensor is advantageously arranged in the driver's seat. An imaging device arranged in the vehicle 5 can be used to capture an image of the interior of the vehicle 5, and the presence of a person can be detected in the image. Another alternative is to use a sensor or antenna that detects whether a key associated with the vehicle 5 is present in the vehicle 5 based on wireless signal communication. In one embodiment, the key is a mobile device, such as a smart phone, which has been associated or paired with the vehicle 5 to replace or supplement the vehicle remote control key. Another alternative is to detect whether the door has been opened by a suitable sensor. One or more of these alternatives can be combined to improve accuracy and / or redundancy when determining whether a user is in the vehicle 5 or has walked out of the vehicle 5.

[0034] In the case where the user is not in the vehicle 5, the method proceeds to the second step S2, in which it is determined whether the user is within a predetermined threshold distance from the vehicle 5. Corresponding to the first scenario described above, if the user is within the predetermined threshold distance, the vehicle status notification to the user is deactivated. For example, if the vehicle 5 is unlocked, no notification is sent because it is assumed that the user is aware of this and is in a position to observe any suspicious activity near the vehicle 5.

[0035] On the other hand, corresponding to the fourth scenario described above, if the user is not within the predetermined threshold distance, the vehicle status notification to the user is enabled. Therefore, if the user forgets to lock the vehicle 5 or does not turn off the lights, the status notification of the vehicle 5 is sent because it is assumed that the user is unaware of this and / or is not in a position to observe any suspicious activities near the vehicle 5.

[0036] Several ways of determining whether a user is within a threshold distance from the vehicle 5 can be used in combination with the method according to the present disclosure. In one embodiment, the distance between the vehicle 5 and the key associated with the vehicle 5 is measured based on a sensor / antenna disposed in the vehicle 5. For example, the time required for the wireless signal from the key to reach the sensor / antenna is used to calculate the distance between them using wireless signal communication between the vehicle 5 and the key associated with the vehicle 5. As explained above, the key can be any suitable mobile device (e.g., a smart phone) that has been associated or paired with the vehicle 5 to replace or supplement the vehicle remote control key. Another alternative is to use the geolocation of the vehicle 5 and the key associated with the vehicle 5, that is, to determine or estimate the geographic location of the vehicle 5 and the key, thereby inferring the distance between them. Geolocation can be performed by using GNSS positioning, mobile device tracking, Wi-Fi positioning, etc. These methods are based on the assumption that the user carries the key so that the user's location corresponds to the location of the key. As described above, another alternative is to determine that the user is present in at least one image of the surrounding environment of the vehicle 5. Image processing can be used to identify and / or determine people in the image. One or more of these alternative approaches may be combined to increase accuracy and / or redundancy in determining whether a user is within a threshold distance from the vehicle 5 .

[0037] To provide an additional layer of security, the method may proceed to a third step S3 in which it is determined whether the user is visible from the vehicle 5, i.e. whether he / she can clearly see the vehicle 5. To this end, at least one image of the environment around the vehicle 5 is captured, and it is determined whether the user is present in the at least one image. Corresponding to the first scenario described above, if the user is present in the at least one image, the vehicle status notification to the user is deactivated.

[0038] On the other hand, corresponding to the second scenario described above, if the user is not present in the at least one image, a vehicle status notification for the user is enabled. In this case, it can be inferred that the user cannot see the vehicle 5. Therefore, if the user forgets to lock the vehicle 5 or does not turn off the lights or heating, a status notification for the vehicle 5 is sent because it is assumed that the user is not aware of this and / or is not in a position to observe any suspicious activities near the vehicle 5.

[0039] To take into account additional conceivable situations in which a user may need or want to receive vehicle status notifications, the method may proceed to a fourth step S4 which considers whether the user is actually observing or looking at the vehicle 5. To this end, it is determined whether the user is facing the vehicle 5 within a predetermined time period. If the user is facing the vehicle 5 within the predetermined time period, vehicle status notifications to the user are deactivated. Depending on the user's preferences, the time period may range from a few seconds to a few minutes (in the range) and may be set by the user.

[0040] On the other hand, corresponding to the third scenario described above, if the user does not face the vehicle 5 within a predetermined period of time, a vehicle status notification is enabled for the user. Thus, if the vehicle 5 is unlocked and the user does not look at the vehicle 5 within a period of, for example, 2 minutes, a vehicle 5 status notification is sent because it is assumed that the user is unaware of this and / or is not in a position to observe any suspicious activity near the vehicle 5.

[0041] Facial recognition can be used to determine whether a user is present in an image and / or whether the user is facing the vehicle 5. The face can be recognized with or without further discrimination. In other words, in order to determine that the user is facing the vehicle 5, it is sufficient to recognize the face without identifying who the user is. In order to improve the accuracy of determining whether a user is present in the captured image of the surrounding environment, the facial image can be matched with a second facial image of the user, for example, the second facial image can be a facial image stored in a database, or a facial image extracted from an image captured in the vehicle 5 that is very close in time to the first facial image. As described above, another way to determine whether a user is present in an image is to associate the position of the person in the image with the position of the key to the vehicle 5. This may be useful in the case where more than one person is present in the image. As described above, the image can be captured by using at least one imaging device arranged in the vehicle 5, the at least one imaging device being, for example, an external camera for imaging the surrounding environment of the vehicle 5.

[0042] The vehicle 5 status includes a door lock status, a lighting system status, and / or a heating system status. The method for notifying includes sending a signal to at least one mobile device (e.g., a smart phone) associated with the vehicle 5. The third step and the fourth step may be optional. Figure 2 The corresponding boxes are shown in the hatching.

[0043] The different steps of the method may be repeated, for example, at regular (fixed) intervals to ensure that a decision is made whether to enable or disable vehicle status notifications based on updated information about the user's position and viewing direction relative to the vehicle 5. During such repetitions, not all steps of the method are necessarily performed again. For example, if the user has exited the vehicle 5, the first step S1 need not be performed again unless other steps indicate that the user has re-entered the vehicle 5. Similarly, if any of the steps S2-S4 returns a negative response, the subsequent steps are considered unnecessary.

[0044] Reference now Figure 3 , shows a schematic diagram of a system 100 for controlling vehicle status notification to a user according to the present disclosure. The system 100 includes a processing circuit 110 and a memory 120, wherein the memory 120 contains instructions that can be executed by the processing circuit 110. Moreover, the system 100 is arranged to communicate with at least one sensor 10, antenna 20 and / or imaging device 30 arranged in the vehicle 5 to receive data via a suitable communication device 140 known in the art. The system 100 is configured to perform the above combined Figure 2 The method described. The system 100 may be included in a vehicle, wherein some components or functions may be separate from the vehicle 5 (e.g., located on a remote server (not shown) or executed on a remote server. Moreover, the system is also arranged to send a vehicle status notification to a user's mobile device via wireless signal communication (directly via the communication device 140 or via a remote server).

[0045] The present disclosure also relates to a computer program 130 comprising instructions, which, when executed by a computer, cause the computer to perform the above method. The computer program 130 may be stored in a memory 120, and the instructions may be executed by a processing circuit 110. The present disclosure also relates to a computer-readable data carrier having the computer program 120 stored thereon. The computer-readable data carrier may be one of a storage medium of an electrical signal, an optical signal, a radio signal, a computer-readable storage medium, or other suitable computer-readable data carriers known to those skilled in the art.

[0046] Although the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description should be considered illustrative or exemplary, and the invention is not limited to the disclosed embodiments. For example, the window and emergency release mechanism may be arranged in a building or other structure instead of a vehicle.

[0047] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and the invention is not limited to the disclosed embodiments.

Claims

1. A method for controlling vehicle status notification to a user, comprising: determining whether the user is in the vehicle; as well as If the user is not in the vehicle, determining whether the user is within a predetermined threshold distance from the vehicle; in if the user is within the predetermined threshold distance from the vehicle, deactivating the vehicle status notification to the user; and If the user is not within the predetermined threshold distance from the vehicle, the vehicle status notification to the user is enabled.

2. The method according to claim 1, further comprising: If the user is not in the vehicle, capturing at least one image of an environment surrounding the vehicle; determining whether the user is present in the at least one image; in If the user is present in the at least one image, deactivating the vehicle status notification to the user; and If the user is not in the at least one image, enabling notification of the vehicle status to the user.

3. The method according to claim 2, further comprising: If the user is present in the at least one image, determining whether the user faces the vehicle within a predetermined time period; in deactivating the vehicle status notification to the user if the user is facing the vehicle within a predetermined time period; and If the user is not facing the vehicle within a predetermined time period, notification of the vehicle status to the user is enabled.

4. The method according to claim 3 or 4, wherein: Facial recognition is used to determine whether the user is present in the at least one image and / or whether the user is facing the vehicle.

5. The method according to claim 3, wherein: The presence of the user in the at least one image is determined by correlating the location of the person in the at least one image with the location of a key associated with the vehicle.

6. The method according to any one of claims 2 to 5, wherein: The at least one image is captured using at least one imaging device disposed within the vehicle.

7. A method according to any one of the preceding claims, wherein: Determining whether the user is in the vehicle is done by: a pressure sensor disposed in at least one seat of the vehicle; at least one interior imaging device within said vehicle; and / or At least one sensor or antenna is arranged in the vehicle and is configured to detect the presence of a key associated with the vehicle and / or whether a door has been opened.

8. A method according to any one of the preceding claims, wherein: The step of determining whether the user is within a threshold distance from the vehicle comprises: measuring a distance between the vehicle and a key associated with the vehicle based on a sensor / antenna disposed within the vehicle; The presence of the user in at least one image surrounding the vehicle, wireless signal communication between the vehicle and the key associated with the vehicle, and / or geographic location of the vehicle and the key associated with the vehicle is determined.

9. The method according to claim 8, wherein: Geolocation includes the use of GNSS positioning, mobile device tracking and / or Wi-Fi positioning.

10. The method according to any one of the preceding claims, wherein: The vehicle status includes a door lock status, a lighting system status and / or a heating system status.

11. A method according to any one of the preceding claims, wherein: Notifying the user of the vehicle status includes sending a signal to at least one mobile device associated with the vehicle.

12. A system for controlling vehicle status notification to a user, comprising: Processing circuit; as well as a memory, wherein the memory contains instructions executable by the processing circuit; wherein the system is arranged to communicate with at least one sensor, antenna and / or imaging device arranged in the vehicle to receive data; Wherein, the system is configured to be able to perform the method according to any of the preceding claims.

13. A vehicle comprising a system according to claim 12.

14. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 11.

15. A computer-readable data carrier on which a computer program according to claim 14 is stored.

Citation Information

Patent Citations

  • Methods and systems for closure notification

    CN104670123A

  • Notification managing apparatus and notification managing method for vehicle

    US20050270152A1

  • Vehicle outside monitoring apparatus, monitoring service center, and vehicle outside monitoring method

    US20240096189A1

  • Intelligent voice assistant system, apparatus, and method for vehicle

    WO2016154777A1

  • Method for alerting a user of a vehicle, controller and vehicle

    WO2020126882A1