Face recognition and object detection for vehicle unlocking scenarios

Abstract

Face recognition and object detection for vehicle unlocking scenarios are disclosed. According to various embodiments of the disclosed technology, a biometric access control system and method for a vehicle may include: capturing images of two or more individuals approaching the vehicle; analyzing the captured images to determine whether the one or more individuals approaching the vehicle are authorized users of the vehicle; in the event that at least one but not all of the two or more individuals approaching the vehicle are identified as authorized users of the vehicle, analyzing the captured images to determine an emotion level between the identified authorized users of the vehicle and unauthorized users of the vehicle; and performing a vehicle access security operation based on the determined emotion level between the authorized users of the vehicle and the unauthorized users of the vehicle.

Description

Technical Field

[0001] The present disclosure relates generally to vehicle security and, more particularly, some implementations may relate to facial recognition and image analysis for determining vehicle unlock context. Background Art

[0002] Access to many modern vehicles is accomplished by using the vehicle key to control an actuator to operate the door locking devices. Such systems operate in response to a user pressing a button on the vehicle key to lock or unlock the vehicle doors, open the front or rear trunk, enter an emergency mode, etc. Some more advanced systems provide a hands-free method in which the vehicle key is recognized as the user approaches the vehicle, automatically unlocking the vehicle doors without the user having to press any buttons.

[0003] Some systems allow the user flexibility as to which doors are opened by the car key. For example, certain vehicles can be programmed so that one click of the unlock button unlocks only the driver's door, and a second click is required to unlock all passenger seat doors.

[0004] Improving vehicle functionality and design to provide users with a more satisfying or enjoyable user experience or simply to make accessing the vehicle less cumbersome is an ongoing challenge for the automotive industry. Automotive designers have been struggling with ways to improve the user experience of their vehicles and provide features that can differentiate their vehicles from those of other manufacturers. Vehicle designers are also seeking ways to make systems less prone to misuse and provide enhanced safety for vehicles and their operators. Summary of the invention

[0005] According to various embodiments of the technology disclosed herein, a biometric access control method for a vehicle using artificial intelligence (AI) with high-precision determination logic may include: using an image sensor to capture images of two or more individuals approaching the vehicle; analyzing the captured images to determine whether one or more individuals approaching the vehicle are authorized users of the vehicle; in a case where at least one but not all of the two or more individuals approaching the vehicle are identified as authorized users of the vehicle, analyzing the captured images to determine an emotion level between the identified authorized users of the vehicle and unauthorized users of the vehicle; and performing a vehicle access security operation based on the determined emotion level between the authorized users of the vehicle and the unauthorized users of the vehicle.

[0006] The vehicle access operation security may include: opening the vehicle door for each approaching individual if the interaction between the authorized user of the vehicle and the unauthorized user of the vehicle is determined to be a non-hostile interaction; and opening the vehicle door only for the authorized user of the vehicle when the interaction between the authorized user of the vehicle and the unauthorized user of the vehicle is determined to be a hostile interaction. Opening the vehicle door may include unlocking the vehicle door or physically opening the vehicle door.

[0007] In some embodiments, vehicle access security operations may include not opening any doors of the vehicle when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

[0008] The method may also include activating an alarm when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

[0009] Analyzing the captured images using AI to determine whether one or more individuals approaching the vehicle are authorized users of the vehicle may include performing facial recognition on the captured images of the one or more individuals to determine the identity of the one or more individuals approaching the vehicle.

[0010] The method may also include determining a user profile of the identified individual and using the user profile to determine a seat position for the identified individual and a vehicle door to be opened corresponding to the seat position. The method may also include using the user profile to adjust a vehicle configuration setting at the seat position determined for the identified individual.

[0011] The method may also include: using AI to analyze the captured images to identify individuals among multiple individuals approaching the vehicle; and, based on the identification of one or more authorized users of the vehicle approaching the vehicle, opening a specific door of the vehicle for each identified individual approaching the vehicle as an authorized user of the vehicle.

[0012] The method may also include analyzing the captured image using AI to determine a category of the unidentified individual, and determining a door of the vehicle to be opened for the individual based on the determined category of the individual.

[0013] A vehicle control system may include: a processor; and a memory coupled to the processor to store instructions, which, when executed by the processor, cause the processor to perform operations, the operations including: capturing images of two or more individuals approaching the vehicle; analyzing the captured images to determine whether one or more individuals approaching the vehicle are authorized users of the vehicle; in a case where at least one but not all of the two or more individuals approaching the vehicle are identified as authorized users of the vehicle, analyzing the captured images to determine an emotion level between the identified authorized users of the vehicle and unauthorized users of the vehicle; and performing a vehicle access security operation based on the determined emotion level between the authorized users of the vehicle and the unauthorized users of the vehicle.

[0014] In some embodiments, the vehicle access security operation may include: opening the door for each approaching individual if the interaction between the authorized user of the vehicle and the unauthorized user of the vehicle is determined to be a non-hostile interaction; and opening the door only for the authorized user of the vehicle when the interaction between the authorized user of the vehicle and the unauthorized user of the vehicle is determined to be a hostile interaction. Opening the door may include unlocking the door or physically opening the door.

[0015] In some embodiments, vehicle access security operations may include not opening any doors of the vehicle when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

[0016] The operations may also include activating an alarm when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

[0017] In some embodiments, analyzing the captured images using AI to determine whether one or more individuals approaching the vehicle are authorized users of the vehicle may include: performing facial recognition on the captured images of the one or more individuals to determine the identity of the one or more individuals approaching the vehicle. The operation may also include determining a user profile for the identified individual, and using the user profile to determine a seat position for the identified individual and a door to be opened corresponding to the seat position. In some embodiments, the operation may also include using the user profile to adjust a vehicle configuration setting at the seat position determined for the identified individual.

[0018] In some embodiments, the operation may also include: using AI to analyze the captured image to identify an individual among multiple individuals approaching the vehicle; and, based on the identification of one or more authorized users of the vehicle approaching the vehicle, opening a specific door of the vehicle for each identified individual approaching the vehicle who is an authorized user of the vehicle.

[0019] In some embodiments, the operations may also include analyzing the captured image using AI to determine a category of the unrecognized individual, and determining a door of the vehicle to be opened for the individual based on the determined category of the individual.

[0020] Other features and aspects of the disclosed technology become apparent from the following detailed description, which illustrates features consistent with embodiments of the disclosed technology by way of example in conjunction with the accompanying drawings. The Summary of the Invention is not intended to limit the scope of any invention described herein, which is limited only by the claims attached hereto. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] According to one or more various embodiments, the present disclosure is described in detail with reference to the following figures. The figures provided are for illustration only and depict only typical or example embodiments.

[0022] Figure 1 is a schematic diagram of an example hybrid vehicle with which embodiments of the systems and methods disclosed herein may be implemented.

[0023] Figure 2 An example process for a vehicle unlocking context based on face recognition is illustrated, according to one embodiment.

[0024] Figure 3 The figure shows a top view of an example vehicle equipped with sensors for detecting approaching individuals.

[0025] Figure 4 The diagram shows Figure 3 Side view of the example vehicle and seven approaching individuals.

[0026] Figure 5 An example architecture of a facial recognition system for vehicle access is shown, according to one embodiment.

[0027] Figure 6 The diagram illustrates an example process for facial recognition access control according to one embodiment.

[0028] Figure 7 are example computing components that may be used to implement various features of the embodiments described in this disclosure.

[0029] The drawings are not exhaustive and do not limit the disclosure to the precise form disclosed. DETAILED DESCRIPTION

[0030] Embodiments of the systems and methods disclosed herein may provide systems and methods for enhanced vehicle access. Embodiments may use image data to verify a driver or passenger's authority to automatically access a vehicle as the driver approaches the vehicle. Image data may be collected from a person approaching the vehicle and analyzed to verify the authority of the approaching person. Artificial intelligence (AI) may be used to perform the analysis. The vehicle may evaluate the verified individual and automatically unlock or open the door as an identified operator or identified passenger approaches the vehicle.

[0031] When an individual operator approaches the vehicle and the operator is recognized by the vehicle as an authorized driver of the vehicle, the vehicle can be configured to unlock and open only the driver's door to allow the driver access. When an identified "couple" approaches the vehicle, the vehicle can automatically unlock and open both front doors, determine that one of the couple is usually always the driver, and configure the driver and passenger settings (not only seats and mirrors, but also zoned climate control and other settings) or the driver settings. When parents or one of them approaches the vehicle with one or more identified children, the vehicle can open the appropriate door based on the learned seat position and configure the vehicle for these passengers. The system can also be configured to determine which person or people in an identified couple, family or other group are approaching which door of the vehicle, and configure the driver and passenger settings accordingly.

[0032] In further embodiments, image analysis may be used to evaluate interactions between approaching individuals to determine if adjustments should be made for additional unidentified individuals. For example, the vehicle may detect a known operator and 3 other individuals approaching the vehicle. The adults may be discussing or chatting with each other in a friendly manner. (As opposed to ignoring each other) In this case, the vehicle may assume that these 4 adults are about to enter the vehicle together and all 4 doors should be unlocked and opened. If the vehicle identifies through image analysis that an argument is taking place, the vehicle may only unlock the driver's door and issue a vehicle alarm at the same time.

[0033] User habits can be collected and remembered over time so that the vehicle can learn and automatically respond to the preferences of the operator and passengers. These preferences can be stored on the vehicle or in a storage location external to the vehicle (e.g., a cloud-based storage facility) and remembered when the passenger is recognized. The technology can also learn different habits or preferences on different days of the week or different times of the day. For example, a couple approaching a vehicle on the weekend may result in different user preference settings than the same couple approaching the vehicle on a weekday morning or weekday evening. Similarly, the system can be configured to learn that when a couple approaches the vehicle together on a weekday morning, only one of the couple enters the vehicle and the other continues to pass the vehicle; however, when the couple approaches the vehicle together on a weekend evening, they typically both enter the vehicle, and a particular individual in the couple is typically the driver and the other is the passenger.

[0034] The systems and methods disclosed herein can be implemented on any of many different vehicles and vehicle types. For example, the systems and methods disclosed herein can be used for cars, trucks, motorcycles, recreational vehicles, and other similar road or off-road vehicles. In addition, the subject matter disclosed herein can also be extended to other vehicle types. Figure 1 An example hybrid electric vehicle (HEV) is shown in which embodiments of the disclosed technology may be implemented. Figure 1 The example described is a hybrid vehicle, but the systems and methods of vehicle access control may be implemented in other types of vehicles, including gasoline or diesel powered vehicles, fuel cell vehicles, electric vehicles, or other vehicles.

[0035] Figure 1 The figure shows a drive system of a vehicle 2, which may include an internal combustion engine 14 as a power source and one or more electric motors 22 (which may also be used as generators). The driving force generated by the internal combustion engine 14 and the motor 22 can be transmitted to one or more wheels 34 through a torque converter 16, a transmission 18, a differential gear device 28 and a pair of shafts 30.

[0036] As a HEV, the vehicle 2 may be driven / powered by one or both of the engine 14 and the motor 22 as driving sources for travel. For example, the first travel mode may be an engine-only travel mode using only the internal combustion engine 14 as a power source. The second travel mode may be an EV travel mode using only the motor 22 as a power source. The third travel mode may be an HEV travel mode using both the engine 14 and the motor 22 as power sources. In the engine-only and HEV travel modes, the vehicle 2 relies on power generated by at least the internal combustion engine 14, and the clutch 15 may be included to engage the engine 14. In the EV travel mode, the vehicle 2 is powered by power generated by the motor 22, while the engine 14 may be stopped and the clutch 15 may be disengaged.

[0037] The engine 14 may be an internal combustion engine, such as a gasoline, diesel or similar powered engine, in which fuel is injected and burned in a combustion chamber. A cooling system 12 may be provided to cool the engine 14, such as, for example, to remove excess heat from the engine 14. For example, the cooling system 12 may be implemented by including a radiator, a water pump and a set of cooling channels. In operation, the water pump circulates coolant through the engine 14 to absorb excess heat from the engine. The heated coolant is circulated through the radiator to remove heat from the coolant, and then the cooled coolant may be recirculated through the engine. A fan may also be included to increase the cooling capacity of the radiator. The water pump, and in some cases the fan, may be operated by a drive shaft directly or indirectly coupled to the engine 14. In other applications, one or both of the water pump and the fan may be operated by an electric current such as from a battery 44.

[0038] The output control circuit 14A may be provided to control the drive (output torque) of the engine 14. The output control circuit 14A may include a throttle actuator to control an electronic throttle valve that controls fuel injection, an ignition device that controls ignition timing, etc. The output control circuit 14A may perform output control of the engine 14 as described below according to a command control signal provided from the electronic control unit 50. Such output control may include, for example, throttle control, fuel injection control, and ignition timing control.

[0039] The motor 22 may also be used to provide power in the vehicle 2 and is powered by a battery 44. The battery 44 may be implemented as one or more batteries or other electrical storage devices, including, for example, lead-acid batteries, lithium-ion batteries, and capacitive storage devices. The battery 44 may be charged by a battery charger 45 that receives energy from the internal combustion engine 14. For example, an alternator or generator may be directly or indirectly coupled to the drive shaft of the internal combustion engine 14 to generate electrical current as a result of the operation of the internal combustion engine 14. A clutch may be included to engage / disengage the battery charger 45. The battery 44 may also be charged by the motor 22 during the period when the motor 22 is acting as a generator, such as, for example, by regenerative braking or by coasting.

[0040] The motor 22 may be powered by a battery 44 to generate power to move the vehicle and adjust the speed of the vehicle. The motor 22 may also be used as a generator to generate electricity, such as, for example, when coasting or braking. The battery 44 may also be used to power other electrical or electronic systems in the vehicle. The motor 22 may be connected to the battery 44 via a converter 42. The battery 44 may include, for example, one or more batteries, a capacitive storage unit, or other storage device suitable for storing electrical energy, which may be used to power the motor 22. When the battery 44 is implemented using one or more batteries, the battery may include, for example, a nickel metal hydride battery, a lithium ion battery, a lead acid battery, a nickel cadmium battery, a lithium ion polymer battery, and other types of batteries.

[0041] An electronic control unit 50 (described below) may be included and may control the electric drive components of the vehicle as well as other components of the vehicle. For example, the electronic control unit 50 may control the inverter 42, adjust the drive current provided to the motor 22, and adjust the current received from the motor 22 during regenerative coasting and braking. As a more specific example, the output torque of the motor 22 may be increased or decreased by the electronic control unit 50 through the inverter 42.

[0042] A torque converter 16 may be included to control the application of power from the engine 14 and the motor 22 to the transmission 18. The torque converter 16 may include a viscous fluid coupling that transfers the rotational power from the power source to the drive shaft through the transmission. The torque converter 16 may include a conventional torque converter or a locking torque converter. In other embodiments, a mechanical clutch may be used instead of the torque converter 16.

[0043] A clutch 15 may be included to engage and disengage the engine 14 with the vehicle's drive train. In the example shown, the crankshaft 32, which is an output member of the engine 14, may be selectively coupled to the motor 22 and the torque converter 16 via the clutch 15. The clutch 15 may be implemented as, for example, a multi-plate hydraulic friction engagement device, the engagement of which is controlled by an actuator such as a hydraulic actuator. The clutch 15 may be controlled so that its engagement state is fully engaged, slippingly engaged, and fully disengaged, depending on the pressure applied to the clutch. For example, the maximum torque of the clutch 15 may be controlled according to the hydraulic pressure supplied from the hydraulic control circuit (not shown). When the clutch 15 is engaged, power transmission is provided in the power transmission path between the crankshaft 32 and the torque converter 16. On the other hand, when the clutch 15 is disengaged, the power from the engine 14 is not transmitted to the torque converter 16. In the slipping engagement state, the clutch 15 is engaged, and power is provided to the torque converter 16 according to the maximum torque (transmission torque) of the clutch 15.

[0044] As described above, the vehicle 2 may include an electronic control unit 50. The electronic control unit 50 may include circuits for controlling various aspects of vehicle operation. The electronic control unit 50 may include, for example, a microcomputer including one or more processing units (e.g., a microprocessor), a memory (e.g., RAM, ROM, etc.), and an I / O device. The processing unit of the electronic control unit 50 executes instructions stored in the memory to control one or more electrical systems or subsystems in the vehicle. The electronic control unit 50 may include a plurality of electronic control units, such as, for example, an electronic engine control module, a powertrain control module, a transmission control module, a suspension control module, and a body control module. As a further example, an electronic control unit may be included to control systems and functions, such as doors and door locks, lighting, a human-machine interface, cruise control, telematics, a braking system (e.g., ABS or ESC), and a battery management system, etc. These various control units may be implemented using two or more separate electronic control units or using a single electronic control unit.

[0045] exist Figure 1 In the example shown, the electronic control unit 50 receives information from a plurality of sensors included in the vehicle 2. For example, the electronic control unit 50 may receive signals indicating vehicle operating conditions or characteristics, or signals that may be used to derive vehicle operating conditions or characteristics. These signals may include, but are not limited to, the accelerator operation amount A CC , the speed N of the internal combustion engine 14 E (engine RPM), the speed N of the motor 22 MG (motor speed) and vehicle speed N V These signals may also include the output N of the torque converter 16 T(e.g., output amperes representing motor output), brake operation amount / pressure B, battery SOC (e.g., the charge amount of battery 44 detected by the SOC sensor). Accordingly, the vehicle 2 may include a plurality of sensors 52, which may be used to detect various conditions inside or outside the vehicle and provide the sensed conditions to the engine control unit 50 (the engine control unit 50 may also be implemented as one or more separate control circuits). In one embodiment, the sensor 52 may be included to directly or indirectly detect one or more conditions, such as, for example, fuel efficiency, E F , motor efficiency, E MG , hybrid power (internal combustion motor 14 + MG12) efficiency, acceleration, A CC wait.

[0046] In some embodiments, one or more sensors 52 may include their own processing capabilities to calculate results that may be provided to the electronic control unit 50 for additional information. In other embodiments, one or more sensors may be data-only sensors that provide only raw data to the electronic control unit 50. In further embodiments, hybrid sensors may be included that provide a combination of raw and processed data to the electronic control unit 50. The sensors 52 may provide analog outputs or digital outputs.

[0047] Sensors 52 may be included to detect not only vehicle conditions, but also external conditions. Sensors that may be used to detect external conditions may include, for example, sonar, radar, lidar or other vehicle proximity sensors, and cameras or other image sensors. Image sensors may be used to detect, for example, traffic signs indicating current speed limits, road curvature, and obstacles. Other sensors may still include those that can detect road slopes. Some sensors may be used to actively detect passive environmental objects, while other sensors may be included and used to detect dynamic objects, such as those used to implement smart roads that can actively send and / or receive data or other information.

[0048] For illustration purposes only Figure 1 The example of is used as an example of a vehicle system that can implement the embodiments of the disclosed technology. A person of ordinary skill in the art who reads this specification will understand how to use this platform and other vehicle platforms to implement the embodiments of the disclosed technology.

[0049] Figure 2 The figure illustrates an example process for a vehicle unlocking scenario based on face recognition according to one embodiment. Figure 2, at operation 212, the system detects an individual approaching the subject vehicle. For example, one or more of a variety of sensors may be used to detect the approaching individual. At operation 214, images of the approaching individuals may be collected, and the system may use facial recognition or other recognition techniques based on image analysis to determine whether the system has identified one or more approaching individuals. If the approaching individual is identified as an operator of the vehicle or other authorized entrant into the vehicle, the system may unlock the appropriate one or more doors corresponding to the individual or individuals. In other embodiments, the appropriate one or more doors may be automatically opened to allow the identified one or more individuals to access the vehicle.

[0050] In various embodiments, authorized vehicle operators and passengers may be registered as authorized users with the vehicle through various techniques. For example, an authorized person may register as an authorized operator or passenger using a smartphone application, website, or other similar access application. The system may collect identification data as well as biometric data to be stored in the system (e.g., ECU 50 ( Figure 1 ), vehicle access control system 340( Figure 5 ) or other storage location). In some applications, the vehicle itself can be used to register authorized operators or passengers. For example, an onboard camera or other biometric sensor can be used to capture and store biometric information of authorized users. In some embodiments, the vehicle can learn and automatically add authorized operators or passengers over time based on the history of vehicle access and use.

[0051] In the case where the system does not identify all the approaching individuals, the system may perform further analysis on the approaching group to determine whether it should unlock the vehicle, and if so, to what extent it should unlock the vehicle. For example, the system may be configured to analyze the video or other moving images of the approaching individuals captured by one or more vehicle cameras to determine the nature of the interaction between individuals. The system may analyze the image to determine whether the unrecognized individual is engaging in cordial or friendly behavior with the identified individual. If so, the system may determine that it is appropriate to unlock all doors or unlock the doors for each approaching individual. The system may evaluate the interaction between individuals to determine whether they are participating in an interaction, such as holding hands, walking arm in arm, and looking at each other to talk and smile. Face recognition can be used to determine whether the identified individuals show happiness or satisfaction, or whether they look frightened. AI algorithms can be used to perform analysis of individuals and their interactions to determine the emotional level of the interaction and determine whether the unrecognized individual is a friendly or hostile (or other category) individual. Machine learning techniques can be used to train models for analysis based on data from previous interactions between identified and unrecognized individuals. Result confirmation from authorized individuals can be used to refine the model.The data can be collected from a subject vehicle, or can be collected from multiple different vehicles across a wide range of participants.

[0052] The system may automatically register or store biometric data of unrecognized individuals as visitors to the vehicle. This information may be called up in the future when the same individuals are identified approaching the vehicle at a later date or time. Accordingly, these individuals may be registered as authorized visitors and the AI ​​algorithm may use this information when analyzing approaching individuals and their interactions for vehicle access control.

[0053] The system may also analyze the image to determine whether the unidentified individual is engaging in hostile behavior toward the identified individual. For example, image analysis may be used to determine whether the unidentified individual is pushing or bumping the identified individual. The system may also determine facial expressions of the identified and unidentified individuals to determine whether the identified individual is unhappy or appears to be fearful. If the behavior is more hostile, the system may be configured to unlock only enough doors for the identified individuals. In the case of only one identified individual, the system may be configured to unlock only the driver's door so that the identified driver can enter the vehicle, while other obviously hostile unidentified individuals remain locked out of the vehicle. The system may be further configured to immediately relock one or more unlocked doors when the corresponding identified individual enters the vehicle. In the case of hostility, the system may be further configured to trigger an audible or visual alarm to warn the hostile individual to leave or draw attention to the vehicle. In a further embodiment, the system may be configured to send a message alerting others such as law enforcement agencies or friends and family to a potential dangerous situation.

[0054] The system may automatically register or store biometric data of unrecognized individuals as hostile individuals. This information may be called up in the future when the same individuals are identified approaching the vehicle at a later date or time. Accordingly, these individuals may be registered as unauthorized individuals and the AI ​​algorithms may use this information when analyzing approaching individuals and their interactions for vehicle access control.

[0055] In these and other embodiments, the system may be configured to also recognize other signals provided by the identified individual to confirm or override the system's access decision. In some implementations, the system may use gesture recognition to enable the identified individual to communicate with the vehicle using a variety of gestures, including innocuous gestures that may not be recognized by a hostile party. The system may also be configured so that a signal from a car key operated by the identified user can override the system decision.

[0056] At operation 218, the system may be further configured to analyze the images to evaluate other characteristics of the objects or approaching individuals. Objects in the hands of individuals may be evaluated to determine the type of object and whether the object poses a threat or requires special handling. For example, the system may be configured to analyze images of objects and compare them to other known objects to determine whether the individual is holding a gun or knife or other weapon. The system may also determine whether one or more approaching individuals are carrying a package, and the system may use this determination to further determine whether to open the trunk, hatch, or cargo door of the vehicle so that the individual can place the package in the cargo hold.

[0057] As shown in the above example, at operation 220, the system may control vehicle access based on the image recognition and determinations made at operations 214 to 218. As described above, in the event that only one approaching individual is identified, the system may only open one door unless the individual's behavior indicates otherwise. In various applications, the system may be configured to use facial recognition to determine not only whether the approaching individual is an authorized passenger, but also where these authorized passengers typically sit in the vehicle so that the system can open the appropriate door for the appropriate passenger. For example, the system may determine that the driver and his spouse are approaching the vehicle, in which case the system will unlock and possibly automatically open the driver's door and the door corresponding to the seat position where his spouse typically sits.

[0058] The system may also be able to determine the category of individuals, such as adults, children, toddlers, infants, disabled people, elderly people, ordinary passengers, and trusted passengers. For example, the system may determine that the driver and his two children are approaching the vehicle. The system may further identify the children as child A and child B, with child A always sitting in the left rear seat position, and child B always sitting in the front passenger position when the spouse is not present, and child B always sitting in the right rear seat position when the spouse is present. Accordingly, when the driver and the two identified children approach, the system may use this information to open the two front doors and the left rear door, or when the driver and his spouse and the two children approach, the system may use this information to open the two front doors and the two rear doors. As a further example, the system may identify the approaching driver and child A and child C. The system may further determine that child C is different from child B and always sits in the right rear seat position. Accordingly, in this case, the system may be configured to open the front driver's door and the two rear passenger doors.

[0059] Special use cases may also be provided depending on the class or type of vehicle or the purpose for which the vehicle is used. For example, in the case of using a vehicle for a ride-sharing service, only trusted passengers are allowed to open the door of the vehicle at the time of service request. Users of a ride-sharing application may be requested to register using a ride-sharing application or website, which may include performing facial recognition on the user of the rights sharing at the time of registration. Accordingly, vehicles used for ride-sharing services may perform facial recognition on approaching individuals to confirm that they are registered users of the service and that they are users requesting the corresponding vehicle. This may add security measures to ride-sharing services. In addition, an onboard camera may capture an image of the vehicle driver and perform facial recognition on it to confirm that the vehicle driver is an authorized driver of the ride-sharing subject vehicle. The system may send a confirmation to the ride-sharing service and the requesting passenger to confirm that the driver is an authorized driver of the vehicle.

[0060] Figure 3 The figure shows a top view of an example vehicle equipped with sensors for detecting approaching individuals. Figure 4 The diagram shows Figure 3 Side view of the example vehicle and seven approaching entities in . Now refer to Figure 3 and Figure 4, the example subject vehicle 2 includes a plurality of image sensors 64 placed around the subject vehicle 2 so that they can capture a substantially 360° field of view around the vehicle. In various embodiments, other sensors may be used to detect approaching individuals in place of or in addition to image sensors. For example, radar, lidar, infrared, ultrasonic, or other sensors may be used to detect approaching individuals. As a further example, in addition to or in place of the image sensor 64, parking sensors or adaptive cruise control sensors may also be used to detect the presence of approaching individuals. The image sensor 64 may be a camera or other image sensor dedicated to a vehicle access control system. In other embodiments, some or all cameras may be shared with other functions, such as, for example, a reversing camera, a surround view camera, a front view camera, or other cameras for other functions of the vehicle.

[0061] Although the image sensor 64 is not included at the left rear quarter of the vehicle 2 in this example, an image sensor 64 may also be provided there. In addition, many other sensors may be used to detect the approach of an individual and capture an image of the approaching individual. In addition to or including Figure 3 and Figure 4 Although the image sensor is shown in the example of FIG. 6 , the image sensor 64 may also be mounted at other locations in the vehicle.

[0062] Figure 4 The example diagram of shows a side mounted image sensor 64 for detecting approaching individuals and capturing images of detected approaching individuals. As described above, other sensors besides the image sensor 64 may be used to detect approaching individuals. The image sensor 64 need not be mounted on the front window of the vehicle, but may be mounted at other locations around the vehicle. In addition, as described above, many other image sensors 64 may be provided. Generally, the greater the coverage provided by the image sensor, the more efficient the system is in detecting approaching individuals and evaluating the identity and nature of the approaching individuals. In some embodiments, a wider angle image sensor may be used, thereby requiring a smaller number of image sensors than in the case of an implementation with an image sensor having a narrower viewing angle.

[0063] Figure 5 The figure shows an example architecture of a face recognition system for vehicle access according to one embodiment. The example includes a camera 332, a sensor 334, a face recognition module 336, a video image recognition module 338, a storage device 337, a vehicle access control system 340, a vehicle locking system 342 and various locking mechanisms 343.

[0064] Sensor 334 can be used to detect the approach of an individual toward the vehicle. Sensor 334 may include, for example, an image sensor, a laser radar, a radar, an ultrasonic detector, an acoustic sensor, and other sensors that can be used to detect the approach of an individual. In the case where sensor 334 includes an image sensor, these sensors may also perform image capture without camera 332. In this example, sensor 334 provides data to vehicle access control system 340, which uses this information to determine whether an individual is approaching. In some embodiments, vehicle access control system 340 may turn on camera 332 when an individual approaches, and may trigger the operation of face recognition module 336 and video image recognition module 338. In some embodiments, vehicle access control system 340 may be implemented as a processor-based system including one or more processors and memory. In some embodiments, vehicle access control system 340 may be implemented by any one of face recognition 336, video image recognition 338, camera 332, sensor 334, memory 337, and vehicle locking system 342.

[0065] A camera 332 may be included to capture images of individuals approaching. The camera 332 may include, for example, various types of cameras, such as a stereo camera, a time-of-flight camera, a depth camera, or other camera technologies. The camera 332 may be included to capture still images for face recognition and object detection, as well as moving images for behavior recognition. Any of a plurality of cameras 332 may be included to capture images of the exterior of the vehicle. The camera 332 may be controlled by the vehicle access control system 340, and image data may be provided to the face recognition module 336 and the video image recognition module 338. In some embodiments, the face recognition module 336 and the video image recognition module 338 may be implemented as part of the vehicle access control system 340 or as part of the camera 332. The camera 332 and the sensor 334 may provide the collected image data and other data to the face recognition module 336 and the video image recognition module 338 and the vehicle access control system 340.

[0066] The face recognition module 336 may be implemented as a processor-based system that includes face recognition software with AI capabilities for analyzing facial features captured by one or more cameras 332 to determine the identity of an approaching individual. The analysis may be performed by an embedded module such as on a cloud server or by edge computing inside or outside the vehicle. In some embodiments, this is accomplished by the face recognition module 336 comparing the captured image with an image stored in a storage device 337. In some embodiments, the face recognition module 336 may also be configured to authenticate an approaching individual once it is identified, while in other embodiments, this may be performed by the vehicle access control system 340.

[0067] The video image recognition module 338 may also be implemented as a processor-based system that includes video image analysis software for analyzing motion, posture, and facial features captured by one or more cameras 332 to determine the nature of the approaching individual, particularly when some approaching individuals are not recognized individuals. AI technology may be used to perform the analysis. The analysis may be performed by an embedded module such as on a cloud server inside or outside the vehicle or by edge computing. The video image recognition module 338 may be configured to analyze body motion, interactions between approaching individuals, facial features or expressions, and other activities. This information may be used by the vehicle access control system 340 to determine whether the unrecognized individual is hostile or friendly. The video image recognition module 338 may also be used to detect the vitality of approaching individuals.

[0068] The vehicle access control system 340 can use information from the facial recognition module 336 and the video image recognition module 338 to determine whether to actuate the door lock / unlock mechanism or open the door. For example, in the event that the facial recognition module determines that all approaching individuals are identified and authorized, the vehicle access control system 340 can instruct the vehicle locking system 342 to actuate the vehicle locking mechanism 343 to unlock the corresponding door for the approaching authorized individual. As another example, in the event that the system determines that only one approaching individual is identified and the accompanying unidentified individual appears hostile, the vehicle access control system 340 can instruct the vehicle locking system 342 to unlock only one door for the identified individual and keep the other doors locked. This can be performed by an embedded module inside or outside the vehicle, such as on a cloud server, or by edge computing. Although in Figure 5 Not shown, but the vehicle access control system 340 may also instruct an alarm system to initiate a vehicle alarm (audible, visual, or otherwise), and may instruct a messaging system to send an alert to an agency, next of kin, or other designated recipient.

[0069] Despite Figure 5 The figure shows, but a communication module may also be included to allow the vehicle access control system to communicate with a server (e.g., a cloud server or other remote server) or with an edge computing resource to perform functions such as image recognition and analysis. Some or all of these functions may be performed outside the vehicle or inside the vehicle depending on system design preferences. Accordingly, an appropriate wireless communication interface (e.g., cellular data, LTE, 5G, etc.) may be provided to enable such communication.

[0070] Figure 6 Figure 1 illustrates an example process for face recognition access control according to one embodiment. Figure 6, at operation 412, the system uses sensors to scan the area around the subject vehicle. In some embodiments, sensors of surrounding vehicles or infrastructure equipment in the area may also be used, and sensor data may be transmitted to the subject vehicle (or to a server such as for a cloud-based system, or to multiple vehicles for edge-based processing) to determine whether one or more individuals are approaching the vehicle. As described above, any of a number of different sensors may be utilized, including radar, lidar, ultrasonic, image sensors, and other sensors.

[0071] The system receives the sensor data and evaluates it to determine whether one or more individuals are approaching the vehicle. The system may also evaluate other data, such as car key data, to detect or verify the presence of approaching individuals. If no approaching individuals are detected at operation 414, the system continues to scan the vicinity of the vehicle. If the system determines that an individual is approaching, the system may use one or more image sensors (e.g., one or more cameras) to collect one or more images of the approaching individuals.

[0072] In operation 416, the system performs image recognition on one or more approaching individuals to determine whether some or all of the individuals are identified as individuals authorized to enter or operate the vehicle. This analysis can be performed on the vehicle or by an external processing function such as a cloud-based service system or edge computing network. Figure 5 In the example shown, the recognition may be performed by a facial recognition module 336 , which may be a standalone processing system or may be integrated with a vehicle access control system 340 , a vehicle locking system 342 , or other vehicle systems.

[0073] At operation 418, if all approaching individuals are identified, the process continues at operation 420, where the system instructs the vehicle locking system to unlock all doors for the occupants. In some embodiments, the system may be configured so that all doors are unlocked or all doors plus any cargo doors and fuel door are unlocked. In another embodiment, the system may be configured to unlock one door for each approaching identified occupant, which may be less than the total number of doors on the vehicle. In the case where the vehicle includes actuators for physically opening and closing various doors, the system may be further configured to unlock the exits and open the doors.

[0074] The system may be further configured to determine the identity of the identified approaching individuals and unlock or open specific doors corresponding to the identified individuals. As an example, the system may identify two approaching individuals and determine that they are specific known individuals. By using stored profiles about the identified individuals, or by using machine learning techniques to identify and learn the behavior of approaching individuals based on their historical practices, the system may determine that one of the approaching individuals is typically driving the vehicle, while the other approaching individual is typically a passenger in the front passenger seat. Based on this determination, the system may open the left and right front doors of the vehicle and configure vehicle settings according to the user profiles corresponding to the identified individuals.

[0075] However, if no approaching individual is identified, then in operation 422, the system does not unlock the vehicle, and can continue to scan more approaching individuals in operation 412. In some applications, the system can be configured to improve vehicle safety without approaching individuals being identified. On the other hand, if some but not all approaching individuals are identified, the system can perform image analysis in operation 424 to determine the nature of the individual. Image analysis can be performed on still images and moving images to determine the interaction between the identified and unidentified individuals approaching the vehicle. The system can use interaction or other information to determine the emotional level between approaching individuals. Machine learning techniques can be used to build and train models to identify different types of behaviors or interactions between individuals. For example, the system can detect signs of hostile or other highly excited emotions between individuals, such as pushing or bumping, unidentified individuals firmly grasping the arms of identified individuals, expressions of fear or worry in the expressions of identified individuals, etc. The system can also detect signs of friendliness or camaraderie between individuals, such as, for example, laughter or playfulness, individuals holding hands are walking arm in arm, smiling or happy, etc. The system may be further configured to determine whether the level of detected hostility exceeds a set threshold level to determine whether the interaction is sufficiently hostile to partially or completely restrict vehicle access.

[0076] In the event that all individuals are friendly, the process may continue at operation 420 where all doors are unlocked, or doors corresponding to each approaching individual may be unlocked. For the identified individuals, the profiles may again be used to open the appropriate doors for those individuals and configure the vehicle settings in their usual seating positions for those vehicles. In the event that the individuals are unidentified friendly individuals and no profile is available for them, the system may make an educated decision about which doors to open and how to configure the seating positions. For example, in the event that an unidentified friendly child is approaching the vehicle with an identified adult, the system may be configured to unlock or open the rear doors for the child and configure the seat or seat belt settings in those rear seats for the child.

[0077] For unidentified individuals who appear to be unfriendly, as shown in operation 428, the system can be configured to react accordingly, such as partially unlocking the vehicle. The system can be configured to use image analysis to determine where the identified individual is approaching the vehicle, and only unlock the doors necessary for the identified individual so that they can enter the vehicle, and the unfriendly approacher will hopefully remain locked out of the vehicle. If the system determines that the friendliness has deteriorated to a necessary point, the system can also be configured to issue a vehicle alarm. For example, the system can determine that a friendly driver and passenger can enter the vehicle and close and lock all doors. In this case, the system will not issue an alarm or any alarm. However, the system can determine that the dispute has deteriorated, and the identified driver and passenger are prevented from entering the vehicle or are potentially dangerous. In this case, the system can issue a vehicle alarm to warn the possible attacker to leave, or otherwise draw attention to the situation, and the system can send an alarm to an agency or appropriate individual.

[0078] Image analysis can also be used to perform object detection in various situations, regardless of whether the image analysis determines that some, all, or no approaching individuals are identified. For example, the system can use image analysis to determine whether any individual approaching the vehicle is brandishing a gun or other actual or potential weapon. In the case where the identified individual is carrying a weapon, the system can determine on this basis that this is an unfriendly and potentially dangerous situation, and respond by partially unlocking the vehicle so that the identified individual can enter the vehicle. As another example, the system can determine other objects, such as the presence of a pet or that the identified individual is carrying a package of a specific size (e.g., exceeding a size threshold). Further to this example, in the case where the identified individual is carrying a large package, pushing a full shopping cart, or a possible load for the vehicle, the vehicle can be configured to automatically open the cargo door so that the identified object can be loaded into the cargo hold.

[0079] In a further scenario, the system may be configured to determine that some unidentified individuals accompanying the identified individuals are friendly, while other unidentified individuals are unfriendly. In this case, the system may unlock the doors for the identified individuals and those identified as friendly, but not for the unidentified unfriendly individuals.

[0080] The system can be programmed (e.g., at the factory, at a service provider, or via an over-the-air (OTA) update) to determine whether an unidentified individual is friendly or unfriendly. As described above, machine learning techniques can be used to further train and refine models that allow the system to determine the nature of an approaching unidentified individual. Data from the crowd or other data from multiple different sources can be collected and used to train the model. For example, facial recognition can be performed by a user on a vehicle, by a user on a smartphone or other application, or at a dealer or service provider to identify and determine the identity of an approaching individual. The user can allow their facial features to be scanned and the scan to be stored as part of the user profile. The vehicle can further establish a profile based on learned user behavior in a system that can be configured to allow the user to input user preferences into his or her profile. These profiles may include, for example, a typical seat position of the user, the user's vehicle setting configuration (e.g., seat settings, mirror settings, radio settings, driving mode settings, etc.) and other user information. The user profile can be securely stored in the vehicle or in a server such as a cloud server.

[0081] Various embodiments may be configured to use additional environmental information to report the determination of various vehicle unlocking scenarios. For example, regional or geographic information may be used to adjust the unlocking scheme based on the safety or security of the geographic area where the vehicle is parked. Date and time information may also be used. If the vehicle is in a safer location or at an updated time that is generally safer (e.g., a relatively safe neighborhood or a safe parking lot during the day), the system may be configured to be more lenient to unidentified passengers, and less lenient to unidentified passengers if the vehicle is parked in a less secure area or a less secure time of the day (e.g., at night in a bad neighborhood or an empty parking lot). Third-party data sources such as crime databases or databases containing information about parking lots may be used, and crowd-sourced information related to security (e.g., crime information reported by vehicles with similar systems or various other users) may also be used to build models to predict the safety of the geographic area where the vehicle is parked. This information may also be used to warn drivers that they are entering or parked in an unsafe area.

[0082] Another example of environmental information that can be used to report the determination of the unlocking context is information related to the user's calendar, or the user's message data, or the user's habits. As an example, a user's calendar entry may indicate that the user is picking up a friend or relative from an airport, school, or other location at a specific place and time. Messages exchanged with friends and family may provide similar information. The system can use this information to weight the prediction of whether an unidentified individual approaching the vehicle with the user at the scheduled pick-up location and time is a friendly companion. The system can also be configured to learn user habits and use this information to weight the prediction. For example, a commercial driver may make frequent late-night pick-ups at an airport or train station. Similarly, parents may frequently pick up high school-age children after soccer practice. The system can learn this behavior and, if the behavior fits the pattern, weight the prediction of whether an unidentified individual approaching the vehicle with the user at the learned pick-up time and location is a friendly companion.

[0083] In various embodiments, the user is provided with the option to configure the system to a "pick-up mode" or other relaxed setting prior to leaving and locking the vehicle at the pick-up location. The system may also be configured to provide settings or user-configurable options to the user to allow the user to configure the system to his or her preferences. For example, the system may enable the user to select from a range of modes from most restrictive to most relaxed; the system may be configured to provide a child-friendly mode to provide more relaxed settings for younger unidentified individuals, etc.

[0084] In further embodiments, information obtained from the system may be shared with law enforcement, security forces, or other entities for purposes outside of the vehicle unlocking context. In the event that the system detects an individual that appears to be engaging in nefarious activity, the system may be configured to provide an alert to a third party, such as law enforcement, that nefarious activity may be occurring. Responsive to the context and the possible identification of individuals involved in the activity, the system may share captured video, still images, and other sensor information with law enforcement to help identify the activity.

[0085] As used herein, the term module can describe a given functional unit, which can be executed according to one or more embodiments of the present application. As used herein, a component can be implemented using any form of hardware, software, or a combination thereof. For example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logic components, software programs, or other mechanisms can be implemented to form a component. The various components described herein can be implemented as discrete components, or the functions and features described can be shared partially or completely between one or more components. In other words, it will be clear to a person of ordinary skill in the art after reading this specification that the various features and functions described herein can be implemented in any given application. They can be implemented in various combinations and arrangements in one or more discrete or shared components. Although various features or functional elements can be described or required as discrete components individually, it should be understood that these features / functions can be shared between one or more common software and hardware elements. Such a description should not require or imply that discrete hardware or software components are used to implement such features or functions.

[0086] Where a module is implemented in whole or in part using software, these software elements may be implemented to operate with a computing or processing component capable of performing the functions described therein. Figure 7 Such an example computing component is shown in . Various embodiments are described with respect to the example computing component 500. After reading this description, it will become clear to a person skilled in the relevant art how to implement an application using other computing components or architectures.

[0087] Reference now Figure 7 , computing components 500 may represent, for example, computing or processing capabilities found in self-adjusting displays, desktops, portable computers, notebook computers, and tablet computers. They may be found in handheld computing devices (tablets, PDAs, smart phones, cell phones, palmtops, etc.). They may be found in workstations or other devices with displays, servers, or any other special or general type of computing device that may be desirable or appropriate for a given application or environment. Computing components 500 may also represent computing capabilities embedded in or otherwise available to a given device. For example, computing components may be found in other electronic devices such as, for example, portable computing devices, and other electronic devices that may include some form of processing capabilities.

[0088] The computing component 500 may include, for example, one or more processors, controllers, control components, or other processing devices. The processor 504 may be implemented using a general or special processing engine such as, for example, a microprocessor, controller, or other control logic. The processor 504 may be connected to the bus 502. However, any communication medium may be used to facilitate interaction with other components of the computing component 500 or for external communication.

[0089] The computing component 500 may also include one or more memory components, referred to herein as main memory 508. For example, a random access memory (RAM) or other dynamic storage may be used to store information and instructions to be executed by the processor 504. The main memory 508 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 504. The computing component 500 may also include a read-only memory ("ROM") or other static storage device coupled to the bus 502 for storing static information and instructions for the processor 504.

[0090] The computing component 500 may also include one or more information storage mechanisms 510 in various forms, which may include, for example, a media drive 512 and a storage unit interface 520. The media drive 512 may include a drive or other mechanism that supports fixed or removable storage media 514. For example, a hard disk drive, a solid state drive, a tape drive, an optical drive, a compact disk (CD) or a digital video disk (DVD) drive (R or RW), or other removable or fixed media drives may be provided. The storage medium 514 may include, for example, a hard disk, an integrated circuit assembly, a magnetic tape, a cassette, an optical disk, a CD or a DVD. The storage medium 514 may be any other fixed or removable medium that is read, written or accessed by the media drive 512. As shown in these examples, the storage medium 514 may include a computer-usable storage medium having computer software or data stored therein.

[0091] In alternative embodiments, the information storage mechanism 510 may include other similar means for allowing computer programs or other instructions or data to be loaded into the computing component 500. Such means may include, for example, a fixed or removable storage unit 522 and an interface 520. Examples of such storage units 522 and interfaces 520 may include cartridge program memory and cartridge memory interfaces, removable memory (e.g., flash memory or other removable memory components), and memory slots. Other examples may include PCMCIA slots and cards, and other fixed or removable storage units 522 and interfaces 520 that allow software and data to be transferred from the storage unit 522 to the computing component 500.

[0092] The computing component 500 may also include a communication interface 524. The communication interface 524 may be used to allow software and data to be transferred between the computing component 500 and external devices. Examples of the communication interface 524 may include a modem or soft modem, a network interface (such as Ethernet, a network interface card, IEEE 802.XX or other interface). Other examples include a communication port (such as, for example, a USB port, an IR port, an RS232 port, a WiFi interface, The software / data transmitted via the communication interface 524 may be carried on a signal, which may be electronic, electromagnetic (including optical), or other signal capable of being exchanged with a given communication interface 524. These signals may be provided to the communication interface 524 via a channel 528. The channel 528 may carry signals and may be implemented using a wired or wireless communication medium. Some examples of a channel may include a telephone line, a cellular link, an RF link, an optical link, a network interface, a local or wide area network, and other wired or wireless communication channels.

[0093] As used herein, the terms "computer program medium" and "computer usable medium" are generally used to refer to either transient or non-transitory media. Such media may be, for example, memory 508, storage unit 520, medium 514, and channel 528. These and other various forms of computer program media or computer usable media may involve carrying one or more sequences of one or more instructions to a processing device for execution. Such instructions embedded on a medium are generally referred to as "computer program code" or "computer program products" (which may be grouped in the form of computer programs or other groupings). When executed, such instructions may enable computing component 500 to perform the features or functions of the present application discussed herein.

[0094] It should be understood that the various features, aspects and functions described in one or more individual embodiments are not limited in their applicability to specific embodiments and their descriptions. Instead, they can be applied to one or more other embodiments individually or in various combinations, regardless of whether such embodiments are described and whether these features are represented as part of the described embodiments. Therefore, the breadth and scope of the present invention should not be limited by any of the above exemplary embodiments.

[0095] Unless otherwise expressly stated, the terms and phrases used in this document and their variations should be interpreted as open rather than restrictive. As an example above, the term "including" should be understood to mean "including but not limited to" and the like. The term "example" is used to provide an illustrative example of the item being discussed, rather than an exhaustive or restrictive list thereof. The term "one" should be understood as "at least one", "one or more", etc.; and adjectives such as "conventional", "traditional", "normal", "standard", "known", and terms of similar meanings should not be interpreted as limiting the items described to items available in a given time period or at a given time. Instead, they should be understood to cover conventional, traditional, normal or standard technologies that are available or known at any time now or in the future. Where this document relates to technologies that will be clear or known to a person of ordinary skill in the art, such technologies cover technologies that are clear or known to a person of ordinary skill in the art at any time now or in the future.

[0096] In some cases, the presence of expanded words and phrases such as "one or more," "at least," "but not limited to," or other similar phrases, where such expanded phrases may be absent, should not be understood to mean or require a narrower case. The use of the term "component" does not mean that the aspects or functions described or claimed as part of the component are configured in a common package. In fact, any or all aspects of the various aspects of the component, whether control logic or other components, may be combined in a single package or maintained separately, and may further be distributed in multiple groups or packages or across multiple locations.

[0097] In addition, various embodiments described herein are described in terms of exemplary block diagrams, flow charts, and other illustrations. After reading this document, it will become clear to one of ordinary skill in the art that the illustrated embodiments and their various alternatives may be implemented without limitation to the illustrated examples. For example, the block diagrams and their accompanying descriptions should not be construed as requiring a particular architecture or configuration.

Claims

1. A biometric access control method for a vehicle, comprising: using an image sensor to capture images of two or more individuals approaching a vehicle; a facial recognition module analyzing the captured images to determine whether one or more individuals approaching the vehicle are authorized users of the vehicle; an image analysis module analyzing the captured image to determine an emotion level between the authorized user of the vehicle and an unauthorized user of the vehicle, where at least one but not all of the two or more individuals approaching the vehicle are identified as authorized users of the vehicle, wherein the determination of the emotion level between the identified authorized user of the vehicle and the unauthorized user of the vehicle is weighted using the environmental information; and performing a vehicle access security operation based on the determined emotion level between an authorized user of the vehicle and an unauthorized user of the vehicle, Among them, vehicle access security operations include: In the event that the interaction between the authorized user of the vehicle and the unauthorized user of the vehicle is determined to be a non-hostile interaction, opening a door for each approaching individual; and When an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is determined to be a hostile interaction, the door is opened only for the authorized user of the vehicle.

2. The method according to claim 1, wherein: Opening a door includes unlocking the door or physically opening the door.

3. The method according to claim 1, wherein: The vehicle access security operation includes not opening any doors of the vehicle when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

4. The method according to claim 1, further comprising: When an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction, an alarm is activated.

5. The method according to claim 1, wherein: Analyzing the captured images to determine whether the one or more individuals approaching the vehicle are authorized users of the vehicle includes performing facial recognition on the captured images of the one or more individuals to determine the identities of the one or more individuals approaching the vehicle.

6. The method according to claim 5, further comprising: A user profile of the identified individual is determined, and the user profile is used to determine a seating position of the identified individual and a vehicle door to open corresponding to the seating position.

7. The method according to claim 6, further comprising: The user profile is used to adjust vehicle configuration settings at the seating position determined for the identified individual.

8. The method according to claim 1, further comprising: analyzing the captured images to identify an individual among a plurality of individuals approaching the vehicle; And, based on identifying one or more authorized users of the vehicle approaching the vehicle, a specific door of the vehicle is opened for each identified individual approaching the vehicle who is an authorized user of the vehicle.

9. The method according to claim 1, further comprising: The captured image is analyzed to determine a class of the unidentified individual, and a door of the vehicle is determined to be opened for the individual based on the determined class of the individual.

10. The method according to claim 1, wherein: The environmental information includes at least one of the following items: Area-defining information or geographic information for adjusting vehicle access security operations based on the security of the geographic area in which the vehicle is parked; and Information related to the authorized user's calendar data, or the authorized user's message data, or the authorized user's habits.

11. A vehicle control system, comprising: processor; as well as A memory coupled to the processor to store instructions that, when executed by the processor, cause the processor to perform operations including: capturing images of two or more individuals approaching a vehicle; analyzing the captured images to determine whether one or more individuals approaching the vehicle are authorized users of the vehicle; analyzing the captured images to determine an emotion level between the authorized user of the vehicle and an unauthorized user of the vehicle, where at least one but not all of the two or more individuals approaching the vehicle are identified as authorized users of the vehicle, wherein the determination of the emotion level between the identified authorized user of the vehicle and the unauthorized user of the vehicle is weighted using the environmental information; and performing a vehicle access security operation based on the determined emotion level between an authorized user of the vehicle and an unauthorized user of the vehicle, Among them, vehicle access security operations include: In the event that the interaction between the authorized user of the vehicle and the unauthorized user of the vehicle is determined to be a non-hostile interaction, opening a door for each approaching individual; and When an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is determined to be a hostile interaction, the door is opened only for the authorized user of the vehicle.

12. The system according to claim 11, wherein: Opening a door includes unlocking the door or physically opening the door.

13. The system according to claim 11, wherein: The vehicle access security operation includes not opening any doors of the vehicle when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

14. The system according to claim 11, wherein: The operations also include activating an alarm when an interaction between an authorized user of the vehicle and an unauthorized user of the vehicle is deemed to be a hostile interaction.

15. The system according to claim 11, wherein: Analyzing the captured images to determine whether the one or more individuals approaching the vehicle are authorized users of the vehicle includes performing facial recognition on the captured images of the one or more individuals to determine the identities of the one or more individuals approaching the vehicle.

16. The system of claim 15, wherein: The operations also include determining a user profile for the identified individual and using the user profile to determine a seat position for the identified individual and a door to open corresponding to the seat position.

17. The system of claim 16, wherein: The operations also include using the user profile to adjust vehicle configuration settings at a seating position determined for the identified individual.

18. The system of claim 11, wherein: The operation also includes: analyzing the captured image to identify an individual among multiple individuals approaching the vehicle; and, based on the identification of one or more authorized users of the vehicle approaching the vehicle, opening a specific door of the vehicle for each identified individual approaching the vehicle as an authorized user of the vehicle.

19. The system of claim 11, wherein: The operations also include analyzing the captured image to determine a category of the unidentified individual, and determining a door of the vehicle to open for the individual based on the determined category of the individual.

20. The system of claim 11, wherein: The environmental information includes at least one of the following items: Area-defining information or geographic information for adjusting vehicle access security operations based on the security of the geographic area in which the vehicle is parked; and Information related to the authorized user's calendar data, or the authorized user's message data, or the authorized user's habits.

Images