Camera sensor actuation

Abstract

The present disclosure provides "camera sensor actuation". An example system may include a computer having a processor coupled to a memory. The memory may store instructions executable by the processor to process the received Doppler radar signal to extract a feature representative of a characteristic of the moving object. In response to determining that the moving object does not include the authorized user based on the extracted features, the computer may actuate the camera sensor.

Description

Technical Field

[0001] The present disclosure relates to camera sensor actuation in a vehicle system. Background Art

[0002] Systems or environments such as vehicles may be equipped with security features. For example, security features of a vehicle may determine whether a person attempting to enter and / or operate the vehicle is an authorized user. In some cases, security-related components such as camera sensors and computerized image processing may represent significant vehicle sensing and processing resources. Summary of the invention

[0003] The present disclosure describes techniques for camera sensor actuation in a system, which may include a vehicle system. However, although the exemplary systems disclosed herein may include one or more camera sensors mounted on a vehicle, camera sensor actuation may be used to control access to other types of systems, such as systems for controlling access to facilities within a building or a building, or for controlling access to complex machinery other than a vehicle system. The camera sensor may have a field of view that allows the sensor to observe stationary or moving objects located in any direction relative to the vehicle when the vehicle is traveling along a path or when the vehicle is deactivated, parked (i.e., stationary) and unattended. The camera sensor may be actuated in response to the vehicle being deactivated, parked, and unattended by a vehicle user (e.g., an operator or a passenger) to operate to monitor an area outside the vehicle for an extended period of time. Such monitoring of the environment outside the vehicle may deter vehicle theft, vehicle intrusion, damage, vandalism, etc., as well as provide a reporting channel from the vehicle to the vehicle's owner, insurance company, security service company, etc. The onboard camera sensor may also be used in image recognition vehicle applications, where the image processing programming of the vehicle computer may be utilized to capture and process images of people approaching the vehicle. Such image processing may allow the vehicle computer to determine whether an approaching person is an authorized user of the vehicle, such as the operator or an authorized passenger.

[0004] However, camera sensor monitoring of an area external to a deactivated vehicle may result in a significant drain on the vehicle's battery resources. Accordingly, the duration of camera sensor operation may be reduced in order to conserve battery resources so that such resources may be used to activate (e.g., actuate) the vehicle's ignition, and / or activate various battery-powered vehicle systems. In response to reducing the duration of camera sensor monitoring of an environment external to a deactivated vehicle, the vehicle may be more susceptible to theft, intrusion, damage, vandalism, etc. Additionally, the ability to obtain still or video images of persons involved in such incidents (e.g., for subsequent investigations) may be diminished. Furthermore, other vehicle convenience features (e.g., illuminating cabin lighting when an authorized user (e.g., an authorized operator or passenger) approaches the vehicle, unlocking vehicle doors to allow keyless entry to the vehicle, etc.) may be unavailable due to the loss of battery resources to perform these functions.

[0005] As described herein, vehicle radar can be used to monitor the external environment of the vehicle. Radar can be used to replace or supplement the camera sensor monitoring of the environment outside the deactivated vehicle. Therefore, the vehicle radar (which can be used to detect static or moving objects in the vehicle environment during driving operation) can also be used to provide monitoring of the external environment of the vehicle when the vehicle is parked, deactivated and unattended. As described herein, the vehicle radar can be operated in a Doppler mode, which can be operated to transmit a Doppler signal and determine whether the mobile object represents an authorized user of the vehicle, such as an operator or a passenger, based on the Doppler signal received from the mobile object. In an example, in response to the Doppler radar detecting that the person approaching the vehicle does not represent an authorized user, the vehicle computer can selectively actuate the camera sensor to record an image of the approaching mobile object, and then deactivate the camera sensor after recording the image of the unauthorized person and / or transmitting the image to the server via a wireless communication link. In another example, in response to the Doppler radar detecting that the person approaching the vehicle represents an authorized user, the vehicle computer can maintain the camera sensor in a deactivated state. Due to this selective use of the camera sensor and computerized image processing, battery resources can be conserved to make them available so that the vehicle computer can actuate the vehicle engine ignition and actuate convenience features such as allowing keyless entry to the vehicle, illuminating the interior lighting, etc.

[0006] In an example, a system may include a computer having a processor coupled to a memory. The memory may store instructions executable by the processor to process a received Doppler radar signal to extract features representative of characteristics of a moving object. The instructions may further actuate a camera sensor in response to determining that the moving object does not include an authorized user based on the extracted features.

[0007] In an example, the characteristic may be at least one of a gait, heel strike, arm swing, head nod, twist, or turn of the authorized user.

[0008] In an example, the Doppler radar signal may include a dominant signal reflected from the moving object, and wherein the extracted features indicate a characteristic gait based on the dominant signal.

[0009] In an example, the instructions may further include instructions for calculating a composite signature of an authorized user based on a relationship between two or more extracted features of the Doppler radar signal.

[0010] In an example, the instructions may further include instructions for determining a trajectory of a moving object based on a dominant characteristic of the Doppler radar signal.

[0011] In an example, the instructions may further include instructions for determining a trajectory of the moving object and assigning a threat index to the moving object based on the trajectory.

[0012] In an example, the instructions may further include instructions for increasing a threat index based on the trajectory indicating movement of the mobile object toward the system.

[0013] In an example, the camera sensor may include an onboard camera sensor having a field of view that includes the location of the moving object.

[0014] In an example, the camera sensor may include an onboard camera sensor having a field of view that includes a moving object, wherein actuation of the camera sensor includes capturing a video image of the moving object.

[0015] In an example, the instructions may further include instructions for maintaining the camera sensor in a deactivated state in response to determining that the moving object includes an authorized user.

[0016] In an example, the instructions further include instructions for transitioning the system from the Doppler radar mode of operation to the Bluetooth mode of operation after executing the instructions for actuating the camera sensor.

[0017] In an example, the instructions may further include instructions for refining the computer model including characteristics of the mobile object based on successfully recognizing that the mobile object includes an authorized user.

[0018] In another example, a method may include processing a received Doppler radar signal to extract features representative of characteristics of a moving object. The method may further include determining that the moving object does not include an authorized user based on the extracted features and actuating a camera sensor.

[0019] In an example, the characteristic may be at least one of a gait, heel strike, arm swing, head nod, twist, or turn of the authorized user.

[0020] In an example, the method may further include determining a trajectory of the moving object based on dominant characteristics of the Doppler radar signal reflected from the moving object.

[0021] In an example, the method may additionally include determining a trajectory of the moving object and assigning a threat index to the moving object based on the trajectory.

[0022] In an example, the method may additionally include assigning an increased threat index based on the trajectory indicating movement of the mobile object toward the camera sensor.

[0023] In an example, the method may additionally include maintaining the camera sensor in a deactivated state in response to determining that the moving object includes an authorized user.

[0024] In an example, the method may additionally include transitioning the Doppler radar to a Bluetooth mode of operation after actuating the camera sensor.

[0025] In an example, the method may further include refining the computer model of the characteristics comprising the authorized user based on successfully identifying that the mobile object comprises the authorized user. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 An exemplary vehicle system is shown.

[0027] Figure 2 Exemplary components involved in performing camera sensor actuation are shown.

[0028] Figure 3 to Figure 4 Exemplary Doppler signal characteristics are shown.

[0029] Figure 5 is an exemplary flow chart for camera sensor actuation. DETAILED DESCRIPTION

[0030] Figure 1An exemplary system 100 for operating sensors 108 of a vehicle 102 is shown. A computer 104 of the vehicle 102 may be programmed to receive and process signals representing data output from various sensors 108, including camera sensors 108A. In an example, the output data from the camera sensors 108A may include data representing fixed or moving objects relative to the vehicle 102 when the vehicle is in motion or when the vehicle is parked, deactivated, and unattended. The output data from the camera sensors 108A may be transmitted to the computer 104 using, for example, a vehicle network 106. In response to receiving the digitized image, the programming of the computer 104 may execute any suitable image processing programming that operates to extract features of the received static or video image. Thus, in an example, in response to a person 120 approaching the vehicle 102, one or more of the camera sensors 108A may capture an image of facial features of the person 120. The digitized image may be processed to extract facial features of the person 120. The extracted facial features of the person 120 may then be compared with extracted facial features of an authorized user (e.g., an operator or passenger) of the vehicle 102. In response to a threshold level of correlation (e.g., 0.85, 0.9, 0.95) between the extracted facial features of person 120 and facial features of an authorized user (e.g., an operator or passenger) of vehicle 102 stored in a memory accessible to computer 104, the vehicle may illuminate the vehicle's cabin lighting, start the vehicle's engine, actuate the vehicle's horn, deactivate (e.g., unlock) a rear or front vehicle trunk, actuate a vehicle liftgate, activate a vehicle's climate control system, actuate a pattern of exterior vehicle lighting, illuminate areas near a vehicle entrance or on a door handle, liftgate, front trunk, and / or rear trunk, etc. In some cases, in response to a lower correlation, certain vehicle features may be actuated, such as illuminating an area near a vehicle entry point. In response to a higher correlation, other vehicle features may be actuated, such as actuation of door locks, a vehicle ignition, etc. In an example, a correlation greater than a threshold level (eg, 0.85, 0.9, 0.95) indicates a high level of confidence that the extracted facial features of person 120 correspond to facial features of an authorized operator or occupant of vehicle 102 .

[0031] The sensors 108 of the vehicle 102 additionally include a radio frequency (RF) sensor 108B. Figure 1In an example of , RF sensor 108B may include a radar sensor, such as a Doppler radar sensor, that operates to provide recognition of person 120 based on a Doppler signal signature representative of the person. In this context, a person's "Doppler signal signature" means a signature of a reflected Doppler signal that can identify a person based on the person's movement in an environment, such as an environment outside vehicle 102. The Doppler signal signature includes a "dominant Doppler signal signature," which means a reflected Doppler signal that has an amplitude proportional to the body size of person 120 and has a frequency that is shifted relative to the transmitted Doppler radar signal, i.e., generally increases in proportion to the person's gait speed (e.g., 5.0 km / hour, 6.0 km / hour, 7.0 km / hour, etc.). Thus, the dominant Doppler signal signature provides a measure of the body size and gait speed of person 120. The Doppler signal characteristics representing a person additionally include Doppler sideband characteristics, which means frequencies outside the frequency of the dominant Doppler signal characteristics, which provide a measure of the speed of the limbs of the person 120 (e.g., the speed of a characteristic arm swing or a characteristic leg movement) and the amplitude of the return signal from the limbs of the person 120. In this context, the term "characteristic" means a distinguishable movement pattern of an object (such as a person 120). Therefore, when a person (e.g., person 120) walks toward or away from the vehicle 102, the person exhibits a characteristic movement. Therefore, in an example, when the person 120 approaches the vehicle 102, the limbs of the person 120 may move intermittently and characteristically at a speed greater than the gait speed of the person 120. In another example, the characteristic intermittent movement of the legs of the person 120 may start when the person moves the first leg forward and end when the heel of the first leg strikes or contacts the ground. In another example, when the arms of the person swing while walking, the characteristic movement of the arms and hands of the person 120 may move intermittently at a speed greater than the gait speed of the person. In other examples, when person 120 walks toward or away from vehicle 102, or walks in a lateral direction of vehicle 102 (e.g., in a direction from the left side of vehicle 102 to the right side of vehicle 102), other intermittent movements (such as turning, nodding, twisting, hip swinging, etc.) may produce Doppler signal characteristics that can characterize person 120.

[0032] In addition, and as referenced below Figure 3Described in more detail, in response to person 120 being of medium stature, the gait speed of first person 120 may be less than the gait speed of a relatively tall person (e.g., second person 220). Therefore, the dominant Doppler signal characteristic representing the gait of person 120 may include a frequency shift that is less than the Doppler signal characteristic of a relatively tall person (e.g., person 220). In addition, also in response to person 120 being of medium stature, the Doppler sideband characteristics that provide a measure of the characteristic arm and leg movements of person 120 may be less than the Doppler sideband characteristics of a relatively tall person. In addition, in response to person 120 having relatively large-sized arms and / or legs, the amplitude of the Doppler sideband characteristics may be greater than the amplitude of the Doppler sideband characteristics of a person of similar stature having arms and / or legs that are smaller than the arms and / or legs of person 120.

[0033] Similar to extracting facial features to determine whether person 120 represents an authorized user (e.g., an authorized operator or passenger) of vehicle 102, Doppler signal features may be utilized to determine whether person 120 represents an authorized user (e.g., an authorized operator or passenger) of vehicle 102. In an example, in response to a first correlation threshold level (e.g., 0.85, 0.9, 0.95) between the Doppler signal features representing person 120 and Doppler signal features representing an authorized user (e.g., an operator or passenger) of vehicle 102 stored in a memory accessible to computer 104, the vehicle may illuminate cabin lighting of the vehicle, start the engine of the vehicle, actuate the horn of the vehicle, etc. In response to determining the first correlation level between the Doppler signal features of person 120 and the Doppler signal features of the authorized user, camera sensor 108A may remain deactivated, thereby conserving battery resources of vehicle 102.

[0034] In another example, in response to a second threshold level of correlation (e.g., 0.7, 0.75, 0.8) between the Doppler signal characteristics of the person 120 and the Doppler signal characteristics of the authorized user of the vehicle 102, the programming of the computer 104 may actuate one or more of the camera sensors 108A. Actuation of one or more of the camera sensors 108A may cause the camera sensors to capture a still image or video image of the person 120 for transmission to the server 140 over the network 130 via the communication component 114. The server 140 may store the features extracted from the still image or video image of the person 120, 220 for use by the owner of the vehicle 102, an insurance company, a security organization, etc.

[0035] The RF sensor 108B can include at least two operating modes. In a first operating mode, the sensor 108B can perform Doppler radar functions, including transmitting Doppler radar signals to an area outside the vehicle 102 and processing received Doppler radar signals reflected from moving objects within the field of view 115. With respect to moving objects, the RF sensor 108B can determine Doppler signal characteristics, which include a dominant Doppler signal characteristic and one or more Doppler side-frequency characteristics as discussed above. In addition, the RF sensor 108B can transition from the Doppler mode to the Bluetooth positioning mode in response to a control signal from the computer 104. In response to this transition, the RF sensor 108B can locate an authorized user, such as an operator or passenger, based on credentials stored in the authorized user's mobile communication device 122. Figure 1 In the example of , the Doppler radar operation utilizes a frequency substantially similar to the frequency used for the Bluetooth operating mode, such as a frequency between 5.0 GHz and 20.0 GHz. Thus, in the example, the RF sensor 108B can be operated to perform Doppler radar signal transmission, reception, and Doppler signal feature extraction, and then transition to a Bluetooth positioning mode that allows the mobile communication device 122 to operate in a phone-as-key mode. In the phone-as-key mode, the computer 104 can actuate a door unlocking mechanism to allow keyless entry to the vehicle 102, start the engine of the vehicle 102, and / or actuate lighting devices, such as interior lights, headlights, running lights. Etc. Thus, the person 120 can be allowed to operate the vehicle 102 without using a physical key, key fob, or other dedicated device.

[0036] The sensors 108 of the vehicle 102 may include sensors other than the camera sensor 108A and the RF sensor 108B, such as sensors for monitoring and detecting various aspects of the driving environment of the vehicle. Such sensors may include long-range radar sensors, lidar sensors, ultrasonic transducers, motion detectors, etc. In addition, the vehicle 102 may include sensors for measuring internal vehicle states and modes, such as sensors for measuring engine speed (rpm), such as tachometers, wheel speed sensors, tire pressure sensors, etc.

[0037] The computer 104 may generally be programmed to communicate over a vehicle network 106, which may include a communication bus such as a CAN bus, a LIN bus, etc., and / or other wired and / or wireless technologies such as Ethernet, WIFI, Ultra-wideband (UWB), etc. The vehicle network 106 may represent one or more mechanisms by which the computer 104 can communicate with a remote computer (such as a remote server). Thus, the vehicle network 106 may include one or more of a variety of wired or wireless communication mechanisms, including any desired combination of wired (e.g., cable and fiber optic) and / or wireless (e.g., cellular, wireless, satellite, microwave, and radio frequency) communication mechanisms and any desired network topology (or multiple topologies when multiple communication mechanisms are used). Exemplary communication networks include wireless communication networks that provide data communication services (e.g., using Low Energy (BLE), UWB, IEEE 802.11, Vehicle-to-Vehicle (V2V) such as Dedicated Short Range Communication (DSRC), etc.), Local Area Network (LAN) and / or Wide Area Network (WAN) including the Internet.

[0038] Alternatively or additionally, in an example where the computer 104 includes multiple devices, the vehicle network 106 may be used for communication between devices represented as the computer 104. In an example, the computer 104 may be a general-purpose computer having a processor and memory as described above, and / or may include an electronic control unit (ECU) or controller, etc., for a specific function or set of functions, and / or may include a dedicated electronic circuit including an ASIC manufactured for a specific operation, such as an ASIC for processing sensor data and / or transmitting sensor data. In another example, the computer 104 may include an FPGA (field programmable gate array), which is an integrated circuit manufactured to be configurable by an authorized user. Typically, hardware description languages ​​such as VHDL (very high speed integrated circuit hardware description language) are used in electronic design automation to describe digital and mixed signal systems such as FPGAs and ASICs. For example, an ASIC is manufactured based on VHDL programming provided before manufacturing, and the logic components inside the FPGA may be configured based on, for example, VHDL programming stored in a memory electrically connected to the FPGA circuit. In some examples, a combination of a processor, an ASIC, and / or an FPGA circuit may be included in the computer 104. Additionally, the computer 104 may be programmed to communicate with the communication component 114, the human interface (HMI) 112, and vehicle components using a vehicle network 106, as described below, which may include various wired and / or wireless networking technologies, such as cellular, Low energy (BLE), UWB, wired and / or wireless packet networks, etc.

[0039] The vehicle 102 may include various additional devices, such as vehicle components 110, which may utilize the vehicle network 106 to transmit and receive data related to, for example, vehicle speed, acceleration, position, subsystem and / or component status, etc. In an example, additional sensors of the vehicle 102 may provide data for controlling the operation of a component in a set of vehicle components 110, such as a component for controlling an aspect of vehicle operation. The vehicle components 110 may include other hardware entities suitable for performing mechanical or electromechanical functions or operations, such as moving the vehicle 102, slowing or stopping the vehicle, steering the vehicle, etc. The vehicle components 110 may include transmission components, steering components (which may include one or more of a steering wheel, a steering rack, etc.), brake components, parking aid components, adaptive cruise control components, adaptive steering components, movable seats, etc. The vehicle components 110 may also include computing devices, such as electronic control units (ECUs), etc. and / or computing devices such as those described above with respect to the computer 104, and they also communicate via the vehicle network 106.

[0040] Figure 2 An exemplary subsystem 200 involved in performing camera sensor actuation is shown. Figure 2 As seen in FIG. 1 , computer 104 includes user identification programming 104A, which may be stored in a memory accessible to computer 104. User identification programming 104A may include a table or other suitable data structure storing facial features of authorized operators or passengers of vehicle 102. Thus, in response to receiving, for example, a user identification program 104A from camera sensor 108A, Figure 1 In response to determining a threshold correlation (e.g., 0.85, 0.9, 0.95, etc.) between the extracted features of the received image and the stored extracted features of the authorized user (e.g., operator or passenger) of the vehicle 102, the programming of the computer 104 may actuate keyless entry of the vehicle 102, illuminate the cabin lights of the vehicle 102, etc. In response to determining a lower correlation (e.g., less than 0.85), the user identification programming 104A may deactivate the keyless entry of the vehicle 102 and maintain the cabin lights in a deactivated state.

[0041] The user identification programming 104A may additionally include a table storing Doppler signal characteristics (e.g., dominant Doppler signal characteristics and Doppler sideband characteristics) of authorized operators and passengers of the vehicle 102, etc. Thus, in response to receiving a Doppler signal characteristic representing the person 120 from the RF sensor 108B, the user identification programming 104A may determine whether the received Doppler signal characteristic correlates with a stored Doppler signal characteristic of an authorized user (e.g., operator or passenger) of the vehicle 102. In an example, in response to determining a threshold correlation (e.g., 0.85, 0.9, 0.95, etc.) between the received Doppler signal characteristic and the stored Doppler signal characteristic representing the authorized user of the vehicle 102, the programming 104A may actuate keyless entry of the vehicle 102, illuminate the cabin lights of the vehicle 102, etc. In addition, the user identification programming 104A may maintain the camera sensor 108A in a deactivated state. In another example, in response to determining a lower correlation (e.g., less than 0.85), the user identification programming 104A may disable the keyless entry of the vehicle 102 and maintain the cabin lights in a disabled state. In addition, the user identification programming 104A may actuate the camera sensor 108A to capture a still image or video image of the person 120 for transmission to the server 140.

[0042] As reference Figure 1 As stated, Figure 2 The RF sensor 108B can operate in Doppler radar mode (which allows Doppler radar signals to be transmitted to and received from mobile objects), and can also operate in Bluetooth positioning mode. During the Doppler radar mode operation, the RF sensor 108B can transmit signals to mobile objects located outside the vehicle 102 and receive signals reflected from the mobile objects. The RF sensor 108B can then process the reflected Doppler signal to extract Doppler signal features, such as dominant Doppler signal features and Doppler sideband features. The Doppler signal features extracted by the RF sensor 108B can then be transmitted to the computer 104 using the vehicle network 106 to compare them with Doppler signal features representing authorized users of the vehicle 102 (e.g., operators or passengers). The RF sensor 108B can then transition to a Bluetooth phone-as-key mode, in which the RF sensor 108B receives a beacon signal from a mobile communication device 122 to allow the person 120 to operate the vehicle 102 without using a physical key, key fob, or other dedicated device. In an example, the RF sensor 108B transitions between the Doppler radar mode and the Bluetooth phone-and-key mode at intervals between 100 milliseconds and 2.0 seconds.

[0043] Such transitions between operating modes of the RF sensor 108B may allow the sensor 108B to perform dual or double functions under the control of the user identification programming 104A. For example, during a first interval, the user identification programming 104A may command the RF sensor 108B to operate in a Doppler radar mode, in which Doppler radar signals are transmitted to and received from a mobile object outside the vehicle 102. The RF sensor 108B may then receive the signal reflected from the mobile object and extract Doppler signal features, such as the dominant Doppler signal feature 310 and the Doppler sideband feature 315, from the received Doppler radar signal. The RF sensor 108B may then transition to a Bluetooth handset-as-a-key mode to detect a beacon transmitted from the mobile communication device 122.

[0044] Figure 3 1 is a graph showing exemplary Doppler signal characteristics 300 that may be included in a return signal from a moving object, such as person 120. Figure 3 , the vertical axis in each graph represents Doppler signal amplitude, and the horizontal axis represents the frequency of the received Doppler signal. Signal profile 305 shows the amplitude and frequency of a dominant signal generated by a Doppler radar signal reflected from person 120 approaching vehicle 102, as well as first and second frequencies outside of (or beside) the dominant Doppler signal. Signal profile 355 includes a dominant Doppler signal feature (310) and side frequencies (315) generated by a Doppler radar signal reflected from person 220 also approaching vehicle 102. With respect to both persons 120 and 220, and as described with reference to FIG. Figure 1As described, the reflected Doppler radar signal can be shifted in frequency relative to the transmitted Doppler radar signal. Such frequency shift increases in proportion to the gait speed of the approaching person (e.g., 120, 220). Therefore, in response to the people 120 and 220 walking toward the vehicle 102 with their corresponding gait speeds, the received Doppler radar signal includes a frequency that increases relative to the transmitted Doppler radar signal. Therefore, in response to the people 120 and 220 walking away from the vehicle 102, the received Doppler radar signal includes a frequency that decreases relative to the transmitted Doppler radar signal. In addition, when the people 120 and 220 approach the vehicle 102, the limbs (e.g., arms, legs, hands, feet) of the people 120 and 220 can move intermittently at a speed greater than the gait speed of the people. This movement of the legs of the people 120 and 220 can start when each person moves the first leg forward and end when the heel of the first leg contacts or hits the ground (e.g., the heel lands). In addition, when the person's arms swing while walking, the characteristic movements of the arms and hands of the persons 120 and 220 may intermittently move at a speed greater than the person's gait speed. In another example, another part of the body of the person 120 may intermittently move while the person walks, such as the head of the person 120 (e.g., nodding the head), the hips of the person 120 (e.g., twisting or turning the hips), etc.

[0045] Therefore, if Figure 3 As shown, a first exemplary received Doppler radar signal having a profile 305 includes a dominant Doppler signal feature 310 having an amplitude D1 and a frequency F1. In the example, the amplitude (D1) of the dominant Doppler signal feature 310 indicates the body size of the person 120, which can establish the amount of Doppler radar signals reflected by the body of the person 120. For example, in response to the person 120 having a medium build, for example, relative to a larger or taller person (e.g., person 220), the person 120 can reflect a first amplitude of the Doppler signal transmitted by the RF sensor 108B. In addition, according to the gait (walking) speed of the person 120, for example, 5.0 km / hour, 6.0 km / hour, 7.0 km / hour, etc., the frequency (F1) of the dominant Doppler signal feature 310 can be relative to the transmitted Doppler radar signal transmitted by the RF sensor 108B.

[0046] Signal profile 305 further shows a first Doppler side-frequency feature 315 having a frequency of F2 and an amplitude of D2. Figure 3 In the example of FIG. 1 , Doppler sideband feature F2 may correspond to a characteristic arm swing of person 120 as the person approaches vehicle 102. Doppler sideband feature F3 may correspond to a characteristic leg movement of person 120 as the person's legs move toward vehicle 102. Figure 3, first Doppler side-frequency signature 315 (e.g., characteristic arm swing) includes a reduced amplitude signature relative to second Doppler side-frequency signature 320 (F3) (e.g., characteristic leg movement), indicating that the legs of person 120 reflect a greater portion of the transmitted Doppler radar signal than the arms of person 120. Second Doppler side-frequency signature 320 increases relative to first Doppler side-frequency signature 315 (e.g., characteristic arm swing), indicating that the legs of person 120 are moving intermittently at a greater speed than the arms of person 120.

[0047] Figure 3 Also depicted is a graph of a second exemplary signal profile 355 having an amplitude D4 and a frequency F4, the second exemplary signal profile including a dominant Doppler signal signature 360. In an example, the dominant Doppler signal signature 360 ​​is indicative of the stature of the person 220, which may establish the amount of Doppler radar signals reflected by the body of the person 220. For example, in response to the person 220 having a relatively tall or large stature, e.g., relative to the person 120, the dominant Doppler signal signature 360 ​​may be greater than the dominant Doppler signal signature 310. Furthermore, depending on the gait (walking) speed of the person 220, e.g., 6.0 km / hour, 7.0 km / hour, 8.0 km / hour, etc., the frequency (F4) of the dominant Doppler signal signature 360 ​​may be shifted relative to the dominant Doppler radar signature 310 (of the person 120).

[0048] Signal profile 355 additionally shows a Doppler side-frequency characteristic 365 having a frequency of F5 and an amplitude of D5. Signal profile 355 also shows a Doppler side-frequency characteristic 370 having a frequency of F6 and an amplitude of D6. Figure 3 In the example of FIG. 1 , Doppler sideband 365 may correspond to the characteristic arm swing of person 220 as the person approaches vehicle 102. Doppler sideband F6 may correspond to the characteristic leg movement of person 220 as the person's legs move toward vehicle 102. Figure 3 As seen in FIG. 1 , Doppler sidefrequency feature F5 (e.g., characteristic arm swing) includes a reduced amplitude relative to Doppler sidefrequency (F6) (e.g., characteristic leg movement), indicating that the legs of person 120 reflect a larger portion of the Doppler signal transmitted by RF sensor 108B than the arms of person 220.

[0049] Therefore, from Figure 3It can be seen that for the Doppler radar signal emitted by the RF sensor 108B, the signal distribution curves (e.g., 305, 355) including the dominant Doppler signal features (e.g., 310, 360) and the Doppler side-frequency features (e.g., 315, 320, 365, 370) are different according to the physical characteristics of the persons 120 and 220 (i.e., the characteristic arm swing and the characteristic leg movement). In the example where the person 120 is of medium build, the dominant Doppler signal feature 310 includes a frequency F1 and an amplitude D1. In addition, the Doppler side-frequency features 315 and 320 generated by the characteristic arm swing and leg movement of the person 120 exhibit amplitude features D2 and D3, respectively. In addition, when the person 120 moves toward the vehicle 102, the ratio (or other type of numerical relationship) between D1 and D2 and between D1 and D3 remains constant because the cross-sectional area presented by the features of the person 120 (e.g., head, chest, arms, hands, legs, feet, etc.) remains constant.

[0050] Thus, two or more features (e.g., dominant Doppler signal feature 310, Doppler side-frequency feature 315, Doppler side-frequency feature 320) may be combined to represent a composite feature of person 120 calculated or derived from the two or more features. Similarly, in an example where person 220 is larger than person 120, dominant Doppler signal feature 360 ​​includes frequency F4 and amplitude D4. Additionally, Doppler side-frequency features 365 and 370 generated by characteristic arm swing and leg motion exhibit amplitudes D5 and D6, respectively. In an example, a composite amplitude feature of person 120 may be calculated in response to an amplitude (D1) of dominant Doppler signal feature 310 being -60 decibels (dBm) relative to 1 milliwatt to an amplitude (D2) of Doppler side-frequency feature 315 being -75 dBm. In this example, the composite amplitude feature of person 120 may be a ratio D1 / D2, which would be equal to 0.8 (dimensionless). In another example, the composite frequency signature of person 120 may be calculated in response to the frequency (F1) of the dominant Doppler signal signature 310 being 6.01 gigahertz to the frequency (F2) of the Doppler side frequency signature 315 being 6.02 gigahertz. In such an example, the composite frequency signature of person 120 may be the ratio F1 / F2, which would be equal to 0.998 (dimensionless).

[0051] Thus, similar to the case of person 120, as person 220 moves toward vehicle 102, the ratios (or other types of numerical relationships) between D4 and D5 and between D4 and D6 remain constant because the cross-sectional areas presented by the various parts of person 220 (e.g., head, chest, arms, hands, legs, feet, etc.) remain constant. Thus, two or more features (e.g., dominant Doppler signal feature 360, Doppler sideband feature 365, Doppler sideband 370) may form a composite feature of person 220 calculated or derived from the two or more features. Thus, as persons 120 and 220 move from a point relatively far from vehicle 102 to a point relatively close to the vehicle, the overall shape of signal profiles 305 and 355 remains constant, even though each signal profile (305, 355) may be scaled along the received Doppler signal amplitude (vertical) axis.

[0052] Likewise Figure 3 As can be seen in FIG. 2 , the frequency (F4) of the dominant Doppler signal signature 360 ​​of person 220 is shifted relative to the frequency (F1) of the dominant Doppler signal signature 310 of person 120. The frequency difference between the dominant Doppler signal signatures 310 and 320 is given by ΔF D In addition, due to the characteristic arm swing of person 220, Doppler side-frequency signature 365 (F5) of person 220 is shifted in frequency relative to first Doppler side-frequency signature 315 (F2) of person 120, as indicated by ΔF A In addition, due to the characteristic leg motion of person 220, Doppler side-frequency signature 370 (F6) of person 220 is shifted in frequency relative to second Doppler side-frequency signature 320 (F3) of person 120, as indicated by ΔF L Therefore, according to Figure 3 For example, user identification programming 104A can use such differences to identify and distinguish between persons 120 and 220.

[0053] Figure 4 An exemplary Doppler signal characteristic 400 is shown that may be included in a return signal from a moving object, such as a person 120. A signal profile 405 indicates a Doppler signal characteristic generated by processing a received Doppler radar signal reflected from a person 120 leaving or walking away from a vehicle 102. Figure 4As can be seen in FIG. 1 , in response to person 120 walking away from vehicle 102, Doppler side-frequency features 415 and 420 decrease relative to dominant Doppler signal feature 410. For example, Doppler side-frequency feature 415 is indicated as having a lower frequency than dominant Doppler signal feature 410 due to the characteristic arm swing of person 120. Additionally, Doppler side-frequency feature 415 due to the characteristic leg motion of person 120 as the person walks away from vehicle 102 is also indicated as having a lower frequency relative to dominant Doppler signal feature 410 due to the intermittent motion of the legs of person 120.

[0054] Therefore, if Figure 4 , signal profile 405 includes a similar but opposite overall shape relative to signal profile 305. This similarity in the overall shape of signal profiles 405 and 305 is caused by the invariant characteristic gait of person 120, whether person 120 is approaching or leaving vehicle 102. Therefore, dominant Doppler signal feature 410 and Doppler side-frequency features 415 and 420 may be utilized to identify person 120, whether person is approaching or leaving vehicle 102.

[0055] As previously described, the dominant Doppler signal features (e.g., 310, 360, and 410) may be utilized to determine the gait or walking speed of a person (e.g., 120, 220). Thus, in response to detecting that a moving object (e.g., a person) is moving at a speed greater than a walking speed (e.g., 6.0 km / hour, 7.0 km / hour, 8.0 km / hour), the dominant Doppler signal features may indicate that the person (e.g., 120, 220) is running toward the vehicle 102. Thus, based on the speed indicated by the dominant Doppler signal features (310, 360) (e.g., the person 120, 220 is running toward the vehicle 102), the user identification programming 104A may assign a threat index, for example, indicating possible imminent damage to the vehicle 102. In the example, a person is running toward the vehicle 102, which can be indicated by examining the dominant Doppler signal features (310, 360) that are frequency shifted in proportion to the speed of the approaching person, as well as Doppler side frequencies (315, 320, 365, 370) that indicate high-speed arm swinging and leg motion. In such an example, the programming 104A can actuate the camera sensor 108A to begin capturing still images or video images of the approaching person (e.g., 120, 220) for transmission to the server 140 over the network 130 via the communication component 114. The server 140 can store the features extracted from the still images or video images of the person 120, 220 for use by the owner of the vehicle 102, an insurance company, a security organization, etc.

[0056] In another example, in response to a plurality of dominant Doppler signal features indicating that a group of people (e.g., a chaotic group of people) appear to be approaching the vehicle 102, the programming 104A may assign a threat index, e.g., indicating possible imminent damage to the vehicle 102. In this case, a group of chaotic people may be determined by detecting a set of dominant Doppler signal features and a set of Doppler side frequencies by the RF sensor 108B. In response, the programming 104A may actuate the camera sensor 108A so that a still image or video image of the approaching and possibly chaotic group of people may be transmitted to the server 140. In an example, the threat index may be based on a variety of factors, such as the speed of the approaching person (e.g., as determined by the frequency shift of the dominant Doppler signal features (e.g., 310, 360)), the distance between the approaching person and the vehicle 102 (e.g., as determined by the amplitude of the dominant Doppler signal features (e.g., 310, 360) and / or the plurality of Doppler side frequency features (e.g., 315, 320, 365, 370)). The threat index may include values ​​between 0.0 and 1.0, depending on various factors, such as the speed at which the person (120, 220) approaches the vehicle 102, the number of people approaching the vehicle, etc. In an example, a single person (120) approaching the vehicle 102 that has not been identified as an authorized user (e.g., an operator or passenger of the vehicle 102) may be assigned a threat index of 0.4. In another example, a group of people that appear to be moving quickly toward the vehicle 102 may be assigned an increased threat index, such as 0.9. In an example, as shown in FIG. Figure 4 As described, in response to an approaching person (eg, 120 , 220 ) moving away from the vehicle 102 , the programming of the computer 104 may remove the threat index, deactivate the camera sensor 108A, and cease transmitting still and video images to the server 140 .

[0057] In an example, the user identification programming 104A can utilize machine learning programming to improve a computer model that correlates dominant Doppler signal features (e.g., 310, 360, 410) and Doppler side frequency features (e.g., 315, 320, 365, 370, 415, 420) with known or registered users of the vehicle 102. Alternatively or additionally, the machine learning program can be trained to correlate dominant Doppler signal features with Doppler side frequencies to distinguish between a person or group of people running toward the vehicle and a person or group of people walking casually toward the vehicle. In an example, a user of the vehicle 102 (e.g., an operator or a passenger) can undergo a registration process, which allows the operator or passenger to be recognized by the user identification programming 104A. During the registration process, the RF sensor 108B of the vehicle 102 can capture a reflected radar signal from an expected user (e.g., an operator or a passenger of the vehicle). Then, when the expected user (e.g., operator or passenger) approaches the vehicle 102, the RF sensor 108B can extract the dominant Doppler signal features (310, 360, 410) and the Doppler side-frequency features (315, 320, 365, 370, 415, 420). The dominant Doppler signal and side-frequency features can then be used as a training set of input data for a computer model implementing machine learning programming. The machine learning programming of the user identification programming 104A can further train and / or improve the computer model on instances of successful identification (e.g., true positives), incorrect identification (e.g., false positives), and related identification events regarding authorized users (e.g., 120, 220). Thus, over time, the machine learning programming can enhance the computer model of the user so as to improve the programmatic ability of the computer 104 to identify authorized users (e.g., 120, 220).

[0058] In an example, a suitable machine learning program for implementing the computer model may include a deep neural network (DNN) that can be trained and then used to output, filter, and / or predict whether an approaching person (e.g., 120, 220) includes an authorized user (e.g., an operator or a passenger) of the vehicle 102. The DNN may include programming that can be loaded into a memory and executed by a processor included in the computer 104 of the vehicle 102 and / or a processor of the RF sensor 108B. In an exemplary embodiment, the DNN may include, but is not limited to, a convolutional neural network CNN, an R-CNN (region-based CNN), a fast R-CNN, and a faster R-CNN. The DNN includes a plurality of nodes or neurons. The neurons are arranged so that the DNN includes an input layer, one or more hidden layers, and an output layer. The input layer and the output layer may also include more than one node.

[0059] As an example, a DNN can be used to train a computer model using ground truth data (i.e., data about real-world conditions or states). For example, a DNN is used to train a computer model using ground truth data and / or to update it using additional data. For example, weights can be initialized using a Gaussian distribution, and the bias of each node can be set to zero. Training a computer model via a DNN can include updating weights and biases via suitable techniques such as back propagation and optimization. Ground truth data refers to data that is considered to represent a real-world environment (e.g., conditions and / or objects in the environment). Thus, ground truth data can include sensor data that depicts the environment (e.g., time of day, location, weather conditions, etc.), and one or more tags that describe the environment, such as tags that describe data (e.g., calendar events), received previously accessed messages, etc.

[0060] In an example, the user identification program 104A may additionally utilize the time sequence of the identification of the person (eg, 120, 220) to determine whether the person is facing (eg, referring to Figure 3 The vehicle 102 moves forward or moves away from the vehicle (as shown in the reference Figure 4 ). Thus, over time, the user identification programming 104A can determine a trajectory or other type of path over a time series of Doppler signal profiles (305, 355, 405). In an example, in response to the person 120 moving in a zigzag manner (where the person briefly moves toward the vehicle 102 and then moves away from the vehicle 102), the user identification programming 104A can detect such movement by evaluating changes in the amplitude of the dominant Doppler signal features (e.g., 310, 360, 410). The time series can extend over any suitable duration, such as 5 seconds, 10 seconds, 15 seconds, etc.

[0061] Figure 51 is an exemplary flow chart for camera sensor actuation. In an exemplary embodiment, the programming of the computer 104 utilizes the camera sensor 108A to view an area outside the vehicle after the vehicle 102 is deactivated. In the process of determining whether to actuate the camera sensor 108A, the vehicle 102 may utilize the RF sensor 108B to transmit a Doppler radar signal to the environment outside the vehicle 102. In response to receiving a signal reflected from a moving object (e.g., a person 120, 220) in the external environment, the RF sensor 108B may extract features from the received Doppler radar signal. Such features may include dominant Doppler signal features (e.g., 310, 360, 410), whose amplitude (e.g., D1, D4) may be used to indicate the body size of the person (120, 220) approaching the vehicle 102. The Doppler signal features may additionally include Doppler sideband features, such as 315, 320, 365, 370, 415, 420, to determine whether the moving object includes an authorized user of the vehicle 102, such as an operator or a passenger. In response to determining that the moving object does not include an authorized user, the user recognition programming 104A may actuate the camera sensor 108A to record a still image or a video image for transmission to the server 140 via the communication component 114 and the network 130. In response to determining that the moving object includes an authorized user, the programming of the computer 104 may maintain the camera sensor 108A in a deactivated state, and may actuate components to unlock one or more doors of the vehicle 102, actuate interior lighting of the vehicle 102, etc. In an example, the RF sensor 108B may operate in a Doppler radar mode, or in a Bluetooth-based phone-as-a-key mode of operation, and may intermittently transition between the Doppler radar mode and the Bluetooth mode of operation.

[0062] Prior to initiating process 500, the programming of computer 104 and / or server 140 may execute a machine learning program as a separate process to detect authorized users (e.g., operators or passengers) of vehicle 102. In an example, RF sensor 108B may capture a reflected radar signal from an intended user (e.g., an operator or passenger of vehicle 102). Then, as the intended user (e.g., operator or passenger) approaches vehicle 102, RF sensor 108B may develop a dominant Doppler signal signature (310, 360, 410) and Doppler sideband signatures (315, 320, etc.). The machine learning programming of user identification programming 104A may further train and improve the computer model on instances of successful identifications (e.g., true positives), incorrect identifications (e.g., false positives), and related identification events of authorized users.

[0063] Process 500 may begin at block 510, which includes transmitting a Doppler radar signal to an area outside of vehicle 102, such as via RF sensor 108B. Transmission of the Doppler radar signal may be performed when vehicle 102 is disabled, parked, and unattended as a safety measure to monitor the area outside of vehicle 102.

[0064] Process 500 may continue at block 515, which includes processing signals received from a moving object, such as a person 120, 220. Block 515 may additionally include processing the received signals to determine dominant Doppler signal characteristics (310, 360, 410) and Doppler side frequency characteristics (315, 320, 365, 370, 415, 420).

[0065] Process 500 may continue at block 520 where the Doppler signal characteristics, such as the dominant Doppler signal characteristics (310, 360, 410) and the Doppler sideband characteristics (315, 320, 365, 370, 415, 420), are transmitted to the user identification programming 104A. The programming 104A may utilize a table or other data structure stored in a memory accessible to the programming 104A to determine a correlation between the Doppler signal characteristics and signal characteristics of an authorized user (e.g., an operator or a passenger) of the vehicle 102.

[0066] Process 500 may continue at block 525, where programming 104A may utilize the time series of the person (120, 220) to determine a trajectory or other type of path over the time series of the signal profile (305, 355, 405). The time series of identification of the person (120, 220) may extend over any suitable duration, such as 5 seconds, 10 seconds, 15 seconds, etc.

[0067] Process 500 may continue at block 530, where programming 104A may utilize the trajectory determined at block 525 to determine whether the person (120, 220) is moving toward or away from the vehicle 102. In an example, such as reference Figure 3 As described, in response to the Doppler sideband characteristics (315, 320, 365, 370, 415, 420) including a frequency greater than the dominant Doppler signal characteristic (310, 360, 410), it can be determined that the person (120, 220) is moving toward the vehicle 102. In another example, such as reference Figure 4 In response to the Doppler sideband characteristics including a frequency less than the dominant Doppler signal characteristic, it can be determined that the person is moving away from the vehicle 102 (moving away). In response to the decision at block 530, the process 500 can proceed to block 545, where the programming of the computer 104 can convert the RF sensor 108B to perform the phone-as-key function as discussed below.

[0068] In response to the decision at block 530 that the mobile object is moving toward the vehicle 102, process 500 may continue at block 535, where the user identification program 104A may assign a threat index to the mobile object. In an example, the threat index may be based on a variety of factors, such as the speed of the approaching person (e.g., as determined by the frequency shift of the dominant Doppler signal feature (310, 360)), the distance between the approaching person and the vehicle 102 (e.g., as determined by the amplitude of the dominant Doppler signal feature (310, 360) and / or the plurality of different dominant Doppler signal features (310, 360, 410) and the Doppler side-frequency features (315, 320, 365, 370)).

[0069] Process 500 may continue at block 540 where the programming of computer 104 may actuate camera sensor 108A, which causes the camera sensor to capture a still image or video image of person 120 for transmission to server 140 over network 130 via communication component 114 .

[0070] Process 500 may continue at block 545 where programming of computer 104 may transition RF sensor 108B from Doppler radar mode to Bluetooth handset-or-key mode, which allows a person (eg, person 120 ) to unlock vehicle doors, illuminate interior lighting, etc. using mobile communication device 122 .

[0071] Process 500 may continue at block 550 where RF sensor 108B may perform a Bluetooth phone-as-a-key function. Such a function may include receiving a signal from a Bluetooth low energy beacon from a mobile communication device 122 of person 120.

[0072] Process 500 may continue at block 555 where the programming of computer 104 may transition RF sensor 108B to perform Doppler radar functionality. In response to the transition at block 555, process 500 may return to block 510.

[0073] After block 555, process 500 ends.

[0074] Descriptions of various examples and embodiments have been presented for purposes of illustration, but the descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, practical applications, or technical enhancements to technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0075] It should be understood that the described methods and systems may be implemented as a computer program product. The computer program product may include a computer-readable storage medium (or multiple computer-readable storage media) having computer-readable program instructions thereon for causing a processor to perform the operations discussed herein.

[0076] The computer readable storage medium may be a tangible device that can store and store instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes the following: a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), a memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device (such as a punched card or a raised structure in a groove on which instructions are recorded) and any suitable combination of the foregoing. The computer readable storage medium as used herein should not be interpreted as being a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted by a wire.

[0077] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, transmission optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0078] The computer-readable program instructions for implementing the operation can be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional process programming languages, such as "C" programming language or similar programming languages. The computer-readable program instructions can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer by any type of network including a local area network (LAN) or a wide area network (WAN), or can be (for example, using an Internet service provider to form a connection with an external computer through the Internet). In some embodiments, the electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) can perform computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0079] Various embodiments of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products. It should be understood that each box in the flowchart illustration and / or block diagram and the combination of boxes in the flowchart illustration and / or block diagram can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a means for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device, and / or other device to function in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other apparatus, to cause a series of operating steps to be performed on the computer, other programmable device, or other apparatus to produce a computer-implemented process, such that the instructions executed on the computer, other programmable device, or other apparatus implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0082] The flow charts and block diagrams in the accompanying drawings illustrate the architecture, functions and operations of possible implementations of systems, methods and computer program products. In this regard, each box in the flow chart or block diagram can represent a module, a fragment or a portion of an instruction, and the instruction includes one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the function indicated in the box can appear in a different order than that indicated in the accompanying drawings. For example, the two boxes shown in succession can in fact be executed substantially in parallel, or the boxes can sometimes be executed in reverse order, depending on the functionality involved. It will also be noted that each box in the block diagram and / or the flow chart diagram and the combination of boxes in the block diagram and / or the flow chart diagram can be implemented by a system based on dedicated hardware that performs a specified function or action or implements a combination of dedicated hardware and computer instructions.

[0083] Unless expressly indicated to the contrary herein, all terms used in the claims are intended to be given their ordinary and customary meanings as understood by those skilled in the art. Specifically, unless a claim recites an express limitation to the contrary, the use of singular articles such as "a," "an," "the," and "said" should be interpreted as reciting one or more of the indicated elements. The use of "in response to" and "upon determining..." indicates a causal relationship, not just a temporal relationship.

[0084] The present disclosure has been described in an illustrative manner, and it is to be understood that the terminology which has been used is intended to be in the nature of words of description rather than limitation. Many modifications and variations of the present disclosure are possible in light of the above teachings, and the present disclosure may be practiced in other ways than specifically described.

[0085] According to the present invention, a system is provided, the system having: a computer, the computer including a processor connected to a memory, the memory storing instructions, the instructions executable by the processor to: process a received Doppler radar signal to extract features representing characteristics of a moving object; and actuate a camera sensor in response to determining that the moving object does not include an authorized user based on the extracted features.

[0086] According to an embodiment, the characteristic is at least one of a gait, a heel strike, an arm swing, a nod, a twist or a rotation of the authorized user.

[0087] According to an embodiment, the Doppler radar signal comprises a dominant signal reflected from the moving object, and wherein the extracted features are indicative of a characteristic gait based on the dominant signal.

[0088] According to an embodiment, the instructions further comprise instructions for calculating a composite feature of the moving object based on a relationship between two or more extracted features of the Doppler radar signal.

[0089] According to an embodiment, the instructions further comprise instructions for determining a trajectory of the mobile object based on a dominant characteristic of the Doppler radar signal.

[0090] According to an embodiment, the instructions further include instructions for: determining a trajectory of the mobile object; and assigning a threat index to the mobile object based on the trajectory.

[0091] According to an embodiment, the instructions further include instructions for increasing the threat index based on the trajectory indicating movement of the mobile object towards the system.

[0092] According to an embodiment, the camera sensor comprises an on-board camera sensor having a field of view including the position of the moving object.

[0093] According to an embodiment, the camera sensor comprises an onboard camera sensor having a field of view that includes the moving object, and wherein actuation of the camera sensor comprises capturing a video image of the moving object.

[0094] According to an embodiment, the instructions further include instructions for maintaining the camera sensor in a deactivated state in response to determining that the moving object includes the authorized user.

[0095] According to an embodiment, the instructions further include instructions for transitioning the system from a Doppler radar mode of operation to a Bluetooth mode of operation after executing the instructions for actuating the camera sensor.

[0096] According to an embodiment, the instructions further comprise instructions for refining the computer model comprising the characteristics of the mobile object based on successfully recognizing that the mobile object comprises the authorized user.

[0097] According to the present invention, a method includes: processing a received Doppler radar signal to extract features representative of characteristics of a moving object; determining that the moving object does not include an authorized user based on the extracted features; and actuating a camera sensor.

[0098] In one aspect of the invention, the characteristic is at least one of a gait, heel strike, arm swing, head nod, twist or turn of the authorized user.

[0099] In one aspect of the invention, the method comprises determining a trajectory of the mobile object based on dominant characteristics of the Doppler radar signal reflected from the mobile object.

[0100] In one aspect of the invention, the method comprises determining a trajectory of the mobile object and assigning a threat index to the mobile object based on the trajectory.

[0101] In one aspect of the invention, the method includes assigning an increased threat index based on the trajectory indicating movement of the mobile object towards the camera sensor.

[0102] In one aspect of the invention, the method includes maintaining the camera sensor in a deactivated state in response to determining that the moving object includes the authorized user.

[0103] In one aspect of the invention, the method comprises transitioning the Doppler radar to a Bluetooth mode of operation after actuating the camera sensor.

[0104] In one aspect of the invention, the method includes refining the computer model of the characteristics including the authorized user based on successfully identifying that the mobile object includes the authorized user.

Claims

1. A method comprising: processing the received Doppler radar signal to extract features indicative of characteristics of the moving object; determining, based on the extracted features, that the mobile object does not include an authorized user; as well as Actuate the camera sensor.

2. The method of claim 1, wherein the characteristic is at least one of a gait, heel strike, arm swing, head nod, twist, or rotation of the authorized user. 3 . The method of claim 1 , wherein the Doppler radar signal comprises a dominant signal reflected from the moving object, and wherein the extracted features indicate a characteristic gait based on the dominant signal.

4. The method according to claim 1, further comprising: A composite feature of the moving object is calculated based on a relationship between two or more extracted features of the Doppler radar signal.

5. The method according to claim 1, further comprising: The trajectory of the moving object is determined based on dominant characteristics of the Doppler radar signal.

6. The method according to claim 1, further comprising: determining a trajectory of the moving object; as well as A threat index is assigned to the mobile object based on the trajectory.

7. The method according to claim 6, further comprising: The threat index is increased based on the trajectory indicating movement of the mobile object toward a vehicle on which the camera sensor is mounted. 8 . The method of claim 1 , wherein the camera sensor comprises an onboard camera sensor having a field of view that includes the location of the moving object.

9. The method of claim 1, wherein the camera sensor comprises an onboard camera sensor having a field of view that includes the moving object, and wherein actuating the camera sensor comprises capturing a video image of the moving object.

10. The method according to claim 1, further comprising: The camera sensor is maintained in a deactivated state in response to determining that the moving object includes the authorized user.

11. The method according to claim 1, further comprising: After actuating the camera sensor, the radar generating the Doppler radar signal is transitioned from a Doppler radar mode of operation to a Bluetooth mode of operation.

12. The method according to claim 1, further comprising: A computer model including the characteristics of the mobile object is improved based on successfully identifying that the mobile object includes the authorized user.

13. A computer programmed to perform the method according to any one of claims 1 to 12.

14. A vehicle comprising a computer according to claim 13.

Images