SYSTEM FOR INTEGRATING VEHICLE SENSORS WITH REMOTE SYSTEM SENSORS

Abstract

A system (10) for integrating vehicle sensors (20) with remote system sensors, the system (10) comprising: a vehicle (12) containing: a vehicle controller (16); at least one vehicle sensor (20) in communication with the vehicle controller (16), the at least one vehicle sensor (20) configured to generate vehicle sensor data associated with a vehicle threat warning; and a vehicle communication device (18) in communication with the vehicle controller (16) and configured to transmit the vehicle sensor data; a remote system (14) containing: a remote network controller (38); at least one remote sensor (42) in communication with the remote network controller (38), the at least one remote sensor (42) configured to generate remote sensor data associated with a remote threat warning; and a communication hub (40) connected to the remote network controller (38) and the vehicle communication device (18), wherein the vehicle (12) and the remote system (14) communicate via the communication hub (40), the vehicle (12) is operable to forward the vehicle threat warnings to the remote system (14) and to receive the remote threat warnings from the remote system (14), the vehicle controller (16) checks the received remote threat warnings and modifies an operating mode of the at least one vehicle sensor (20) based on the received remote threat warnings, characterized in that the remote network controller (38) checks the received vehicle threat warnings and modifies a detection mode of the remote system (14) based on the received vehicle threat warnings, wherein the at least one vehicle sensor (20) includes a vehicle camera (24) operable to capture images and send the images to the vehicle controller (16), and a vehicle access detector (26) operable to detect the opening of a vehicle access component, wherein the vehicle controller (16) includes a vehicle processor (50) and a vehicle memory (52), wherein the vehicle memory (52) includes instructions such that the vehicle processor (50) is programmed to: transmit a location of the vehicle (12) to the remote system (14); determine the vehicle threat warnings based on the images from the vehicle camera (24) and the status of the vehicle access detector (26); to communicate with the remote system (14); to control the operation of the vehicle camera (24); to check the transmissions received from the remote system (14); and to forward the vehicle threat warnings to the remote system (14), wherein the vehicle processor (50) is further programmed to control the operating mode, the operating mode including: a relaxed mode of operation wherein the vehicle camera (24) is turned off and the vehicle access detector (26) is turned on; a normal operating mode wherein the vehicle access detector (26) is turned on, the vehicle camera (24) is turned on if the vehicle (12) is turned on, and the vehicle camera (24) is periodically turned on at a normally set schedule to capture images if the vehicle (12) is turned off; and a cautious mode of operation, wherein the vehicle access detector (26) is turned on, the vehicle camera (24) is turned on if the vehicle (12) is turned on, and if the vehicle (12) is turned off, the vehicle camera (24) is periodically turned on at a cautiously set schedule that is more frequent than the normally set schedule so that the vehicle camera (24) captures images more frequently, and the images captured by the vehicle camera (24) are transmitted to the remote system (14).

Description

INTRODUCTION

[0001] The present disclosure relates to integrating vehicle sensors with remote system sensors.

[0002] The document DE 10 2020 113 175 A1 discloses a system for integrating vehicle sensors with remote system sensors according to the preamble of claim 1. The documents DE 10 2020 210 210 A1 and DE 10 2017 200 160 A1 disclose related systems.

[0003] Vehicles have sensors configured to generate data associated with a vehicle threat warning. The vehicle sensors include vehicle cameras and vehicle access detectors. Determining a vehicle threat warning is limited to the data provided by the vehicle sensors. Remote terrain systems have sensors configured to generate data associated with a remote terrain threat warning. The remote sensors include cameras and motion detectors.

[0004] While these systems are useful for their intended purpose, an object of the invention is to provide collaborative threat detection between these systems. SUMMARY

[0005] The aforementioned object is achieved by the features of claim 1. Advantageous further developments are defined in the subclaims.

[0006] A system for integrating vehicle sensors with remote system sensors is provided. The system includes a vehicle having a vehicle controller, at least one vehicle sensor in communication with the vehicle controller and configured to generate vehicle sensor data associated with a vehicle threat alert, and a vehicle communication device in communication with the vehicle controller and configured to transmit the vehicle sensor data. A remote system having a remote network controller, at least one remote sensor in communication with the remote network controller and configured to generate remote sensor data associated with a remote threat alert, and a communications hub in communication with the remote network controller and the vehicle communication device.The vehicle and the remote system communicate via the communication hub, the vehicle is operable to forward the vehicle threat alerts to the remote system and to receive the remote threat alerts from the remote system, the vehicle controller reviews the received remote threat alerts and modifies an operating mode of the at least one vehicle sensor based on the received remote threat alerts, and the remote network controller reviews the received vehicle threat alerts and modifies a detection mode of the remote system based on the received vehicle threat alerts.

[0007] The at least one vehicle sensor includes a vehicle camera operable to capture images and transmit the images to the vehicle controller, and a vehicle access detector operable to detect the opening of a vehicle access component, wherein the vehicle controller includes a vehicle processor and a vehicle memory. The vehicle memory contains instructions such that the vehicle processor is programmed to communicate a location of the vehicle to the remote system, determine the vehicle threat alerts based on the images from the vehicle camera and the status of the vehicle access detector, communicate with the remote system, control the operation of the vehicle camera, review the transmissions received from the remote system, and communicate the vehicle threat alerts to the remote system.

[0008] The vehicle processor is further programmed to control the operating mode, wherein the operating mode comprises a relaxed operating mode, wherein the vehicle camera is turned off and the vehicle access detector is turned on, a normal operating mode, wherein the vehicle access detector is turned on, the vehicle camera is turned on if the vehicle is turned on, and the vehicle camera is periodically turned on at a normally set schedule to take images if the vehicle is turned off, and a cautious operating mode, wherein the vehicle access detector is turned on, the vehicle camera is turned on if the vehicle is turned on, and if the vehicle is turned off, the vehicle camera is periodically turned on at a cautiously set schedule that is more frequent than the normally set schedule so that the camera takes images more frequently,and the images captured by the vehicle camera are transmitted to the remote system.

[0009] According to one aspect, the vehicle includes a battery, wherein the vehicle processor is further programmed to determine a voltage status of the battery, control the operating mode based on the voltage status, place the vehicle in the relaxed operating mode when the voltage status is a low voltage, limit changes in the operating mode from the relaxed operating mode when the voltage status is a low voltage until receipt of a remote threat alert or a threat alert from the vehicle access detector, and inform the remote system of the operating mode.

[0010] According to another aspect, the remote system generates the remote threat warning when a remote person is detected by at least one remote sensor, notifying the vehicle when the remote person moves beyond a sensor range of the at least one remote sensor, and the controller places the vehicle in the cautious mode, wherein the vehicle generates the vehicle threat warning when a vehicle person is detected by the at least one vehicle sensor and notifies the remote system when the vehicle person moves beyond the sensor range of the at least one vehicle sensor, wherein the remote system changes the detection mode to a high-precision threat detection mode or an ultra-precision threat detection mode.

[0011] According to another aspect, the remote system detection mode includes an occasional threat detection mode, wherein the at least one remote sensor is turned off for a limited period of time based on input from a user of the remote system, a high-precision threat detection mode, which is a standard detection mode and wherein the at least one remote sensor can be informed to activate and generate remote sensor data associated with a remote threat alert, and an ultra-precision threat detection mode, wherein the at least one remote sensor is continuously active to generate remote sensor data associated with a remote threat alert.

[0012] In another aspect, the vehicle processor is further programmed to limit the information passed to the remote system based on an information release provided by an owner of the vehicle.

[0013] According to another aspect, there are multiple vehicles and remote systems, and a vehicle communication service that communicates with the vehicles via cellular networks. The vehicles relay the vehicle threat alerts and vehicle locations to the vehicle communication service. The vehicle communication service identifies a geographic area of ​​concern based on the received vehicle threat alerts and transmits a threat detection to the vehicles in the geographic area of ​​concern. The vehicles receiving the threat detection transmit the threat detection to the associated remote systems.

[0014] In another aspect, the vehicle communication service provides information about threat images that are threat concerns based on a person or dangerous object in an image, wherein the vehicle processor is further programmed to identify a threat alert by determining whether the image captured by the vehicle camera includes threat images provided by the vehicle communication service, the remote systems provide information about person images that are not a threat concern, and the vehicle processor is further programmed to identify a camera image as not a threat alert based on the person images provided by the remote systems.

[0015] In another aspect, the vehicle communication service changes the geographic area of ​​concern, transmitting the threat detection to the vehicles in the changed geographic area of ​​concern, wherein the vehicles receiving the threat detection transmit the threat detection to the associated remote systems.

[0016] According to another embodiment, a method is provided for integrating a vehicle having at least one vehicle sensor configured to generate vehicle sensor data associated with a vehicle threat warning with a remote system having at least one remote sensor configured to generate remote sensor data associated with a remote threat warning. The method includes transmitting a location of the vehicle to the remote system, transmitting the vehicle threat warnings to the remote system, transmitting the remote threat warnings to the vehicle, modifying an operating mode of the at least one vehicle sensor based on received remote threat warnings, and modifying a detection mode of the remote system based on received vehicle threat warnings.

[0017] In one aspect, the at least one vehicle sensor includes a vehicle camera operable to capture images and a vehicle access detector operable to detect the opening of a vehicle access component, the method further including determining the vehicle threat alerts based on the images from the vehicle camera and the status of the vehicle access detector, controlling the operation of the vehicle camera based on the operating mode, and reviewing the transmissions received from the remote system.

[0018] According to another aspect, the vehicle has a battery, and the method further includes determining a voltage status of the battery, setting the operating mode based on the voltage status such that the vehicle camera is disabled when the voltage status is a low voltage, and limiting activation of the vehicle camera when the voltage status is a low voltage until receipt of a remote threat alert or a threat alert from the vehicle access detector, and informing the remote system of the operating mode.

[0019] In another aspect, the method further includes the remote system notifying the vehicle of a remote person concern when a person detected by the remote system moves beyond a sensor range of a remote sensor, and activating the vehicle camera when notified of the remote person concern.

[0020] In another aspect, the method further includes the vehicle notifying the remote system of a concern about a vehicle occupant when a person detected by the vehicle moves beyond a sensor range of the vehicle camera, and activating the at least one remote sensor when notified of the concern about the vehicle occupant.

[0021] According to another aspect, there are multiple vehicles and remote systems and a vehicle communication service that communicates with the vehicles via cellular networks, the method further including the vehicle communication service providing information about threat images that are threat concerns based on a person or dangerous object in an image, the remote systems providing information about person images that are not a threat concern, the vehicles identifying a camera image as not being a threat based on the information about person images provided by the remote systems, the vehicles identifying threat alerts by determining whether the image captured by the vehicle camera contains threat images as provided by the vehicle communication service, the vehicles communicating the vehicle threat alerts and the vehicle locations to the vehicle communication service,the vehicle communication service identifies a geographical area of ​​concern based on the received vehicle threat alerts, the vehicle communication service transmits a threat detection to the vehicles in the geographical area of ​​concern, and the vehicles receiving the threat detection transmit the threat detection to the associated remote systems.

[0022] According to yet another embodiment, a vehicle is provided that integrates vehicle sensors with a remote system. The vehicle includes a vehicle controller including a processor and memory, vehicle sensors configured to generate vehicle sensor data associated with a vehicle threat warning, the vehicle sensors including vehicle cameras operable to capture images and send the images to the vehicle controller, and vehicle access detectors operable to detect the opening of a vehicle access component, and a vehicle communication device in communication with the vehicle controller and configured to transmit the vehicle sensor data and in communication with a communication hub of a remote system.The memory contains instructions such that the processor is programmed to communicate with the remote systems, verify remote threat alerts received from the remote system, control an operating mode of the vehicle sensors and activate the vehicle cameras if necessary based on the transmission received from the remote system, relay the images from the vehicle cameras to the remote system if the vehicle cameras are activated and the vehicle is communicating with the remote system, determine the vehicle threat alerts based on the images from the vehicle camera and the status of the vehicle access detector, relay the vehicle threat alerts to the remote system, and transmit a location of the vehicle to the remote system.

[0023] According to another aspect, the processor is further programmed to modify the operating mode based on the received remote threat alerts, the operating mode including a relaxed mode where the vehicle cameras are off and the vehicle entry detectors are on, a normal mode where the vehicle entry detectors are on and the vehicle cameras are randomly turned on and off and turned on when the vehicle entry detectors sense a threat and the vehicle camera images are transmitted to the remote system, and a cautious mode where the vehicle cameras are active and the vehicle camera images are transmitted to the remote system and the vehicle entry detectors are on.

[0024] According to another aspect, the vehicle includes a battery, and the vehicle processor is further programmed to determine a voltage status of the battery, control the operating mode based on the voltage status, place the vehicle in the relaxed operating mode when the voltage status is a low voltage, limit changes to the operating mode from the relaxed operating mode when the voltage status is a low voltage until receipt of a remote threat alert or a vehicle threat alert from the vehicle access detectors, and inform the remote system of the operating mode.

[0025] According to another aspect, the vehicle includes a wireless communication device operable to communicate with a vehicle communication service that communicates with other vehicles having a cellular network device, the controller being in communication with the cellular network device, and the processor being further programmed to communicate the threat detections to the vehicle communication service and the remote system, receive notification of threats from the vehicle communication service, transmit the received threats to the remote system, determine the status of the vehicle and activate the vehicle camera based on the status of the vehicle when notified of the threats, and communicate the detected threats when notified of a threat.

[0026] Further areas of application will become apparent from the description provided herein. It should be understood that the description and specific examples are provided for illustrative purposes only. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described here are for illustrative purposes only; they show: Fig. 1 is a schematic view of a system for integrating vehicle sensors with remote system sensors according to an exemplary embodiment; Fig. 2 is a process flow diagram of operation of an integrated vehicle according to an exemplary embodiment; Fig. 3 is a process flow diagram of operation of an integrated remote system according to an exemplary embodiment; Fig. 4 is a process flow diagram of a vehicle communication service according to an exemplary embodiment; Fig. 5 is a schematic view of a generated geographical area of ​​concern according to an exemplary embodiment; and Fig. 6 is a schematic view of a generated geographic area for random security activation according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] In Fig. 1, a system 10 is shown that integrates the sensors of a vehicle 12 with the sensors of a remote system 14 according to the principles of the present disclosure. Although the vehicle 12 is shown as a sedan, it is understood that the vehicle 12 may be any other type of vehicle, such as a pickup truck, a coupe, a sport utility vehicle (SUV), a recreational vehicle (RV), a semi-trailer truck, etc. The vehicle 12 includes a vehicle controller 16, a vehicle communication device 18, a plurality of vehicle sensors 20, and a battery 22. The vehicle sensors 20 communicate with the vehicle controller 16 and collect and communicate vehicle sensor data to the vehicle controller 16. The vehicle sensors 20 include vehicle cameras 24 and vehicle entry detectors 26. It should be appreciated that other sensors, including motion detectors, radar, or lidar, may be employed.The vehicle cameras 24 capture images and send the images to the vehicle controller 16. The vehicle access detectors 26 detect the opening of a vehicle access component and send the opening warning to the vehicle controller 16. The vehicle access components may include doors, a trunk, a hood, or other components. The vehicle controller 16 communicates with the battery 22 and determines a voltage status of the battery 22 based on the voltage level. The voltage level used to determine the voltage status may relate to various functional capabilities, such as the energy in the battery 22 and the ability of the vehicle 12 to operate in a normal manner, such as the ability to be driven. The vehicle controller 16 is operable to determine a location of the vehicle 12.The vehicle controller 16 communicates with the vehicle communication device 18 and utilizes the vehicle communication device 18 to communicate, as described below. The vehicle controller 16 and the vehicle communication device 18 may operate jointly as a vehicle communication hub.

[0029] The remote system 14 may take various formats, such as a security system, a surveillance system, a data collection system, etc. The remote system 14 is associated with a site 30 that includes a building 32 and its lot 34, which may include a driveway 36 or other parking features for the vehicle 12. The building 32 may be any type of building, such as an individual home, apartments, condominiums, an office building, a factory, a school, a store, etc. The remote system 14 includes a remote network controller 38, a communications hub 40, and a plurality of remote sensors 42. The remote sensors 42 are in communication with the remote network controller 38 and are configured to generate remote sensor data associated with a remote threat alert and to collect and communicate the remote sensor data to the remote network controller 38.The remote sensors 42 may include the remote motion detectors 43, which detect motion, and the remote cameras 44, which capture images and send the images to the remote network controller 38. The remote motion detectors 43 may be separate elements or may be integrated with the remote cameras 44. The communications hub 40 communicates with the remote network controller 38 and the vehicle communications device 18. The remote system 14 may include additional components and capabilities, such as smart speakers that can provide alerts and lights, such as strobe lights, that can be activated to indicate a threat.

[0030] The communication hub 40 and the vehicle communication device 18 may include and utilize various communication technologies to enable the vehicle controller 16 to communicate with the remote system 14, the vehicle controller 16 to communicate directly with the remote sensors 42, the vehicle controller 16 to communicate with the vehicle sensors 20, and the remote network controller 38 to communicate with the remote sensors 42. According to an exemplary embodiment, the communication technologies included in and utilized by the vehicle communication device 18 and the communication hub 40 may be, for example, a Wi-Fi device, a Bluetooth device, and other wireless communication devices. Communication between the vehicle controller 16 and the remote system 14 may be via Wi-Fi or Bluetooth or other wireless technologies, such as, but not limited to, wireless mesh networks or high-bandwidth wireless sensor networks, and may utilize the Internet of Things (IOT) Object Protocol. The IOT Object Protocol may be used, for example, to couple the vehicle 12 to a remote system 14. Electrical connections may also be used by the vehicle controller 16 to communicate with the vehicle sensors 20 and by the remote network controller 38 to communicate with the remote sensors 42. The vehicle communication device 18 may also include and utilize various communication techniques to enable the vehicle controller 16 to communicate directly with other vehicles 12. It should be recognized that various additional wired and wireless techniques and communication protocols for communicating are within the scope of the present disclosure.The remote system 14 can selectively approve which remote sensors 42 can communicate with the vehicle controller 16, wherein the vehicle controller 16 can communicate with and receive data from the approved remote sensors 42 either via the communication hub 40 or by communicating directly with the remote sensors 42.

[0031] The system 10 may include a cloud-based vehicle communication service 46 that communicates with all vehicles 12. The vehicle communication service 46 communicates with the vehicle communication device 18, which enables the vehicle controller 16 to communicate with the vehicle communication service 46. The vehicle communication device 18 and the vehicle communication service 46 communicate using wireless communication and may include a cellular device and communicate using cellular communication. It should be appreciated that various additional wireless techniques and communication protocols for communicating are within the scope of the present disclosure.

[0032] The vehicle controller 16 is used to implement a vehicle method 100 for integrating the vehicle sensors 20 of the vehicle 12 with the remote sensors 42 of the remote system 14 according to the principles of the present disclosure, as described below. The vehicle controller 16 includes at least one processor 50 and a non-transitory computer-readable storage device or non-transitory computer-readable storage media 52. The processor 50 may be a custom-built or off-the-shelf processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among a plurality of processors associated with the vehicle controller 16, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally any device for executing instructions.The computer-readable storage device or computer-readable storage media 52 may include, for example, volatile and non-volatile memory in read-only memory (ROM), random access memory (RAM), and persistent memory (RAM). A RAM is persistent or non-volatile memory that can be used to store various operating variables while the processor 50 is powered off. The computer-readable storage device or computer-readable storage media 52 may be implemented using a number of storage devices, such as:PROMs (Programmable Read-Only Memory), EPROMs (Electrical PROMs), EEPROMs (Electrically Erasable PROMs), flash memory, or other electrical, magnetic, optical, or combination storage devices capable of storing data, some of which represents executable instructions used by the vehicle controller 16 to control various systems of the vehicle 12. The vehicle controller 16 may also be comprised of multiple controllers that are in electrical communication with one another. The vehicle controller 16 may be interconnected with additional systems and / or controllers of the vehicle 12, allowing the vehicle controller 16 to access data, such as the location of the vehicle 12.

[0033] In Fig. 2, a vehicle method 100 for integrating the vehicle sensors 20 of the vehicle 12 with the remote sensors 42 of the remote system 14 is shown in accordance with the principles of the present disclosure. The vehicle method 100 begins at step 102, where the vehicle controller 16 receives data from the owner of the vehicle 12, the vehicle battery 22, the vehicle sensors 20, the vehicle communication service 46, and the remote system 14 (which are described in Fig. 1). The data includes vehicle credentials set by the owner or operator of the vehicle 12, a voltage status of the vehicle battery 22, images from the vehicle cameras 24, opening alerts from the vehicle access detectors 26, threat alerts from the vehicle communication service 46, and threat alerts or instructions from the remote system 14. The vehicle method 100 then proceeds to step 104.

[0034] In step 104, the vehicle controller 16 determines an operating mode to be set for the vehicle 12. The operating mode is a vehicle condition that determines the activity level of the plurality of vehicle sensors 20 and whether the sensor input data should be recorded or transmitted. The operating mode includes a relaxed operating mode, a normal operating mode, and a cautious operating mode.

[0035] In the relaxed mode, the vehicle cameras 24 are turned off, thus reducing consumption of the vehicle battery 22, while the vehicle access detectors 26 are still active. In the normal mode, the vehicle access detectors 26 are active and the vehicle cameras 24 are turned on if the engine of the vehicle 12 is turned on or if the vehicle is charging. When the engine of the vehicle 12 is turned off or the vehicle is not charging, the vehicle cameras 24 are periodically turned on according to a normal, predetermined schedule to capture images. The periodic schedule may, for example, include capturing images every ten seconds, every five minutes, every ten minutes, etc. The periodic schedule may also be randomized.In the cautious mode, the vehicle access detectors 26 are active and the vehicle cameras 24 are turned on if the engine of the vehicle 12 is turned on, or the vehicle 12 is charging, or if the voltage status exceeds a charge threshold, as discussed below. Otherwise, the vehicle cameras 24 are periodically turned on according to a cautious predetermined schedule that is more frequent than the normal predetermined schedule, so that the vehicle cameras 24 capture images more frequently than in the normal mode.

[0036] The determination of the operating mode is based on the voltage status of the battery 22 if the vehicle's engine 12 is on or the vehicle 12 is charging, if the vehicle 12 has previously determined that there is a vehicle threat (as defined in step 110 below), if a threat warning has been provided by the remote system 14 or the vehicle communication service 46 and transmitted to the vehicle 12, or if an activation request to set a specific operating mode has been received from the vehicle communication service 46. For example, the vehicle controller 16 compares the voltage status received from the vehicle battery 22 to a charge threshold. The charge threshold is defined as 50% charge according to the provided example. However, other charge levels may be used.The voltage status is defined as low voltage when the battery charge level is less than the charge threshold.

[0037] By default, the vehicle controller 16 places the vehicle 12 in normal operating mode when the engine of the vehicle 12 is on or the vehicle 12 is charging. If there is a vehicle threat, a provided threat warning, or an activation request, the vehicle controller 16 places the vehicle 12 in cautious operating mode. If the battery 22 has a low voltage, the vehicle controller 16 places the vehicle in relaxed operating mode. However, the relaxed operating mode can be overridden if there is a vehicle threat detected by the vehicle 12, a threat warning provided by the vehicle communication service 46 or the remote system 14, or an activation request provided by an operator of the vehicle 12, the vehicle communication service 46, or the remote system 14. For example, if the vehicle 12 isIf the vehicle is at a remote system 14 that is actively monitoring for threats, the vehicle controller 16 sets the operating mode to the cautious mode. The vehicle method 100 then proceeds to step 106.

[0038] In step 106, the vehicle controller 16 determines what information or data can be shared with the remote system 14 and the vehicle communication service 46. Generally, any information that is not restricted by the owner of the vehicle 12 and that is not restricted based on the mode of operation can be shared. For example, the owner of the vehicle 12 can determine what information can and cannot be shared, and can set permissions that restrict the sharing of images, limit a location of the vehicle, etc. This allows the owner of the vehicle 12 to address any privacy concerns. The mode of operation restricts what information may be shared, where, for example,the vehicle battery 22 has a low voltage, the sharing of images from the vehicle cameras 24 and other data can be restricted to sharing only during an active threat (as defined in step 110 below). The vehicle method 100 then proceeds to step 108.

[0039] In step 108, the vehicle controller 16 communicates the information to the vehicle communication service 46 and to the remote system 14 based on what information was determined to be communicable in step 106. For example, the vehicle controller 16 communicates a location of the vehicle 12, the operating mode, a detection mode of the remote system 14, threat alerts to the vehicle communication service 46, and sensor data from the vehicle sensors 20. The vehicle method 100 then proceeds to step 110.

[0040] In step 110, the vehicle controller 16 performs a vehicle threat determination based on the sensor data from the vehicle sensors 20. The vehicle controller 16 determines the presence of a vehicle threat based on the sensor data from the vehicle sensors 20. For example, a vehicle threat is determined to be present if a vehicle access detector 26 indicates opening of a vehicle access component when the vehicle 12 is locked. It is also determined that a vehicle threat is present when the vehicle controller 16 reviews the images from the vehicle cameras 24 to perform a vehicle threat determination based on the images. The vehicle controller 16 determines whether the images from the vehicle cameras 24 include a dangerous object, a concerned face or person, or a known face or person.A dangerous object and a concerned face or person are threat concerns, whereas a known face or person is not a threat concern. The information about dangerous objects and a concerned face or person is provided by the vehicle communication service 46 and may be based on information from social media, a government, and security agencies, such as a police department, a sheriff's office, and the Federal Bureau of Investigation (the FBI). Information about a known face or person is generated by the user of the remote system 14 and the owner of the vehicle 12 and is provided by the owner of the vehicle 12 and the remote system 14. A dangerous object is an object that may be associated with a criminal or harmful activity, such asa gun, rifle, knife, sword, explosive device, or other type of weapon. A concerned face or person is based on a person or the face of someone who is a threat concern, such as a criminal, a wanted person, or a jail escapee. A familiar face or person is based on a person or the face of someone who is considered safe and not a threat concern, such as a family member, friend, or neighbor.

[0041] The vehicle controller 16 determines that a vehicle threat exists if the images from the vehicle cameras 24 include a dangerous object or a concerned face or person. The vehicle controller 16 determines that no vehicle threat exists if the images from the vehicle cameras 24 include only known faces or people. The vehicle controller 16 may determine a possibility of a vehicle threat if the images from the vehicle cameras 24 include an unknown face or person and do not include a dangerous object or a concerned face or person. If no vehicle threat is determined, the vehicle method 100 returns to step 102. If a vehicle threat is determined, the vehicle method 100 proceeds to step 112.

[0042] In step 112, the vehicle controller 16 transmits the vehicle threat warning to the remote system 14 and the vehicle communication service 46. The vehicle method 100 then proceeds to step 114.

[0043] In step 114, the vehicle controller 16 communicates the sensor data from the vehicle sensors 20 used to determine the vehicle threat warning to the remote system 14. For example, the vehicle controller 16 communicates the images from the vehicle cameras 24 to the remote system 14. The operating mode may limit what is communicated to the remote system 14 and limit the frequency of communication to the remote system 14. The operating modes control which vehicle cameras 24 are active and the frequency of activation and deactivation of the vehicle cameras 24, so continuous communication of images from the vehicle cameras 24 may not be possible. The vehicle method 100 then proceeds to step 116.

[0044] In step 116, the vehicle controller 16 monitors the identified threat, recording the sensor data from the vehicle sensors 20. The vehicle method 100 then proceeds to step 118.

[0045] In step 118, the vehicle controller 16 determines whether the identified threat has left a sensor range of the vehicle sensors 20 monitoring the identified threat. If the vehicle controller 16 determines that the identified threat has not left the sensor range of the vehicle sensors 20 monitoring the identified threat, the vehicle method 100 returns to step 116. If the vehicle controller 16 determines that the identified threat has left the sensor range of the vehicle sensors 20 monitoring the identified threat, the vehicle method 100 proceeds to step 120.

[0046] In step 120, the vehicle controller 16 notifies the remote system 14 that the identified threat has left the sensor range of the vehicle sensors 20 monitoring the identified threat. The vehicle method 100 then returns to step 102.

[0047] Once the remote network controller 38 receives the vehicle threat warning and any relayed data, the remote network controller 38 responsively increases to an ultra-precision mode in which the remote cameras 44 are active and recording.

[0048] In Fig. 3 shows the remote system method 200 for integrating the vehicle sensors 20 of the vehicle 12 with the remote sensors 42 of the remote system 14 according to the principles of the present disclosure. The remote system method 200 begins at step 202, where the remote network controller 38 receives data from a user of the remote system 14, the remote sensors 42, and the vehicle 12. The data from the user may reflect that the user restricts the activation of some remote sensors 42 due to approved activities on the premises 30, in the building 32, on the lot 34, and in the driveway 36. The user may also restrict or prevent the sharing of information from specific remote sensors 42 with the vehicle 12 to address any privacy concerns. The user restrictions may prevent false or inaccurate remote threat alerts from the approved activities.The user may also provide information about known faces or known individuals, which the remote system 14 may communicate to the vehicle 12 and use to determine that an image captured by a remote camera 44 is not a threat, as described below. The remote system method 200 then proceeds to step 204.

[0049] In step 204, the remote network controller 38 determines a detection mode for the remote system 14, placing the remote system 14 in the determined detection mode. The detection mode includes an occasional threat detection mode, a high-precision threat detection mode, and an ultra-precision threat detection mode. The occasional threat detection mode includes disabling some or all of the remote sensors 42 for a limited period of time based on the user 45 desiring to restrict the activation of some remote sensors 42 due to authorized activities on the premises 30, in the building 32, on the lot 34, or in the driveway 36.The high-precision threat detection mode is the normal mode and has all remote sensors 42 active, allowing the remote motion detectors 43 to detect motion and inform the associated remote cameras 44 to turn on and capture images to detect and report threats, and uses the normal algorithm. The ultra-precision threat detection mode has all remote sensors 42 active, allowing the remote motion detectors 43 to detect motion, and all remote cameras 44 turned on and capturing images all the time to detect and report threats using a highly accurate algorithm to analyze, detect, and report images from the remote cameras 44. The determination of the detection mode other than the standard high-precision threat detection mode is based on user input or a desire to provide increased threat detection.The default operating mode is the high-precision threat detection mode, with the remote network controller 38 placing the remote system 14 in the high-precision threat detection mode unless the remote network controller 38 receives a request from the user, a threat is detected, a threat alert is received, or an activation request is received. If a request is received from the user 45 to decrease the detection mode, the remote network controller 38 places the remote system 14 in the occasional threat detection mode. If a threat is detected by the remote system 14, a threat alert is received from the vehicle 12, or an activation request is received from the vehicle 12, the remote network controller 38 places the remote system 14 in the ultra-precision threat detection mode. The remote system process 200 then proceeds to step 206.

[0050] In step 206, the remote network controller 38 communicates the information to the vehicle 12. The remote network controller 38 communicates the detection mode to the vehicle 12 and communicates the information about known faces or known individuals that the user has indicated as non-threats. The remote system method 200 then proceeds to step 208.

[0051] In step 208, the remote network controller 38 performs a remote threat determination based on the sensor data from the remote sensors 42. The remote network controller 38 determines the presence of a remote threat based on the sensor data from the remote sensors 42. The remote network controller 38 examines the images from the remote cameras 44 to perform a remote threat determination based on the images. The remote network controller 38 determines whether the images from the remote cameras 44 include a dangerous object, a concerned face or person, or a known face or person, as discussed above. The information about dangerous objects, a concerned face or person, and a known face or person may be provided by the vehicle 12.The remote network controller 38 determines that a remote threat exists if the images from the remote cameras 44 include a dangerous object or a concerned face or person. The remote network controller 38 determines that no remote threat exists if the images from the remote cameras 44 include only known faces or people. The remote network controller 38 may determine a possibility of a remote threat if the images from the remote cameras 44 include an unknown face or person and do not include a dangerous object or a concerned face or person. If no remote threat is determined, the remote system method 200 returns to step 202. If a remote threat is determined, the remote system method 200 proceeds to step 210.

[0052] In step 210, the remote network controller 38 has determined the presence of a remote threat and communicates the presence of the determined remote threat as a remote threat alert. The remote network controller 38 communicates the remote threat alert to the vehicle 12. The remote system method 200 then proceeds to step 212. Once the vehicle 12 receives the remote threat alert and any communicated data, the vehicle controller 16 verifies the received remote threat alert, modifying an operating mode of the at least one vehicle sensor based on the received remote threat alert.

[0053] In step 212, the remote network controller 38 monitors the identified threat, recording the sensor data from the remote sensors 42. The remote system method 200 then proceeds to step 214.

[0054] In step 214, the remote network controller 38 determines whether the identified threat has left the sensor range of the remote sensors 42 capable of monitoring the identified threat. If the remote network controller 38 determines that the identified threat has not left the sensor range of the remote sensors 42 capable of monitoring the identified threat, the remote system method 200 returns to step 212. If the remote network controller 38 determines that the identified threat has left the sensor range of the remote sensors 42 capable of monitoring the identified threat, the remote system method 200 proceeds to step 216.

[0055] In step 216, the remote network controller 38 notifies the vehicle 12 that the identified threat has left the sensor range of the remote sensors 42 capable of monitoring the identified threat. The remote system method 200 then returns to step 202.

[0056] In Fig. 4, the vehicle communication service method 300 for integrating the vehicle sensors 20 of the vehicle 12 with the remote sensors 42 of the remote system 14 is shown in accordance with the principles of the present disclosure. The vehicle communication service method 300 begins at step 302, where the vehicle communication service 46 receives data from the vehicles 12. The data from the vehicles 12 includes vehicle threat alerts, a vehicle location, a vehicle operating mode, and a remote system detection mode. The vehicle communication service method 300 then proceeds to step 304.

[0057] In step 304, the vehicle communication service 46 communicates the information about dangerous objects and concerned faces or people that can be detected in an image and are threat concerns to the vehicles 12. The vehicle communication service method 300 then proceeds to step 306 and step 312.

[0058] In step 306, the vehicle communication service 46 determines whether a threat has been detected by reviewing the data provided by the vehicles 12. The vehicle communication service 46 determines that a threat has been detected if a threat alert has been provided by a vehicle 12. The vehicle communication service 46 determines that no threat has been detected if no threat alerts have been provided by a vehicle 12. If no threats have been detected, the vehicle communication service method 300 returns to step 302. If a threat has been detected, the vehicle communication service method 300 proceeds to step 308.

[0059] In Fig. 4 and Fig. 5, in step 308, the vehicle communication service 46 determines a geographic area 60 of concern based on the location of the vehicle 12 that provided the threat alert. The geographic area 60 of concern identifies risk zones with varying risk levels based on the distance from the location of the vehicle 12 that provided the threat alert. The geographic area 60 of concern includes a high-risk zone 62 containing and surrounding the location of the vehicle 12 that provided the threat alert, a medium-risk zone 64 outside and surrounding the high-risk zone 62, and a low-risk zone 66 outside and surrounding the medium-risk zone 64. The area, size, and shape of the risk zones 62, 64, and 66 may vary based on the number and locations of multiple threat alerts, the characteristics of the areas, and the type of threat alerts.The risk zones 62, 64, and 66 may each include the vehicles 12 and the remote systems 14 that communicate with the vehicles 12. The vehicle communication service method 300 then proceeds to step 310.

[0060] At step 310, the vehicle communication service 46 communicates the threat detections to the vehicles 12 in the geographic area 60 of concern, and may communicate a risk level based on the risk zones 62, 64, and 66 in which the vehicles 12 are located. The vehicles 12 may, as described above, communicate the received threat alerts to the remote systems 14 that are in communication. The vehicle communication service method 300 then returns to step 302.

[0061] In Fig. 4 and Fig.6, steps 312 through 320 are used to provide an unpredictable location and activation of threat detection systems, such as the vehicles 12 and the remote systems 14, such that it is difficult or impossible for intruders and criminals to know or predict the areas not protected by active threat detection systems. In step 312, the vehicle communication service 46 determines a geographic area 70 for the random activation of the vehicles 12 and the remote systems 14 within the geographic area 70. The geographic area 70 may vary in size and location. The vehicle communication service method 300 then proceeds to step 314.

[0062] In step 314, the vehicle communication service 46 determines a desired operating mode for the vehicles 12 and a desired detection mode for the remote systems 14 in the geographic area 70. The vehicle communication service method 300 then proceeds to step 316.

[0063] In step 316, the vehicle communication service 46 determines the vehicles 12 located within the geographic area 70. The vehicle communication service method 300 then proceeds to step 318.

[0064] In step 318, the vehicle communication service 46 randomly selects vehicles 12 located within the geographic area 70. The vehicle communication service method 300 then proceeds to step 320.

[0065] In step 320, the vehicle communication service 46 sends an activation request to the vehicles 12 selected in step 318. The activation request includes the desired operating mode and the desired detection mode as determined in step 314. The vehicles 12 may initiate the requested operating mode, communicating the requested activation to the associated remote systems 14. The remote systems 14 may initiate the requested detection mode. The vehicle communication service method 300 then returns to step 302.

[0066] Through system 10, the vehicle controller 16, the remote network controller 38, and the vehicle communication service 46 may be able to determine the presence of a threat based on the sensor data received from any source. For example, the vehicle controller 16 may be able to determine that a threat is present based on the images and data received from the remote sensors 42, the remote network controller 38 may be able to determine that a threat is present based on the images and data from the vehicle sensors 20, and the vehicle communication service 46 may be able to determine that a threat is present based on the images and data from the vehicle sensors 20 and / or the remote sensors 42.While the vehicle controller 16, the remote network controller 38, and the vehicle communication service 46 individually perform threat determinations, they may additionally cooperate to perform threat determinations.

[0067] The system 10 for integrating vehicle sensors with remote system sensors provides many advantages. The system 10 improves vehicle safety by leveraging the sensing and processing capabilities of remote systems (at home, office, etc.). Additionally, detected threats can be uploaded to the cloud and further assist neighboring vehicles and homes with situational awareness and detection of potential threats.

Claims

[1] A system (10) for integrating vehicle sensors (20) with remote system sensors, the system (10) comprising: a vehicle (12) containing: a vehicle controller (16); at least one vehicle sensor (20) in communication with the vehicle controller (16), the at least one vehicle sensor (20) configured to generate vehicle sensor data associated with a vehicle threat warning; and a vehicle communication device (18) in communication with the vehicle controller (16) and configured to transmit the vehicle sensor data; a remote system (14) containing: a remote network controller (38); at least one remote sensor (42) in communication with the remote network controller (38), the at least one remote sensor (42) configured to generate remote sensor data associated with a remote threat warning; and a communication hub (40) connected to the remote network controller (38) and the vehicle communication device (18), wherein the vehicle (12) and the remote system (14) communicate via the communication hub (40), the vehicle (12) is operable to forward the vehicle threat warnings to the remote system (14) and to receive the remote threat warnings from the remote system (14), the vehicle controller (16) checks the received remote threat warnings and modifies an operating mode of the at least one vehicle sensor (20) based on the received remote threat warnings, characterized by , that the remote network controller (38) checks the received vehicle threat warnings and modifies a detection mode of the remote system (14) based on the received vehicle threat warnings, wherein the at least one vehicle sensor (20) includes a vehicle camera (24) operable to capture images and send the images to the vehicle controller (16), and a vehicle access detector (26) operable to detect the opening of a vehicle access component, wherein the vehicle controller (16) includes a vehicle processor (50) and a vehicle memory (52), wherein the vehicle memory (52) includes instructions such that the vehicle processor (50) is programmed to: transmit a location of the vehicle (12) to the remote system (14); determine the vehicle threat warnings based on the images from the vehicle camera (24) and the status of the vehicle access detector (26); to communicate with the remote system (14); to control the operation of the vehicle camera (24); to check the transmissions received from the remote system (14); and to forward the vehicle threat warnings to the remote system (14), wherein the vehicle processor (50) is further programmed to control the operating mode, the operating mode including: a relaxed mode of operation wherein the vehicle camera (24) is turned off and the vehicle access detector (26) is turned on; a normal operating mode wherein the vehicle access detector (26) is turned on, the vehicle camera (24) is turned on if the vehicle (12) is turned on, and the vehicle camera (24) is periodically turned on at a normally set schedule to capture images if the vehicle (12) is turned off; and a cautious mode of operation, wherein the vehicle access detector (26) is turned on, the vehicle camera (24) is turned on if the vehicle (12) is turned on, and if the vehicle (12) is turned off, the vehicle camera (24) is periodically turned on at a cautiously set schedule that is more frequent than the normally set schedule so that the vehicle camera (24) captures images more frequently, and the images captured by the vehicle camera (24) are transmitted to the remote system (14). [2] The system (10) of claim 1, wherein the vehicle (12) includes a battery (22) and the vehicle processor (50) is further programmed: to determine a voltage status of the battery (22); control the operating mode based on the voltage status; to put the vehicle (12) into the relaxed mode when the voltage status is a low voltage; limiting changes in operating mode from the relaxed operating mode when the voltage status is low voltage until receipt of a remote threat warning or a threat warning from the vehicle access detector (26); and to inform the remote system (14) about the operating mode. [3] The system (10) of claim 1, wherein the remote system (14) generates the remote threat warning when a remote person is detected by the at least one remote sensor (42) and notifies the vehicle (12) when the remote person moves beyond a sensor range of the at least one remote sensor (42), and the vehicle controller (16) places the vehicle (12) in the cautious mode, and the vehicle (12) generates the vehicle threat warning when a vehicle person is detected by the at least one vehicle sensor (20) and notifies the remote system (14) when the vehicle person moves beyond the sensor range of the at least one vehicle sensor (20), and the remote system (14) changes the detection mode to a high-precision threat detection mode or an ultra-precision threat detection mode. [4] The system (10) of claim 1, wherein the remote system detection mode includes: an occasional threat detection mode wherein the at least one remote sensor (42) is disabled for a limited period of time based on input from a user of the remote system (14); a high-precision threat detection mode, which is a standard detection mode and wherein the at least one remote sensor (42) can be informed to activate and generate remote sensor data associated with a remote threat warning; and an ultra-precision threat detection mode, wherein the at least one remote sensor (42) is continuously active to generate remote sensor data associated with a remote threat warning. [5] The system (10) of claim 1, wherein the vehicle processor (50) is further programmed to limit the information passed to the remote system (14) based on an information release provided by an owner of the vehicle (12). [6] The system (10) of claim 1, wherein there are a plurality of vehicles (12) and remote systems (14), and the system (10) further comprises a vehicle communication service (46) that communicates with the vehicles (12) via cellular networks, and wherein the vehicles (12) communicate the vehicle threat alerts and the vehicle locations to the vehicle communication service (46), the vehicle communication service (46) identifies a geographical area (60) of concern based on the received vehicle threat alerts and transmits a threat detection to the vehicles (12) in the geographical area (60) of concern, and the vehicles (12) that receive the threat detection transmit the threat detection to the associated remote systems (14). [7] The system (10) of claim 6, wherein the vehicle communication service (46) provides information about threat images that are threat concerns based on a person or dangerous object in an image, and the vehicle processor (50) is further programmed to: identify a threat alert by determining whether the image captured by the vehicle camera (24) includes threat images provided by the vehicle communication service (46), wherein the remote systems (14) provide information about person images that are not a threat concern, and the vehicle processor (50) is further programmed to: identify a camera image as not being a threat alert based on the person images provided by the remote systems (14). [8] The system (10) of claim 6, wherein the vehicle communication service (46) changes the geographical area (60) of concern and transmits the threat detection to the vehicles (12) in the changed geographical area (60) of concern, wherein the vehicles (12) receiving the threat detection transmit the threat detection to the associated remote systems (14).

Images