Integration of vehicle sensors with remote system sensors

Abstract

A system for integrating a vehicle sensor with a remote system sensor includes a vehicle having a controller in communication with a vehicle sensor that generates data associated with a vehicle threat alert; a vehicle communication device that communicates with the vehicle controller and transmits vehicle sensor data; a remote system, the remote system comprising: a controller in communication with a remote sensor that generates data associated with a remote threat alert; and a communication hub in communication with the remote controller and the vehicle communication device. A vehicle communicates with a remote system via a communication hub, shares a vehicle threat alert with the remote system, receives and checks a remote threat alert from the remote system, and modifies an operating mode of a vehicle sensor based on the remote threat alert. The remote network controller checks the received vehicle threat alerts and modifies a detection mode of the remote system based on the received vehicle threat alerts.

Description

Technical Field

[0001] The present disclosure relates to integrating vehicle sensors with remote system sensors. Background Art

[0002] A vehicle has sensors configured to generate data associated with a vehicle threat alert. The vehicle sensors include a vehicle camera and a vehicle access detector. The determination of a vehicle threat alert is limited to the data provided by the vehicle sensors. A remote system for a premises has sensors to generate data associated with a remote threat alert at the premises. The remote sensors include a camera and a motion detector. While these systems are useful for their intended purposes, there is a need in the art to provide coordinated threat detection between these systems. Summary of the invention

[0003] A system for integrating vehicle sensors with remote system sensors is provided. The system includes a vehicle having: a vehicle controller, at least one vehicle sensor that communicates with the vehicle controller and is configured to generate vehicle sensor data associated with a vehicle threat alert, a vehicle communication device that communicates with the vehicle controller and is configured to send the vehicle sensor data, and a remote system; the remote system having: a remote network controller that communicates with the remote network controller and is configured to generate at least one remote sensor associated with the remote threat alert, and a communication hub that communicates with the remote network controller and the vehicle communication device. The vehicle and the remote system communicate through the communication hub, the vehicle can be operated to share the vehicle threat alert with the remote system and receive the remote threat alert from the remote system, the vehicle controller checks the received remote threat alert and modifies the operation mode of at least one vehicle sensor based on the received remote threat alert, and the remote network controller checks the received vehicle threat alert and modifies the detection mode of the remote system based on the received vehicle threat alert.

[0004] In one aspect, the at least one vehicle sensor includes: a vehicle camera operable to capture images and send images to a vehicle controller; and a vehicle access detector operable to detect the opening of a vehicle access feature. And the vehicle controller includes a vehicle processor and a vehicle memory. The vehicle memory includes instructions such that the vehicle processor is programmed to: share the location of the vehicle with a remote system, determine a vehicle threat alert based on an image from the vehicle camera and a state of the vehicle access detector, communicate with the remote system, control the operation of the vehicle camera, check communications received from the remote system, and share the vehicle threat alert with the remote system.

[0005] In another aspect, the vehicle processor is further programmed to control an operating mode, and the operating mode includes a relaxed mode, a normal mode, and a cautious mode, in which the relaxed mode the vehicle camera is turned off and the vehicle access detector is activated, in which the normal mode the vehicle access detector is activated, if the vehicle is started, the vehicle camera is activated, and if the vehicle is turned off, the vehicle camera is periodically activated at a normally set schedule to capture images, and in the cautious mode the vehicle access detector is activated, if the vehicle is started, the vehicle camera is activated, and when the vehicle is turned off, the vehicle camera is periodically activated at a cautiously set schedule that is more frequent than the normally set schedule, so that the camera captures images more frequently, and so that the images captured by the vehicle camera are sent to the remote system.

[0006] In another aspect, the vehicle has a battery and the vehicle processor is further programmed to: determine a voltage state of the battery, control an operating mode based on the voltage state, place the vehicle in a relaxation mode when the voltage state is low voltage, limit a change in the operating mode from the relaxation mode to a remote threat alert or receipt of a threat alert from a vehicle access detector when the voltage state is low voltage, and notify a remote system of the operating mode.

[0007] In another aspect, when a remote person is detected by the at least one remote sensor, the remote system creates the remote threat alert, and when the remote person moves beyond the sensor range of the at least one remote sensor and the controller puts the vehicle into the cautious mode, the remote system notifies the vehicle, when a vehicle person is detected by the at least one vehicle sensor, the vehicle creates the vehicle threat alert, and when the vehicle person moves beyond the sensor range of the at least one vehicle sensor, the vehicle notifies the remote system, and the remote system changes the detection mode to a high-precision threat detection mode or an ultra-precision threat detection mode.

[0008] In another aspect, the remote system detection mode includes an occasional threat detection mode, a high-precision threat detection mode, and an ultra-precision threat detection mode; wherein in the occasional threat detection mode, based on the input of a user of the remote system, at least one remote sensor is turned off for a limited time period; the high-precision threat detection mode is the default detection mode, and wherein at least one remote sensor can be notified to activate and generate remote sensor data associated with a remote threat alert; in the ultra-precision threat detection mode, at least one remote sensor is activated on a continuous basis to generate remote sensor data associated with a remote threat alert.

[0009] In another aspect, the vehicle processor is further programmed to limit the information shared with the remote system based on information approval provided by the vehicle owner.

[0010] In another aspect, there are multiple vehicles and remote systems and a vehicle communication service that communicates with the vehicles via a cellular network. The vehicles share vehicle threat alerts and vehicle locations with the vehicle communication service, the vehicle communication service identifies a geographic area of ​​interest based on received vehicle threat alerts and transmits the threat detection to vehicles in the geographic area of ​​interest, and the vehicles that receive the threat detection transmit the threat detection to an associated remote system.

[0011] In another aspect, a vehicle communication service provides information about a threat image that is a threat concern based on a person or dangerous object in the image, and the vehicle processor is further programmed to: identify a threat alert by determining whether an image captured by a vehicle camera includes a threat image provided by the vehicle communication service, a remote system provides information about an image of a person that is not a threat concern, and the vehicle processor is further programmed to: identify the camera image as not a threat alert based on the image of the person provided by the remote system.

[0012] In another aspect, the vehicle communication service changes the geographic area of ​​interest and transmits the threat detection to vehicles in the changed geographic area of ​​interest, and the vehicles receiving the threat detection transmit the threat detection to an associated remote system.

[0013] In another embodiment, a method of integrating a vehicle having at least one vehicle sensor configured to generate vehicle sensor data associated with a vehicle threat alert and a remote system having at least one remote sensor configured to generate remote sensor data associated with a remote threat alert is provided. The method includes transmitting a location of the vehicle to the remote system, transmitting the vehicle threat alert with the remote system, transmitting the remote threat alert with the vehicle, modifying an operating mode of at least one vehicle sensor based on the received remote threat alert, and modifying a detection mode of the remote system based on the received vehicle threat alert.

[0014] In one aspect, the at least one vehicle sensor includes: a vehicle camera operable to capture images; and a vehicle access detector operable to detect the opening of a vehicle access feature. The method further includes: determining the vehicle threat alert based on the image from the vehicle camera and the state of the vehicle access detector; controlling the operation of the vehicle camera based on the operating mode; and checking communications received from the remote system.

[0015] In another aspect, the vehicle has a battery, and the method further includes: determining a voltage state of the battery; adjusting the operating mode based on the voltage state such that: when the voltage state is low voltage, the vehicle camera is deactivated; and when the voltage state is low voltage, activation of the vehicle camera is limited to receipt of a remote threat alert or threat alert from the vehicle access detector; and notifying the remote system of the operating mode.

[0016] In another aspect, the method further includes the remote system notifying the vehicle of the presence of a remote person of interest when the person detected by the remote system moves beyond the sensor range of the remote sensor, and activating the vehicle camera when notified of the presence of the remote person of interest.

[0017] In another aspect, the method further includes: when the person detected by the vehicle moves beyond the sensor range of the vehicle camera, the vehicle notifies the remote system of the presence of vehicle person attention; and when notified of the presence of the vehicle person attention, activating the at least one remote sensor.

[0018] On the other hand, there are multiple vehicles and remote systems, a vehicle communication service communicates with the vehicle via a cellular network, and the method also includes the vehicle communication service providing information about a threat image that is a threat concern based on a person or dangerous object in the image; the remote system providing information about a person image that is not a threat concern; the vehicle identifies the camera image as not a threat based on the information about the person image provided by the remote system; the vehicle identifies the threat alert by determining whether the image captured by the vehicle camera includes the threat image provided by the vehicle communication service; the vehicle shares the vehicle threat alert and the vehicle location with the vehicle communication service; the vehicle communication service identifies a geographic area of ​​concern based on the received vehicle threat alert; the vehicle communication service transmits the threat detection to vehicles in the geographic area of ​​concern; and the vehicle receiving the threat detection transmits the threat detection to the associated remote system.

[0019] In yet another embodiment, a vehicle is provided that integrates a vehicle sensor with a remote system. The vehicle includes: a vehicle controller, the vehicle controller including a processor and a memory; a vehicle sensor, the vehicle sensor configured to generate vehicle sensor data associated with a vehicle threat alert. The vehicle sensor includes: a vehicle camera, the vehicle camera can be operated to capture images and send images to the vehicle controller; and a vehicle access detector, the vehicle access detector can be operated to detect the opening of a vehicle access component; and a vehicle communication device, the vehicle communication device communicates with the vehicle controller, is configured to send the vehicle sensor data, and communicates with a remote system communication hub. The memory includes instructions so that the processor is programmed to: communicate with the remote system, check the remote threat alert received from the remote system, control the operating mode of the vehicle sensor based on the communication received from the remote system and activate the vehicle camera as needed, if the vehicle camera is activated and the vehicle is communicating with the remote system, share the image from the vehicle camera with the remote system, determine the vehicle threat alert based on the image from the vehicle camera and the state of the vehicle access detector, share the vehicle threat alert with the remote system, and communicate the location of the vehicle with the remote system.

[0020] In another aspect, the processor is further programmed to: modify the operating mode based on a received remote threat alert, and the operating mode includes a relaxed mode, a normal mode, and a cautious mode. In the relaxed mode, the vehicle camera is off and the vehicle access detector is activated; in the normal mode, the vehicle access detector is activated, and the vehicle camera is randomly activated and off, and when the vehicle access detector senses a threat and the vehicle camera image is sent to the remote system, the vehicle camera is activated; and in the cautious mode, the vehicle camera is active, the vehicle camera image is transmitted to the remote system, and the vehicle access detector is activated.

[0021] In another aspect, the vehicle has a battery and the vehicle processor is further programmed to: determine a voltage state of the battery, control an operating mode based on the voltage state, place the vehicle in a relaxation mode when the voltage state is low voltage, limit a change in the operating mode from the relaxation mode to receiving a remote threat alert or a vehicle threat alert from the vehicle access detector when the voltage state is low voltage, and notify a remote system of the operating mode.

[0022] In another aspect, the vehicle includes a wireless communication device operable to communicate with a vehicle communication service that communicates with other vehicles having a cellular network device, the controller communicates with the cellular network device, and the processor is further programmed to: share threat detection with the vehicle communication service and the remote system, receive threat notifications from the vehicle communication service, transmit received threats to the remote system, determine a status of the vehicle, and activate the vehicle camera based on the status of the vehicle when notified of a threat, and share the detected threat when notified of a threat.

[0023] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0025] Figure 1 is a schematic diagram of a system for integrating vehicle sensors with remote system sensors according to an exemplary embodiment;

[0026] Figure 2 is a process flow diagram of the operation of an integrated vehicle according to an exemplary embodiment;

[0027] Figure 3 is a process flow diagram of the operation of an integrated remote system according to an exemplary embodiment;

[0028] Figure 4 is a process flow chart of a vehicle communication service according to an exemplary embodiment;

[0029] Figure 5 is a schematic diagram of a geographic area of ​​interest created in accordance with an exemplary embodiment; and

[0030] Figure 6 is a schematic diagram of a geographic area for creation of random security activations according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0032] refer to Figure 1, a system 10 is shown that integrates sensors of a vehicle 12 with sensors of a remote system 14 according to the principles of the present disclosure. Although the vehicle 12 is shown as a sedan, it is contemplated that the vehicle 12 may be any other type of vehicle, such as a light truck, a coupe, a sport utility vehicle (SUV), a recreational vehicle (RV), a semi-truck, etc. The vehicle 12 has: a vehicle controller 16, a vehicle communication device 18, a plurality of vehicle sensors 20, and a battery 22. The vehicle sensors 20 communicate with the vehicle controller 16 and capture vehicle sensor data and share the vehicle sensor data with the vehicle controller 16. The vehicle sensors 20 include a vehicle camera 24 and a vehicle access detector 26. It should be understood that other sensors may be employed, including motion detectors, radars, or lidars. The vehicle camera 24 captures images and sends the images to the vehicle controller 16. The vehicle access detector 26 detects the opening of a vehicle access component and sends an opening alert to the vehicle controller 16. The vehicle access component may include a door, a trunk, a hood, or other components. The vehicle controller 16 communicates with the battery 22 and determines the voltage state of the battery 22 based on the voltage level. The voltage level used to determine the voltage state may relate to various functional capabilities, such as the energy in the battery 22 and the ability of the vehicle 12 to operate in a normal manner, such as the ability to be driven. The vehicle controller 16 may be operated to determine the location of the vehicle 12. The vehicle controller 16 communicates with the vehicle communication device 18 and communicates using the vehicle communication device 18 as described below. The vehicle controller 16 and the vehicle communication device 18 may cooperatively function as a vehicle communication hub.

[0033] The remote system 14 may be in various formats, such as a security system, a surveillance system, a data collection system, etc. The remote system 14 is associated with a house 30 including a building 32 and its ground 34, which may include a driveway 36 or other parking features for the vehicle 12. The building 32 may be any type of building, such as a personal residence, an apartment building, an office building, a factory, a school, a store, etc. The remote system 14 has: a remote network controller 38, a communication hub 40, and a plurality of remote sensors 42. The remote sensors 42 communicate with the remote network controller 38 and are configured to generate remote sensor data associated with a remote threat alert, and to capture and share the remote sensor data with the remote network controller 38. The remote sensors 42 may include a remote motion detector 43 that detects motion and a remote camera 44 that captures images and sends the images to the remote network controller 38. The remote motion detector 43 may be a separate item or may be integrated into the remote camera 44. The communication hub 40 communicates with the remote network controller 38 and the vehicle communication device 18. The remote system 14 may include additional components and capabilities, such as a smart speaker that can provide warnings and lights that can be activated to indicate a threat, such as a flashing floodlight.

[0034] The communication hub 40 and the vehicle communication device 18 may include and utilize various communication technologies to enable the vehicle controller 16 to communicate with the remote system 14, to enable the vehicle controller 16 to communicate directly with the remote sensor 42, to enable the vehicle controller 16 to communicate with the vehicle sensor 20, and to enable the remote network controller 38 to communicate with the remote sensor 42. In an exemplary embodiment, the communication technology included and used by the vehicle communication device 18 and the communication hub 40 may be, for example, a Wi-Fi device, a Bluetooth device, and other wireless communication devices. The communication between the vehicle controller 16 and the remote system 14 may be communicated via Wi-Fi or Bluetooth or other wireless technology, such as but not limited to a wireless mesh network or a high-bandwidth wireless sensor network, and may use an Internet of Things (IOT) transaction protocol. For example, the IOT Matter protocol may be used to pair the vehicle 12 with the remote system 14. The electrical connection may also be used by the vehicle controller 16 to communicate with the vehicle sensor 20, and by the remote network controller 38 to communicate with the remote sensor 42. The vehicle communication device 18 may also include and utilize various communication technologies to enable the vehicle controller 16 to communicate directly with other vehicles 12. It should be understood that various additional wired and wireless technologies and communication protocols for communication are within the scope of the present disclosure. The remote system 14 can selectively approve which remote sensors 42 can communicate with the vehicle controller 16, and the vehicle controller 16 can communicate with and receive data from the approved remote sensors 42 through the communication hub 40 or through direct communication with the remote sensors 42.

[0035] The system 10 may include a cloud-based vehicle communication service 46 that communicates with all vehicles 12. The vehicle communication service 46 communicates with the vehicle communication device 18 that enables the vehicle controller 16 to communicate with the vehicle communication service 46. The vehicle communication device 18 and the vehicle communication service 46 communicate using wireless communications and may include a cellular device and communicate using cellular communications. It should be understood that various additional wireless technologies and communication protocols for communication are within the scope of the present disclosure.

[0036] The vehicle controller 16 is used to implement a vehicle method 100 for integrating the vehicle sensors 20 of the vehicle 12 with the remote sensors 42 of the remote system 14 according to the principles of the present disclosure, as described below. The vehicle controller 16 includes at least one processor 50 and a non-transitory computer-readable storage device or medium 52. The processor 50 can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the vehicle controller 16, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or a device generally used to execute instructions. The computer-readable storage device or medium 52 can include volatile and non-volatile storage in, for example, read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operating variables when the processor 50 is powered off. The computer readable storage device or medium 52 may be implemented using a plurality of storage devices, such as a PROM (Programmable Read Only Memory), an EPROM (Electrical PROM), an EEPROM (Electrically Erasable PROM), flash memory, or another electrical, magnetic, optical, or combination storage device capable of storing data, some of which represent executable instructions that may be used by the vehicle controller 16 to control the various systems of the vehicle 12. The vehicle controller 16 may also include a plurality of controllers in electrical communication with each other. The vehicle controller 16 may be interconnected with additional systems and / or controllers of the vehicle 12, allowing the vehicle controller 16 to access data, such as the location of the vehicle 12.

[0037] Reference now Figure 2 , a vehicle method 100 for integrating the vehicle sensor 20 of the vehicle 12 with the remote sensor 42 of the remote system 14 according to the principles of the present disclosure is shown. The vehicle method 100 begins at step 102, where the vehicle controller 16 receives information from the owner of the vehicle 12, the vehicle battery 22, the vehicle sensor 20, the vehicle communication service 46, and the remote system 14 ( Figure 1 The data includes vehicle permissions set by the owner or operator of the vehicle 12, the voltage status of the vehicle battery 22, images from the vehicle camera 24, an open alarm from the vehicle access detector 26, a threat alarm from the vehicle communication service 46, and a threat alarm or command from the remote system 14. The vehicle method 100 then proceeds to step 104.

[0038] At step 104, the vehicle controller 16 determines the operating mode set for the vehicle 12. The operating mode is a vehicle state that sets the activity level of the plurality of vehicle sensors 20 and whether sensor input data should be recorded or transmitted. The operating modes include a relaxed mode, a normal mode, and a cautious mode.

[0039] In the relaxed mode, the vehicle camera 24 is turned off, so that the use of the vehicle battery 22 is reduced, and the vehicle access detector 26 is activated. In the normal mode, if the engine of the vehicle 12 is started or if the vehicle is charging, the vehicle access detector 26 is activated and the vehicle camera 24 is activated. If the engine of the vehicle 12 is turned off or not charging, the vehicle camera 24 is periodically activated to capture images with a normal set schedule. For example, the periodic schedule may include capturing images every ten seconds, every five minutes, every ten minutes, etc. The periodic schedule may also be randomized. In the cautious mode, if the engine of the vehicle 12 is started or the vehicle 12 is charging or if the voltage state exceeds the charging threshold, the vehicle access detector 26 is activated and the vehicle camera 24 is activated, as described below. Otherwise, the vehicle camera 24 is periodically activated with a more frequent cautious setting schedule than the normal setting schedule, so that the vehicle camera 24 captures images more frequently than the normal mode.

[0040] The determination of the operating mode is based on the voltage state of the battery 22, whether the engine of the vehicle 12 is started or the vehicle 12 is charging, whether the vehicle 12 has previously determined that a vehicle threat exists (as defined in the following step 110), whether the remote system 14 or the vehicle communication service 46 has provided a threat alert and transmitted it to the vehicle 12, or whether an activation request for setting a specific operating mode has been received from the vehicle communication service 46. For example, the vehicle controller 16 compares the voltage state received from the vehicle battery 22 to a charge threshold. In the example provided, the charge threshold is defined as 50% charged. However, other charge levels may be used. When the battery charge level is less than the charge threshold, the voltage state is defined as low voltage.

[0041] As a default, the vehicle controller 16 will place the vehicle 12 in normal mode when the engine of the vehicle 12 is started or charging. The vehicle controller 16 will place the vehicle 12 in cautious mode when there is a vehicle threat, a threat alert provided, or an activation request. The vehicle controller 16 will place the vehicle in relaxed mode when the battery 22 is low voltage. However, the relaxed mode can be overridden when there is a vehicle threat detected by the vehicle 12, a threat alert provided by the vehicle communication service 46 or the remote system 14, or an activation request provided by an operator of the vehicle 12, the vehicle communication service 46, or the remote system 14. For example, when the vehicle 12 is located next to a remote system 14 that is actively monitoring for threats, the vehicle controller 16 sets the operating mode to cautious mode. The vehicle method 100 then proceeds to step 106.

[0042] At step 106 , the vehicle controller 16 determines what information or data can be shared with the remote system 14 and the vehicle communication service 46 . In general, any information that is not restricted by the owner of the vehicle 12 and is not restricted based on the operating mode can be shared. For example, the owner of the vehicle 12 can determine what information can be shared and what information can not be shared, and can set permissions that limit images to be shared, limit the location of the vehicle, etc. This allows the owner of the vehicle 12 to address any privacy issues. The operating mode limits what information can be shared, for example, when the vehicle battery 22 is low voltage, and the sharing of images and other data from the vehicle camera 24 can be limited to only sharing during active threats (as defined in the following step 110 ). The vehicle method 100 then proceeds to step 108 .

[0043] At step 108, the vehicle controller 16 shares information with the vehicle communication service 46 and the remote system 14 based on what information is determined to be shareable at step 106. For example, the vehicle controller 16 shares the location of the vehicle 12, the operating mode, the detection mode of the remote system 14, the threat alerts with the vehicle communication service 46, and the sensor data from the vehicle sensors 20. The vehicle method 100 then proceeds to step 110.

[0044] At step 110, the vehicle controller 16 makes a vehicle threat determination based on the sensor data from the vehicle sensors 20. The vehicle controller 16 determines the presence of a vehicle threat based on the sensor data from the vehicle sensors 20. For example, if the vehicle access detector 26 indicates that the vehicle access feature is on when the vehicle 12 is locked, a vehicle threat is determined to be present. A vehicle threat is also determined to be present when the vehicle controller 16 checks the image from the vehicle camera 24 to make a vehicle threat determination based on the image. The vehicle controller 16 determines whether the image from the vehicle camera 24 includes a hazardous item, a face or person of concern, or a known face or person. Hazardous items and faces or persons of concern are threat concerns, while known faces or persons are not threat concerns. Information about dangerous items and faces or persons of concern is provided by the vehicle communication service 46 and may be based on information from social media, government and security agencies such as police departments, security offices, and the Federal Bureau of Investigation (FBI). Information about known faces or persons is created by the user of the remote system 14 and the owner of the vehicle 12 and provided by the owner of the vehicle 12 and the remote system 14. Dangerous items are items that may be associated with criminal or harmful behavior, such as guns, rifles, knives, swords, explosive devices, or other types of weapons. Faces or persons of concern are faces of persons or persons of concern based on threats, such as criminals, wanted persons, or fugitives. Known faces or persons are faces of persons or persons of concern that are considered safe and not threatening, such as family members, friends, or neighbors.

[0045] If the image from the vehicle camera 24 includes a hazardous object or a face or person of interest, the vehicle controller 16 determines that a vehicle threat exists. If the image from the vehicle camera 24 includes only known faces or persons, the vehicle controller 16 determines that a vehicle threat does not exist. If the image from the vehicle camera 24 includes unknown faces or persons and does not include a hazardous object or a face or person of interest, the vehicle controller 16 may determine the likelihood of a vehicle threat. If it is determined that there is no vehicle threat, the vehicle method 100 returns to step 102. If a vehicle threat is determined, the vehicle method 100 proceeds to step 112.

[0046] At step 112 , the vehicle controller 16 communicates the vehicle threat alert to the remote system 14 and the vehicle communication service 46 , and the vehicle method 100 then proceeds to step 114 .

[0047] At step 114, the vehicle controller 16 shares sensor data from the vehicle sensors 20 used to determine vehicle threat alerts with the remote system 14. For example, the vehicle controller 16 shares images from the vehicle cameras 24 with the remote system 14. The operating mode may limit what is shared with the remote system 14 and how often it is shared. The operating mode controls which vehicle cameras 24 are active and the frequency of activation and deactivation of the vehicle cameras 24, such that continuous sharing of images from the vehicle cameras 24 is not possible. The vehicle method 100 then proceeds to step 116.

[0048] At step 116 , the vehicle controller 16 monitors for identified threats and records sensor data from the vehicle sensors 20 . The vehicle method 100 then proceeds to step 118 .

[0049] At step 118, the vehicle controller 16 determines whether the identified threat has left the sensor range of the vehicle sensors 20 monitoring the identified threat. If the vehicle controller 16 determines that the identified threat has not left the sensor range of the vehicle sensors 20 monitoring the identified threat, the vehicle method 100 returns to step 116. If the vehicle controller 16 determines that the identified threat has left the sensor range of the vehicle sensors 20 monitoring the identified threat, the vehicle method 100 proceeds to step 120.

[0050] At step 120 , the vehicle controller 16 notifies the remote system 14 that the identified threat has left the sensor range of the vehicle sensors 20 monitoring the identified threat. The vehicle method 100 then returns to step 102 .

[0051] Once the remote network controller 38 receives the vehicle threat alert and any shared data, the remote network controller 38 responsively advances to an ultra-precision mode, wherein the remote camera 44 is active and recording.

[0052] Reference now Figure 3 , a remote system method 200 for integrating the vehicle sensor 20 of the vehicle 12 with the remote sensor 42 of the remote system 14 according to the principles of the present disclosure is shown. The remote system method 200 begins at step 202, where the remote network controller 38 receives data from the remote system 14, the remote sensor 42, and the user of the vehicle 12. The data from the user can reflect that the user restricts the activation of certain remote sensors 42 due to approved activities on the premises 30, in the building 32, on the land 34, and in the lane 36. The user can also limit or prevent information from specific remote sensors 42 from being shared with the vehicle 12 to address any privacy issues. This user restriction can prevent false or inaccurate remote threat alerts from approved activities. The user can also provide information about known faces or people, which the remote system 14 can share with the vehicle 12, and the information about the known faces or people can be used to determine that the image captured by the remote camera 44 is not a threat, as described below. The remote system method 200 then proceeds to step 204.

[0053] At step 204, the remote network controller 38 determines the detection mode of the remote system 14 and places the remote system 14 in the determined detection mode. The detection modes include an occasional threat detection mode, a high-precision threat detection mode, and an ultra-precision threat detection mode. The occasional threat detection mode causes some or all remote sensors 42 to be turned off for a limited period of time based on the user 45 wanting to limit the activation of some remote sensors 42 due to approved activities on the house 30, in the building 32, on the land 34, and in the driveway 36. The high-precision threat detection mode is a normal mode, and all remote sensors 42 are activated so that the remote motion detector 43 can detect motion and notify the associated remote camera 44 to start and capture images to detect and notify threats, and use normal algorithms. The ultra-precision threat detection mode causes all remote sensors 42 to be activated, the remote motion detector 43 always detects motion, all remote cameras 44 are always in working state, and capture images to use the most accurate algorithm to analyze images from the remote cameras 44 to detect and notify threats. The determination of a detection mode different from the default high accuracy threat detection mode is based on user input or a desire to provide enhanced threat detection. The default mode is the high accuracy threat detection mode, and the remote network controller 38 places the remote system 14 in the high accuracy threat detection mode unless the remote network controller 38 receives a request from the user, or detects a threat, or receives a threat alert, or receives an activation request. If a request for a reduced detection mode is received from the user 45, the remote network controller 38 will place the remote system 14 in the occasional threat detection mode. If the remote system 14 detects a threat or receives a threat alert from the vehicle 12 or receives an activation request from the vehicle 12, the remote network controller 38 will place the remote system 14 in the ultra-accuracy threat detection mode. The remote system method 200 then proceeds to step 206.

[0054] At step 206 , the remote network controller 38 shares information with the vehicle 12 . The remote network controller 38 shares the detection pattern with the vehicle 12 and shares information about the known faces or persons that have been indicated by the user as not being a threat. The remote system method 200 then proceeds to step 208 .

[0055] At step 208, the remote network controller 38 performs a remote threat determination based on the sensor data from the remote sensor 42. The remote network controller 38 determines the presence of a remote threat based on the sensor data from the remote sensor 42. The remote network controller 38 checks the image from the remote camera 44 to perform a remote threat determination based on the image. As described above, the remote network controller 38 determines whether the image from the remote camera 44 includes a dangerous object, a face or person of interest, or a known face or person. Information about dangerous objects, faces or persons of interest, and known faces or persons may be provided by the vehicle 12. If the image from the remote camera 44 includes a dangerous object or a face or person of interest, the remote network controller 38 determines that a remote threat exists. If the image from the remote camera 44 includes only known faces or persons, the remote network controller 38 determines that a remote threat does not exist. If the image from the remote camera 44 includes an unknown face or person and does not include a dangerous object or a face or person of interest, the remote network controller 38 may determine the likelihood of a remote threat. If it is determined that there is no remote threat, the remote system method 200 returns to step 202. If a remote threat is determined, the remote system method 200 proceeds to step 210 .

[0056] At step 210, the remote network controller 38 has determined the presence of a remote threat and will share the determined presence of the remote threat as a remote threat alert. The remote network controller 38 shares the remote threat alert with the vehicle 12. The remote system method 200 then proceeds to step 212. Once the vehicle 12 receives the remote threat alert and any shared data, the vehicle controller 16 verifies the received remote threat alert and modifies the operating mode of at least one vehicle sensor based on the received remote threat alert.

[0057] At step 212 , the remote network controller 38 monitors for identified threats and records sensor data from the remote sensors 42 . The remote system method 200 then proceeds to step 214 .

[0058] At step 214, the remote network controller 38 determines whether the identified threat has left the sensor range of the remote sensor 42 capable of monitoring the identified threat. If the remote network controller 38 determines that the identified threat has not left the sensor range of the remote sensor 42 capable of monitoring the identified threat, the remote system method 200 returns to step 212. If the remote network controller 38 determines that the identified threat has left the sensor range of the remote sensor 42 capable of monitoring the identified threat, the remote system method 200 proceeds to step 216.

[0059] At step 216 , the remote network controller 38 notifies the vehicle 12 that the identified threat has left sensor range and the remote sensor 42 is able to monitor the identified threat. The remote system method 200 then returns to step 202 .

[0060] Reference now Figure 4 , a vehicle communication service method 300 for integrating the vehicle sensor 20 of the vehicle 12 with the remote sensor 42 of the remote system 14 according to the principles of the present disclosure is shown. The vehicle communication service method 300 begins at step 302, where the vehicle communication service 46 receives data from the vehicle 12. The data from the vehicle 12 includes vehicle threat alerts, vehicle location, vehicle operating mode, and remote system detection mode. Then, the vehicle communication service method 300 proceeds to step 304.

[0061] At step 304 , the vehicle communication service 46 shares information regarding hazardous items and faces or persons of interest that may be found in the image and are threat concerns for the vehicle 12 . The vehicle communication service method 300 then proceeds to steps 306 and 312 .

[0062] In step 306, the vehicle communication service 46 determines whether a threat has been detected by reviewing data provided by the vehicle 12. If a threat alert has been provided by the vehicle 12, the vehicle communication service 46 determines that a threat has been detected. If a threat alert has not been provided by the vehicle 12, the vehicle communication service 46 determines that no threat has been detected. If no threat has been detected, the vehicle communication service method 300 returns to step 302. If a threat has been detected, the vehicle communication service method 300 then proceeds to step 308.

[0063] refer to Figure 4 and Figure 5 In step 308, the vehicle communication service 46 determines a geographic area of ​​interest 60 based on the location of the vehicle 12 that provided the threat alert. The geographic area of ​​interest 60 identifies risk areas with different risk levels based on the distance from the location of the vehicle 12 that provided the threat alert. The geographic area of ​​interest 60 includes a high risk area 62 that includes and surrounds the location of the vehicle 12 that provided the threat alert, a medium risk area 64 that is outside and surrounds the high risk area 62, and a low risk area 66 that is outside and surrounds the medium risk area 64. The area, size, and shape of the risk areas 62, 64, and 66 may vary based on the number and location of multiple threat alerts, the characteristics of the area, and the nature of the threat alert. The risk areas 62, 64, and 66 may each include a vehicle 12 and a remote system 14 that communicates with the vehicle 12. The vehicle communication service method 300 then proceeds to step 310.

[0064] At step 310, the vehicle communication service 46 shares threat detections with vehicles 12 in the geographic area of ​​interest 60 and may share risk levels based on the risk zones 62, 64, and 66 in which the vehicle 12 is located. As described above, the vehicle 12 may share received threat alerts with the communicating remote system 14. The vehicle communication service method 300 then returns to step 302.

[0065] refer to Figure 4 and Figure 6 , steps 312 to 320 are used to provide unpredictable positioning and activation of threat detection systems (such as vehicles 12 and remote systems 14) so ​​that it is difficult or impossible for intruders and criminals to know or predict areas that are not protected by active threat detection systems. In step 312, the vehicle communication service 46 determines a geographic area 70 for randomly activating vehicles 12 and remote systems 14 in the geographic area 70. The size and location of the geographic area 70 can vary. The vehicle communication service method 300 then proceeds to step 314.

[0066] In step 314 , the vehicle communication service 46 determines the desired operating mode for the vehicles 12 and the desired detection mode for the remote systems 14 in the geographic area 70 . The vehicle communication service method 300 then proceeds to step 316 .

[0067] In step 316 , the vehicle communication service 46 determines the vehicles 12 that are located in the geographic area 70 . The vehicle communication service method 300 then proceeds to step 318 .

[0068] In step 318 , the vehicle communication service 46 randomly selects a vehicle 12 that is located in the geographic area 70 . The vehicle communication service method 300 then proceeds to step 320 .

[0069] In step 320, the vehicle communication service 46 sends an activation request to the vehicle 12 selected in step 318. The activation request includes the desired operating mode and detection mode determined in step 314. The vehicle 12 may initiate the requested operating mode and will share the requested activation with the associated remote system 14. The remote system 14 may initiate the requested detection mode. The vehicle communication service method 300 then returns to step 302.

[0070] Through the system 10, the vehicle controller 16, the remote network controller 38, and the vehicle communication service 46 can determine the presence of a threat based on sensor data received from any source. For example, the vehicle controller 16 can determine the presence of a threat based on images and data received from the remote sensor 42, the remote network controller 38 can determine the presence of a threat based on images and data from the vehicle sensor 20, and the vehicle communication service 46 can determine the presence of a threat based on images and data from the vehicle sensor 20 and / or the remote sensor 42. In addition, while the vehicle controller 16, the remote network controller 38, and the vehicle communication service 46 can make threat determinations individually, they can also work together to make threat determinations.

[0071] The system 10 for integrating vehicle sensors with remote system sensors provides many advantages. The system 10 enhances vehicle safety by leveraging remote (home, office, etc.) system sensing and processing capabilities. In addition, detected threats can be uploaded to the cloud and further assist vehicles and homes in the neighborhood in situational awareness and potential threat detection.

[0072] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to be within the scope of the present disclosure. Such variations should not be regarded as departing from the spirit and scope of the present disclosure.

Claims

1. A system for integrating a vehicle sensor with a remote system sensor, the system comprising: A vehicle, the vehicle comprising: Vehicle controller; at least one vehicle sensor in communication with the vehicle controller and configured to generate vehicle sensor data associated with a vehicle threat alert; and a vehicle communication device in communication with the vehicle controller and configured to transmit the vehicle sensor data; A remote system, the remote system comprising: Remote network controller; at least one remote sensor in communication with the remote network controller and configured to generate remote sensor data associated with a remote threat alert; and a communication hub in communication with the remote network controller and the vehicle communication device; wherein the vehicle and the remote system communicate via the communication hub, the vehicle is operable to share vehicle threat alerts with the remote system and to receive remote threat alerts from the remote system, the vehicle controller checks the received remote threat alerts and modifies an operating mode of the at least one vehicle sensor based on the received remote threat alerts, and the remote network controller checks the received vehicle threat alerts and modifies a detection mode of the remote system based on the received vehicle threat alerts.

2. The system according to claim 1, wherein: The at least one vehicle sensor includes: a vehicle camera operable to capture an image and transmit the image to the vehicle controller; and a vehicle access detector operable to detect opening of a vehicle access feature; and the vehicle controller includes a vehicle processor and a vehicle memory, the vehicle memory including instructions such that the vehicle processor is programmed to: sharing the location of the vehicle with the remote system; determining the vehicle threat alert based on an image from the vehicle camera and a status of the vehicle access detector; communicating with the remote system; controlling the operation of the vehicle camera; reviewing communications received from the remote system; and The vehicle threat alert is shared with the remote system.

3. The system according to claim 2, wherein: The vehicle processor is further programmed to control the operating modes, which include: a relaxation mode, wherein the vehicle camera is off and the vehicle access detector is activated; a normal mode in which the vehicle access detector is activated, the vehicle camera is activated if the vehicle is on, and the vehicle camera is periodically activated on a normally set schedule to capture images if the vehicle is off; and a discreet mode in which the vehicle access detector is activated, the vehicle camera is activated if the vehicle is started, and when the vehicle is turned off, the vehicle camera is periodically activated at a discreet set schedule that is more frequent than the normal set schedule, causing the camera to capture images more frequently, and causing images captured by the vehicle camera to be sent to the remote system.

4. The system according to claim 3, wherein: The vehicle has a battery, and the vehicle processor is further programmed to: determining a voltage state of the battery; controlling the operating mode based on the voltage state; placing the vehicle in the relaxation mode when the voltage state is a low voltage; when the voltage state is low voltage, limiting the change in the operating mode from the relaxed mode to receiving a remote threat alert or a threat alert from the vehicle access detector; as well as The remote system is notified of the operating mode.

5. The system according to claim 3, wherein: the remote system creating the remote threat alert when a remote person is detected by the at least one remote sensor, and notifying the vehicle when the remote person moves beyond the sensor range of the at least one remote sensor and the controller is to place the vehicle into the cautious mode; the vehicle creating the vehicle threat alert when a vehicle occupant is detected by the at least one vehicle sensor; And when the vehicle occupant moves beyond the sensor range of the at least one vehicle sensor, the vehicle notifies the remote system, and the remote system changes the detection mode to a high-precision threat detection mode or an ultra-precision threat detection mode.

6. The system according to claim 1, wherein: The remote system detection mode includes: an occasional threat detection mode, wherein the at least one remote sensor is turned off for a limited period of time based on input from a user of the remote system; a high accuracy threat detection mode, the high accuracy threat detection mode being a default detection mode and wherein the at least one remote sensor is enabled to be notified to activate and generate remote sensor data associated with a remote threat alert; and An ultra-precision threat detection mode, wherein the at least one remote sensor is activated on a continuous basis to generate remote sensor data associated with a remote threat alert.

7. The system according to claim 2, wherein: The vehicle processor is also programmed to limit information shared with the remote system based on information approval provided by an owner of the vehicle.

8. The system according to claim 2, wherein: There are multiple vehicles and the remote system, and also include a vehicle communication service that communicates with the vehicles through a cellular network, and wherein the vehicles share vehicle threat alerts and vehicle locations with the vehicle communication service, the vehicle communication service identifies a geographic area of ​​concern based on received vehicle threat alerts, and transmits threat detections to vehicles in the geographic area of ​​concern, and the vehicles that receive the threat detections transmit the threat detections to the associated remote system.

9. The system according to claim 8, wherein: The vehicle communication service provides information about threat images, which are threat concerns based on people or dangerous objects in the images, and the vehicle processor is also programmed to: identify threat alerts by determining whether the images captured by the vehicle camera include threat images provided by the vehicle communication service, and the remote system provides information about images of people that are not threat concerns, and the vehicle processor is also programmed to: identify the camera image as not a threat alert based on the images of people provided by the remote system.

10. The system according to claim 8, wherein: The vehicle communication service changes the geographic area of ​​interest and transmits the threat detection to vehicles in the changed geographic area of ​​interest, and the vehicles receiving the threat detection transmit the threat detection to an associated remote system.

Images