Vehicle safety system

Abstract

Method comprising: Determining that a vehicle is unoccupied; Detecting an unexpected object in the vehicle and / or an intrusion into a vehicle compartment; Classifying the unexpected object and / or intrusion according to a risk, determining a route to a location selected according to the unexpected object and / or intrusion; wherein a destination of the vehicle is selected at least in part based on one or more respective classifications of an unexpected object and / or intruder; and Navigate the vehicle to the location.

Description

PRIOR ARTUsers may leave items in a shared vehicle upon exiting the vehicle. The shared vehicle cannot be returned to a service shop after each use for inspection or cleaning during which such items could be found. Thus, a user may not be able to recover a lost item before another user has access to the vehicle. Further, there is a risk that a user, although accidental, will leave a hazardous object in the vehicle, e.g. a weapon, a chemical, an explosive material concealed in a bag, etc.The document WO 2016 / 183 241 A1 relates to a method for monitoring a vehicle which is used to provide a transport service. The method includes determining, from one or more sensors of the vehicle, that an object to be removed from the vehicle after completion of the transportation service remains in the vehicle; and in response to determining that the object remains in the vehicle, automatically initiating a remedial action.US 2016 / 0 116 913 A1 discloses a method for detecting the environment for use in an autonomously controllable vehicle, which is designed for the conveyance of human passengers, in which hazardous substances are detected in the vehicle or in the vicinity thereof during the autonomous operation and the autonomous functions and the operation of the vehicle are automatically deactivated in response to the detection of the hazardous material.US 2009 / 0 146 813 A1 discloses a remote notification system for an automobile, which comprises a detector that can determine the presence of a living occupant, as well as a transmitter for transmitting and a receiver for receiving the notification, wherein the transmitter transmits the message as a voice message to a mobile telephone.U.S. Pat. No. 6,922,147 B1 discloses a warning system for detecting a child left in a car seat of a vehicle, which warning system comprises a child occupancy sensor connected to the car seat, which child occupancy sensor has an input for detecting the child and an output for outputting a signal about the presence of the child in the child car seat, and a temperature sensor for detecting an ambient temperature of the vehicle. An alarm control circuit connected to the car seat is configured to provide a warning of whether the child is in the child car seat and the ambient temperature is outside an acceptable range.According to the invention, a method having the features of independent claim 1 and a system having a data processing device in a vehicle having the features of independent claim 10 are proposed. Advantageous embodiments can be found in the dependent claims.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a block diagram of an example system for detecting an object in a vehicle. FIG. 2 is a flowchart of an example process for detecting an object in a vehicle and performing actions. FIG. 3 is a flowchart of an example process for vehicle route restrictions when an unexpected object is detected in a vehicle.DETAILED DESCRIPTIONFIG. 1 shows an example vehicle system 100. A vehicle 110 may include a computing device 115 and sensors 116 having a detection field (not shown) that includes at least a portion of a vehicle interior. The computing device 115 may receive data from the sensors 116. Further, the computing device 115 may include programming, e.g., as a set of instructions stored in and executable by a memory of a processor of the computing device 115, for operations described herein, e.g., to detect an object in a vehicle and cause actuation of one or more vehicle components based on such a determination. In particular, the computing device 115 may be programmed to determine that the vehicle is unoccupied, that an unexpected object is present in the vehicle, for example, by evaluating the sensor data received from the sensors 116, calculating a route to a location selected according to the unexpected object, and navigating the vehicle to the location.Example System ElementsThe vehicle 110 is typically a land vehicle having three or more wheels. The vehicle 110 may be propelled in a variety of known ways, e.g., with an electric motor and / or an internal combustion engine. The vehicle 110 includes the computing device 115, sensors 116, and other elements discussed herein below.The data processing apparatus 115 includes a processor and a memory as are known. The memory further includes one or more forms of computer readable media and stores instructions executable by the processor for performing various operations including those disclosed herein. For example, computing device 115 may include programming to actuate one or more of vehicle brakes, propulsion (e.g., control acceleration in vehicle 110 by controlling an internal combustion engine and / or an electric motor and / or a hybrid motor, etc.), steering, air conditioning, interior and / or exterior lighting, etc., as well as to determine whether and when computing device 115 is to control such operations, as opposed to a human operator.The computing device 115 may include, or be communicatively coupled to, more than one computing device, e.g., via a vehicle communication bus, as further described below, e.g., controllers or the like included in the vehicle 110 to monitor and / or control various vehicle components, e.g., a transmission controller 112, a brake controller 113, a steering controller 114, etc. The computing device 115 is generally configured for communication on a vehicle communication network, such as a bus in the vehicle 110, such as a controller area network (CAN) or the like; the vehicle 110 network may include wired or wireless communication mechanisms known, e.g., Ethernet or other communication protocols.Via the vehicle network, the computing device 115 may send messages to various devices in the vehicle and / or receive messages from the various devices, e.g., controllers, actuators, sensors, etc., including sensors 116. Alternatively or additionally, if the computing device 115 actually comprises multiple devices, the vehicle communication network may be used for communication between devices represented as the computing device 115 in the present disclosure. Further, as mentioned below, various controllers or sensing elements may provide data to computing device 115 via the vehicle communication network.Additionally, the computing device 115 may be configured for communication via a vehicle-to-infrastructure (V-to-I) interface 111 with a remote server computer 120, e.g., a cloud server, via a network 130, which may utilize various wired and / or wireless networking technologies, e.g., cellular, BLUETOOTH® wired and / or wireless packet networks, etc., as described below.As mentioned above, generally included in instructions stored in the memory and executed by the processor of the computing device 115 is programming to actuate one or more components of the vehicle 110, e.g., brakes, steering, propulsion, etc., without human operator intervention. Using data received at computing device 115, e.g., the sensor data from sensors 116, server computer 120, etc., computing device 115 may make various determinations and / or control various components and / or operations of vehicle 110 without a driver to operate vehicle 110. For example, the computing device 115 may include programming to regulate vehicle 110 operational behaviors such as speed, acceleration, deceleration, steering, etc., as well as tactics such as distance between vehicles and / or duration of time between vehicles, lane change minimum distance between vehicles, left turn-over path minimum, time to arrival at a particular location, intersection (without blinking), minimum time to arrival to cross the intersection, etc.Controllers, as used herein, are computing devices that are typically programmed to control a specific vehicle subsystem. Examples include a power transfer controller 112, a brake controller 113, and a steering controller 114. A controller may be an electronic control unit (ECU) as is known, possibly including additional programming as described herein. The controllers may be communicatively connected to and have instructions from the computing device 115 to actuate the subsystem according to the instructions. For example, the brake controller 113 may receive instructions from the computing device 115 to apply the brakes of the vehicle 110.The sensors 116 may include a variety of devices known to provide data over the vehicle communication bus. For example, a radar mounted on a front bumper (not shown) of the vehicle 110 may provide a distance of the vehicle 110 to a next vehicle in front of the vehicle 110, or a global positioning system (GPS) sensor located in the vehicle 110 may provide geographic coordinates of the vehicle 110. The range or geographic coordinates provided by the radar may be used by the computing device 115 to operate the vehicle 110 autonomously or semi-autonomously.As will be described further below with reference to the process 200 illustrated in FIG. 2, the computing device 115 may evaluate the sensor data received from the sensors 116 to detect a status of vehicle occupancy, i.e., whether the vehicle 110 is occupied by a human occupant user. For example, the vehicle 110 may have one or more seats, and the sensors 116 may include weight sensors disposed in the seats for detecting an occupant on the seat. The computing device 115 may receive the status of the vehicle occupancy from a suitable interface, such as is known. As will be discussed further below, various types of sensors 116 could be used to detect the status of the vehicle occupancy.For example, the sensors 116 may include one or more cameras disposed in the vehicle 110 that provide optical image data that includes at least a portion of the vehicle interior. The data processing device 115 may receive the optical image data via a suitable interface, such as is known.Further, the sensors 116 may include microphones disposed in the vehicle, e.g., the interior or a trunk, that provide audio data. The data processing device 115 may receive the audio data by means of a suitable interface, e.g. an analog-to-digital converter.Additionally or alternatively, the sensors 116 may include one or more thermal cameras disposed in the vehicle 110 that provide thermal data from the vehicle interior. The computing device 115 may receive the thermal data via an analog or digital interface or any other interface. Thermal data can be used to generate a map or image, where warmer portions of a region in the thermal camera detection field can be represented with a color that contrasts with cooler regions. This may provide discrimination between the vehicle interior 110 and occupants in the vehicle due to the different temperature of the human body compared to the vehicle interior.Additionally or alternatively, the sensors 116 may include one or more infrared sensors that provide infrared data from the vehicle interior. The infrared sensors may further include excitation sources and infrared receivers. The excitation sources may illuminate the vehicle interior with infrared radiation. The infrared sensor may detect a reflection of the infrared illumination as infrared data. The computing device 115 may receive the infrared data via a suitable interface, such as is known.Further, the sensors 116 may include one or more radio frequency receiving elements to detect a short range data connection from an in-vehicle controller to an external device, such as a key fob in communication with the in-vehicle computing device 115. For example, an active communication from a vehicle key with an immobilization control of the vehicle or a BLUETOOTH® connection with a mobile user device 160 may be interpreted as occupancy of the vehicle, i.e., a driver of the vehicle may be located in the vehicle 110 or may be located in his vicinity, e.g., at a predetermined distance from the vehicle 110.The sensors 116 may further include a GPS module, as is known, included in the vehicle 110 and / or the mobile user device 160. The information processing device 115 may receive the current location of the mobile user device 160 detected by the GPS sensor included in the GPS-equipped mobile user device 160 via a mobile communication network that communicates with the V-to-I interface 111. A GPS coordinate in the vicinity of the vehicle 110 reported by the user mobile device 160 may be considered an indication of vehicle 110 occupancy.ProcessesFIG. 2 shows an example process 200 for determining whether an unexpected object is in a vehicle 110 when the vehicle 110 is not occupied by a user. In the present context, a detected object is "unexpected" when computing device 115 determines that the object should not be in vehicle 110 at a time of detection of the object. For example, computing device 115 could include in its memory a list of objects that could be detected in vehicle 110 when vehicle 110 is unoccupied and / or could be programmed to expect that no object is detected at one or more locations in vehicle 110 when the vehicle is unoccupied.The process 200 begins in a block 201, in which the data processing device 115 receives data from the sensor 116. As discussed above, such data may include audio data and / or infrared, thermal, and / or visual (i.e., camera) image data, etc.Next, in a block 205, the computing device 115 determines whether the vehicle is unoccupied. Such determination may include, as indicated above, processing of optical, infrared, thermal, audio, ultrasound, and / or other sensor data received at block 201. Additionally or alternatively, determining the vehicle occupancy 205 may include receiving location data of a mobile user device 160. If a location of the device 160 is more than a predetermined distance, e.g., 5 meters, from the vehicle 110, it may be determined that the user is not occupying the vehicle 110.If it is determined that the vehicle 110 is occupied, the process 200 returns to block 201. If it is determined that the vehicle 110 is not occupied, the process 200 proceeds to a block 210. It is possible that the computing device 115 could perform steps such as those described below to detect an object in the vehicle 110 when the vehicle 110 is occupied by a user.At block 210, the computing device 115 collects data of the sensor 116 at block 210, e.g., data from sensors 116 such as cameras, weight sensors, microphones, etc. In one example, the data of the sensor 116 collected at block 210 may include different data compared to data of the sensor 116 collected at block 201, e.g., a sensor 116 may use signals that may be harmful to humans and may only be activated after determining that the vehicle 110 is unoccupied.Next, in a block 215, the computing device 115 has determined whether an unexpected object is detected. Computing device 115 may use known object detection techniques, e.g., block 215 may include use of image comparison techniques to compare received image data to reference images, i.e., current image data of the vehicle interior or trunk of the vehicle may be compared to one or more reference images previously captured and stored in the memory of computing device 115. A reference image typically shows a portion of the interior of the vehicle 110 without any unexpected objects and unoccupied. Differences determined by comparing the current image data to the reference image may indicate a presence of an unexpected object. Additionally or alternatively, the thermal image data may be used and compared to thermal reference image data. Use of thermal image data may be useful in detecting specific objects such as explosive material.As another addition or alternative, object detection at block 215 may include analyzing audio data received from sensors 116 using signal processing algorithms, e.g., a sound of a mechanically constructed timer may indicate the presence of a detonation mechanism.Further, for example, performing object detection at block 215 could include combining results of the object detection based on two or more types of sensor 116 data. For example, the computing device 115 could compare a prediction of an expected object determined by three types of sensor data, e.g., infrared image, camera image, and audio data, and predict the presence of an unexpected object only if two or more types of data from the sensor 116 provided an indication of an unexpected object. These or other data fusion techniques may reduce a likelihood of false detections and improve confidence in detecting the unexpected object, e.g., combining the results of the object detection from the audio data and the image data may help better identify the object or avoid false detection.In a block 220, the computing device 115 has determined whether an unexpected object was detected in the block 215. If so, the computing device 115 proceeds to a block 225 to classify the unexpected object. Otherwise, the process 200 proceeds to a block 230.At block 225, the computing device 115 uses image recognition techniques or the like to classify the one or more detected unexpected objects. Classifications could include, for example, "no risk", "moderate risk", "high risk", etc. The data processing device 115 could store reference images of objects in a "no risk" category, e.g., fast food bags, drinking cups, etc. The computing device 115 could also store reference images of objects in other categories. If the computing device 115 is unable to identify an object for classification, a medium or high risk category could be a requirement. Further, at block 225, the computing device 115 may send a message to the server computer 120 indicating the detection of the unexpected object and possibly including its classification as well. In some implementations, block 225 could be omitted and it could be assumed that each detected unexpected object represents a single risk level, justifies the action as described below, and / or server 120 could simply be notified of an unexpected object.In a block 230 that could follow each of the blocks 220, 225, the computing device 115 determines whether a user has attempted to access a restricted area of the vehicle 110. Access to some areas of the vehicle 110 may be restricted to reduce complexity of detecting objects in the vehicle, e.g., disallowing access to a glove box, an engine compartment, or other space. In such a configuration, no additional sensors may be necessary for detecting objects in these restricted areas. However, the user may intentionally attempt to gain access to these restricted areas to place an unexpected object. The sensors 116 may therefore include sensors for detecting intrusion into these restricted areas. As shown in block 230, intrusion into such restricted areas may be detected using sensor 116 data. If intrusion into a restricted area is detected, a block 235 is next executed. If desired, the process 200 proceeds to a block 240.In block 235, a detected intrusion is classified. Intrusion classification may be made according to categories such as those described above regarding unexpected objects and according to properties of a restricted area, e.g. size of the restricted area, i.e. a more hazardous object may be placed in a larger restricted area. In some implementations, block 235 could be omitted and it could be assumed that each detected intrusion represents a single risk level that justifies action as described below.At block 240, the computing device 115 determines whether an unexpected object and / or intrusion into a restricted area has been detected. If not, process 200 ends. However, if an unexpected object and / or intrusion has been detected, the process 200 continues at block 245 to take one or more actions to handle the unexpected object and / or intrusion.Next, in a block 245, the computing device 115 determines one or more actions for the vehicle 110 based on the detected unexpected object or objects and / or the detected intrusion(s). For example, the action could be to plan and navigate a route to a service shop or the like where the vehicle could be inspected. Alternatively, the action could be to drive the vehicle to a safe location, e.g., a location where emergency or service personnel can examine the vehicle 110, where the vehicle 110 may present a lesser risk to surrounding persons and / or other vehicles, buildings, etc. The action could be based on a classification of the object and / or intrusion performed as described above. For example, a classification of a highest level of risk could mean that computing device 115 causes the vehicle to navigate to the next location that can be identified where vehicle 110 presents the smallest risk to the environment. On the other hand, classifications of low or medium risk could mean that the computing device 115 schedules or causes a route to a service shop or the like in which the vehicle 110 can be inspected. A process 300 for planning and implementing such a route is described below with reference to FIG. 3. As another alternative or addition, e.g., if a highest level of risk is indicated, the computing device 115 could determine to provide alerts, e.g., by flashing the vehicle lights, sounding the vehicle horn, and / or directing a message to the server 120, etc.Next, in a block 250, the computing device 115 implements an action determined in the block 245. For example, the computing device 115 could cause the vehicle lamps to blink, sound a horn, send a message to the server 120, etc. Further, the computing device 115 could instruct various ECUs to control the vehicle 110 for navigation on a planned route. That is, the vehicle 110 may be automatically navigated according to the route determined in block 245. The computing device 115 may communicate with the sensors 116, the power transfer controller 112, the steering controller 114, and the brake controller 113 to accelerate, steer, and stop the vehicle on the determined route from the current location to the safe location. Alternatively or additionally, the server computer 120 may communicate with the data processing system 115 and apply a change to the destination, i.e., the particular safe location and / or route constraint.After block 250, the process 200 ends.FIG. 3 shows the details of an example process 300 for determining a route of the vehicle 110, e.g., as mentioned above with respect to the block 245 of the process 200.The process 300 begins in a block 301, in which the computing device 115 identifies a location of the vehicle 110, e.g., geo-coordinates (e.g., latitude and longitude), such as are known by the vehicle 110, according to a GPS sensor 116, for example, in the vehicle 110.Next, in a block 305, a destination of the vehicle 110 may be selected based at least in part on one or more respective classifications of an unexpected object and / or intruder. For example, if an unexpected object is detected and / or a highest risk level is detected, or both an unexpected object and an intrusion are detected, then a target could be selected to reflect the high risk level, e.g., the target could be a closest safe location. The memory of the computing device 115 could thus store a list of predetermined safe locations and, according to a predetermined rule, select the location from the list, e.g., the closest safe location at the current or parking location of the vehicle, a safe location that is unlikely to be surrounded by other vehicles or persons or at least has less dense environments, etc. Alternatively, the computing device 115 could send a request to the server computer 120 with information about the current location of the vehicle 110, the above(s) classification(s), etc. The server computer 120 then responds to the request by providing a route destination, i.e., a recommended safe location.Next, in a block 310, the computing device 115 may take into account the fact that moving the vehicle 110 to the destination selected in the block 305 may be at risk for areas on the planned route. Therefore, at block 310, the computing device 115 may impose one or more restrictions on the scheduled route using, for example, geo-furnaceing or the like. For example, a constraint could impose a minimum distance from the vehicle 110 to schools or shopping centers. In other words, a route constraint could be that the vehicle 110 should never come closer to a school or shopping center than a minimum distance defined by a geofence. Alternatively or additionally, other route constraints may be imposed, e.g., a speed constraint.At block 315, the computing device 115 may consider a constraint that may be imposed above and determine a route of the vehicle 115 from a current location of the vehicle 110 to the identified destination based on an identified destination as indicated above according to route constraints. Computing device 115 may use known route algorithms. In addition, the information processing apparatus 115 may inform the server computer 120 of the scheduled route.Computing devices such as those discussed herein generally each include instructions executable by one or more computing devices such as those identified above and for executing blocks or steps of processes described above. Computer-executable instructions may be compiled or interpreted from computer programs created using a variety of programming languages and / or technologies, including, without limitation, and either alone or in combination, Java™ C, C++, Visual Basic, Java Script, Perl, HTML, etc. A processor (e.g., a microprocessor) generally receives instructions, for example, from a memory, a computer-readable medium, etc., and executes these instructions to thereby execute one or more processes, including one or more of the processes described herein. Such instructions and other data may be stored and transmitted using a variety of computer readable media. A file in a computing device is generally a collection of data stored on a computer readable medium such as a storage medium, random access memory, etc.A computer readable medium includes any medium that participates in providing data (e.g., instructions) that may be read by a computer. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media, etc. Non-volatile media includes, for example, optical or magnetic disks and other persistent storage. Volatile media includes dynamic random access memory (DRAM) which typically constitutes main memory. Common forms of computer readable media include, for example, a floppy disk, a floppy disk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, an EPROM, a flash EEPROM, any other memory chip or cartridge, or any other medium from which a computer can read.With respect to the media, processes, systems, methods, etc. described herein, it should be appreciated that although the steps of such processes, etc. have been described as occurring according to a particular ordered sequence, such processes could be performed with the described steps in an order other than that described herein. It will be further understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of systems and / or processes are provided herein for the purpose of illustrating certain embodiments and should not be construed in any way as limiting the disclosed subject matter.Accordingly, it is to be understood that the present disclosure, including the above description and the accompanying figures and the following claims, is not intended to be limiting, but illustrative. Many embodiments and applications other than the examples given would become apparent to those skilled in the art upon review of the above description. The scope of the invention should not be determined with reference to the above description, but should instead be determined with reference to the claims appended hereto and / or included in a final patent application based thereon, along with the full scope of equivalents to which these claims are entitled. It is anticipated and intended that future developments will occur in the art discussed herein and that the disclosed systems and methods will be incorporated into such future embodiments. In summary, it should be understood that the disclosed subject matter may be modified and modified.All terms used in the claims are intended to have their obvious and ordinary meaning as understood by those skilled in the art, unless explicit reference is made herein to the contrary. In particular, the use of articles in the singular, such as "a", "an", "the", "the", etc., is to be understood as indicating one or more of the indicated elements, unless a claim expressly indicates an opposite limitation.

Claims

A method comprising: determining that a vehicle is unoccupied; detecting an unexpected object in the vehicle and / or intrusion into a vehicle space; classifying the unexpected object and / or intrusion according to a risk, determining a route to a location selected according to the unexpected object and / or intrusion; wherein a destination of the vehicle is selected based at least in part on one or more respective classifications of an unexpected object and / or intruder; and navigating the vehicle to the location.The method of claim 1, wherein the detecting comprises receiving image data from an imaging system and / or receiving audio data from microphones and / or receiving seat occupancy status from seat occupancy sensors and / or detecting a short-range data link to a mobile device and / or receiving a GPS location from a GPS-equipped mobile device.The method of claim 2, wherein the image data is image and / or infrared and / or thermal and / or ultrasound data.The method of claim 1, wherein determining that the unexpected object is present in the vehicle comprises comparing image data to reference images and / or performing object detection on the image data and / or detecting the object based on an audio signal and / or detecting intrusion into a restricted area in the vehicle.The method according to claim 4, wherein the image data is an optical image and / or an infrared image and / or a thermal image and / or an ultrasonic image.The method of claim 1, wherein determining the route to the location further comprises selecting the location based in part on the classification.The method of claim 1, wherein determining the route to the location further comprises assigning a route constraint based on the classification.The method of claim 7, wherein determining the route to the location is based on the route constraint.The method of claim 1, further comprising transmitting a message indicating the detection.A system comprising a computing device in a vehicle, the computing device comprising a processor and a memory, the memory storing instructions executable by the processor including instructions to determine that the vehicle is unoccupied; detect an unexpected object in the vehicle and / or intrusion into a vehicle compartment; classify the unexpected object and / or intrusion according to a risk, determine a route to a location selected according to the unexpected object and / or intrusion; wherein a destination of the vehicle is selected based at least in part on one or more respective classifications of an unexpected object and / or intruder; and navigate the vehicle based on the route.The system of claim 10, wherein the processor is further programmed to receive image data from an imaging system and / or audio data from microphones and / or seat occupancy status from seat occupancy sensors and / or a status of detecting a short-range data link to a mobile device and / or a GPS location from a GPS-equipped mobile device.The system of claim 11, wherein the image data is image and / or infrared and / or thermal and / or ultrasound data.The system of claim 10, wherein the processor is further programmed to compare image data to reference images and / or perform object detection on the image data and / or detect the object based on an audio signal and / or detect intrusion into a restricted area in the vehicle.The system of claim 13, wherein the image data is an optical image and / or an infrared image and / or a thermal image and / or an ultrasound image.The system of claim 10, wherein the process is further programmed to select the location based in part on the classification.The system of claim 10, wherein the processor is further programmed to assign a route constraint based on the classification.The system of claim 16, wherein determining the route to the location is based on the route constraint.The system of claim 10, wherein the processor is further programmed to send a message indicating the detection.

Images