In-car living thing detection sensor
By installing detection sensors in the car and using infrared thermal imaging and feature matching technology, the problem of low detection accuracy of live objects in the car is solved, and an effective warning to children, incapacitated people or pets is achieved, reducing the risk of forgetting.
Patent Information
- Application Number
- CN202210507551.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-11
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-05-11
AI Technical Summary
The existing live organism detection technology has low accuracy and is prone to missed detection, resulting in high risk of children, incapacitated people or pets being left in the car.
A living object detection sensor in the car is designed, including the main/co-pilot position detection module, infrared thermal imaging module, feature recognition module and matching module. By monitoring the interior environment in real time, the living objects in infrared thermal imaging images are extracted and matched. Features: Use the model library to determine whether there are living things in the car and alarm through Bluetooth wireless connection.
It improves the accuracy of live objects recognition and can effectively warn children, special groups of incapacity or pets to stay in the car, reducing the risk of heat stroke or other dangers.
Smart Images

Figure CN115016021B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field, and in particular to a living thing detection sensor in a vehicle. Background Art
[0002] With the popularity of cars, traveling by car has become a common tool for Chinese families, especially on weekends when the whole family goes out for outings, shopping, and shopping. Since children are placed in safety seats in the back seats, some parents may ignore the existence of children and leave them alone in the car because they are busy answering calls or other urgent matters after arriving at the destination. Especially in the hot summer, when the car is turned off, the temperature in the car will rise rapidly. Children are too young to open the car door by themselves, or they are not awake in deep sleep, which can easily cause heatstroke or more serious injuries. When parents find out, it is too late. Such reports are often seen in newspapers and television reports, which is very regrettable. In fact, not only infants or children, but other "incapable" people or pets will also be left in the car by the car owner for "temporary" reasons out of convenience and fluke, and then they are often forgotten. Due to the changes in the car environment (the temperature in the car rises / cools too quickly, lack of oxygen), tragedies may occur. Therefore, it is of great significance to prevent such things from happening through technical means.
[0003] At present, some reminder functions have appeared in the car manufacturers and after-sales market to prevent children and other special people with no behavioral ability or pets from being forgotten in the car after the owner leaves or locks the car. However, these reminder functions are based on the premise of relatively accurate detection of living things in the car. At present, detection of living things in the car is not popular enough, the accuracy is not high, and it is easy to make false detections and missed detections. Summary of the invention
[0004] The object of the present invention is to provide a sensor for detecting living things in a vehicle, so as to solve the problem of low accuracy of detecting living things in a vehicle.
[0005] In order to achieve the above object, the present invention provides an in-vehicle living thing detection sensor, which is connected to an on-vehicle electronic control unit and includes:
[0006] A driver / co-driver seat detection module, used to detect whether the driver / co-driver seat has exited the vehicle and send out a first detection signal;
[0007] an infrared thermal imaging module, which monitors the interior environment of the vehicle in real time in response to the first detection signal and obtains an infrared thermal imaging image of the interior of the vehicle;
[0008] A feature recognition module, used to perform feature recognition of living things on the infrared thermal imaging image and extract a target area suspected to be a living thing;
[0009] The matching module searches and matches the outer contour of the target area in a model library, and determines whether there is a living thing in the car according to the matching result.
[0010] Optionally, the main / co-pilot seat detection module is connected to the vehicle-mounted electronic control unit, a gravity sensor is arranged under the main / co-pilot seat, the gravity sensor is connected to the vehicle-mounted electronic control unit, the vehicle-mounted electronic control unit receives a second detection signal transmitted by the gravity sensor, and sends the second detection signal to the main / co-pilot seat detection module, and the main / co-pilot seat detection module sends the first detection signal according to the second detection signal.
[0011] Optionally, the feature recognition module includes:
[0012] A preprocessing unit, used for preprocessing the infrared thermal imaging image;
[0013] A segmentation unit is used to segment the preprocessed infrared thermal imaging image to obtain a number of candidate regions;
[0014] A feature extraction unit is used to extract living features from the candidate area; and
[0015] The target recognition unit classifies the candidate areas according to the living thing characteristics to obtain a target area suspected to be a living thing.
[0016] Optionally, a K-means clustering segmentation algorithm is used to perform image segmentation on the infrared thermal imaging image, and Fourier descriptors, Haar wavelet features, or directional gradient histogram features are used to extract living creature features in the candidate area.
[0017] Optionally, the matching module includes:
[0018] A searching unit is used to search the model library for the outer contour of the target area, find a feature vector similar to the feature vector of the outer contour of the target area, and use the model corresponding to the feature vector similar to the feature vector of the outer contour of the target area in the model library as the similar model of the target area;
[0019] The judging unit is used to judge the similarity between the target area and the similarity model, and determine whether there is a living thing in the vehicle according to the similarity.
[0020] Optionally, the distance between the feature vector of the outer contour of the target area and the feature vector of the model in the model library is used as a condition for judging the similarity, and when the distance between the feature vector of the outer contour of the target area and the feature vector of the similar model is less than a set value, it is determined that there is a living thing in the vehicle.
[0021] Optionally, also include:
[0022] Input module, used to upload image information of living things;
[0023] The edge extraction module is used to extract the outer contour of the living thing in the image information and store it in the model library as a model in the model library.
[0024] Optionally, the matching module is connected to the vehicle-mounted electronic control unit signal. After the matching module determines that there is a living thing in the vehicle, the matching module sends a third detection signal to the vehicle-mounted electronic control unit, and the vehicle-mounted electronic control unit responds to the third detection signal and issues an alarm.
[0025] Optionally, the matching module and the on-board electronic control unit as well as the driver / passenger seat detection module and the on-board electronic control unit are wirelessly connected via Bluetooth.
[0026] Optionally, also include:
[0027] The evaluation module is used to receive the user's living thing determination result of the matching module, and remind the user to update the model library when the error rate of the living thing determination result of the matching module is greater than a threshold.
[0028] In the in-car living thing detection sensor provided by the present invention, the main / co-driver seat detection module detects whether the main / co-driver seat has gotten off the car and sends a first detection signal; the infrared thermal imaging module monitors the in-car environment in real time in response to the first detection signal and obtains the infrared thermal imaging image in the car; the feature recognition module performs living thing feature recognition on the infrared thermal imaging image and extracts the target area suspected of being a living thing; the matching module searches and matches the outer contour of the target area in a model library and determines whether there is a living thing in the car based on the matching result. The present invention extracts the outer contour of the target area extracted from the infrared thermal imaging image again, and determines whether there is a living thing in the car based on the matching result of the outer contour and the model library, which can improve the accuracy of living thing recognition and can effectively warn children, special people with no behavioral ability or pets who are stranded in the car. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 A structural block diagram of a living thing detection sensor in a vehicle provided by an embodiment of the present invention;
[0030] Figure 2 A structural block diagram of a feature recognition module provided in this embodiment;
[0031] Figure 3 A structural block diagram of a matching module provided in an embodiment of the present invention;
[0032] Wherein, the accompanying drawings are marked as follows:
[0033] 100-driver / co-driver seat detection module; 200-infrared thermal imaging module; 300-feature recognition module; 301-preprocessing unit; 302-segmentation unit; 303-feature extraction unit; 304-target recognition unit; 400-matching module; 401-search unit; 402-judgment unit; 500-on-board electronic control unit; 600-gravity sensor; 700-model library; 800-edge extraction module; 900-input module. DETAILED DESCRIPTION
[0034] The specific implementation of the present invention will be described in more detail below in conjunction with the schematic diagram. The advantages and features of the present invention will become clearer based on the following description. It should be noted that the drawings are all in a very simplified form and are not in exact proportions, and are only used to facilitate and clearly assist in explaining the purpose of the embodiments of the present invention.
[0035] Figure 1 This is a structural block diagram of the in-vehicle living thing detection sensor provided in this embodiment. Figure 1 As shown, the in-vehicle living thing detection sensor is connected to the vehicle-mounted electronic control unit 500Electronic Control Unit (ECU), including the following connected in sequence:
[0036] The main / co-pilot seat detection module 100 is used to detect whether the main / co-pilot seat has exited the vehicle and send out a first detection signal;
[0037] The infrared thermal imaging module 200 monitors the interior environment of the vehicle in real time in response to the first detection signal and obtains an infrared thermal imaging image of the interior of the vehicle;
[0038] The feature recognition module 300 is used to perform feature recognition of living things on the infrared thermal imaging image and extract a target area suspected to be a living thing;
[0039] The matching module 400 searches and matches the outer contour of the target area in a model library 700, and determines whether there is a living thing in the car according to the matching result.
[0040] It should be understood that since children, special people with no behavioral capacity or pets usually cannot sit in the front row of the car, but in the back row or trunk, and the guardian (driver) usually sits in the front row (i.e. the main / co-pilot seat), when the guardian leaves, the children, special people with no behavioral capacity or pets are usually left in the back row. A gravity sensor 600 is usually provided under the main / co-pilot seat to remind the user of the main / co-pilot seat to wear a seat belt.
[0041] The gravity sensor 600 is usually connected to the vehicle-mounted electronic control unit 500, and the gravity sensor 600 transmits a second detection signal to the vehicle-mounted electronic control unit 500, and the second detection signal is used to indicate whether there is someone in the main / co-pilot seat. The vehicle-mounted electronic control unit 500 receives the second detection signal transmitted by the gravity sensor 600 and takes corresponding actions according to the second detection signal, such as reminding the user not to fasten the seat belt on the center console.
[0042] Based on this, in this embodiment, the main / co-pilot seat detection module 100 is connected to the vehicle-mounted electronic control unit 500. When the vehicle-mounted electronic control unit 500 receives the second detection signal transmitted by the gravity sensor 600, and sends the second detection signal to the main / co-pilot seat detection module 100, the main / co-pilot seat detection module 100 sends the first detection signal according to the second detection signal. The first detection signal is a signal that triggers the infrared thermal imaging module 200 to work. For example, when the second detection signal indicates that there is no one in the main / co-pilot seat, the first detection signal triggers the infrared thermal imaging module 200 to start working. On the contrary, when the second detection signal indicates that there is someone in the main / co-pilot seat, the first detection signal can control the infrared thermal imaging module 200 to turn off or be in standby mode.
[0043] As an optional embodiment, the infrared thermal imaging module 200 may be an infrared thermal imager. There may be one or more infrared thermal imagers installed in the vehicle, and specifically installed on the side or top surface of the vehicle.
[0044] The infrared thermal imaging module 200 starts to work in response to the first detection signal, monitors the interior environment of the vehicle in real time, and obtains infrared thermal imaging images of the interior of the vehicle.
[0045] Further, Figure 2 FIG. 3 is a structural block diagram of the feature recognition module 300 provided in this embodiment. Figure 2 As shown, the feature recognition module 300 includes a preprocessing unit 301 , a segmentation unit 302 , a feature extraction unit 303 and a target recognition unit 304 .
[0046] The preprocessing unit 301 is used to preprocess the infrared thermal imaging image. In this embodiment, the preprocessing includes filtering the infrared thermal imaging image to remove noise. For example, the infrared thermal imaging image can be filtered by a smoothing filter algorithm, a low-pass filter algorithm, or a median filter algorithm. Since the useful information is well preserved when the infrared thermal imaging image is filtered by the median filter algorithm, the target is not fragmented, and the edge of the target is well preserved, in this embodiment, the infrared thermal imaging image is filtered by the median filter algorithm.
[0047] The segmentation unit 302 is used to perform image segmentation on the preprocessed infrared thermal imaging image to obtain a number of candidate regions. For example, the image segmentation of the preprocessed infrared thermal imaging image can be performed using an edge detection image segmentation method, a threshold segmentation method, or a clustering segmentation method. Since the K-means clustering segmentation algorithm has a fast running speed, good applicability to samples, and good segmentation effect, in this embodiment, the K-means clustering segmentation algorithm is used to perform image segmentation on the infrared thermal imaging image.
[0048] The feature extraction unit 303 is used to extract the living thing features in the candidate area. In this embodiment, the living thing is a human being, and the living thing features are inherent features of the human body. The inherent features of the human body can be Fourier descriptors, Haar wavelet features, or directional gradient histogram features, etc. The living thing features of the candidate area can be extracted using Fourier descriptors, Haar wavelet features, or directional gradient histogram features. Of course, two or more living thing features can also be extracted.
[0049] The target recognition unit 304 can classify the candidate areas according to the living thing features to obtain a target area suspected of being a living thing. Specifically, the target recognition unit 304 can be a pre-built classifier, such as an SVM classifier, etc., which classifies the candidate areas into several categories. The obtained target area is a target area suspected of being a living thing. Since the accuracy of feature recognition is not high enough, it cannot be determined whether there is a living thing at this time.
[0050] Furthermore, in this embodiment, the models stored in the model library 700 are all models of the outer contours of living things. For example, when the living thing is a person, the models stored in the model library 700 can be models of the outer contours of a child or an elderly person; and the outer contour of the living thing can be the outer contour when sitting, lying, sideways, or frontally, etc.
[0051] In this embodiment, the in-vehicle living thing detection sensor also includes an input module 900, which can, for example, read information in an external USB flash drive. A user can upload image information of living things through the input module 900. At this time, the user can actually upload image information of his or her own children, elderly people or pets from various shooting angles.
[0052] Based on this, the in-vehicle living thing detection sensor further includes an edge extraction module 800, which is connected to the input module 900 and can be used to extract the outer contour of the living thing in the image information uploaded by the user, and store it in the model library 700 as the model in the model library 700. In this way, the user can supplement the model in the model library 700 according to the actual situation of the user, thereby improving the accuracy of the judgment.
[0053] Figure 3 FIG. 4 is a structural block diagram of the matching module 400 provided in this embodiment. Figure 3 As shown, the matching module 400 includes a search unit 401 and a judgment unit 402 which are connected.
[0054] The search unit 401 is used to search the model library 700 for the outer contour of the target area, and find a feature vector similar to the feature vector of the outer contour of the target area. When no similar feature vector is found, it can be preliminarily determined that there is no living thing in the car; when a similar feature vector is found, the model corresponding to the feature vector similar to the feature vector of the outer contour of the target area in the model library 700 is used as the similar model of the target area.
[0055] The judgment unit 402 is connected to the search unit 401, and is used to judge the similarity between the target area and the similarity model, and judge whether there is a living thing in the car according to the similarity. Specifically, if the similarity between the target area and the similarity model is high, it can be judged that there is a living thing in the car; if the similarity between the target area and the similarity model is low, it can be judged that there is no living thing in the car.
[0056] In this embodiment, the distance between the feature vector of the outer contour of the target area and the feature vector of the model in the model library 700 is used as a condition for judging the similarity. For example, when the distance between the feature vector of the outer contour of the target area and the feature vector of the model in the model library 700 is less than a predetermined value, it can be judged that the feature vector of the outer contour of the target area is similar to the feature vector of the model in the model library 700, thereby finding the similar model.
[0057] Furthermore, when the distance between the feature vector of the outer contour of the target area and the feature vector of the similar model is less than a set value, the feature vector representing the outer contour of the target area is very similar to the feature vector of the similar model. Since the models in the model library 700 are all models of the outer contours of living things, it can be determined that there are living things in the car.
[0058] It should be understood that increasing the number of models stored in the model library 700 can improve the accuracy of judgment, but at the same time it will extend the search time.
[0059] Furthermore, the matching module 400 is connected to the vehicle-mounted electronic control unit 500 by signal. When the matching module 400 determines that there is a living thing in the vehicle, it sends a third detection signal to the vehicle-mounted electronic control unit 500. The vehicle-mounted electronic control unit 500 responds to the third detection signal and issues an alarm, thereby being able to provide effective warning for children, special people with no behavioral capacity, or pets stranded in the vehicle.
[0060] As an optional embodiment, the matching module 400 and the on-board electronic control unit 500 as well as the main / co-driver position detection module 100 and the on-board electronic control unit 500 are connected wirelessly via Bluetooth. Of course, they can also be connected via GSM and / or GPRS communication methods, which will not be explained one by one here.
[0061] Furthermore, in this embodiment, the in-vehicle living thing detection sensor also includes an evaluation module. The evaluation module is used to receive the user's living thing determination result of the matching module 400, and when the error rate of the living thing determination result of the matching module 400 is greater than a threshold, remind the user to update the model library 700. Specifically, for the living thing determination result of the matching module 400, the user can send evaluation information each time, such as sending information that the living thing determination result is correct or the living thing determination result of this time. When the error rate of the living thing determination result is greater than a threshold, it indicates that the judgment of the in-vehicle living thing detection sensor is not accurate enough at this time, and some new living thing image information can be uploaded to update the model library 700, thereby improving the accuracy of subsequent detection.
[0062] In summary, in the in-vehicle living thing detection sensor provided in this embodiment, the main / co-pilot seat detection module detects whether the main / co-pilot seat has gotten off the vehicle and sends a first detection signal; the infrared thermal imaging module monitors the in-vehicle environment in real time in response to the first detection signal and obtains the infrared thermal imaging image in the vehicle; the feature recognition module performs living thing feature recognition on the infrared thermal imaging image and extracts the target area suspected of being a living thing; the matching module searches and matches the outer contour of the target area in a model library and determines whether there is a living thing in the vehicle based on the matching result. The present invention can improve the accuracy of living thing recognition by extracting the outer contour of the target area extracted from the infrared thermal imaging image again and determining whether there is a living thing in the vehicle based on the matching result of the outer contour and the model library, and can effectively warn children, special people with no behavioral ability or pets who are stranded in the vehicle.
[0063] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.
Claims
1. A living thing detection sensor in a vehicle, characterized in that: The in-vehicle living thing detection sensor is connected to the vehicle-mounted electronic control unit, and includes: A driver / co-driver seat detection module, used to detect whether the driver / co-driver seat has exited the vehicle and send out a first detection signal; an infrared thermal imaging module, which monitors the interior environment of the vehicle in real time in response to the first detection signal and obtains an infrared thermal imaging image of the interior of the vehicle; A feature recognition module, used to perform feature recognition of living things on the infrared thermal imaging image and extract a target area suspected to be a living thing; A matching module searches and matches the outer contour of the target area in a model library, and determines whether there is a living thing in the car according to the matching result; The main / co-pilot seat detection module is connected to the vehicle-mounted electronic control unit. A gravity sensor is provided under each of the main / co-pilot seats. The gravity sensor is connected to the vehicle-mounted electronic control unit. The vehicle-mounted electronic control unit receives a second detection signal transmitted by the gravity sensor and sends the second detection signal to the main / co-pilot seat detection module. The main / co-pilot seat detection module sends the first detection signal according to the second detection signal. The feature recognition module comprises: A preprocessing unit, used for preprocessing the infrared thermal imaging image; A segmentation unit is used to segment the preprocessed infrared thermal imaging image to obtain a number of candidate regions; A feature extraction unit, used to extract living features from the candidate area; A target recognition unit, which classifies the candidate areas according to the living thing characteristics to obtain a target area suspected of being a living thing; The infrared thermal imaging image is segmented by using a K-means clustering segmentation algorithm, and the living creature features of the candidate area are extracted by using a Fourier descriptor, a Harr wavelet feature or a directional gradient histogram feature; The matching module comprises: A searching unit is used to search the model library for the outer contour of the target area, find a feature vector similar to the feature vector of the outer contour of the target area, and use the model corresponding to the feature vector similar to the feature vector of the outer contour of the target area in the model library as the similar model of the target area; A judging unit, used for judging the similarity between the target area and the similarity model, and judging whether there is a living thing in the vehicle according to the similarity; The distance between the feature vector of the outer contour of the target area and the feature vector of the model in the model library is used as a condition for judging the similarity, and when the distance between the feature vector of the outer contour of the target area and the feature vector of the similar model is less than a set value, it is determined that there is a living thing in the car.
2. The in-vehicle living thing detection sensor as claimed in claim 1, characterized in that Also includes: Input module, used to upload image information of living things; The edge extraction module is used to extract the outer contour of the living thing in the image information and store it in the model library as a model in the model library.
3. The in-vehicle living thing detection sensor as claimed in claim 1, characterized in that: The matching module is connected to the vehicle-mounted electronic control unit by signal. After the matching module determines that there is a living thing in the vehicle, it sends a third detection signal to the vehicle-mounted electronic control unit. The vehicle-mounted electronic control unit responds to the third detection signal and issues an alarm.
4. The in-vehicle living thing detection sensor as claimed in claim 3, characterized in that: The matching module and the on-board electronic control unit as well as the main / co-driver position detection module and the on-board electronic control unit are connected wirelessly via Bluetooth.
5. The in-vehicle living thing detection sensor as claimed in claim 1, characterized in that: Also includes: The evaluation module is used to receive the user's living thing determination result of the matching module, and remind the user to update the model library when the error rate of the living thing determination result of the matching module is greater than a threshold.
