In-vehicle life body monitoring control method, device, equipment, system and vehicle
By integrating multiple sensors and recognition models in the vehicle, accurate detection and monitoring of living organisms in a variety of scenarios is achieved, and the problem of inaccurate identification of single sensors in complex scenarios is solved, ensuring the safety of living organisms in the vehicle.
Patent Information
- Application Number
- CN202310608807.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-05-26
AI Technical Summary
The single sensor detection scheme inside the existing vehicle cannot accurately identify the living organisms in complex scenarios, and there are problems of missed detection and inaccuracy, which leads to tragedy such as high temperature suffocation and other tragedies such as locked in the car.
Using a variety of sensor fusion solutions, including optical cameras, infrared cameras, millimeter-wave radars, temperature sensors and photometric sensors, generate vital sign information through image and microdynamic sensing data, combine category identification models to control windows, ventilation systems and displays and other devices to realize intelligent reminders and remote alarms.
Automatic intelligent switching detection methods in various scenarios improve the accurate identification and monitoring of living organisms in the car, reduce the possibility of missed or missed inspections, and promptly and effectively send reminders to avoid the living organisms being locked in the car Inflict damage.
Smart Images

Figure CN119018080A_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention generally relate to the field of vehicle technology, and more specifically, to a method, device, equipment, system, vehicle and storage medium for monitoring and controlling a living being in a vehicle. Background Art
[0002] In recent years, social tragedies of personal injury caused by locking children in cars have occurred frequently. Parents mistakenly leave their children in the car due to carelessness and other reasons. The temperature will rise quickly in a closed environment (especially in direct sunlight in the summer), so children in the car are very likely to suffer from high temperature suffocation, causing tragedy.
[0003] Currently, the vehicles on the market are usually equipped with only a single sensor to detect life forms inside the vehicle, and the application scenarios are single. Therefore, in complex scenarios, there is a high probability of missing detection of life forms inside the vehicle, and there are certain limitations and detection inaccuracies. Summary of the invention
[0004] In order to solve the above-mentioned problems in the prior art, in a first aspect, an embodiment of the present invention provides a method for monitoring and controlling a life form in a vehicle, the method comprising: in response to receiving a stop signal, acquiring an image captured by a camera arranged in the vehicle, wherein the camera comprises an optical camera and an infrared camera, and the image captured by the camera comprises an optical image and an infrared image; determining whether there is a life form in the vehicle based on the image captured by the camera; in the case of determining that there is a life form in the vehicle, acquiring the image captured by the camera and micro-motion sensing data captured by a millimeter-wave radar arranged in the vehicle, wherein the micro-motion sensing data comprises one or more of a respiratory frequency, a chest cavity fluctuation amplitude, and an abdominal cavity fluctuation amplitude; generating vital sign information based on the image captured by the camera and the micro-motion sensing data; and inputting the vital sign information into a life form category. In the recognition model, the category of the life form in the car is obtained; the current distance between the car key and the vehicle is obtained; when the current distance is less than or equal to a preset distance threshold, the car lights and / or speakers are controlled to perform a reminder operation; when the current distance is greater than the preset distance threshold, remote reminder information is sent to the user client through wireless communication, and, when the category of the life form in the car is the first category, the car windows and door locks are controlled to be opened; when the category of the life form in the car is the second category, one or more of the following steps are performed: the image captured by the camera is transmitted to the user client in real time; the ventilation system in the car is controlled to be turned on; the display and audio facilities set in the car are controlled to be turned on to play a preset video or audio file; voice or video communication is established with the user client via the camera and microphone set in the car.
[0005] In some embodiments, when it is determined that there is a living body in the vehicle, acquiring the image captured by the camera and the micro-motion sensing data collected by the millimeter-wave radar set in the vehicle includes: acquiring the temperature in the vehicle through a temperature sensor set in the vehicle, and comparing the temperature in the vehicle with a preset temperature threshold; acquiring the light intensity in the vehicle through a photometry sensor set in the vehicle, and comparing the light intensity in the vehicle with a preset first light intensity threshold and a second light intensity threshold, wherein the second light intensity threshold is higher than the first light intensity threshold; when the temperature in the vehicle is lower than the temperature threshold and the light intensity in the vehicle is lower than or equal to the first light intensity threshold, acquiring the image captured by the infrared camera and the micro-motion sensing data collected by the millimeter-wave radar set in the vehicle. The infrared image collected by the optical camera and the micro-motion sensing data collected by the millimeter-wave radar are obtained; when the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than the first light intensity threshold and lower than the second light intensity threshold, the optical image collected by the optical camera and the micro-motion sensing data collected by the millimeter-wave radar are obtained; when the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than or equal to the second light intensity threshold, the micro-motion sensing data collected by the millimeter-wave radar are obtained; when the temperature inside the vehicle is higher than or equal to the temperature threshold, the optical image collected by the optical camera and the micro-motion sensing data collected by the millimeter-wave radar are obtained.
[0006] In some embodiments, the method further includes: after determining that there is a living being in the vehicle, in response to receiving a vehicle locking signal, immediately performing a reminder operation.
[0007] In some embodiments, determining whether there is a living being in the vehicle based on an image captured by a camera includes: obtaining moving object information based on changes between different frames of the optical image and / or the infrared image; obtaining temperature information based on the infrared image; and determining whether there is a living being in the vehicle based on the moving object information and the temperature information.
[0008] In some embodiments, generating vital signs information based on the image captured by the camera and the micro-motion sensing data includes: calling a vital signs database, comparing the image captured by the camera and the micro-motion sensing data with the data in the vital signs database, and generating vital signs information based on the comparison results.
[0009] In some embodiments, the first category of in-vehicle life forms includes adults or elderly people; the second category of in-vehicle life forms includes disabled people, children, and pets.
[0010] In a second aspect, an embodiment of the present invention proposes a device for monitoring and controlling a life form in a vehicle, the device comprising: an image acquisition module, configured to acquire an image captured by a camera disposed in the vehicle in response to receiving a parking signal, wherein the camera comprises an optical camera and an infrared camera, and the image captured by the camera comprises an optical image and an infrared image; a life form preliminary determination module, configured to determine whether a life form exists in the vehicle based on the image captured by the camera; a sensing data acquisition module, configured to acquire the image captured by the camera and the micro-motion sensing data captured by a millimeter-wave radar disposed in the vehicle when determining that a life form exists in the vehicle, wherein the micro-motion sensing data comprises one or more of a respiratory frequency, a chest cavity fluctuation amplitude, and an abdominal cavity fluctuation amplitude; a vital sign information generation module, configured to generate vital sign information based on the image captured by the camera and the micro-motion sensing data; a life form category acquisition module, configured to input the vital sign information into a life form category recognition model to acquire the category of the life form in the vehicle; a distance An acquisition module is configured to acquire the current distance between the car key and the vehicle; a self-vehicle reminder module is configured to control the car lights and / or speakers to perform a reminder operation when the current distance is less than or equal to a preset distance threshold; a remote reminder module is configured to send a remote reminder message to a user client through wireless communication when the current distance is greater than the preset distance threshold; a first category handling module is configured to control the opening of the car windows and door locks when the current distance is greater than the preset distance threshold and the category of the life form in the car is the first category; a second category handling module is configured to perform one or more of the following operations when the current distance is greater than the preset distance threshold and the category of the life form in the car is the second category: transmitting the image collected by the camera to the user client in real time; controlling the opening of the ventilation system in the car; controlling the opening of the display and audio facilities set in the car to play a preset video or audio file; and establishing voice or video communication with the user client via the camera and microphone set in the car.
[0011] In a third aspect, an embodiment of the present invention proposes an in-vehicle life monitoring and control device, which is arranged on a vehicle, and includes a memory and a processor. A computer program is stored on the memory, and when the computer program is executed by the processor, the method described in any of the above embodiments is implemented.
[0012] In a fourth aspect, an embodiment of the present invention proposes an in-vehicle life monitoring and control system, the system comprising an in-vehicle life monitoring and control device according to the above embodiment and a client. The client is used to: receive remote reminder information sent by the in-vehicle life monitoring and control device, and display the remote reminder information; receive images transmitted in real time by the in-vehicle life monitoring and control device, and display the images; and establish voice or video communication with the in-vehicle life monitoring and control device.
[0013] In the fifth aspect, the embodiment of the present invention proposes a vehicle, which includes the in-vehicle life monitoring and control device according to the above embodiment, as well as a camera, a millimeter-wave radar, a display and an audio facility. The camera includes an optical camera and an infrared camera, the optical camera is used to collect optical images in the vehicle, and the infrared camera is used to collect infrared images in the vehicle; the millimeter-wave radar is used to collect micro-motion sensing data in the vehicle; the display and the audio facility are used to play a preset video or audio file under the control of the in-vehicle life monitoring and control device.
[0014] In some embodiments, the display is disposed on the back of a front seat of the vehicle.
[0015] In some embodiments, the millimeter wave radar is disposed on a ceiling above a rear row of the vehicle.
[0016] In some embodiments, the vehicle further includes a temperature sensor and a light sensor, wherein the temperature sensor is used to obtain the temperature inside the vehicle, and the light sensor is used to obtain the light intensity inside the vehicle.
[0017] In a sixth aspect, an embodiment of the present invention provides a storage medium storing computer-readable instructions, which, when executed by a processor, executes the method according to any of the above embodiments.
[0018] The in-vehicle life monitoring and control method, device, equipment, system and storage medium proposed in the embodiments of the present invention can automatically and intelligently switch the detection mode in various scenarios through the integration of multiple sensors. It overcomes the defects of the prior art that a single sensor cannot accurately identify the type of life in complex scenarios, thus causing limitations and inaccurate detection. In addition, multiple detection methods cooperate with each other to detect various life signs in a timely and accurate manner, detect different types of life, and issue adaptive control instructions to ensure that the possibility of missed detection or wrong detection is reduced, and more timely and effective prompt information is issued to avoid tragedies caused by life locked in the car. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The above and other objects, features and advantages of the embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:
[0020] Figure 1 A flow chart showing a method for controlling in-vehicle life monitoring according to an embodiment of the present invention is shown;
[0021] Figure 2 A schematic diagram showing an application scenario of an in-vehicle life monitoring and control device according to an embodiment of the present invention;
[0022] Figure 3 A schematic diagram of a vehicle arrangement according to an embodiment of the present invention is shown;
[0023] Figure 4 A block diagram of an in-vehicle life monitoring control device according to an embodiment of the present invention is shown.
[0024] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION
[0025] The principle and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way.
[0026] In one aspect, an embodiment of the present invention provides a method for monitoring and controlling a life form in a vehicle, referring to Figure 1 , which shows a flow chart of a method 100 for controlling in-vehicle life monitoring according to an embodiment of the present invention. The method 100 may include steps S101-S110.
[0027] In step S101, in response to receiving a parking signal, an image captured by a camera disposed in the vehicle is acquired, wherein the camera includes an optical camera and an infrared camera, and the image captured by the camera includes an optical image and an infrared image. As an example, the parking signal may include a gear change, for example, a change from a reverse (R) gear to a parking (P) gear, or a change from a forward (D) gear to a P gear. As another example, the parking signal may also be obtained by considering the vehicle speed, for example, the vehicle speed drops to zero.
[0028] In step S102, it is determined whether there is a living being in the vehicle based on the image captured by the camera.
[0029] Optionally, the existence of a living being can be determined when a living being is detected in an image captured by any one of the optical camera and the infrared camera, thereby avoiding missed detection.
[0030] As an embodiment of the present invention, step S102 may include: obtaining moving object information based on changes between different frames of optical images and / or infrared images; obtaining temperature information based on infrared images; and determining whether there is a living being in the vehicle based on the moving object information and the temperature information. In addition to the optical camera, an infrared camera is also used for life detection, which can cover a variety of environmental conditions and scenarios.
[0031] Optionally, the presence of a living being can be determined when a moving object is detected in an optical image, a moving object is detected in an infrared image, or an object with a corresponding temperature is detected in an infrared image. Alternatively, the presence of a living being can be determined when any two of the above three conditions are met. Also alternatively, the presence of a living being can be determined when all three of the above conditions are met, and in particular, the presence of a living being can be determined when vital signs of temperature and movement are detected. In practical applications, the detection conditions can be flexibly configured according to the detection requirements.
[0032] Through the above steps, a preliminary judgment on whether there is a living being in the car is achieved.
[0033] As an embodiment of the present invention, the method may further include: after determining that there is a life form in the car, in response to receiving a car lock signal, immediately performing a reminder operation. Thus, a reminder can be issued to the user immediately when the user locks the car, and the reminder information is used to remind the user to pay attention to the situation in the car in time. The reminder operation at this time may include self-car reminders and / or remote reminders. Self-car reminders include flashing lights, honking horns, etc. Remote reminders may be, for example, by sending a message to the user's client (e.g., a mobile phone).
[0034] In step S103, when it is determined that there is a living body in the vehicle, the image captured by the camera and the micro-motion sensing data collected by the millimeter-wave radar installed in the vehicle are obtained, wherein the micro-motion sensing data includes one or more of the respiratory rate, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude. Millimeter waves have great advantages in identifying moving objects. Compared with cameras, millimeter-wave radars can process echoes that are close to static, such as chest and abdomen fluctuations.
[0035] As an embodiment of the present invention, appropriate detection elements may be selected according to the environmental conditions at the time and the different sensitivities and accuracies of each detection element in different scenarios. The following Table 1 shows the sensing performance of different sensors in different application environments.
[0036] Table 1. Sensing performance of different sensors in different application environments
[0037]
[0038]
[0039] Based on the above considerations, step S103 may include:
[0040] The temperature inside the vehicle is obtained by a temperature sensor arranged inside the vehicle, and the temperature inside the vehicle is compared with a preset temperature threshold;
[0041] Acquiring the light intensity inside the vehicle through a photometric sensor disposed inside the vehicle, and comparing the light intensity inside the vehicle with a preset first light intensity threshold and a second light intensity threshold, wherein the second light intensity threshold is higher than the first light intensity threshold;
[0042] When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is lower than or equal to the first light intensity threshold, indicating that the temperature inside the vehicle is normal and the vehicle is in a relatively weak light environment, an infrared image collected by the infrared camera and micro-motion sensing data collected by the millimeter wave radar are obtained;
[0043] When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than the first light intensity threshold and lower than the second light intensity threshold, indicating that the temperature inside the vehicle is normal and the vehicle is in a normal brightness environment, an optical image collected by the optical camera and micro-motion sensing data collected by the millimeter wave radar are obtained;
[0044] When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than or equal to the second light intensity threshold, indicating that the temperature inside the vehicle is normal and the light intensity inside the vehicle is too high, the micro-motion sensing data collected by the millimeter wave radar is obtained;
[0045] When the temperature inside the vehicle is higher than or equal to the temperature threshold, that is, indicating that the vehicle is currently in a high temperature environment, an optical image collected by the optical camera and micro-motion sensing data collected by the millimeter-wave radar are acquired.
[0046] In the case where the existence of a living being has been determined, the above steps realize accurate collection of vital signs by selecting a combination of detection elements (ie, detection methods) to perform secondary collection of vital signs to improve accuracy.
[0047] In step S104, vital sign information is generated according to the image collected by the camera and the micro-motion sensing data.
[0048] As an embodiment of the present invention, step S104 may include: calling a vital sign database, comparing the image and micro-motion sensing data collected by the camera with the data in the vital sign database, and generating vital sign information according to the comparison result. The vital sign database may include, for example, a breathing frequency range, a chest fluctuation amplitude range, an abdominal fluctuation amplitude range, a body length range, a ratio between various parts of the body, facial features, behavioral posture features, and the like.
[0049] In step S105, the vital sign information is input into a life form category recognition model to obtain the category of the life form in the vehicle. For example, the life form category recognition model can be obtained by deep learning a large amount of sample data.
[0050] In step S106 , the current distance between the vehicle key and the vehicle is obtained.
[0051] In step S107, when the current distance is less than or equal to the preset distance threshold, the vehicle lights and / or speakers are controlled to perform a reminder operation, for example, the vehicle lights are controlled to flash and / or the horn is sounded. At this time, the user is close to the vehicle and can perceive the flashing lights and horn. This method can effectively serve as a reminder, allowing the user to return to the vehicle in time.
[0052] In step S108, when the current distance is greater than the preset distance threshold, a remote reminder message is sent to the user client via wireless communication. At this time, the user is already far away from the car and cannot perceive the flashing lights and honking of the car horn. Therefore, the reminder is sent to the user's handheld client. The user client can be, for example, a smartphone, a tablet computer, etc. The remote reminder message can be a text message or a message in an application (app) installed on the client.
[0053] In step S109, when the category of the living being in the vehicle is the first category, the vehicle windows and door locks are controlled to be opened. Generally, the first category may be a person who has the ability to take care of himself, for example, the first category may include an adult or an elderly person. For example, when the window and door locks are opened, an adult or an elderly person can get out of the vehicle by themselves.
[0054] In step S110, when the category of the life form in the car is the second category, one or more of the following vehicle control operations are performed: the image captured by the camera is transmitted to the user client in real time, so that the dynamics of the life form can be observed at any time; the ventilation system in the car is controlled to be turned on to maintain air circulation in the car; the display and audio facilities installed in the car are controlled to be turned on to play pre-set video or audio files; voice or video communication is established with the user client via the camera and microphone installed in the car, and the life form in the car is communicated and interacted, and the dynamics of the life form can be observed at any time to make a timely response.
[0055] The second category of life forms in the car can be people who are not able to take care of themselves or other life forms, such as disabled people, children, and pets. When the life form is of the second category, there will be safety hazards if you get out of the car or open the window. Through the above method, for example, when the life form in the car is a child, you can turn on the camera to observe the child's movements at any time, turn on the ventilation system in the car to circulate the air in the car, turn on the display and speakers to play animations to soothe the child's emotions, and wait for the user to rescue as soon as possible. For another example, when the life form is a pet, turn on the ventilation system in the car at the same time, turn on the camera to observe the pet's movements at any time, and use the microphone to speak to soothe the pet's emotions, waiting for the owner to rescue.
[0056] Through the above implementation, according to different types of life forms, corresponding response mechanisms are adopted, and different control signals are sent out by comprehensively considering the distance between the car key and the car and different types of life forms.
[0057] The in-vehicle life monitoring and control method proposed in the embodiment of the present invention can automatically and intelligently switch the detection mode in various scenarios through the integration of multiple sensors. The multiple detection modes cooperate with each other to timely and accurately detect various life signs, detect different types of life, and issue adaptive control instructions to ensure that the possibility of missed detection or wrong detection is reduced, and more timely and effective prompt information is issued to avoid tragedies caused by life locked in the car.
[0058] On the other hand, an embodiment of the present invention proposes an in-vehicle life monitoring and control device, which is arranged on a vehicle. The device includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the in-vehicle life monitoring and control method described in any of the above embodiments is implemented.
[0059] In yet another aspect, an embodiment of the present invention provides an in-vehicle life monitoring and control system, which includes the in-vehicle life monitoring and control device described in any of the above embodiments, and a client.
[0060] The client can be used to: receive remote reminder information sent by the in-vehicle life monitoring and control device, and display the remote reminder information; receive images transmitted in real time by the in-vehicle life monitoring and control device, and display the images; establish voice or video communication with the in-vehicle life monitoring and control device.
[0061] refer to Figure 2, which shows a schematic diagram of an application scenario of an in-vehicle life monitoring and control device according to an embodiment of the present invention. The in-vehicle life monitoring and control device according to an embodiment of the present invention is communicatively connected to a millimeter-wave radar and a camera, and communicates with a temperature sensor, a display, an on-board communication transceiver, and a body control module (BodyControl Module, BCM) through a gateway, and further controls speakers, lights, doors, windows, air conditioning (ventilation system), etc. through the BCM. The in-vehicle life monitoring and control device communicates with the user's client via the gateway and the on-board communication transceiver.
[0062] On the other hand, an embodiment of the present invention provides a vehicle, which includes the in-vehicle life monitoring and control device described in any of the above embodiments, as well as a camera, a millimeter-wave radar, a display, and audio facilities.
[0063] The camera includes an optical camera and an infrared camera. The optical camera is used to collect optical images inside the vehicle, and the infrared camera is used to collect infrared images inside the vehicle.
[0064] Millimeter-wave radar is used to collect micro-motion sensing data inside the vehicle.
[0065] The display and audio facilities are used to play preset video or audio files under the control of the in-vehicle life monitoring control device.
[0066] refer to Figure 3 , which shows a schematic diagram of a vehicle arrangement according to an embodiment of the present invention. Figure 3 In the illustrated embodiment, the display is arranged on the back of the front seat of the vehicle, for example, to facilitate playing video files for children in the back seat. The millimeter wave radar is arranged on the ceiling above the back seat of the vehicle, and can detect the front and rear row areas in the vehicle. The camera can be installed on the front windshield, for example, the DMS camera can be reused, and the field of view is required to cover the back seat and cannot be blocked.
[0067] As an embodiment of the present invention, the vehicle further includes a temperature sensor and a light sensor, wherein the temperature sensor is used to obtain the temperature inside the vehicle, and the light sensor is used to obtain the light intensity inside the vehicle. Figure 3 The positions of the temperature sensor and the vehicle-mounted communication transceiver shown are merely examples, and the present invention does not require the positions of the temperature sensor and the vehicle-mounted communication transceiver.
[0068] On the other hand, the embodiment of the present invention provides a vehicle life monitoring control device, referring to Figure 4 , which shows a block diagram of a vehicle life monitoring control device according to an embodiment of the present invention. Figure 4 As shown, the apparatus may include modules 401 - 410 .
[0069] The image acquisition module 401 may be configured to acquire an image captured by a camera disposed in the vehicle in response to receiving a parking signal, wherein the camera includes an optical camera and an infrared camera, and the image captured by the camera includes an optical image and an infrared image.
[0070] The life form preliminary determination module 402 may be configured to determine whether a life form exists in the vehicle based on images captured by the camera.
[0071] The sensing data acquisition module 403 can be configured to acquire images captured by the camera and micro-motion sensing data collected by a millimeter-wave radar installed in the vehicle when it is determined that there is a living organism in the vehicle, wherein the micro-motion sensing data includes one or more of breathing frequency, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude.
[0072] The vital sign information generating module 404 may be configured to generate vital sign information based on the images and micro-motion sensing data collected by the camera.
[0073] The life form category acquisition module 405 may be configured to input the vital sign information into a life form category recognition model to acquire the category of the life form in the vehicle.
[0074] The distance acquisition module 406 may be configured to acquire the current distance between the vehicle key and the vehicle.
[0075] The vehicle reminder module 407 may be configured to control the vehicle lights and / or speakers to perform a reminder operation when the current distance is less than or equal to a preset distance threshold.
[0076] The remote reminder module 408 may be configured to send remote reminder information to the user client via wireless communication when the current distance is greater than a preset distance threshold.
[0077] The first category handling module 409 may be configured to control the opening of vehicle windows and door locks when the current distance is greater than a preset distance threshold and the category of the living being in the vehicle is the first category.
[0078] The second category handling module 410 can be configured to perform one or more of the following operations when the current distance is greater than a preset distance threshold and the category of the life form in the vehicle is the second category: transmitting the image captured by the camera to the user client in real time; controlling the start of the ventilation system in the vehicle; controlling the start of the display and audio facilities installed in the vehicle to play a pre-set video or audio file; establishing voice or video communication with the user client via the camera and microphone installed in the vehicle.
[0079] It should be noted that the functions implemented by each module in the in-vehicle life monitoring and control device proposed in the embodiment of the present invention correspond one-to-one to the various steps in the in-vehicle life monitoring and control method described above. For its specific implementation methods, examples and technical effects, please refer to the above description of the method.
[0080] In yet another aspect, an embodiment of the present invention provides a storage medium storing computer-readable instructions. When the instructions are executed by a processor, the in-vehicle life monitoring and control method described in any of the above embodiments is executed.
[0081] The in-vehicle life monitoring and control method, device, equipment, system and storage medium proposed in the embodiments of the present invention can automatically and intelligently switch the detection mode in various scenarios through the integration of multiple sensors. It overcomes the defects of the prior art that a single sensor cannot accurately identify the type of life in complex scenarios, thus causing limitations and inaccurate detection. In addition, multiple detection methods cooperate with each other to detect various life signs in a timely and accurate manner, detect different types of life, and issue adaptive control instructions to ensure that the possibility of missed detection or wrong detection is reduced, and more timely and effective prompt information is issued to avoid tragedies caused by life locked in the car.
[0082] For the purpose of illustration, the foregoing description of the embodiments of the present invention has been given, which is not exhaustive nor intended to limit the present invention to the disclosed exact form. It will be appreciated by those skilled in the art that various changes can be made without departing from the scope of the present invention, and that the elements therein can be replaced with equivalents. In addition, without departing from the basic scope of the present invention, many modifications can be made so that specific situations or materials are adapted to the teachings of the present invention. Therefore, the present invention is not intended to be limited to the specific embodiments disclosed as the best mode for realizing the present invention, and the present invention will include all embodiments falling within the scope of the appended claims.
Claims
1. A method for monitoring and controlling a living being in a vehicle, characterized in that: The method comprises: In response to receiving a parking signal, acquiring an image captured by a camera disposed in the vehicle, wherein the camera comprises an optical camera and an infrared camera, and the image captured by the camera comprises an optical image and an infrared image; Determine whether there is a living being in the vehicle based on the images collected by the camera; In the case of determining that there is a living body in the vehicle, acquiring an image captured by the camera and micro-motion sensing data collected by a millimeter-wave radar disposed in the vehicle, wherein the micro-motion sensing data includes one or more of a respiratory frequency, a chest cavity fluctuation amplitude, and an abdominal cavity fluctuation amplitude; generating vital sign information according to the image collected by the camera and the micro-motion sensing data; Inputting the vital sign information into a life form category recognition model to obtain the category of the life form in the vehicle; Get the current distance between the car key and the vehicle; When the current distance is less than or equal to a preset distance threshold, controlling the vehicle lights and / or speakers to perform a reminder operation; When the current distance is greater than the preset distance threshold, a remote reminder message is sent to the user client via wireless communication, and, When the category of the living being in the vehicle is the first category, controlling the opening of the vehicle windows and the door locks; When the category of the in-vehicle life form is the second category, one or more of the following steps are performed: Transmitting the image captured by the camera to the user client in real time; Control to open the ventilation system in the car; Control the display and audio equipment installed in the car to play the pre-set video or audio files; Voice or video communication is established with the user client via a camera and a microphone arranged in the car.
2. The method according to claim 1, characterized in that In the case where it is determined that there is a living being in the vehicle, acquiring the image captured by the camera and the micro-motion sensing data collected by the millimeter wave radar disposed in the vehicle includes: Acquiring the temperature inside the vehicle through a temperature sensor disposed inside the vehicle, and comparing the temperature inside the vehicle with a preset temperature threshold; Acquiring the light intensity inside the vehicle through a photometric sensor disposed inside the vehicle, and comparing the light intensity inside the vehicle with a preset first light intensity threshold and a second light intensity threshold, wherein the second light intensity threshold is higher than the first light intensity threshold; When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is lower than or equal to a first light intensity threshold, acquiring an infrared image collected by the infrared camera and micro-motion sensing data collected by the millimeter wave radar; When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than the first light intensity threshold and lower than the second light intensity threshold, acquiring an optical image collected by the optical camera and micro-motion sensing data collected by the millimeter wave radar; When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than or equal to the second light intensity threshold, acquiring micro-motion sensing data collected by the millimeter wave radar; When the temperature inside the vehicle is higher than or equal to the temperature threshold, an optical image collected by the optical camera and micro-motion sensing data collected by the millimeter-wave radar are acquired.
3. The method according to claim 1, characterized in that: The method further comprises: After determining that there is a living being in the vehicle, in response to receiving a vehicle locking signal, a reminder operation is immediately performed.
4. The method according to claim 1, characterized in that: Based on the images collected by the camera, determining whether there is a living being in the car includes: Obtaining moving object information according to changes between different frames of the optical image and / or the infrared image; Obtaining temperature information according to the infrared image; Determine whether there is a living being in the vehicle according to the moving object information and the temperature sensing information.
5. The method according to claim 1, characterized in that Generating vital sign information according to the image collected by the camera and the micro-motion sensing data includes: The vital sign database is called, the image collected by the camera and the micro-motion sensing data are compared with the data in the vital sign database, and the vital sign information is generated according to the comparison result.
6. The method according to claim 1, characterized in that The first category of in-vehicle life forms includes adults or elderly people; the second category of in-vehicle life forms includes disabled people, children, and pets.
7. A device for monitoring and controlling a life form in a vehicle, characterized in that: The device comprises: an image acquisition module, configured to acquire an image acquired by a camera disposed in the vehicle in response to receiving a parking signal, wherein the camera includes an optical camera and an infrared camera, and the image acquired by the camera includes an optical image and an infrared image; A life form preliminary determination module configured to determine whether there is a life form in the vehicle based on an image captured by the camera; a sensing data acquisition module, configured to acquire, when determining that there is a living being in the vehicle, images acquired by the camera and micro-motion sensing data acquired by a millimeter-wave radar disposed in the vehicle, wherein the micro-motion sensing data includes one or more of a respiratory frequency, a chest cavity fluctuation amplitude, and an abdominal cavity fluctuation amplitude; A vital sign information generating module configured to generate vital sign information based on the image collected by the camera and the micro-motion sensing data; A life form category acquisition module configured to input the life sign information into a life form category recognition model to acquire the category of the life form in the vehicle; A distance acquisition module configured to acquire a current distance between the vehicle key and the vehicle; A self-vehicle reminder module, configured to control the vehicle lights and / or speakers to perform a reminder operation when the current distance is less than or equal to a preset distance threshold; A remote reminder module, configured to send a remote reminder message to a user client via wireless communication when the current distance is greater than the preset distance threshold; A first category handling module, configured to control opening of vehicle windows and door locks when the current distance is greater than the preset distance threshold and the category of the living being in the vehicle is the first category; The second category handling module is configured to perform one or more of the following operations when the current distance is greater than the preset distance threshold and the category of the living body in the vehicle is the second category: Transmitting the image captured by the camera to the user client in real time; Control to open the ventilation system in the car; Control the display and audio equipment installed in the car to play the pre-set video or audio files; Voice or video communication is established with the user client via a camera and a microphone arranged in the car.
8. A vehicle life monitoring and control device, characterized in that: The device is arranged on a vehicle, and comprises a memory and a processor. A computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.
9. A vehicle life monitoring and control system, characterized in that: The system comprises the in-vehicle life monitoring control device according to claim 8 and a client, The client is used to: Receiving remote reminder information sent by the in-vehicle life monitoring and control device, and displaying the remote reminder information; receiving images transmitted in real time by the in-vehicle life monitoring and control device, and displaying the images; Establish voice or video communication with the in-vehicle life monitoring and control device.
10. A vehicle, characterized in that: The vehicle comprises the in-vehicle life monitoring and control device according to claim 8, as well as a camera, a millimeter-wave radar, a display, and an audio facility, The camera includes an optical camera and an infrared camera, wherein the optical camera is used to collect optical images inside the vehicle, and the infrared camera is used to collect infrared images inside the vehicle; The millimeter wave radar is used to collect micro-motion sensing data in the vehicle; The display and audio facilities are used to play preset video or audio files under the control of the in-vehicle life monitoring control device.
11. The vehicle according to claim 10, characterized in that The display is arranged on the back side of the front seats of the vehicle.
12. The vehicle according to claim 10, characterized in that The millimeter wave radar is arranged on the ceiling above the rear row of the vehicle.
13. The vehicle according to claim 10, characterized in that The vehicle further comprises a temperature sensor and a photometric sensor, wherein the temperature sensor is used to obtain the temperature inside the vehicle, and the photometric sensor is used to obtain the light intensity inside the vehicle.
14. A storage medium storing computer-readable instructions, wherein when the instructions are executed by a processor, the method according to any one of claims 1 to 6 is executed.
Citation Information
Patent Citations
Warning system for forgotten living body in vehicle and warning method
CN108216104A
Vehicle control method and device, equipment and storage medium
CN113859171A
In-vehicle life detection method and apparatus, device, and storage medium
WO2022166649A1
Cited By
Remote control method, device and equipment based on in-vehicle living object detection and storage medium
CN119928763A
