In-vehicle life monitoring and control method, device, equipment, system, and vehicle

Abstract

The present invention provides a method, device, equipment, system, vehicle, and storage medium for monitoring and controlling life forms within a vehicle. The method includes: in response to receiving a stop signal, acquiring an image captured by a camera installed within the vehicle; determining the presence of a life form within the vehicle based on the image; acquiring the image captured by the camera and micro-motion sensing data collected by a millimeter-wave radar installed within the vehicle; generating vital sign information based on the image and micro-motion sensing data; obtaining the category of the life form within the vehicle using a life form category recognition model; obtaining the distance between the vehicle key and the vehicle; executing a self-vehicle reminder operation when the distance is less than or equal to a distance threshold; sending a remote reminder message to a user client when the distance is greater than the distance threshold, and, if the life form is of the first category, controlling the opening of the vehicle windows and door locks; and executing one or more vehicle control operations when the life form is of the second category. By integrating multiple sensors, detection accuracy is improved in multiple scenarios.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of vehicle technology, and more specifically, to a method, device, equipment, system, vehicle, and storage medium for monitoring and controlling a living being in a vehicle. Background Art

[0002] In recent years, social tragedies involving personal injuries caused by locking children in cars have occurred frequently. Parents mistakenly leave their children in cars due to carelessness and other reasons. The temperature will rise quickly in a closed environment (especially in direct sunlight in the summer), so children in the car are very likely to suffer from high-temperature suffocation, causing tragedy.

[0003] Currently, the vehicles on the market are usually equipped with only a single sensor to detect life forms inside the vehicle. The application scenarios are limited, so in complex scenarios, there is a high probability of missing detection of life forms inside the vehicle, which has certain limitations and detection inaccuracies. Summary of the Invention

[0004] In order to solve the above-mentioned problems in the prior art, in a first aspect, an embodiment of the present invention provides a method for monitoring and controlling a living being in a vehicle, the method comprising: in response to receiving a stop signal, acquiring an image captured by a camera arranged in the vehicle, wherein the camera comprises an optical camera and an infrared camera, and the image captured by the camera comprises an optical image and an infrared image; determining whether there is a living being in the vehicle based on the image captured by the camera; in the case of determining that there is a living being in the vehicle, acquiring the image captured by the camera and micro-motion sensing data captured by a millimeter-wave radar arranged in the vehicle, wherein the micro-motion sensing data comprises one or more of respiratory frequency, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude; generating vital sign information based on the image captured by the camera and the micro-motion sensing data; and inputting the vital sign information into a living being category. In the recognition model, the category of the life form in the car is obtained; the current distance between the car key and the vehicle is obtained; when the current distance is less than or equal to a preset distance threshold, the car lights and / or speakers are controlled to perform a reminder operation; when the current distance is greater than the preset distance threshold, a remote reminder message is sent to the user client via wireless communication, and when the category of the life form in the car is the first category, the windows and door locks are controlled to be opened; when the category of the life form in the car is the second category, one or more of the following steps are performed: the image captured by the camera is transmitted to the user client in real time; the ventilation system in the car is controlled to be turned on; the display and audio facilities installed in the car are controlled to be turned on to play a preset video or audio file; and voice or video communication is established with the user client via the camera and microphone installed in the car.

[0005] In some embodiments, when it is determined that there is a living body in the car, obtaining the image captured by the camera and the micro-motion sensing data collected by the millimeter-wave radar set in the car includes: obtaining the temperature in the car through a temperature sensor set in the car, and comparing the temperature in the car with a preset temperature threshold; obtaining the light intensity in the car through a photometry sensor set in the car, and comparing the light intensity in the car with a preset first light intensity threshold and a second light intensity threshold, wherein the second light intensity threshold is higher than the first light intensity threshold; when the temperature in the car is lower than the temperature threshold and the light intensity in the car is lower than or equal to the first light intensity threshold, obtaining the image captured by the infrared camera and the micro-motion sensing data collected by the millimeter-wave radar set in the car. The infrared image collected by the optical camera and the micro-motion sensing data collected by the millimeter-wave radar; when the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than the first light intensity threshold and lower than the second light intensity threshold, the optical image collected by the optical camera and the micro-motion sensing data collected by the millimeter-wave radar are obtained; when the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than or equal to the second light intensity threshold, the micro-motion sensing data collected by the millimeter-wave radar are obtained; when the temperature inside the vehicle is higher than or equal to the temperature threshold, the optical image collected by the optical camera and the micro-motion sensing data collected by the millimeter-wave radar are obtained.

[0006] In some embodiments, the method further includes: after determining that a living being exists in the vehicle, in response to receiving a vehicle locking signal, immediately performing a reminder operation.

[0007] In some embodiments, determining whether there is a living organism in the vehicle based on images captured by a camera includes: obtaining moving object information based on changes between different frames of the optical image and / or the infrared image; obtaining temperature information based on the infrared image; and determining whether there is a living organism in the vehicle based on the moving object information and the temperature information.

[0008] In some embodiments, generating vital signs information based on the image captured by the camera and the micro-motion sensing data includes: calling a vital signs database, comparing the image captured by the camera and the micro-motion sensing data with the data in the vital signs database, and generating vital signs information based on the comparison results.

[0009] In some embodiments, the first category of in-vehicle life forms includes adults or elderly people; the second category of in-vehicle life forms includes disabled people, children, and pets.

[0010] In a second aspect, an embodiment of the present invention proposes a device for monitoring and controlling a life form in a vehicle, the device comprising: an image acquisition module configured to acquire an image captured by a camera disposed in the vehicle in response to receiving a parking signal, wherein the camera comprises an optical camera and an infrared camera, and the image captured by the camera comprises an optical image and an infrared image; a preliminary life form determination module configured to determine whether a life form exists in the vehicle based on the image captured by the camera; a sensing data acquisition module configured to acquire, when determining that a life form exists in the vehicle, the image captured by the camera and micro-motion sensing data captured by a millimeter-wave radar disposed in the vehicle, wherein the micro-motion sensing data comprises one or more of respiratory frequency, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude; a vital sign information generation module configured to generate vital sign information based on the image captured by the camera and the micro-motion sensing data; a life form category acquisition module configured to input the vital sign information into a life form category recognition model to obtain the category of the life form in the vehicle; a distance An acquisition module is configured to obtain the current distance between the car key and the vehicle; a self-vehicle reminder module is configured to control the car lights and / or speakers to perform a reminder operation when the current distance is less than or equal to a preset distance threshold; a remote reminder module is configured to send a remote reminder message to the user client via wireless communication when the current distance is greater than the preset distance threshold; a first category handling module is configured to control the opening of the car windows and door locks when the current distance is greater than the preset distance threshold and the category of the life form in the car is the first category; a second category handling module is configured to perform one or more of the following operations when the current distance is greater than the preset distance threshold and the category of the life form in the car is the second category: transmitting the image captured by the camera to the user client in real time; controlling the opening of the ventilation system in the car; controlling the opening of the display and audio facilities set in the car to play a pre-set video or audio file; and establishing voice or video communication with the user client via the camera and microphone set in the car.

[0011] In a third aspect, an embodiment of the present invention proposes an in-vehicle life monitoring and control device, which is arranged on a vehicle. The device includes a memory and a processor, and a computer program is stored on the memory. When the computer program is executed by the processor, the method described in any of the above embodiments is implemented.

[0012] In a fourth aspect, embodiments of the present invention provide an in-vehicle life monitoring and control system, comprising the in-vehicle life monitoring and control device described in the above embodiments and a client. The client is configured to: receive and display remote reminder messages sent by the in-vehicle life monitoring and control device; receive and display real-time images transmitted by the in-vehicle life monitoring and control device; and establish voice or video communication with the in-vehicle life monitoring and control device.

[0013] In a fifth aspect, embodiments of the present invention provide a vehicle comprising the in-vehicle life monitoring and control device described in the above embodiments, as well as a camera, a millimeter-wave radar, a display, and audio equipment. The camera comprises an optical camera and an infrared camera; the optical camera is used to capture optical images of the vehicle interior, while the infrared camera is used to capture infrared images of the vehicle interior; the millimeter-wave radar is used to collect micro-motion sensing data within the vehicle; and the display and audio equipment are used to play pre-set video or audio files under the control of the in-vehicle life monitoring and control device.

[0014] In some embodiments, the display is disposed on the back of a front seat of the vehicle.

[0015] In some embodiments, the millimeter-wave radar is disposed on a ceiling above a rear row of the vehicle.

[0016] In some embodiments, the vehicle further includes a temperature sensor and a light sensor, wherein the temperature sensor is used to obtain the temperature inside the vehicle, and the light sensor is used to obtain the light intensity inside the vehicle.

[0017] In a sixth aspect, an embodiment of the present invention provides a storage medium storing computer-readable instructions, which, when executed by a processor, executes the method according to any of the above embodiments.

[0018] The in-vehicle life monitoring and control methods, devices, equipment, systems, and storage media proposed in the embodiments of the present invention utilize multiple sensors to automatically and intelligently switch detection modes in diverse scenarios. This overcomes the limitations and inaccuracies inherent in existing technologies, where a single sensor cannot accurately identify the type of life in complex scenarios. Furthermore, the coordinated use of multiple detection methods enables timely and accurate detection of various vital signs, identifying different life types and issuing appropriate control commands. This reduces the likelihood of missed or incorrect detections, provides more timely and effective warnings, and prevents tragedies caused by life forms locked inside vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present invention are shown by way of example and not limitation, in which:

[0020] Figure 1 A flow chart of a method for controlling in-vehicle life monitoring according to an embodiment of the present invention is shown;

[0021] Figure 2 A schematic diagram showing an application scenario of an in-vehicle life monitoring and control device according to an embodiment of the present invention;

[0022] Figure 3 shows a schematic diagram of a vehicle arrangement according to an embodiment of the present invention;

[0023] Figure 4 A block diagram of an in-vehicle life monitoring control device according to an embodiment of the present invention is shown.

[0024] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0025] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way.

[0026] In one aspect, an embodiment of the present invention provides a method for monitoring and controlling a life form in a vehicle, referring to Figure 1 , which shows a flow chart of a method 100 for controlling in-vehicle life monitoring according to an embodiment of the present invention. The method 100 may include steps S101-S110.

[0027] In step S101, in response to receiving a stop signal, an image captured by a camera located within the vehicle is acquired. The camera may include an optical camera and an infrared camera, and the image captured by the camera may include an optical image and an infrared image. As an example, the stop signal may include a gear change, such as a shift from reverse (R) to park (P), or a shift from drive (D) to P. As another example, the stop signal may also be derived by considering vehicle speed, such as when the vehicle's speed drops to zero.

[0028] In step S102, it is determined whether there is a living being in the vehicle based on the image captured by the camera.

[0029] Optionally, the presence of a living being may be determined when a living being is detected in an image captured by any one of the optical camera and the infrared camera, thereby avoiding missed detection.

[0030] As one embodiment of the present invention, step S102 may include: obtaining moving object information based on changes between different frames of optical and / or infrared images; obtaining temperature information based on the infrared images; and determining the presence of a living being within the vehicle based on the moving object information and temperature information. Using an infrared camera in addition to an optical camera for life detection can cover a variety of environmental conditions and scenarios.

[0031] Optionally, the presence of a living being can be determined when a moving object is detected in an optical image, a moving object is detected in an infrared image, or an object with a corresponding temperature is detected in an infrared image. Alternatively, the presence of a living being can be determined when any two of the above three conditions are met. Alternatively, the presence of a living being can be determined when all three of the above conditions are met, and in particular, the presence of a living being can be determined when vital signs of temperature and movement are detected. In practical applications, the detection conditions can be flexibly configured according to the detection requirements.

[0032] Through the above steps, a preliminary judgment on whether there is a living organism in the car is achieved.

[0033] As one embodiment of the present invention, the method optionally further includes: after determining the presence of a living being in the vehicle, in response to receiving a vehicle lock signal, immediately executing a reminder operation. Thus, a reminder can be issued to the user the moment the user locks the vehicle, prompting the user to promptly pay attention to the situation inside the vehicle. This reminder operation can include in-vehicle reminders and / or remote reminders. In-vehicle reminders include flashing lights and honking the horn. Remote reminders can, for example, be sent a message to the user's client (e.g., a mobile phone).

[0034] In step S103, if a living being is determined to be present within the vehicle, images captured by the camera and micro-motion data collected by a millimeter-wave radar installed within the vehicle are acquired. The micro-motion data includes one or more of respiratory rate, chest and abdominal fluctuations. Millimeter waves offer significant advantages in identifying moving objects. Compared to cameras, millimeter-wave radars can process nearly static echoes, such as chest and abdominal fluctuations.

[0035] As an embodiment of the present invention, appropriate detection elements can be selected based on the environmental conditions at the time and the different sensitivities and accuracies of each detection element in different scenarios. Table 1 below shows the sensing performance of different sensors in different application environments.

[0036] Table 1. Sensing performance of different sensors in different application environments

[0037]

[0038]

[0039] Based on the above considerations, step S103 may include:

[0040] The temperature inside the vehicle is obtained by a temperature sensor provided in the vehicle, and the temperature inside the vehicle is compared with a preset temperature threshold;

[0041] Obtaining the light intensity inside the vehicle through a photometric sensor disposed inside the vehicle, and comparing the light intensity inside the vehicle with a preset first light intensity threshold and a second light intensity threshold, wherein the second light intensity threshold is higher than the first light intensity threshold;

[0042] When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is lower than or equal to the first light intensity threshold, indicating that the temperature inside the vehicle is normal and the vehicle is in a relatively dim light environment, an infrared image captured by the infrared camera and micro-motion sensing data collected by the millimeter-wave radar are acquired;

[0043] When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than the first light intensity threshold and lower than the second light intensity threshold, indicating that the temperature inside the vehicle is normal and the vehicle interior is in a normal brightness environment, obtaining an optical image captured by the optical camera and micro-motion sensing data collected by the millimeter-wave radar;

[0044] When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than or equal to the second light intensity threshold, indicating that the temperature inside the vehicle is normal and the vehicle is in an environment with excessive light intensity, obtaining micro-motion sensing data collected by the millimeter-wave radar;

[0045] When the temperature inside the vehicle is higher than or equal to the temperature threshold, that is, it indicates that the vehicle is currently in a high temperature environment, an optical image captured by the optical camera and micro-motion sensing data captured by the millimeter-wave radar are acquired.

[0046] In the case where the presence of a living being has been confirmed, the above steps perform a secondary collection of vital signs by selecting a combination of detection elements (ie, detection methods) to improve accuracy, thereby achieving accurate collection of vital signs.

[0047] In step S104 , vital sign information is generated based on the image collected by the camera and the micro-motion sensing data.

[0048] As one embodiment of the present invention, step S104 may include: accessing a vital signs database, comparing the images and micro-motion sensing data captured by the camera with the data in the vital signs database, and generating vital signs information based on the comparison results. The vital signs database may include, for example, respiratory rate range, chest and abdominal fluctuation range, body length range, proportions between various body parts, facial features, behavioral and posture features, and the like.

[0049] In step S105, the vital sign information is input into a life form classification recognition model to obtain the category of the life form in the vehicle. For example, the life form classification recognition model can be obtained by deep learning a large amount of sample data.

[0050] In step S106 , the current distance between the vehicle key and the vehicle is obtained.

[0051] In step S107, if the current distance is less than or equal to a preset distance threshold, the vehicle lights and / or speaker are controlled to perform a reminder operation, such as controlling the vehicle lights to flash and / or honking. In this case, the user is close to the vehicle and can perceive the flashing lights and honking horn. This can effectively serve as a reminder, allowing the user to return to the vehicle in a timely manner.

[0052] In step S108, if the current distance is greater than a preset distance threshold, a remote reminder message is sent to the user client via wireless communication. At this point, the user is already far away from the vehicle and cannot perceive the flashing lights and horn. Therefore, the reminder is sent to the user's handheld client. The user client can be, for example, a smartphone or tablet. The remote reminder message can be a text message or a message in an application (app) installed on the client.

[0053] In step S109, if the category of the living being in the vehicle is the first category, the windows and door locks are controlled to be opened. Typically, the first category may be people who are able to take care of themselves, such as adults or the elderly. For example, when the windows and door locks are opened, the adult or elderly person can exit the vehicle on their own.

[0054] In step S110, when the category of the life form in the vehicle is the second category, one or more of the following vehicle control operations are performed: the image captured by the camera is transmitted to the user client in real time, so that the dynamics of the life form can be observed at any time; the ventilation system in the vehicle is controlled to be turned on to maintain air circulation in the vehicle; the display and audio facilities installed in the vehicle are controlled to be turned on to play pre-set video or audio files; voice or video communication is established with the user client via the camera and microphone installed in the vehicle, so as to communicate and interact with the life form in the vehicle, and the dynamics of the life form can be observed at any time to make timely responses.

[0055] The second category of life forms in the vehicle can be people or other life forms that are unable to take care of themselves, such as disabled people, children, and pets. If the life form is in the second category, leaving the vehicle on their own or opening the car window may pose a safety hazard. Through the above method, for example, when the life form in the vehicle is a child, turning on the camera can monitor the child's movements at any time, while turning on the ventilation system to circulate the air in the vehicle, turning on the display and audio to play animations to soothe the child, and waiting for the user to rescue the child as soon as possible. For another example, when the life form is a pet, turning on the ventilation system and turning on the camera to monitor the pet's movements at any time, speaking through the microphone to soothe the pet, and waiting for the owner to rescue it.

[0056] Through the above implementation, according to different types of life forms, corresponding response mechanisms are adopted, and different control signals are sent out by comprehensively considering the distance between the car key and the car and different types of life forms.

[0057] The in-vehicle life monitoring and control method proposed in the embodiments of the present invention can automatically and intelligently switch detection modes in various scenarios through the integration of multiple sensors. The multiple detection modes cooperate with each other to timely and accurately detect various vital signs, detect different types of life forms, and issue adaptive control instructions to ensure that the possibility of missed detection or false detection is reduced, and more timely and effective prompt information is issued to avoid tragedies caused by life forms locked in the car.

[0058] On the other hand, an embodiment of the present invention proposes an in-vehicle life monitoring and control device, which is set on a vehicle. The device includes a memory and a processor. A computer program is stored in the memory. When the computer program is executed by the processor, the in-vehicle life monitoring and control method described in any of the above embodiments is implemented.

[0059] In another aspect, an embodiment of the present invention provides an in-vehicle life monitoring and control system, which includes the in-vehicle life monitoring and control device described in any of the above embodiments, and a client.

[0060] The client can be used to: receive remote reminder information sent by the in-vehicle life monitoring and control device and display the remote reminder information; receive real-time images transmitted by the in-vehicle life monitoring and control device and display the images; establish voice or video communication with the in-vehicle life monitoring and control device.

[0061] refer to Figure 2, which shows a schematic diagram of an application scenario for an in-vehicle life monitoring and control device according to an embodiment of the present invention. The in-vehicle life monitoring and control device according to an embodiment of the present invention is communicatively connected to a millimeter-wave radar and a camera, and communicates with a temperature sensor, a display, an on-board communication transceiver, and a body control module (BCM) via a gateway. The BCM further controls speakers, lights, doors, windows, air conditioning (ventilation system), and other components. The in-vehicle life monitoring and control device communicates with the user's client via the gateway and the on-board communication transceiver.

[0062] In yet another aspect, an embodiment of the present invention provides a vehicle comprising the in-vehicle life monitoring and control device described in any of the above embodiments, as well as a camera, a millimeter-wave radar, a display, and audio facilities.

[0063] The camera includes an optical camera and an infrared camera. The optical camera is used to collect optical images inside the car, and the infrared camera is used to collect infrared images inside the car.

[0064] Millimeter-wave radar is used to collect micro-motion sensing data inside the vehicle.

[0065] The display and audio facilities are used to play pre-set video or audio files under the control of the in-vehicle life monitoring control device.

[0066] refer to Figure 3 , which shows a schematic diagram of a vehicle arrangement according to an embodiment of the present invention. Figure 3 In the illustrated embodiment, a display is mounted on the back of the vehicle's front seats, for example, to facilitate playing video files for children in the back seat. A millimeter-wave radar is mounted on the ceiling above the rear seats, capable of detecting the front and rear areas of the vehicle. A camera can be mounted on the front windshield, for example, reusing a DMS camera, with a clear field of view extending to the rear seats.

[0067] As an embodiment of the present invention, the vehicle further includes a temperature sensor and a light sensor, wherein the temperature sensor is used to obtain the temperature inside the vehicle, and the light sensor is used to obtain the light intensity inside the vehicle. Figure 3 The positions of the temperature sensor and the vehicle-mounted communication transceiver shown are merely examples, and the present invention does not require the positions of the temperature sensor and the vehicle-mounted communication transceiver.

[0068] On the other hand, the embodiment of the present invention provides a vehicle life monitoring control device, referring to Figure 4 , which shows a block diagram of a vehicle life monitoring control device according to an embodiment of the present invention. Figure 4 As shown, the apparatus may include modules 401 - 410 .

[0069] The image acquisition module 401 may be configured to acquire an image captured by a camera disposed in the vehicle in response to receiving a parking signal, wherein the camera includes an optical camera and an infrared camera, and the image captured by the camera includes an optical image and an infrared image.

[0070] The life form preliminary determination module 402 may be configured to determine whether a life form exists in the vehicle based on images captured by the camera.

[0071] The sensing data acquisition module 403 can be configured to obtain images captured by the camera and micro-motion sensing data collected by the millimeter-wave radar installed in the vehicle when it is determined that there is a living organism in the vehicle, wherein the micro-motion sensing data includes one or more of respiratory frequency, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude.

[0072] The vital sign information generating module 404 may be configured to generate vital sign information based on images and micro-motion sensing data collected by the camera.

[0073] The life form category acquisition module 405 may be configured to input the vital sign information into a life form category recognition model to acquire the category of the life form in the vehicle.

[0074] The distance acquisition module 406 may be configured to acquire a current distance between the vehicle key and the vehicle.

[0075] The vehicle reminder module 407 may be configured to control the vehicle lights and / or speakers to perform a reminder operation when the current distance is less than or equal to a preset distance threshold.

[0076] The remote reminder module 408 may be configured to send a remote reminder message to the user client via wireless communication when the current distance is greater than a preset distance threshold.

[0077] The first category handling module 409 may be configured to control the opening of vehicle windows and door locks when the current distance is greater than a preset distance threshold and the category of the living being in the vehicle is the first category.

[0078] The second category handling module 410 can be configured to perform one or more of the following operations when the current distance is greater than a preset distance threshold and the category of the living body in the vehicle is the second category: transmitting the image captured by the camera to the user client in real time; controlling the opening of the ventilation system in the vehicle; controlling the opening of the display and audio facilities installed in the vehicle to play a pre-set video or audio file; establishing voice or video communication with the user client via the camera and microphone installed in the vehicle.

[0079] It should be noted that the functions implemented by each module in the in-vehicle life monitoring and control device proposed in the embodiment of the present invention correspond one-to-one to the various steps in the in-vehicle life monitoring and control method described above. For its specific implementation methods, examples and technical effects, please refer to the above description of the method.

[0080] In yet another aspect, an embodiment of the present invention provides a storage medium storing computer-readable instructions. When the instructions are executed by a processor, the in-vehicle life monitoring and control method described in any of the above embodiments is executed.

[0081] The in-vehicle life monitoring and control methods, devices, equipment, systems, and storage media proposed in the embodiments of the present invention utilize multiple sensors to automatically and intelligently switch detection modes in diverse scenarios. This overcomes the limitations and inaccuracies inherent in existing technologies, where a single sensor cannot accurately identify the type of life in complex scenarios. Furthermore, the coordinated use of multiple detection methods enables timely and accurate detection of various vital signs, identifying different life types and issuing appropriate control commands. This reduces the likelihood of missed or incorrect detections, provides more timely and effective warnings, and prevents tragedies caused by life forms locked inside vehicles.

[0082] For illustrative purposes, the foregoing description of the embodiments of the present invention has been given, which is not exhaustive nor intended to limit the present invention to disclosed exact forms. It will be appreciated by those skilled in the art that various changes may be made without departing from the scope of the present invention, and that elements therein may be replaced with equivalents. In addition, without departing from the basic scope of the present invention, many modifications may be made so that specific situations or materials are adapted to the teachings of the present invention. Therefore, the present invention is not intended to be limited to the specific embodiments disclosed as the best mode for realizing the present invention, and the present invention will include all embodiments within the scope of the appended claims.

Claims

1. A method for monitoring and controlling a living being in a vehicle, characterized in that: The method comprises: In response to receiving a stop signal, acquiring an image captured by a camera disposed in the vehicle, wherein the camera includes an optical camera and an infrared camera, and the image captured by the camera includes an optical image and an infrared image; Determine whether there is a living being in the car based on images collected by the camera; When it is determined that there is a living being in the vehicle, acquiring images captured by the camera and micro-motion sensing data captured by a millimeter-wave radar disposed in the vehicle, wherein the micro-motion sensing data includes one or more of respiratory rate, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude; generating vital sign information based on the image captured by the camera and the micro-motion sensing data; Inputting the vital sign information into a life form category recognition model to obtain the category of the life form in the vehicle; Get the current distance between the car key and the vehicle; When the current distance is less than or equal to a preset distance threshold, controlling the vehicle lights and / or speakers to perform a reminder operation; When the current distance is greater than the preset distance threshold, a remote reminder message is sent to the user client via wireless communication, and, When the category of the living being in the vehicle is the first category, controlling the opening of the vehicle windows and the locking of the vehicle doors; If the category of the in-vehicle life form is the second category, perform one or more of the following steps: Transmitting the image captured by the camera to the user client in real time; Control to open the vehicle ventilation system; Control the display and audio equipment installed in the car to play pre-set video or audio files; Voice or video communication is established with the user client via a camera and a microphone arranged in the car.

2. The method according to claim 1, characterized in that When it is determined that there is a living being in the vehicle, obtaining the image captured by the camera and the micro-motion sensing data collected by the millimeter-wave radar disposed in the vehicle includes: Acquiring the vehicle interior temperature through a temperature sensor disposed in the vehicle, and comparing the vehicle interior temperature with a preset temperature threshold; Acquiring the light intensity inside the vehicle through a photometric sensor disposed inside the vehicle, and comparing the light intensity inside the vehicle with a preset first light intensity threshold and a second light intensity threshold, wherein the second light intensity threshold is higher than the first light intensity threshold; When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is lower than or equal to a first light intensity threshold, acquiring an infrared image captured by the infrared camera and micro-motion sensing data captured by the millimeter-wave radar; When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than the first light intensity threshold and lower than the second light intensity threshold, acquiring an optical image captured by the optical camera and micro-motion sensing data captured by the millimeter-wave radar; When the temperature inside the vehicle is lower than the temperature threshold and the light intensity inside the vehicle is higher than or equal to the second light intensity threshold, acquiring micro-motion sensing data collected by the millimeter-wave radar; When the temperature inside the vehicle is higher than or equal to the temperature threshold, an optical image captured by the optical camera and micro-motion sensing data captured by the millimeter-wave radar are acquired.

3. The method according to claim 1, characterized in that The method further comprises: After determining that there is a living being in the vehicle, a reminder operation is immediately performed in response to receiving a vehicle locking signal.

4. The method according to claim 1, wherein Determining whether there is a living being in the vehicle based on the image captured by the camera includes: obtaining moving object information according to changes between different frames of the optical image and / or the infrared image; obtaining temperature information according to the infrared image; Determine whether there is a living being in the vehicle based on the moving object information and the temperature sensing information.

5. The method according to claim 1, wherein Generating vital sign information according to the image captured by the camera and the micro-motion sensing data includes: The vital signs database is called, the image collected by the camera and the micro-motion sensing data are compared with the data in the vital signs database, and vital signs information is generated according to the comparison result.

6. The method according to claim 1, characterized in that The first category of living things in the car includes adults or elderly people; the second category of living things in the car includes disabled people, children, and pets.

7. A vehicle-mounted life monitoring and control device, characterized in that: The device comprises: an image acquisition module configured to acquire an image captured by a camera disposed in the vehicle in response to receiving a stop signal, wherein the camera includes an optical camera and an infrared camera, and the image captured by the camera includes an optical image and an infrared image; a life form preliminary determination module configured to determine whether a life form exists in the vehicle based on images captured by the camera; a sensing data acquisition module configured to acquire, upon determining the presence of a living being in the vehicle, images captured by the camera and micro-motion sensing data captured by a millimeter-wave radar disposed in the vehicle, wherein the micro-motion sensing data includes one or more of respiratory rate, chest cavity fluctuation amplitude, and abdominal cavity fluctuation amplitude; a vital sign information generating module configured to generate vital sign information based on the image captured by the camera and the micro-motion sensing data; a life form category acquisition module configured to input the vital sign information into a life form category recognition model to acquire the category of the life form in the vehicle; A distance acquisition module configured to acquire a current distance between the vehicle key and the vehicle; A vehicle reminder module configured to control the vehicle lights and / or speakers to perform a reminder operation when the current distance is less than or equal to a preset distance threshold; A remote reminder module configured to send a remote reminder message to the user client via wireless communication when the current distance is greater than the preset distance threshold; a first category handling module configured to control opening of the vehicle windows and door locks when the current distance is greater than the preset distance threshold and the category of the living being in the vehicle is the first category; The second category handling module is configured to perform one or more of the following operations when the current distance is greater than the preset distance threshold and the category of the living being in the vehicle is the second category: Transmitting the image captured by the camera to the user client in real time; Control to open the vehicle ventilation system; Control the display and audio equipment installed in the car to play pre-set video or audio files; Voice or video communication is established with the user client via a camera and a microphone arranged in the car.

8. A vehicle-mounted life monitoring and control device, characterized in that: The device is arranged on a vehicle, and includes a memory and a processor. A computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.

9. A vehicle-based life monitoring and control system, characterized in that: The system comprises the in-vehicle life monitoring and control device according to claim 8 and a client, The client is used to: receiving remote reminder information sent by the in-vehicle life monitoring and control device, and displaying the remote reminder information; receiving images transmitted in real time by the in-vehicle life monitoring and control device, and displaying the images; Establish voice or video communication with the in-vehicle life monitoring and control device.

10. A vehicle, characterized in that: The vehicle includes the in-vehicle life monitoring and control device according to claim 8, as well as a camera, a millimeter-wave radar, a display, and an audio facility. The camera includes an optical camera and an infrared camera, the optical camera is used to collect optical images inside the vehicle, and the infrared camera is used to collect infrared images inside the vehicle; The millimeter wave radar is used to collect micro-motion sensing data inside the vehicle; The display and audio facilities are used to play preset video or audio files under the control of the in-vehicle life monitoring control device.

11. The vehicle according to claim 10, characterized in that The display is arranged on the back side of the front seat of the vehicle.

12. The vehicle according to claim 10, characterized in that The millimeter-wave radar is arranged on the ceiling above the rear row of the vehicle.

13. The vehicle according to claim 10, characterized in that The vehicle further includes a temperature sensor and a light sensor, wherein the temperature sensor is used to obtain the temperature inside the vehicle, and the light sensor is used to obtain the light intensity inside the vehicle.

14. A storage medium storing computer-readable instructions, wherein when the instructions are executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images