In-vehicle living body detection method, living body detection system, and vehicle

Abstract

This application relates to an in-vehicle liveness detection method, a liveness detection system, and a vehicle. After the vehicle is locked, if at least one liveness detection verification signal is detected, cabin image data of the target cabin area corresponding to the liveness detection verification signal is acquired. The liveness detection verification signal is generated by the liveness detection module performing preliminary liveness detection on the corresponding target cabin area. Based on the cabin image data, local liveness detection is performed to obtain the corresponding liveness detection result. Through a two-level collaborative detection mechanism of primary perception and visual verification, the risk of false alarms or missed alarms caused by environmental interference or sensitivity limitations due to the use of a single sensor in traditional technologies is avoided, effectively improving the reliability and accuracy of in-vehicle liveness detection.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to an in-vehicle liveness detection method, a liveness detection system, and a vehicle. Background Technology

[0002] The intelligent cockpit, through the linkage of intelligent cockpit interior and cockpit electronics, enables intelligent interaction between people, roads, and vehicles, redefining the relationship between humans and cars. The car cockpit is the most direct embodiment of automotive technology. With the continued strengthening of personalized and high-end consumption trends, cockpit electronics, characterized by intelligence and a sense of technology, can meet consumers' demands for intelligent driving configurations. Among these, cockpit monitoring is a new application that has emerged in recent years, aiming to improve driving safety by sensing the in-vehicle environment in real time.

[0003] However, current related technologies mostly rely on a single sensor for cockpit monitoring, which is susceptible to interference from ambient temperature, noise, etc., leading to frequent false triggers and a high false alarm rate, making it difficult to meet the reliability requirements of practical applications. Summary of the Invention

[0004] Therefore, it is necessary to provide an in-vehicle liveness detection method, a liveness detection system, and a vehicle that can improve the reliability of in-vehicle liveness detection in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides an in-vehicle liveness detection method, applied to a vehicle liveness detection system, wherein the liveness detection system includes at least one liveness detection module, and the method includes:

[0006] After the vehicle is locked, if at least one liveness detection verification signal is detected, cabin image data of the target cabin area corresponding to the liveness detection verification signal is acquired; the liveness detection verification signal is generated by the liveness detection module performing preliminary liveness detection on the corresponding target cabin area.

[0007] Based on the cockpit image data, local liveness detection is performed to obtain the corresponding liveness detection results.

[0008] In one embodiment, the method further includes:

[0009] If all the liveness detection results indicate that there are no live bodies, then enter low-power standby mode;

[0010] If the liveness detection result indicates the presence of a live body, then the cockpit image data corresponding to the target cockpit area where the live body is present, as well as the audio data associated with the cockpit image data, are encoded to generate a corresponding cockpit area video stream.

[0011] The cockpit area video stream is sent to the remote communication terminal, and after the cockpit area video stream is sent, it enters a low-power standby mode.

[0012] In one embodiment, the liveness detection system further includes at least one camera module; the method for acquiring the cockpit image data corresponding to the target cockpit area where a live body exists includes:

[0013] Identify the target camera module corresponding to the target cockpit area;

[0014] The frame rate of the target camera module is increased from the initial frame rate to a preset frame rate, so that the target camera module can acquire images of the target cockpit area according to the preset frame rate, thereby obtaining the corresponding cockpit image data.

[0015] In one embodiment, the method further includes:

[0016] The liveness detection result is sent to the vehicle's onboard electronic control unit (ECU) so that the ECU can perform a corresponding response operation based on the liveness detection result.

[0017] In one embodiment, the liveness detection system further includes at least one camera module; the method further includes:

[0018] According to a preset cycle, fault detection is performed on the liveness detection module and the camera module respectively, and corresponding fault detection results are generated for each.

[0019] According to the preset fault diagnosis protocol, the fault detection results are sent to the vehicle electronic control unit.

[0020] Secondly, this application provides a liveness detection system, the system comprising a control unit, at least one liveness detection module, and at least one camera module; the control unit is connected to both the liveness detection module and the camera module.

[0021] The liveness detection module is used to perform preliminary liveness detection on the corresponding target cabin area after the vehicle is locked, generate a liveness detection verification signal, and transmit the liveness detection verification signal to the control unit.

[0022] The camera module is used to acquire cockpit image data of the target cockpit area corresponding to the liveness detection verification signal, and transmit the cockpit image data to the control unit;

[0023] The control unit is used to perform the in-vehicle liveness detection method as described above.

[0024] In one embodiment, the liveness detection module includes at least one passive infrared sensor and / or at least one audio sensor.

[0025] Thirdly, this application provides a vehicle that includes the liveness detection system described above.

[0026] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0027] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0028] The aforementioned in-vehicle liveness detection method, liveness detection system, and vehicle are described above. The in-vehicle liveness detection method is applied to the vehicle's liveness detection system, which includes at least one liveness detection module. After the vehicle is locked, the liveness detection module performs preliminary liveness detection on the corresponding target cabin area, thereby sensing liveness signs in the target cabin area in real time and generating a liveness detection verification signal. This triggers the liveness detection system to acquire cabin image data of the target cabin area corresponding to the liveness detection verification signal. Then, local liveness detection is performed based on the cabin image data to verify the accuracy of the liveness detection verification signal, generating a more reliable liveness detection result. Through a two-level collaborative detection mechanism of primary perception and visual verification, the risk of false alarms or missed alarms caused by environmental interference or sensitivity limitations due to the use of a single sensor in traditional technologies is avoided, effectively improving the reliability and accuracy of in-vehicle liveness detection. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a schematic diagram of the structure of a liveness detection system in one embodiment;

[0031] Figure 2 This is a flowchart illustrating an in-vehicle liveness detection method in one embodiment;

[0032] Figure 3 This is a flowchart illustrating the in-vehicle liveness detection method in another embodiment;

[0033] Figure 4This is a flowchart illustrating the cockpit image data acquisition steps in one embodiment;

[0034] Figure 5 This is a flowchart illustrating a liveness detection method for an in-vehicle system in a specific embodiment.

[0035] Figure 6 This is a schematic diagram of the structure of a liveness detection system in a specific embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0037] The in-vehicle liveness detection method provided in this application embodiment is applied to, for example, Figure 1 The liveness detection system shown is installed inside the vehicle's cabin. The location of the liveness detection system needs to be determined comprehensively based on actual safety requirements, vehicle structural layout, and detection coverage, and is not specifically limited here. The vehicle can be, but is not limited to, passenger cars, commercial vehicles, special-purpose vehicles, autonomous vehicles, electric vehicles, hybrid vehicles, driverless minibuses, logistics delivery vehicles, shared mobility vehicles, and other vehicles with enclosed cabin structures. Different types of vehicles can be configured with appropriate liveness detection systems according to their usage scenarios and safety requirements to effectively prevent safety risks such as accidental detention of people or animals or illegal intrusion.

[0038] The liveness detection system includes at least one liveness detection module. This module can be positioned within a target cabin area of ​​the vehicle. In an exemplary embodiment, the target cabin area may, but is not limited to, the driver's seat area, the front passenger seat area, the left / right / middle seats in the second row, the third-row seats, the trunk area, or other enclosed spaces within the vehicle where people or pets may reside. It is understood that by deploying liveness detection modules in the aforementioned key areas, the system can achieve comprehensive coverage and accurate monitoring of the entire vehicle cabin environment, providing a reliable data foundation for vehicle safety warnings.

[0039] The liveness detection system also includes a control unit; the control unit is communicatively connected to the liveness detection module; the control unit may include, but is not limited to, a microcontroller unit (MCU) or a system on chip (SOC); the control unit is used to execute the in-vehicle liveness detection method of this application.

[0040] In one embodiment, such as Figure 2 As shown, Figure 2This is a flowchart illustrating an in-vehicle liveness detection method in one embodiment; the liveness detection method in this embodiment includes the following steps:

[0041] Step S201: After the vehicle is locked, if at least one liveness detection verification signal is detected, then the cockpit image data of the target cockpit area corresponding to the liveness detection verification signal is acquired.

[0042] Understandably, after the vehicle is locked, the entire vehicle enters a low-power or partially power-off state; at this time, the liveness detection system is powered on and continues to operate in low-power standby mode to ensure that potential risks in the cabin are effectively monitored while energy consumption is controlled.

[0043] The liveness detection verification signal is generated by the liveness detection module after performing preliminary liveness detection on the corresponding target cockpit area. The liveness detection module is used to monitor the target cockpit area for signs of liveness in real time when the system is in low-power standby mode. It should be noted that the liveness detection module is configured corresponding to the target cockpit area; the number of liveness detection modules can be one or more, depending on actual security requirements, and the specific number and installation location are not limited here.

[0044] Understandably, the liveness detection verification signal is a preliminary judgment signal generated by the liveness detection module based on the perceived physiological characteristics of the liveness (such as body temperature, sound, etc.). It is used to indicate that there may be a liveness in the target cabin area and serves as a trigger condition to start the subsequent vision-based liveness presence verification process.

[0045] In an exemplary embodiment, the liveness detection module may be a non-visual sensing device for sensing physical signals of vital signs within the target cockpit area. For example, the liveness detection module may include, but is not limited to, one or more of the following: a passive infrared sensor, an audio sensor, a carbon dioxide gas sensor, etc.

[0046] The cockpit image data includes multiple consecutive frames of cockpit images. This data can be obtained by the camera module corresponding to the target cockpit area in the liveness detection system, acquiring images of the target cockpit area in real time at an initial frame rate. The initial frame rate refers to the frame rate of the camera module after initialization; it can be, but is not limited to, 1 frame per second. It is understood that using a lower initial frame rate can effectively avoid wasting system power consumption, storage, and network resources due to high-frequency image acquisition while ensuring basic monitoring capabilities.

[0047] It should be noted that the camera module must be installed in a position that ensures its field of view can completely cover the corresponding target cockpit area to ensure the accuracy of liveness detection.

[0048] Step S202: Based on the cockpit image data, perform local liveness detection to obtain the corresponding liveness detection results.

[0049] Localized liveness detection may include, but is not limited to, intrusion detection and / or liveness detection. Intrusion detection may be implemented through algorithms such as moving target detection to identify unauthorized personnel entering the cabin after the vehicle is locked; liveness detection may be implemented through artificial intelligence algorithms to accurately determine whether a child or pet has been left inside the vehicle.

[0050] Understandably, leaving a living person inside a vehicle (such as a child or pet forgotten in the car) poses an imminent safety risk. Because the interior of a car is enclosed and temperatures can rise or fall rapidly, if a living person remains inside for too long after the vehicle is locked, it could lead to suffocation, heatstroke, hypothermia, or even death. Therefore, the system needs to detect the presence of a living person within a preset critical time period after the vehicle is locked (e.g., the first 20 minutes) to identify and warn of this risk as early as possible. In contrast, while intrusion attempts (such as unauthorized entry into the vehicle) also require prevention, their security threat does not have the same immediate urgency. Therefore, intrusion detection can be performed at any time after the vehicle is locked, and a response can be triggered upon detecting abnormal activity.

[0051] Among them, the liveness detection results are used to characterize whether a live person actually exists in the corresponding target cockpit area.

[0052] Understandably, the liveness detection result is obtained by performing a secondary liveness detection on the target cockpit area based on cockpit image data. Its accuracy is higher than that of the liveness detection verification signal generated by the liveness detection module alone. This effectively avoids the risk of false alarms or missed alarms caused by the susceptibility of a single sensor to environmental interference in traditional technologies.

[0053] In an exemplary embodiment, taking the front passenger seat area (i.e., the target cabin area being the front passenger seat area) as an example, after the vehicle is locked, the liveness detection system powers on and enters a low-power standby mode. At this time, the liveness detection module senses signs of life in the front passenger seat area in real time. Upon sensing the presence of live physiological characteristics in the front passenger seat area, it generates a liveness detection verification signal and feeds it back to the control unit. After detecting the liveness detection verification signal, the control unit wakes up the camera module corresponding to the target cabin area corresponding to the liveness detection verification signal, so that the camera module can collect cabin image data of the target cabin area in real time according to the initial frame rate and feed the cabin image data back to the control unit. Subsequently, the control unit in the liveness detection system switches from the low-power standby mode to the working mode, performs local liveness detection based on the acquired cabin image data, and obtains the corresponding liveness detection result.

[0054] The aforementioned in-vehicle liveness detection method, after the vehicle is locked, performs preliminary liveness detection on the corresponding target cabin area based on the liveness detection module, thereby sensing liveness signs in the target cabin area in real time and generating a liveness detection verification signal. This triggers the control unit in the liveness detection system to acquire cabin image data of the target cabin area corresponding to the liveness detection verification signal. Then, based on the cabin image data, local liveness detection is performed to verify the accuracy of the liveness detection verification signal, generating a more reliable liveness detection result. Through a two-level collaborative detection mechanism of primary perception and visual verification, the risk of false alarms or missed alarms caused by environmental interference or sensitivity limitations due to the use of a single sensor in traditional technologies is avoided, effectively improving the reliability and accuracy of in-vehicle liveness detection.

[0055] In one embodiment, Figure 3 This is a flowchart illustrating an in-vehicle liveness detection method in another embodiment; the in-vehicle liveness detection method further includes the following steps:

[0056] Step S301: If the liveness detection results all indicate that there are no live bodies, then enter the low-power standby mode.

[0057] Understandably, the control unit is in working mode when executing the local liveness detection process. When the liveness detection results all indicate that there are no live bodies, the control unit exits the working mode and enters a low-power standby mode to reduce the overall power consumption of the system.

[0058] Step S302: If the liveness detection result indicates the presence of a live body, then the cockpit image data corresponding to the target cockpit area where the live body exists and the audio data associated with the cockpit image data are encoded to generate the corresponding cockpit area video stream.

[0059] Understandably, the cockpit area video stream can serve as crucial evidence of a live event, allowing users, vehicle owners, or security platforms to remotely view the scene after a safety alarm is triggered, assisting in determining the authenticity of the event and providing a reliable basis for subsequent emergency response.

[0060] In an exemplary embodiment, the cockpit image data corresponding to the target cockpit area where a living person exists, as well as the audio data associated with the cockpit image data, are encoded to generate a corresponding cockpit area video stream, including the following steps:

[0061] Step 1: Starting from the trigger time of the liveness detection verification signal, synchronously acquire cockpit image data corresponding to the target cockpit area where a live person exists, as well as audio data time-aligned with the cockpit image data, within a preset time window. The preset time window can be set according to the internal storage capacity of the control unit, the data sampling rate, and the system response requirements; no specific limitations are imposed here.

[0062] Step 2: Encode the time-aligned cockpit image and audio data to generate the corresponding cockpit area video stream.

[0063] Step S303: Send the cockpit area video stream to the remote communication terminal, and enter low-power standby mode after the cockpit area video stream is sent.

[0064] Among them, the remote communication terminal is used to realize data interaction between the vehicle and external networks (such as the cloud, mobile terminals, etc.).

[0065] In one exemplary embodiment, the liveness detection system may include one or more liveness detection modules. This embodiment uses multiple liveness detection modules as an example, with the multiple liveness detection modules respectively set in different target cabin areas of the vehicle. After the vehicle is locked, the multiple liveness detection modules perform preliminary liveness detection on their respective target cabin areas, and generate corresponding liveness detection verification signals after detecting live physiological characteristics, so as to trigger the control unit in the liveness detection system to execute the in-vehicle liveness detection method of this application, thereby obtaining the liveness detection results corresponding to different target cabin areas.

[0066] Furthermore, based on the liveness detection results corresponding to different target cockpit areas, the system determines whether a live person actually exists in each target cockpit area. If all liveness detection results indicate that no live person exists, the system enters a low-power standby mode to avoid unnecessary energy consumption. If any liveness detection result indicates that a live person exists, the cockpit image data corresponding to the target cockpit area with a live person, as well as the audio data associated with the cockpit image data, are encoded to generate a corresponding cockpit area video stream. The cockpit area video stream is then sent to a remote communication terminal. After the cockpit area video stream is sent, the system automatically enters a low-power standby mode and simultaneously uploads the cockpit area video stream to the cloud via the remote communication terminal to trigger a safety alarm mechanism. This promptly sends alarms to the vehicle owner, emergency contacts, or relevant security platforms and simultaneously provides the user with on-site video of the liveness event, providing evidence for rapid response and effective intervention.

[0067] In this embodiment, the system only initiates multimodal data acquisition and encoding tasks when a real live event occurs, generating a cockpit area video stream for safety alarms. This not only effectively improves the accuracy of safety alarms and event backtracking capabilities, but also effectively reduces the overall power consumption of the system and avoids the problem of network bandwidth waste caused by unnecessary data acquisition.

[0068] In one embodiment, such as Figure 4 As shown, Figure 4 This is a flowchart illustrating the cockpit image data acquisition steps in one embodiment; the method for acquiring cockpit image data corresponding to a target cockpit area containing a living person includes the following steps:

[0069] Step S401: Determine the target camera module corresponding to the target cockpit area.

[0070] The liveness detection system also includes at least one camera module; the camera module is used to monitor at least one target cockpit area; the camera module may include, but is not limited to, an RGB image sensor.

[0071] It is understood that each camera module, based on its installation location, field of view, and coverage area, can be clearly assigned to a target cabin area, thereby providing an accurate and reliable image data source for local liveness detection. In an exemplary embodiment, the installation location of the camera module may be, but is not limited to, the front reading lights, the central control screen, or other similar locations within the vehicle cabin; no specific limitation is made here.

[0072] Among them, the target camera module refers to a camera module whose field of view can completely cover the target cockpit area.

[0073] Step S402: Increase the frame rate of the target camera module from the initial frame rate to the preset frame rate, so that the target camera module can acquire images of the target cockpit area at the preset frame rate and obtain the corresponding cockpit image data.

[0074] The initial frame rate refers to the frame rate of the target camera module after initialization. The initial frame rate can be, but is not limited to, 1 frame per second. It can be understood that using a lower initial frame rate can effectively avoid the waste of system power consumption, storage and network resources caused by high-frequency image acquisition while ensuring basic monitoring capabilities.

[0075] The preset frame rate is greater than the initial frame rate. The preset frame rate can be, but is not limited to, 10 to 30 frames per second; for example, 30 frames per second can be used. The preset frame rate needs to be set according to the actual video data integrity requirements, image smoothness requirements, system storage capacity, and subsequent analysis needs to ensure that clear and continuous video evidence can be obtained when a live event occurs.

[0076] In an exemplary embodiment, after the vehicle is locked, if at least one liveness detection verification signal is detected, cabin image data of the target cabin area corresponding to the liveness detection verification signal is first acquired at an initial frame rate. Based on the cabin image data, local liveness detection is performed to obtain the corresponding liveness detection result. If the liveness detection result indicates the presence of a live person, the frame rate of the target camera module corresponding to the target cabin area corresponding to the liveness detection verification signal is increased to a preset frame rate so that the target camera module can acquire images of the target cabin area at a higher frame rate, thereby obtaining continuous and clear cabin image data.

[0077] In this embodiment, by identifying the target camera module corresponding to the target cockpit area and updating the frame rate of the target camera module, it is possible to ensure that sufficient visual information is captured in a timely manner during the critical period when a real liveness event occurs, providing high-quality and timely data support for subsequent local liveness detection and cockpit area video stream generation.

[0078] In one embodiment, the in-vehicle liveness detection method further includes the following steps:

[0079] The liveness detection results are sent to the vehicle's onboard electronic control unit (ECU) so that the ECU can perform corresponding response operations based on the liveness detection results.

[0080] The vehicle electronic control unit (ECU) may include, but is not limited to, vehicle infotainment systems, air conditioning control units, door lock control units, and other ECUs. Response actions may include, but are not limited to, horn activation, hazard warning lights, automatic window lowering (or sunroof opening for ventilation), remote or local door unlocking, and sending in-vehicle status alerts to the user terminal. These actions enable timely and effective safety responses to risk scenarios such as unauthorized presence or intrusion, maximizing the protection of occupants' lives and vehicle property.

[0081] In one embodiment, the in-vehicle liveness detection method further includes the following steps:

[0082] Step 1: Perform fault detection on the liveness detection module and the camera module according to the preset cycle, and generate the corresponding fault detection results for each.

[0083] The preset cycle can be set according to the actual fault detection needs, and no specific limit is made here.

[0084] Step 2: According to the preset fault diagnosis protocol, send the detection results of each fault to the vehicle electronic control unit.

[0085] The preset fault diagnosis protocol can be set according to vehicle communication bus standards, etc., to ensure reliable transmission of fault detection results. No specific limitations are made here.

[0086] In one example embodiment, the control unit of the liveness detection system periodically monitors the working status of all connected liveness detection modules and camera modules according to a preset cycle to detect faults in the liveness detection modules and camera modules. When a fault is detected in the liveness detection module and / or camera module, the control unit updates the corresponding internal fault register, generates the corresponding fault detection result, and sends the fault detection result to the vehicle electronic control unit via the CAN bus according to a preset fault diagnosis protocol, so that the vehicle system can record the fault, provide an alarm, or perform function degradation processing.

[0087] In other embodiments, the liveness detection system also includes a power module, and the control unit is also used to periodically monitor the working status of the power module, generate corresponding power fault detection results, and then send the power fault detection results to the vehicle electronic control unit according to a preset fault diagnosis protocol.

[0088] In this embodiment, by periodically monitoring the working status of key components such as the liveness detection module, camera module, and power module in the liveness detection system, abnormalities or faults (such as sensor failure, communication interruption, power supply abnormality, etc.) can be detected in a timely manner, and corresponding fault detection results can be generated. Then, the fault detection results are reported to the vehicle electronic control unit through a preset fault diagnosis protocol, so as to realize the rapid location and response of faults and ensure the reliability, availability and functional safety of the liveness detection system.

[0089] In one specific embodiment, such as Figure 5 As shown, Figure 5 This is a flowchart illustrating a liveness detection method inside a vehicle in a specific embodiment; the example uses a liveness detection system including a liveness detection module. The liveness detection method inside a vehicle includes the following steps:

[0090] Step S501: After the vehicle is locked, the liveness detection system is powered on, begins hardware self-test, and initializes all connected liveness detection modules and camera modules.

[0091] For example, the frame rate of the camera module is configured to 1 frame / second, and the system cyclically stores the RGB images of the corresponding target cockpit area; the liveness detection module starts working; the system enters a low-power standby mode.

[0092] Step S502: If the control unit detects a liveness detection verification signal, then acquire the cockpit image data of the target cockpit area corresponding to the liveness detection verification signal.

[0093] Among them, the liveness detection verification signal is generated by the liveness detection module when performing preliminary liveness detection on the corresponding target cockpit area.

[0094] The cockpit image data is collected by the target camera module corresponding to the target cockpit area at the initial frame rate.

[0095] Step S503: Based on the cockpit image data, perform local liveness detection to obtain the corresponding liveness detection results.

[0096] Step S504: Determine whether a living being actually exists in the target cockpit area.

[0097] Step S505: If the liveness detection result indicates that there is no live body, then enter the low-power standby mode.

[0098] Step S506: If the liveness detection result indicates the presence of a live body, the frame rate of the target camera module corresponding to the target cockpit area is increased from the initial frame rate to the preset frame rate, so that the target camera module can acquire images of the target cockpit area at the preset frame rate to obtain the corresponding cockpit image data; at the same time, audio data that is time-aligned with the cockpit image data is acquired.

[0099] Step S507: Encode the time-aligned cockpit image data and audio data to generate the corresponding cockpit area video stream, send the cockpit area video stream to the remote communication terminal, and enter the low-power standby mode after the cockpit area video stream is sent.

[0100] It is understandable that when the liveness detection system includes multiple liveness detection modules, the implementation principle of the in-vehicle liveness detection method is the same as that described in the specific embodiments above, and will not be repeated here.

[0101] In one embodiment, see Figure 1 A liveness detection system is provided, which includes a control unit, at least one liveness detection module, and at least one camera module; the control unit is connected to the liveness detection module and the camera module respectively.

[0102] The liveness detection module is used to perform preliminary liveness detection on the corresponding target cabin area after the vehicle is locked, generate a liveness detection verification signal, and transmit the liveness detection verification signal to the control unit.

[0103] The camera module is used to acquire cockpit image data of the target cockpit area corresponding to the liveness detection and verification signal, and transmit the cockpit image data to the control unit.

[0104] The control unit is used to execute the in-vehicle liveness detection method described in any of the above embodiments. Specific limitations of the in-vehicle liveness detection method can be found in the limitations of the in-vehicle liveness detection method in the above embodiments, and will not be repeated here.

[0105] The installation location of the liveness detection system can be flexibly set according to actual needs. For example, it can be placed above the front dome light, the rearview mirror, the central control screen, the rear reading light, or the rear door handle. The liveness detection system meets automotive-grade requirements and can realize functional safety, diagnostics, and OTA (Over-The-Air) functions. In some embodiments, the standby power consumption of the liveness detection system is less than 50mW.

[0106] The liveness detection module is set in the target cabin area of ​​the vehicle; the target cabin area may include, but is not limited to, the front driver's seat area, the front passenger seat area, the left / right / middle seat area of ​​the second row, the third row seat area, the trunk area, or other enclosed spaces in the vehicle where people or pets may stay.

[0107] The camera module can be, but is not limited to, an RGB image sensor. The installation position of the camera module must ensure that its field of view can completely cover the corresponding target cockpit area to ensure the accuracy of liveness detection.

[0108] The control unit may include, but is not limited to, MCU and SOC.

[0109] In this embodiment, the liveness detection system performs preliminary liveness detection on the corresponding target cabin area based on the liveness detection module after the vehicle is locked. This allows for real-time perception of liveness signs in the target cabin area, generating a liveness detection verification signal. This triggers the liveness detection system to acquire cabin image data of the target cabin area corresponding to the liveness detection verification signal. Subsequently, local liveness detection is performed based on the cabin image data to verify the accuracy of the liveness detection verification signal, generating a more reliable liveness detection result. Through a two-level collaborative detection mechanism of primary perception and visual verification, the system avoids the risk of false alarms or missed alarms caused by environmental interference or sensitivity limitations of traditional technologies using a single sensor, effectively improving the reliability and accuracy of in-vehicle liveness detection.

[0110] In one exemplary embodiment, the liveness detection module includes at least one passive infrared sensor and / or at least one audio sensor. For example, the liveness detection module includes two passive infrared sensors, or two audio sensors, or two passive infrared sensors and one audio sensor. The audio sensor may be, but is not limited to, a microphone sensor.

[0111] It should be noted that by using passive infrared sensors and / or audio sensors as the sensing units of the liveness detection module, the system hardware cost can be effectively reduced and the system's economy and deployability can be improved while meeting the basic needs for detecting signs of liveness (such as human thermal radiation, breathing sounds, crying sounds, etc.).

[0112] The passive infrared sensor includes a power supply, a PIR (Passive Infrared) probe, and a PIR sensor control chip. The power input, after passing through a power converter chip, supplies power to the PIR sensor control chip. The chip receives data from the PIR probe and performs ADC sampling and judgment internally. When the PIR data exceeds a certain threshold (indicating the possible presence of live physiological characteristics), it outputs a PIR interrupt signal, also known as a liveness detection verification signal, to notify the SOC or MCU. The sensitivity of the PIR probe and the duration of the delay after the PIR interrupt signal are generated can be adjusted via external circuitry.

[0113] The microphone sensor includes a power supply, a microphone (MIC), and a comparator chip. The power input, after passing through a power converter chip, powers the comparator chip. The comparator chip receives data from the microphone and compares it to a threshold value set by the chip. When the received sound decibel value exceeds the threshold, it outputs a MIC interrupt signal, also known as a liveness detection verification signal, to notify the SOC or MCU. The comparator chip's discrimination threshold can be adjusted via external circuitry.

[0114] It is understood that when the liveness detection module includes at least one passive infrared sensor and at least one audio sensor, the liveness detection verification signal may be generated by at least one passive infrared sensor and / or at least one audio sensor.

[0115] In one specific embodiment, such as Figure 6 As shown, Figure 6 This is a schematic diagram of a liveness detection system in a specific embodiment. The liveness detection module includes two passive infrared sensors, i.e., PIR sensors. In this embodiment, the audio sensor is a microphone sensor; the camera module is an RGB image sensor. The control unit includes an MCU and a SOC, and the MCU and SOC are communicatively connected.

[0116] In practical applications, after the system powers on, it begins a hardware self-test and initializes all connected PIR / MIC / RGB image sensors. The RGB image sensors are configured to operate at 1 frame / second, and the system cyclically stores RGB images. The PIR / MIC sensors then begin operation. The system enters a low-power standby mode. Based on the liveness detection verification signal fed back by the PIR / MIC sensors and the cockpit image data of the target cockpit area corresponding to the liveness detection verification signal acquired by the RGB image sensors, the SOC wakes up the NPU (Neural Processing Unit) and the hardware encoder, runs a local liveness detection algorithm, and determines whether a live event has actually occurred.

[0117] If the algorithm's result is true, the SOC configures the RGB image sensor to output images at 30 frames per second, storing the images from the previous N seconds. The ECU (Electronic Control Unit) cyclically stores audio and video, records liveness events, and wakes up the TBOX (Telematics Box) via the CAN bus. After waking up, the audio and video are encoded and decoded and streamed to the TBOX. The TBOX forwards the stream to the cloud and, after completing the streaming, shuts down the hardware encoder and NPU unit, and the system re-enters low-power standby mode. If the algorithm's result is false, the system shuts down the hardware encoder and NPU unit, and the system re-enters low-power standby mode. The hardware encoder is used to encode and process audio and video to generate the corresponding video stream.

[0118] Furthermore, in embodiments where the SOC does not support CAN bus communication, the SOC transmits the liveness detection result to the MCU via the SPI / UART interface, and the MCU then sends it to the vehicle's infotainment system or other vehicle ECUs via the CAN bus. In other embodiments where the SOC supports CAN bus communication, the SOC directly sends the liveness detection result to the vehicle's infotainment system or other vehicle ECUs via the CAN bus.

[0119] In one embodiment, a vehicle is provided, which includes the liveness detection system described in any of the above embodiments. The specific limitations of the liveness detection system can be referred to the limitations of the liveness detection system in the above embodiments, and will not be repeated here.

[0120] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0121] Based on the same inventive concept, this application also provides an in-vehicle liveness detection device for implementing the in-vehicle liveness detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more in-vehicle liveness detection device embodiments provided below can be found in the limitations of the in-vehicle liveness detection method described above, and will not be repeated here.

[0122] In one exemplary embodiment, an in-vehicle liveness detection device is provided, comprising: an acquisition module and a detection module, wherein:

[0123] The acquisition module is used to acquire cabin image data of the target cabin area corresponding to the liveness detection verification signal if at least one liveness detection verification signal is detected after the vehicle is locked; the liveness detection verification signal is generated by the liveness detection module performing preliminary liveness detection on the corresponding target cabin area;

[0124] The detection module is used to perform local liveness detection based on cockpit image data and obtain the corresponding liveness detection results.

[0125] The aforementioned in-vehicle liveness detection device, after the vehicle is locked, uses a two-level collaborative detection mechanism of primary perception and visual verification to avoid the risk of false alarms or missed alarms caused by environmental interference or sensitivity limitations of traditional technologies that use a single sensor, thus effectively improving the reliability and accuracy of in-vehicle liveness detection.

[0126] In one embodiment, the in-vehicle liveness detection device further includes a control module, the control module being used for:

[0127] If the liveness detection results all indicate that there are no live subjects, then enter low-power standby mode;

[0128] If the liveness detection result indicates the presence of a live body, the cockpit image data corresponding to the target cockpit area where the live body is present, as well as the audio data associated with the cockpit image data, are encoded to generate the corresponding cockpit area video stream.

[0129] The cockpit area video stream is sent to the remote communication terminal, and after the cockpit area video stream is sent, it enters a low-power standby mode.

[0130] In one embodiment, the control module is further configured to:

[0131] Identify the target camera module corresponding to the target cockpit area;

[0132] The frame rate of the target camera module is increased from the initial frame rate to the preset frame rate, so that the target camera module can acquire images of the target cockpit area at the preset frame rate and obtain the corresponding cockpit image data.

[0133] In one embodiment, the in-vehicle liveness detection device further includes a communication module, the communication module being used for:

[0134] The liveness detection results are sent to the vehicle's onboard electronic control unit (ECU) so that the ECU can perform corresponding response operations based on the liveness detection results.

[0135] In one embodiment, the in-vehicle liveness detection device further includes a fault detection module, the fault detection module being used for:

[0136] According to the preset cycle, fault detection is performed on the liveness detection module and the camera module respectively, and corresponding fault detection results are generated.

[0137] According to the preset fault diagnosis protocol, the results of each fault detection are sent to the vehicle electronic control unit.

[0138] Each module in the aforementioned in-vehicle liveness detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0139] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0140] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0144] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting liveness inside a vehicle, characterized in that, A liveness detection system for vehicles, the liveness detection system including at least one liveness detection module, the method comprising: After the vehicle is locked, if at least one liveness detection verification signal is detected, cabin image data of the target cabin area corresponding to the liveness detection verification signal is acquired; the liveness detection verification signal is generated by the liveness detection module performing preliminary liveness detection on the corresponding target cabin area. Based on the cockpit image data, local liveness detection is performed to obtain the corresponding liveness detection results.

2. The method according to claim 1, characterized in that, The method further includes: If all the liveness detection results indicate that there are no live bodies, then enter low-power standby mode; If the liveness detection result indicates the presence of a live body, then the cockpit image data corresponding to the target cockpit area where the live body is present, as well as the audio data associated with the cockpit image data, are encoded to generate a corresponding cockpit area video stream. The cockpit area video stream is sent to the remote communication terminal, and after the cockpit area video stream is sent, it enters a low-power standby mode.

3. The method according to claim 2, characterized in that, The liveness detection system also includes at least one camera module; A method for acquiring cockpit image data corresponding to the target cockpit area where a living person exists includes: Identify the target camera module corresponding to the target cockpit area; The frame rate of the target camera module is increased from the initial frame rate to a preset frame rate, so that the target camera module can acquire images of the target cockpit area according to the preset frame rate, thereby obtaining the corresponding cockpit image data.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The liveness detection result is sent to the vehicle's onboard electronic control unit (ECU) so that the ECU can perform a corresponding response operation based on the liveness detection result.

5. The method according to any one of claims 1 to 3, characterized in that, The liveness detection system further includes at least one camera module; the method further includes: According to a preset cycle, fault detection is performed on the liveness detection module and the camera module respectively, and corresponding fault detection results are generated for each. According to the preset fault diagnosis protocol, the fault detection results are sent to the vehicle electronic control unit.

6. A liveness detection system, characterized in that, The system includes a control unit, at least one liveness detection module, and at least one camera module; the control unit is connected to both the liveness detection module and the camera module. The liveness detection module is used to perform preliminary liveness detection on the corresponding target cabin area after the vehicle is locked, generate a liveness detection verification signal, and transmit the liveness detection verification signal to the control unit. The camera module is used to acquire cockpit image data of the target cockpit area corresponding to the liveness detection verification signal, and transmit the cockpit image data to the control unit; The control unit is used to perform the in-vehicle liveness detection method according to any one of claims 1 to 5.

7. The system according to claim 6, characterized in that, The liveness detection module includes at least one passive infrared sensor and / or at least one audio sensor.

8. A vehicle, characterized in that, The vehicle includes the liveness detection system as described in any one of claims 6 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images