Living body detection method and device, product and vehicle

Abstract

The invention provides a living body detection method and device, a product and a vehicle, and belongs to the technical field of living body detection. The method comprises the following steps: sending a plurality of different wireless signals in a vehicle, and obtaining channel state information corresponding to the plurality of different wireless signals; and constructing a multi-dimensional feature vector according to the channel state information, and determining a living body detection result in the vehicle according to the multi-dimensional feature vector. The objective of the invention is to improve the accuracy of living body detection.

Description

Technical Field

[0001] This application relates to the technical field of liveness detection, and more specifically, to a liveness detection method, apparatus, product, and vehicle. Background Technology

[0002] With the development of vehicle safety technology, it is essential to conduct liveness detection on vehicles in order to avoid serious safety consequences caused by people left inside them.

[0003] When performing liveness detection on vehicles, the common method is to acquire images inside the vehicle and identify whether there are any living beings inside based on the images. However, this image-based method not only requires the deployment of image acquisition equipment, such as cameras, inside the vehicle, which has high installation and maintenance costs, but is also easily affected by factors such as lighting conditions, obstructions, and installation location, which reduces the accuracy of liveness detection. Summary of the Invention

[0004] This application provides a liveness detection method, apparatus, product, and vehicle, aiming to improve the accuracy of liveness detection.

[0005] In a first aspect, embodiments of this application provide a method for detecting liveness, the method comprising: The system transmits multiple different wireless signals within the vehicle and acquires the channel state information corresponding to the multiple different wireless signals. A multidimensional feature vector is constructed based on the channel state information, and the liveness detection result inside the vehicle is determined based on the multidimensional feature vector.

[0006] Optionally, multiple different wireless signals may be transmitted within the vehicle, including: The first wireless signal and the second wireless signal are transmitted in the vehicle based on a preset channel multiplexing method, wherein the channel multiplexing method includes any one of time division multiplexing, frequency division multiplexing and code division multiplexing. The first wireless signal is a continuous wave signal, and the second wireless signal is a chirped pulse signal.

[0007] Optionally, a multi-dimensional feature vector is constructed based on the channel state information, including: Based on the first channel state information corresponding to the first wireless signal in the channel state information, a first feature is determined, and the first feature is used to characterize the presence of a liveness detection target in the vehicle. Based on the second channel state information corresponding to the second wireless signal in the channel state information, a second feature is determined, which is used to characterize the life characteristics of the target for liveness detection inside the vehicle. A multidimensional feature vector is constructed based on the first feature and the second feature.

[0008] Optionally, the first feature includes amplitude and phase; the second feature includes microDoppler frequency shift and phase change rate.

[0009] Optionally, a multi-dimensional feature vector is constructed based on the first feature and the second feature, including: The first feature and the second feature are aligned to construct an initial multidimensional feature vector; The initial multidimensional feature vector is denoised by wavelet transform to obtain the multidimensional feature vector.

[0010] Optionally, determining the liveness detection result within the vehicle based on the multidimensional feature vector includes: The multidimensional feature vector is input into the lightweight classification model of the vehicle to determine the liveness detection result of the vehicle, which includes human body, animal and still object; The lightweight classification model of the vehicle is obtained by training multiple multidimensional feature vector samples of the vehicle. The multidimensional feature vector samples are determined based on channel state information samples corresponding to the multiple different wireless signals obtained on the vehicle.

[0011] Optionally, the method further includes: The training samples corresponding to different vehicle models are obtained, and federated learning is performed on the preset lightweight classification neural network to obtain a global lightweight classification model. The training sample of any vehicle model is a multi-dimensional feature vector sample determined based on the channel state information samples corresponding to multiple different wireless signals obtained on the vehicle. Multiple multidimensional feature vector samples of the vehicle are obtained, and the global lightweight classification model is fine-tuned and trained to obtain the lightweight classification model corresponding to the vehicle.

[0012] Optionally, training samples corresponding to different vehicle models are obtained, and federated learning is performed on a pre-defined lightweight classification neural network to obtain a global lightweight classification model, including: Initialize the preset model parameters of the lightweight classification neural network; For any vehicle model, multiple training samples of that vehicle are obtained, and the preset lightweight classification neural network is trained independently to determine the model parameters corresponding to that vehicle. The global lightweight classification model is obtained by weighted averaging of the model parameters corresponding to multiple vehicle models.

[0013] Optionally, before obtaining the channel state information corresponding to the plurality of different wireless signals, the method further includes: Obtain the initial channel state information corresponding to the multiple different wireless signals; Based on the initial channel state information, the environmental reflection map of the vehicle is extracted, and the target detection area inside the vehicle is determined. The target detection area is the area where a liveness detection target exists. Calculate the phase offset of each antenna element in the antenna array used to transmit the multiple different wireless signals relative to the target detection area; Based on the phase offset of each antenna element relative to the target detection area, multiple different wireless signals are alternately transmitted to control the antenna array to focus on the target detection area, thereby enhancing the signal-to-noise ratio of each signal within the target detection area.

[0014] Optionally, after determining the liveness detection result inside the vehicle based on the multidimensional feature vector, the method further includes: When the liveness detection result is a human body, a preset human body detection processing strategy is executed. The human body detection alarm strategy is used to send alarm prompt information and / or start the vehicle's air conditioning ventilation system. When the liveness detection result is an animal, a preset animal detection alarm strategy is executed, which is used to notify that an animal is present in the vehicle.

[0015] Secondly, embodiments of this application provide a liveness detection device, the device comprising: The channel state information acquisition module is used to send multiple different wireless signals inside the vehicle and acquire the channel state information corresponding to the multiple different wireless signals. The liveness detection result determination module is used to construct a multi-dimensional feature vector based on the channel state information, and determine the liveness detection result inside the vehicle based on the multi-dimensional feature vector.

[0016] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the liveness detection method as described in the first aspect of the embodiments.

[0017] Fourthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the liveness detection method as described in the first aspect of the embodiments.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the liveness detection method described in the first aspect of the embodiments.

[0019] Sixthly, embodiments of this application provide a vehicle for performing the liveness detection method described in the first aspect of the embodiments, or including the liveness detection device described in the second aspect of the embodiments.

[0020] Beneficial effects: In the liveness detection method provided in this application embodiment, multiple different wireless signals are sent inside the vehicle to obtain channel state information corresponding to the multiple different wireless signals. A multi-dimensional feature vector is constructed based on the channel state information, and the liveness detection result inside the vehicle is determined based on the multi-dimensional feature vector. Since the channel state information corresponding to multiple different signals is richer, a more dimensional multi-dimensional feature vector can be constructed. Liveness detection based on the dimensional multi-dimensional feature vector can improve the accuracy of liveness detection. At the same time, compared with the common image-based liveness detection method, it has a stronger anti-interference ability and is not easily affected by external factors, which can further improve the accuracy of liveness detection. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of the steps of a liveness detection method proposed in an embodiment of this application; Figure 2 This is a schematic diagram showing the location of a wireless communication module according to an embodiment of this application; Figure 3 This is a functional block diagram of a liveness detection device provided in an embodiment of this application; Figure 4 This is a schematic diagram of an electronic device according to an embodiment of this application; Figure 5 This is a schematic diagram of a readable storage medium proposed in an embodiment of this application; Figure 6 This is a schematic diagram of a computer program product provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] TDM: Time Division Multiplexing; CSI: Channel State Information; CAN: Controller Area Network; OBD: On-Board Diagnostics, an in-vehicle automatic diagnostic system; TSP: Telematics Service Provider.

[0026] Reference Figure 1 The diagram illustrates a flowchart of a liveness detection method provided in an embodiment of this application. The method specifically includes the following steps: S101: Send multiple different wireless signals inside the vehicle and obtain the channel state information corresponding to the multiple different wireless signals.

[0027] Multiple different wireless signals use different modulation methods, and the number of wireless signals and the modulation method of each wireless signal can be selected according to the actual application requirements.

[0028] The Channel State Information (CSI) corresponding to any wireless signal is used to describe the fading and scattering effects of the signal on the transmission path. Since multiple different wireless signals use different modulation methods, the obtained CSI includes the CSI corresponding to each wireless signal.

[0029] In actual implementation, multiple different wireless signals can be transmitted by determining a preset channel multiplexing method based on channel multiplexing technology. For example, the channel multiplexing method can include any one of time division multiplexing, frequency division multiplexing, and code division multiplexing. The method can be selected according to the needs of the actual application, and the embodiments of this application do not impose any restrictions.

[0030] Specifically, time division multiplexing divides the transmission time of a channel into many time periods, and then different wireless signals can be transmitted in turn, so as to achieve the simultaneous transmission of multiple signals on the same channel, that is, multiple different wireless signals occupy the same bandwidth at different times.

[0031] Frequency division multiplexing divides a frequency band into multiple non-overlapping sub-bands, with each wireless signal corresponding to one sub-band. This means that multiple different wireless signals occupy different frequency bandwidths at the same time.

[0032] Code division multiplexing (CDM) allows the simultaneous transmission of multiple different wireless signals based on the same frequency by using different spreading codes.

[0033] This application does not impose any restrictions on the channel multiplexing method for transmitting multiple different wireless signals, which can be selected according to the actual application requirements.

[0034] Taking the time-division multiplexing channel multiplexing method for transmitting two different wireless signals in a vehicle as an example, when transmitting two different wireless signals alternately in a vehicle based on time-division multiplexing, the first wireless signal and the second wireless signal are transmitted alternately according to a preset transmission cycle.

[0035] The first wireless signal is a continuous wave signal, i.e., a CW signal, and the second wireless signal is a chirped pulse signal. The first wireless signal and the second wireless signal each have a preset transmission duration and transmission frequency within a single transmission cycle.

[0036] In actual implementation, the transmission period, transmission duration and transmission frequency of each signal can be set according to the needs of the actual application. This application embodiment does not impose any restrictions.

[0037] For example, the transmission period corresponding to the CW signal is T1, and the transmission period of the chirped pulse signal is T2. T1 and T2 can both be set to 50ms.

[0038] In the transmission period T1 corresponding to the CW signal, the transmission frequency f0 of the CW signal can be set to 2.4GHz, and the transmission duration t1 can be set to 30ms.

[0039] During the transmission period T2 of the chirped pulse signal, the transmission frequency of the chirped pulse signal is linearly swept from f1 to f2, where f1 can be set to 2.4 GHz, f2 can be set to 2.41 GHz, and the duration t2 can be set to 20 ms.

[0040] In actual implementation, wireless communication modules for sending and receiving multiple different wireless signals can be installed inside the vehicle. The number, deployment location, and deployment method of the wireless communication modules can be selected according to the actual application requirements, and this application embodiment does not impose any restrictions.

[0041] Reference Figure 2 The diagram shows the location of the wireless communication module provided in the embodiment of this application. The wireless communication module can be installed inside the vehicle by embedding or attaching it, according to the needs of actual application. The locations where the wireless communication module can be deployed include, but are not limited to, location 1, location 2, location 3 and location 4.

[0042] The location 1 includes, but is not limited to, the dashboard, A-pillar, and rearview mirror of the vehicle; the location 2 includes, but is not limited to, the four corners of the roof and B-pillar of the vehicle; the location 3 includes, but is not limited to, the four doors and seats; and the location 4 includes, but is not limited to, the left / right side of the rear of the vehicle and the tailgate.

[0043] In practical applications, wireless communication modules can be integrated transmitters and receivers, or they can completely separate the transmission and reception. For example, a wireless communication module can include clock-synchronized transmitter and receiver units. In scenarios where the transmission and reception are separated, the receiver unit will not be interfered with by the strong direct signal of the transmitter unit itself, and it only processes the signal of its paired transmitter unit. It can ignore interference from other wireless signals in the vehicle, such as mobile phone Bluetooth and in-vehicle Wi-Fi, ensuring the purity of CSI data. This allows it to fully detect extremely weak signal changes caused by micro-movements, resulting in higher quality and sensitivity of CSI data for any wireless signal captured.

[0044] Taking a wireless communication module, which includes a transmitting unit and a receiving unit, as an example, when deploying a wireless communication module that can be used for liveness detection inside a vehicle, the core of its layout is to ensure that the effective propagation path of the wireless signal passes through the chest cavity and body area of ​​the live body (driver / passenger) to be detected as much as possible.

[0045] For example, three sets of wireless communication modules can be used for liveness detection, and their layout is as follows: The first set of wireless communication modules is mainly used to form a signal transmission path for the front row. Specifically, its transmitting unit can be installed on the roof near the rearview mirror, and the receiving unit can be installed in the center of the rear of the roof, near the rear windshield. The first set of wireless communication modules can form a signal path from front to center. Its strong signal path directly covers the chest cavity of the driver and front passenger, and the signal will also be scattered to the central area of ​​the rear row to provide auxiliary information.

[0046] The second set of wireless communication modules is mainly used to establish a signal transmission path on the left diagonal of the rear seats. Specifically, the transmitting unit can be installed on the roof of the vehicle on the driver's side A-pillar or the top of the left front door; the receiving unit can be installed on the roof of the vehicle on the passenger side C-pillar or the top of the right rear door. The second set of wireless communication modules can form a long diagonal signal transmission path from the left front to the right rear, which is one of the longest paths in the vehicle. It can stably pass through the body of the rear right passenger and at the same time provide the driver with a second perception path.

[0047] The third set of wireless communication modules is mainly used to establish a signal transmission path on the right diagonal of the rear seats. Specifically, the transmitting unit can be installed on the roof of the passenger side A-pillar or the top of the right front door, and the receiving unit can be installed on the roof of the driver side C-pillar or the top of the left rear door. The third set of wireless communication modules can form a long diagonal signal transmission path from the right front to the left rear. Compared with the signal transmission path of the second set of wireless communication modules, it can stably pass through the body of the rear left passenger and at the same time provide a second perception path for the front passengers.

[0048] Based on three sets of wireless communication modules, full coverage and redundant detection of people in different seats inside the vehicle can be achieved. This layout ensures that each seat has at least one high-quality, high-sensitivity detection link, improving the reliability of liveness detection. It also has high sensitivity. The roof deployment provides a top-down signal path, which can be highly sensitive to the slight fluctuations of the chest cavity (breathing, heartbeat). Moreover, the long diagonal path causes the signal to be reflected multiple times inside the vehicle, containing rich environmental information, which is beneficial for capturing micro-movements.

[0049] In practice, to reduce the deployment cost of the wireless communication module, a diagonal layout of the second and third sets of wireless communication modules can be adopted, which can cover all four main seats.

[0050] In one feasible implementation, a reconfigurable antenna array, such as a 4×4 MIMO antenna array, can also be integrated into the wireless communication module, and beamforming optimization of the antenna array can be implemented to directionally enhance wireless signal energy, reduce the power consumption of the wireless communication module, enhance spatial resolution, and further improve the accuracy of liveness detection.

[0051] Specifically, before acquiring the channel state information corresponding to the multiple different wireless signals, the method may further include the following steps: A1: Obtain the initial channel state information corresponding to the multiple different wireless signals.

[0052] For example, when alternating the transmission of the first and second wireless signals based on time division multiplexing, the first and second wireless signals are first transmitted in an omnidirectional transmission mode, and the first channel state information corresponding to the first wireless signal and the second channel state information corresponding to the second wireless signal are captured as initial channel state information.

[0053] Omnidirectional transmission mode refers to the fact that the antenna array of a wireless communication module radiates the same amount of radio wave energy in all directions in the horizontal plane. In omnidirectional transmission mode, the initial channel state information captured can reflect the CSI amplitude distribution in different areas inside the vehicle.

[0054] A2: Based on the initial channel state information, extract the environmental reflection map of the vehicle and determine the target detection area inside the vehicle, wherein the target detection area is the area where a live target is detected.

[0055] Based on the captured initial channel state information, CSI analysis can identify strong reflective areas inside the vehicle, such as seat frames and door metal parts, and extract the environmental reflectance spectrum inside the vehicle.

[0056] In actual implementation, the environmental reflection spectrum of the vehicle can be determined based on the first channel state information corresponding to the first wireless signal, or based on the second channel state information corresponding to the second wireless signal, or simultaneously based on the first channel state information and the second channel state information.

[0057] Furthermore, based on the first channel state information and the second channel state information, the area where the liveness detection target is located can be determined, i.e., the target detection area. For example, if there is a target in the position of the left rear seat, the reflection or slight movement of the target will cause the first channel state information and the second channel state information to change accordingly. Thus, the target can be located in the left rear seat, and the left rear seat can be used as the target detection area.

[0058] A3: Calculate the phase offset of each antenna element in the antenna array used to transmit the multiple different wireless signals relative to the target detection area.

[0059] After determining the target detection area, the target azimuth angle of each antenna element in the antenna array relative to the target detection area can be determined based on the current position of the antenna array. Then, for the target detection area, the phase offset Δφ_i of each antenna element in the antenna array can be calculated separately, as shown in the following formula:

[0060] in, For the first iThe spacing between antenna elements; The target azimuth angle; The wavelength is either the first or the second wireless signal.

[0061] A3: Based on the phase offset of each antenna element relative to the target detection area, multiple different wireless signals are alternately transmitted to control the antenna array to focus on the target detection area, thereby enhancing the signal-to-noise ratio of each signal within the target detection area.

[0062] Specifically, after determining the phase offset of each antenna element in the antenna array relative to the target detection area, the corresponding phase offset is applied to each antenna element. For example, for each antenna element, the first and second wireless signals to be transmitted are multiplied by a complex weight corresponding to a phase offset, and then the signal is transmitted through the antenna array to form a beam pointing towards the target detection area. This causes the synthesized beam to form constructive interference in the target detection area, thereby enhancing the energy of each signal in the target detection area. For example, the signal-to-noise ratio in the target detection area can be improved by 8-12 dB.

[0063] In actual implementation, the channel state information of the first and second wireless signals can be used to determine whether the target of liveness detection has moved. If the target moves, the beam direction of the antenna array can be updated by Kalman filtering. Specifically, the new position of the target is predicted by Kalman filtering, and the phase offset of each antenna element in the antenna array is updated in real time according to the new position to maintain beam focusing. If the target does not move, the current beam parameters are maintained.

[0064] After optimizing the beamforming of the antenna array to directionally enhance the wireless signal energy within the target detection area, channel state information for liveness detection can be further obtained.

[0065] S102: Construct a multi-dimensional feature vector based on the channel state information, and determine the liveness detection result inside the vehicle based on the multi-dimensional feature vector.

[0066] Specifically, constructing a multi-dimensional feature vector based on the channel state information includes the following steps: B1: Determine a first feature based on the first channel state information corresponding to the first wireless signal in the channel state information; determine a second feature based on the second channel state information corresponding to the second wireless signal in the channel state information.

[0067] Specifically, CSI is usually estimated based on the received signal and the known transmitted signal. It describes the channel frequency response that the signal experiences from transmission to reception and usually includes amplitude and phase information of multiple subcarriers.

[0068] The first wireless signal is a continuous wave signal, or CW signal. A continuous wave signal is a continuous signal with a single frequency. The CSI corresponding to a single-frequency signal is actually the response at that frequency point. The first channel state information corresponding to the CW signal is a complex number. Its data content includes amplitude A_cw and phase φ_cw. Among them, the phase is extremely sensitive to changes in distance. If there are slight movements of the human body, such as breathing and heartbeat, it will cause periodic changes in the phase.

[0069] Therefore, based on the first channel state information corresponding to the first wireless signal, the amplitude and phase can be extracted as the first feature, which can be used to characterize the presence of a liveness detection target within the vehicle.

[0070] The second wireless signal is a chirped pulse signal, which is a pulse signal whose frequency changes linearly with time, such as sweeping from 2.4 GHz to 2.41 GHz, with a coverage bandwidth of 10 MHz.

[0071] After receiving a chirped pulse signal, the receiving unit performs correlation processing with the transmitted chirped pulse signal, such as using a matched filter to obtain the channel frequency response within 10MHz, and obtains the second channel state information.

[0072] The data content of the second channel state information includes amplitude response and phase response. The amplitude response is a vector that represents the attenuation change of the channel at different frequencies; the phase response is also a vector that represents the phase change of the channel at different frequencies.

[0073] In practical implementation, a short-time Fourier transform can be performed on the second channel state information to extract the micro-Doppler frequency shift Δf and the phase change rate dφ / dt as the second feature. The second feature can be used to characterize the life characteristics of the target for liveness detection inside the vehicle.

[0074] The Short Time Fourier Transform (STFT) can convert the second channel state information in the time domain into a time-frequency representation, resulting in a time-spectrum graph. Since the micro-Doppler effect is frequency modulation caused by minute movements such as breathing and heartbeat, the time-spectrum graph can be analyzed to find periodically changing frequency patterns. For example, breathing typically involves slow modulation of 0.1-0.5 Hz, while heartbeat typically involves fast modulation of 0.8-2.0 Hz, thus obtaining micro-Doppler characteristics. For instance, breathing typically produces a micro-Doppler frequency shift of about 0.1-0.3 Hz, and heartbeat typically produces a micro-Doppler frequency shift of about 1-2 Hz, which appears as sidebands around the main frequency in the time-spectrum graph, allowing the extraction of the micro-Doppler frequency shift.

[0075] In actual implementation, the phase change rate can be calculated based on differential calculation or instantaneous frequency estimation based on STFT, and the embodiments of this application are not limited thereto.

[0076] B2: Construct a multidimensional feature vector based on the first and second features.

[0077] First, the first feature (amplitude A_cw, phase φ_cw) and the second feature (micro-Doppler frequency shift Δf, phase change rate dφ / dt) are aligned and synchronized by timestamp to construct an initial multidimensional feature vector F=[A_cw, φ_cw, Δf, dφ / dt].

[0078] Then, the initial multidimensional feature vector is denoised by wavelet transform to obtain the multidimensional feature vector. Wavelet transform can eliminate phase jumps caused by environmental reflections and retain micro-motion features related to vital signs.

[0079] After constructing a multidimensional feature vector, the liveness detection result inside the vehicle can be determined based on the multidimensional feature vector. In this embodiment, an in-vehicle edge computing network is constructed to determine the liveness detection result.

[0080] Specifically, by deploying a lightweight classification model on the vehicle side, the multidimensional feature vector is input into the vehicle's lightweight classification model. The lightweight classification model can determine and output the vehicle's liveness detection results, which include human bodies, animals, and still objects.

[0081] The lightweight classification model of the vehicle is obtained by training multiple multidimensional feature vector samples of the vehicle. The multidimensional feature vector samples are determined based on channel state information samples corresponding to the multiple different wireless signals obtained on the vehicle.

[0082] In practice, lightweight classification models can be trained based on the MobileNet-V2 neural network.

[0083] In one feasible implementation, considering that the liveness detection process for different vehicle models is strongly correlated with the vehicle model, such as the insufficient generalization of the lightweight classification model due to the difference in interior reflection between SUVs and sedans, this application embodiment also uses federated learning to train a global lightweight classification model that can be applied to multiple vehicle models in order to improve the generalization and applicability of the liveness detection method provided in this application embodiment. When actually applied to a vehicle of a certain model, it is only necessary to fine-tune and optimize the global lightweight classification model based on the data of that vehicle. Specifically, it may include the following steps: C1: Obtain training samples corresponding to different vehicle models, perform federated learning on the preset lightweight classification neural network, and obtain a global lightweight classification model.

[0084] In practical implementation, a globally lightweight classification model can be obtained based on the FedAvg algorithm. The specific process is as follows: First, initialize the model parameters of the preset lightweight classification neural network.

[0085] Then, for any vehicle model, multiple training samples of the vehicle are obtained. The training samples of any vehicle are multi-dimensional feature vector samples determined based on channel state information samples corresponding to multiple different wireless signals obtained on the vehicle. For example, the first wireless signal and the second wireless signal are alternately transmitted on the vehicle using time division multiplexing. Channel state information is obtained in scenarios where there is no target for liveness detection and in scenarios where there is a target for liveness detection, respectively. Multi-dimensional feature vector samples are constructed based on the channel state information.

[0086] Then, for any vehicle model, the preset lightweight classification neural network is independently trained based on multiple training samples of that vehicle to determine the model parameters corresponding to that vehicle.

[0087] Finally, the model parameters corresponding to multiple vehicle models are weighted and averaged to obtain the global lightweight classification model.

[0088] Suppose there are K vehicles, and the number of training samples for the k-th vehicle is... n k Let N be the total number of training samples for K vehicles. In the t-th round, the global lightweight classification model update formula is as follows:

[0089] in, Let be the model parameters for the k-th vehicle in round t.

[0090] C2: Obtain multiple multidimensional feature vector samples of the vehicle, fine-tune and train the global lightweight classification model to obtain the lightweight classification model corresponding to the vehicle.

[0091] For the current vehicle, when deploying the model on the vehicle, multiple multi-dimensional feature vector samples of the vehicle are obtained, such as 10-20 multi-dimensional feature vector samples. The global lightweight classification model is then fine-tuned and trained to obtain the lightweight classification model corresponding to the vehicle.

[0092] In one feasible implementation, after determining the liveness detection result inside the vehicle based on the multidimensional feature vector, the method further includes: When the liveness detection result is a human, a preset human detection processing strategy is executed. The human detection alarm strategy is used to send alarm prompt information and / or start the vehicle's air conditioning ventilation system. When the liveness detection result is an animal, a preset animal detection alarm strategy is executed. The animal detection alarm strategy is used to indicate that there is an animal in the vehicle to prevent safety accidents caused by leaving people or pets behind in the vehicle.

[0093] In practice, if the live detection result is a human, a high-priority warning can be triggered to avoid serious harm to the remaining personnel. If the live detection result is an animal, a low-priority warning can be selectively triggered to avoid false alarms.

[0094] For example, if the vehicle is in a turned-off and locked state, and the liveness detection result is a human body, the preset human body detection processing strategy can be set to first trigger the buzzer alarm and headlight flashing via the vehicle's CAN bus, and then send an alarm prompt message to the driver, including remotely sending an alarm prompt message to the driver via a mobile APP or SMS; it can also be further linked with the vehicle's OBD or TSP platform to automatically dial emergency calls and start the air conditioning ventilation system.

[0095] In practical implementation, in scenarios where liveness detection is continuous, a delayed reminder mechanism can be set to avoid frequent alarms. For example, the time interval between two alarm reminders can be set. When the liveness detection result is a human, the time interval can be set to 5 minutes, and when the liveness detection result is a pet, the time interval can be set to 10 minutes. The delayed reminder mechanism can be set according to the actual application requirements, and this application embodiment does not impose any restrictions.

[0096] The liveness detection method provided in this application can achieve in-vehicle liveness detection at low cost, and has at least the following beneficial effects: 1. By using channel multiplexing technology, such as time division multiplexing (TDM), the transmission cycles of different wireless signals are divided in a single frequency band. For example, continuous wave (CW) signals and chirped pulse signals are transmitted sequentially. Multimodal channel state information corresponding to different wireless signals is collected. Based on the multimodal channel state information, a more dimensional multi-feature vector is constructed to determine the liveness detection result. Without increasing hardware costs, the feature dimension is expanded, which can improve the accuracy of liveness detection. It also has strong anti-interference ability and is not easily affected by external factors.

[0097] 2. Integrating a reconfigurable antenna array into the wireless communication module enables beamforming optimization of the antenna array, directional enhancement of signal energy, reduction of power consumption, and enhancement of spatial resolution, thereby further improving the accuracy and reliability of liveness detection.

[0098] 3. Construct an in-vehicle edge computing network and train it through federated learning to obtain a global lightweight classification model that can be used across vehicle models. This can solve the problem of large differences in detection performance between different vehicle models, such as the problem of insufficient model generalization caused by differences in interior reflections between SUVs and sedans. At the same time, it also achieves data privacy protection for vehicles and allows for real-time model iteration and optimization.

[0099] Reference Figure 3 The diagram illustrates a functional block diagram of a liveness detection device according to an embodiment of this application. The device includes: The channel state information acquisition module 100 is used to send multiple different wireless signals inside the vehicle and acquire the channel state information corresponding to the multiple different wireless signals. The liveness detection result determination module 200 is used to construct a multi-dimensional feature vector based on the channel state information, and determine the liveness detection result inside the vehicle based on the multi-dimensional feature vector.

[0100] Optionally, the channel state information acquisition module includes: The transmitting unit is used to transmit a first wireless signal and a second wireless signal in the vehicle based on a preset channel multiplexing method, wherein the channel multiplexing method includes any one of time division multiplexing, frequency division multiplexing, and code division multiplexing. The first wireless signal is a continuous wave signal, and the second wireless signal is a chirped pulse signal.

[0101] Optionally, the liveness detection result determination module includes a multi-dimensional feature vector construction unit, used for: Based on the first channel state information corresponding to the first wireless signal in the channel state information, a first feature is determined, and the first feature is used to characterize the presence of a liveness detection target in the vehicle. Based on the second channel state information corresponding to the second wireless signal in the channel state information, a second feature is determined, which is used to characterize the life characteristics of the target for liveness detection inside the vehicle. A multidimensional feature vector is constructed based on the first feature and the second feature.

[0102] Optionally, the first feature includes amplitude and phase; the second feature includes microDoppler frequency shift and phase change rate.

[0103] Optionally, the multidimensional feature vector construction unit is further used for: The first feature and the second feature are aligned to construct an initial multidimensional feature vector; The initial multidimensional feature vector is denoised by wavelet transform to obtain the multidimensional feature vector.

[0104] Optionally, the liveness detection result determination module includes a liveness detection result determination unit, used for: The multidimensional feature vector is input into the lightweight classification model of the vehicle to determine the liveness detection result of the vehicle, which includes human body, animal and still object; The lightweight classification model of the vehicle is obtained by training multiple multidimensional feature vector samples of the vehicle. The multidimensional feature vector samples are determined based on channel state information samples corresponding to the multiple different wireless signals obtained on the vehicle.

[0105] Optionally, the device further includes: The model training module is used to obtain training samples corresponding to different vehicle models, perform federated learning on the preset lightweight classification neural network, and obtain a global lightweight classification model. The training samples of any vehicle model are multi-dimensional feature vector samples determined based on channel state information samples corresponding to multiple different wireless signals obtained on the vehicle. The model fine-tuning module is used to obtain multiple multi-dimensional feature vector samples of the vehicle, fine-tune and train the global lightweight classification model, and obtain the lightweight classification model corresponding to the vehicle.

[0106] Optionally, the model training module is used for: Initialize the preset model parameters of the lightweight classification neural network; For any vehicle model, multiple training samples of that vehicle are obtained, and the preset lightweight classification neural network is trained independently to determine the model parameters corresponding to that vehicle. The global lightweight classification model is obtained by weighted averaging of the model parameters corresponding to multiple vehicle models.

[0107] Optionally, the device further includes a beamforming module for: Obtain the initial channel state information corresponding to the multiple different wireless signals; Based on the initial channel state information, the environmental reflection map of the vehicle is extracted, and the target detection area inside the vehicle is determined. The target detection area is the area where a liveness detection target exists. Calculate the phase offset of each antenna element in the antenna array used to transmit the multiple different wireless signals relative to the target detection area; Based on the phase offset of each antenna element relative to the target detection area, multiple different wireless signals are alternately transmitted to control the antenna array to focus on the target detection area, thereby enhancing the signal-to-noise ratio of each signal within the target detection area.

[0108] Optionally, the device further includes a warning module for: When the liveness detection result is a human body, a preset human body detection processing strategy is executed. The human body detection alarm strategy is used to send alarm prompt information and / or start the vehicle's air conditioning ventilation system. When the liveness detection result is an animal, a preset animal detection alarm strategy is executed, which is used to notify that an animal is present in the vehicle.

[0109] The liveness detection device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0110] Reference Figure 4 The diagram illustrates an electronic device provided in an embodiment of this application. The electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the liveness detection method embodiment described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0111] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0112] Reference Figure 5 The diagram illustrates a readable storage medium provided in an embodiment of this application. The readable storage medium stores a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described liveness detection method embodiment and achieve the same technical effect. To avoid repetition, the details will not be repeated here.

[0113] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0114] Reference Figure 6 The diagram illustrates a computer program product provided in an embodiment of this application, including a computer program / instruction. When the computer program / instruction is executed by a processor, it implements the various processes of the above-described liveness detection method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0115] This application also provides a vehicle for performing the various processes of the above-described liveness detection method embodiments, or includes the liveness detection device described in this embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0116] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0118] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. The description of the embodiments above is only for the purpose of helping to understand the method and core idea of ​​this application. Those skilled in the art can make many forms under the guidance of this application without departing from the spirit and scope of protection of the claims, and all of these are within the protection scope of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting liveness, characterized in that, The method includes: The system transmits multiple different wireless signals within the vehicle and acquires the channel state information corresponding to the multiple different wireless signals. A multidimensional feature vector is constructed based on the channel state information, and the liveness detection result inside the vehicle is determined based on the multidimensional feature vector.

2. The method according to claim 1, characterized in that, Sending multiple different wireless signals inside the vehicle, including: The first wireless signal and the second wireless signal are transmitted in the vehicle based on a preset channel multiplexing method, wherein the channel multiplexing method includes any one of time division multiplexing, frequency division multiplexing and code division multiplexing. The first wireless signal is a continuous wave signal, and the second wireless signal is a chirped pulse signal.

3. The method according to claim 2, characterized in that, A multi-dimensional feature vector is constructed based on the channel state information, including: Based on the first channel state information corresponding to the first wireless signal in the channel state information, a first feature is determined, and the first feature is used to characterize the presence of a liveness detection target in the vehicle. Based on the second channel state information corresponding to the second wireless signal in the channel state information, a second feature is determined, which is used to characterize the life characteristics of the target for liveness detection inside the vehicle. A multidimensional feature vector is constructed based on the first feature and the second feature.

4. The method according to claim 3, characterized in that, The first feature includes amplitude and phase; the second feature includes microDoppler frequency shift and phase change rate.

5. The method according to claim 3 or 4, characterized in that, Based on the first feature and the second feature, a multidimensional feature vector is constructed, including: The first feature and the second feature are aligned to construct an initial multidimensional feature vector; The initial multidimensional feature vector is denoised by wavelet transform to obtain the multidimensional feature vector.

6. The method according to claim 1, characterized in that, Determining the liveness detection result inside the vehicle based on the multidimensional feature vector includes: The multidimensional feature vector is input into the lightweight classification model of the vehicle to determine the liveness detection result of the vehicle, which includes human body, animal and still object; The lightweight classification model of the vehicle is obtained by training multiple multidimensional feature vector samples of the vehicle. The multidimensional feature vector samples are determined based on channel state information samples corresponding to the multiple different wireless signals obtained on the vehicle.

7. The method according to claim 6, characterized in that, The method further includes: The training samples corresponding to different vehicle models are obtained, and federated learning is performed on the preset lightweight classification neural network to obtain a global lightweight classification model. The training sample of any vehicle model is a multi-dimensional feature vector sample determined based on the channel state information samples corresponding to multiple different wireless signals obtained on the vehicle. Multiple multidimensional feature vector samples of the vehicle are obtained, and the global lightweight classification model is fine-tuned and trained to obtain the lightweight classification model corresponding to the vehicle.

8. The method according to claim 7, characterized in that, Obtain training samples corresponding to different vehicle models, perform federated learning on a pre-defined lightweight classification neural network, and obtain a global lightweight classification model, including: Initialize the preset model parameters of the lightweight classification neural network; For any vehicle model, multiple training samples of that vehicle are obtained, and the preset lightweight classification neural network is trained independently to determine the model parameters corresponding to that vehicle. The global lightweight classification model is obtained by weighted averaging of the model parameters corresponding to multiple vehicle models.

9. The method according to claim 1, characterized in that, Before obtaining the channel state information corresponding to the multiple different wireless signals, the method further includes: Obtain the initial channel state information corresponding to the multiple different wireless signals; Based on the initial channel state information, the environmental reflection map of the vehicle is extracted, and the target detection area inside the vehicle is determined. The target detection area is the area where a liveness detection target exists. Calculate the phase offset of each antenna element in the antenna array used to transmit the multiple different wireless signals relative to the target detection area; Based on the phase offset of each antenna element relative to the target detection area, multiple different wireless signals are alternately transmitted to control the antenna array to focus on the target detection area, thereby enhancing the signal-to-noise ratio of each signal within the target detection area.

10. The method according to claim 1, characterized in that, After determining the liveness detection result inside the vehicle based on the multidimensional feature vector, the method further includes: When the liveness detection result is a human body, a preset human body detection processing strategy is executed. The human body detection alarm strategy is used to send alarm prompt information and / or start the vehicle's air conditioning ventilation system. When the liveness detection result is an animal, a preset animal detection alarm strategy is executed, which is used to notify that an animal is present in the vehicle.

11. A liveness detection device, characterized in that, The device includes: The channel state information acquisition module is used to send multiple different wireless signals inside the vehicle and acquire the channel state information corresponding to the multiple different wireless signals. The liveness detection result determination module is used to construct a multi-dimensional feature vector based on the channel state information, and determine the liveness detection result inside the vehicle based on the multi-dimensional feature vector.

12. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the liveness detection method as described in any one of claims 1-10.

13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the liveness detection method as described in any one of claims 1-10.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the liveness detection method according to any one of claims 1-10.

15. A vehicle, characterized in that, The vehicle is used to perform the liveness detection method according to any one of claims 1-10, or includes the liveness detection device according to claim 11.

Images