Vehicle safe starting method based on multiple modes, electronic equipment and storage medium

Abstract

The invention discloses a vehicle safe starting method based on multiple modes, electronic equipment and a storage medium. The method comprises the steps of obtaining a heartbeat signal, a face image and a fingerprint image in a preset area where a vehicle is located; fusing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result; performing matching processing based on the heartbeat signal to obtain a matching result; and finally, determining whether a target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on a feature fusion result and a matching result. Therefore, the starting safety of the vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a vehicle safety starting method based on multi-modality, an electronic device and a storage medium. Background Art

[0002] In recent years, with the rapid development of the automobile industry, smart cars have become a new wave of social digital transformation. One of the biggest concerns about smart cars is whether the vehicle is safe enough. On the one hand, vehicle safety is reflected in driving safety. Ensuring driving safety during the driving of smart vehicles can protect life and vehicle property. On the other hand, it is reflected in the safety of vehicle opening. Only when the authorized object of the vehicle is guaranteed to open the vehicle can the vehicle property be better protected.

[0003] At present, the method of opening a vehicle is usually to use a wireless remote control key, NFC (Near Field Communication) card, RFID (Radio Frequency IDentification) card or mechanical key to open the door and start the vehicle. However, unauthorized persons can also start the vehicle after obtaining the key or other unlocking tools. Therefore, this method of starting the vehicle is less secure. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a vehicle safety starting method, electronic device and storage medium based on multi-modality, which can improve the safety of vehicle starting.

[0005] In order to solve the above technical problems, the present application provides a vehicle safety starting method based on multi-modality.

[0006] In one embodiment, a multi-modal vehicle safety starting method is applied to a vehicle, the method comprising:

[0007] Acquire a heartbeat signal, a facial image, and a fingerprint image within a preset area where the vehicle is located;

[0008] Performing fusion processing on the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result;

[0009] Perform matching processing based on the heartbeat signal to obtain a matching result;

[0010] Based on the feature fusion result and the matching result, it is determined whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle.

[0011] In one embodiment, the feature fusion result includes a distance score between the facial feature and / or fingerprint feature and a corresponding preset feature in a preset feature database, and the step of fusing the facial feature of the facial image and the fingerprint feature of the fingerprint image to obtain the feature fusion result includes:

[0012] Normalizing the facial features of the facial image and the fingerprint features of the fingerprint image respectively to obtain normalized facial features and normalized fingerprint features;

[0013] Performing weighted fusion processing on the normalized facial features and the normalized fingerprint features to obtain a feature fusion vector;

[0014] The feature fusion vector is subjected to distance classification processing by using the nearest neighbor distance classification technology to obtain a distance score.

[0015] In one embodiment, the facial features of the facial image include feature vectors of the facial image. Before the step of normalizing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain normalized facial features and normalized fingerprint features, the method includes:

[0016] Performing denoising on the facial image to obtain a denoised facial image;

[0017] A two-dimensional principal component analysis is performed on the facial image after the denoising process to obtain a feature vector of the facial image.

[0018] In one embodiment, the step of determining whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the feature fusion result and the matching result includes:

[0019] Normalizing the feature fusion result and the matching result respectively to obtain a normalized feature fusion result and a normalized matching result;

[0020] Performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result;

[0021] In response to the discrimination result being consistent with a preset discrimination threshold, it is determined that the target object to which the heartbeat signal belongs is a safe startup object for the vehicle.

[0022] In one embodiment, the step of performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain the discrimination result includes:

[0023] Obtaining a first probability density of the feature fusion result after the normalization processing;

[0024] Obtaining a second probability density of the normalized matching result;

[0025] Fitting the first probability density and the second probability density respectively to obtain a fitted first probability density and a fitted second probability density;

[0026] The Bayesian decision technology is used to discriminate the fitted first probability density and the fitted second probability density to obtain a discrimination result.

[0027] In one embodiment, the vehicle includes an image acquisition device and a fingerprint acquisition device, and the step of acquiring a heartbeat signal, a facial image, and a fingerprint image in a preset area where the vehicle is located includes:

[0028] Continuously detecting the heartbeat signal within the preset area where the vehicle is located;

[0029] In response to detecting the presence of a heartbeat signal in the preset area where the vehicle is located, controlling the image acquisition device and the fingerprint acquisition device to turn on, and controlling the image acquisition device and the fingerprint acquisition device to perform continuous acquisition;

[0030] In response to the fingerprint collection device collecting a fingerprint image, a facial image of a target object corresponding to the fingerprint image is collected.

[0031] In one embodiment, the matching result includes a matching score between the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library, and the step of performing matching processing based on the heartbeat signal to obtain the matching result includes:

[0032] Calculating the similarity between the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library to obtain a similarity;

[0033] The maximum similarity is determined as the matching score.

[0034] In one embodiment, the step of performing matching processing based on the heartbeat signal to obtain a matching result includes:

[0035] Performing feature processing on the heartbeat signal to obtain heartbeat features;

[0036] The heartbeat feature is input into a preset matching model to obtain a matching result.

[0037] In order to solve the above technical problems, the present application provides an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above multi-modal based vehicle safety startup method.

[0038] In order to solve the above technical problems, the present application provides a computer-readable storage medium, including: storing program data, and the program data is used to implement the above multi-modal based vehicle safety startup method when executed by a processor.

[0039] The above scheme obtains the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located; then fuses the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result; performs matching processing based on the heartbeat signal to obtain a matching result; and finally determines whether the target object to which the heartbeat signal belongs is the safe startup object of the vehicle based on the feature fusion result and the matching result. Thus, the safe startup object of the vehicle is determined by the feature fusion result between the facial image and the fingerprint image and the matching result of the heartbeat signal, while taking into account the facial image, fingerprint image and heartbeat signal, realizing multiple identity authentications of the target object and improving the safety of vehicle startup. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0041] Figure 1 is a flow chart of an exemplary embodiment of a multi-modal vehicle safety starting method shown in the present application;

[0042] Figure 2 is a schematic diagram of the network structure of the preset matching model shown in this application;

[0043] Figure 3 A flowchart of another exemplary embodiment of a multi-modal vehicle safety starting method according to the present application;

[0044] Figure 4 A flowchart of another exemplary embodiment of a multi-modal vehicle safety starting method according to the present application;

[0045] Figure 5 is a block diagram of a multi-modal vehicle safety starting device shown in an exemplary embodiment of the present application;

[0046] Figure 6 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0047] Figure 7 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be appreciated that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present application.

[0049] First of all, it should be noted that in recent years, with the rapid development of the automobile industry, smart cars have become a new wave of social digital transformation. One of the biggest concerns about smart cars is whether the vehicle is safe enough. On the one hand, vehicle safety is reflected in driving safety. Ensuring driving safety during the driving of smart vehicles can protect life and vehicle property. On the other hand, it is reflected in the safety of vehicle opening. Only by ensuring that the authorized object of the vehicle can open the vehicle can the vehicle property be better protected.

[0050] At present, the method of opening a vehicle is usually to use a wireless remote control key, NFC (Near Field Communication) card, RFID (Radio Frequency IDentification) card or mechanical key to open the door and start the vehicle. However, unauthorized persons can also start the vehicle after obtaining the key or other unlocking tools. Therefore, this method of starting the vehicle is less secure.

[0051] Based on this, the present application provides a multi-modal vehicle safety start method, electronic device and storage medium. Figure 1 , Figure 1 It is a flowchart of an exemplary embodiment of a multi-modal vehicle safety starting method shown in the present application.

[0052] The execution subject of the multimodal vehicle safety startup method may be a terminal device or a server or other processing device, wherein the terminal device may be a computer, a mobile device, a terminal, a computing device, a vehicle-mounted device, etc. The execution subject of the multimodal vehicle safety startup method may also be a multimodal vehicle safety startup device. In some possible implementations, the multimodal vehicle safety startup method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0053] Specifically, a multi-modal vehicle safety starting method of this embodiment is applied to a vehicle, and the multi-modal vehicle safety starting method includes the following steps:

[0054] Step S110: Acquire the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located.

[0055] The preset area where the vehicle is located can be a circular area with the vehicle as the center and a preset length as the radius, or it can be a circular area with the vehicle as the center and the farthest distance at which the radar monitoring equipment, image acquisition equipment and fingerprint acquisition equipment on the vehicle can collect information as the boundary.

[0056] The heartbeat signal is a signal that characterizes the heartbeat in the subject. Specifically, the multimodal vehicle safety starting device includes a radar monitoring device. The multimodal vehicle safety starting device uses the radar monitoring device to transmit a signal to the outside, and receives an echo signal fed back by the heart in the subject, and mixes the transmitted signal and the echo signal to obtain a heartbeat signal. Among them, the multimodal vehicle safety starting device can use a mixer to mix the transmitted signal and the echo signal to obtain a heartbeat signal. The radar monitoring device can be a millimeter wave radar.

[0057] In some embodiments, a millimeter-wave radar is installed on the B-pillar of the vehicle, and a multi-modal vehicle safety starting device transmits a signal to the outside in real time through the millimeter-wave radar. When an object appears in a preset area where the vehicle is located, the signal emitted by the millimeter-wave radar passes through the heart of the object and reflects an echo signal. After the millimeter-wave radar receives the echo signal, a mixer is used to mix the emitted signal with the echo signal to obtain a heartbeat signal.

[0058] A facial image refers to an image containing the face of an object. Specifically, the multimodal vehicle safety starting device includes an image acquisition device, and the multimodal vehicle safety starting device can use the image acquisition device to acquire a facial image of an object in a preset area where the vehicle is located to obtain a facial image. The image acquisition device can be a camera.

[0059] A fingerprint image refers to an image containing the fingerprint of an object. Specifically, a multimodal vehicle safety start device includes a fingerprint acquisition device, and the multimodal vehicle safety start device can use the fingerprint acquisition device to acquire the fingerprint of an object in a preset area where the vehicle is located to obtain a fingerprint image. The fingerprint acquisition device can be a screen fingerprint detection and recognition device. It should be noted that the fingerprint acquisition device can be installed on the handle of a vehicle door.

[0060] The multi-modal vehicle safety starting device obtains the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located. As an example, the multi-modal vehicle safety starting device continuously detects the heartbeat signal in the preset area where the vehicle is located; in response to detecting the presence of a heartbeat signal in the preset area where the vehicle is located, the image acquisition device and the fingerprint acquisition device are controlled to turn on; and the facial image and fingerprint image are respectively acquired by the image acquisition device and the fingerprint acquisition device.

[0061] As another example, the vehicle includes an image acquisition device and a fingerprint acquisition device. The steps of acquiring the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located based on the multimodal vehicle safety start device include: continuously detecting the heartbeat signal in the preset area where the vehicle is located; in response to detecting the presence of a heartbeat signal in the preset area where the vehicle is located, controlling the image acquisition device and the fingerprint acquisition device to turn on, and controlling the image acquisition device and the fingerprint acquisition device to continuously acquire; in response to the fingerprint acquisition device acquiring a fingerprint image, acquiring the facial image of the target object corresponding to the fingerprint image.

[0062] Step S120: fusing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result.

[0063] Facial features are features that characterize facial information in a facial image. Specifically, the multimodal vehicle safety starting device uses 2DPCA (Principal Components Analysis) dimensionality reduction technology to extract facial features in a facial image to obtain facial features.

[0064] Fingerprint features are features that characterize fingerprint information in a fingerprint image. Specifically, the multi-modal vehicle safety starting device uses 2DPCA (Principal Components Analysis) dimensionality reduction technology to extract fingerprint features in a fingerprint image to obtain fingerprint features.

[0065] Fusion processing refers to the process of integrating facial features and fingerprint features into one feature.

[0066] The multimodal vehicle safety starting device fuses the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result. Specifically, the multimodal vehicle safety starting device performs weighted fusion on the facial features and the fingerprint features according to preset weights to obtain a feature fusion result.

[0067] Step S130: Perform matching processing based on the heartbeat signal to obtain a matching result.

[0068] The multimodal vehicle safety starting device performs matching processing based on the heartbeat signal to obtain a matching result. Specifically, the matching result includes a matching score between the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library. The multimodal vehicle safety starting device performs similarity calculation on the heartbeat signal and the preset heartbeat signal in the preset heartbeat signal library to obtain the similarity; and the maximum similarity is determined as the matching score.

[0069] Step S140: Determine whether the target object to which the heartbeat signal belongs is a safe starting object for the vehicle based on the feature fusion result and the matching result.

[0070] A safe start object is an object that authorizes the vehicle to start.

[0071] The multimodal vehicle safety start device determines whether the target object to which the heartbeat signal belongs is the vehicle's safety start object based on the feature fusion result and the matching result. Specifically, the multimodal vehicle safety start device compares the feature fusion result with the first preset score threshold to obtain a first comparison result; compares the matching result with the second preset score threshold to obtain a second comparison result; in response to the first comparison result indicating that the feature fusion result is greater than the first preset score threshold, and the second comparison result indicating that the matching result is greater than the second preset score threshold, it is determined that the target object to which the heartbeat signal belongs is the vehicle's safety start object; otherwise, it is determined that the target object to which the heartbeat signal belongs is not the vehicle's safety start object.

[0072] It can be seen that by acquiring the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located; then fusing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain the feature fusion result; matching processing is performed based on the heartbeat signal to obtain the matching result; finally, based on the feature fusion result and the matching result, it is determined whether the target object to which the heartbeat signal belongs is the safe startup object of the vehicle. Therefore, the safe startup object of the vehicle is determined by the feature fusion result between the facial image and the fingerprint image and the matching result of the heartbeat signal, while taking into account the facial image, fingerprint image and heartbeat signal, realizing multiple identity authentications of the target object, and improving the safety of vehicle startup.

[0073] In one embodiment, the feature fusion result includes the distance score between the facial features and / or fingerprint features and the corresponding preset features in the preset feature database, and the steps of fusing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain the feature fusion result include: normalizing the facial features of the facial image and the fingerprint features of the fingerprint image respectively to obtain the normalized facial features and the normalized fingerprint features; performing weighted fusion processing on the normalized facial features and the normalized fingerprint features to obtain a feature fusion vector; and performing distance classification processing on the feature fusion vector using the nearest neighbor distance classification technology to obtain a distance score.

[0074] Specifically, the multimodal vehicle safety starting device normalizes the facial features of the facial image and the fingerprint features of the fingerprint image, respectively, to obtain the normalized facial features and the normalized fingerprint features. As an example, the multimodal vehicle safety starting device can use the Min-Max normalization technology to normalize the facial features of the facial image and the fingerprint features of the fingerprint image. The Min-Max normalization technology includes the formula (y`=(xx min ) / (x max -x min ))Convert facial features or fingerprint features to the range of [0, 1], where y' is the normalized feature, which can be the normalized facial feature or fingerprint feature, x is the original feature, which can be the facial feature or fingerprint feature, x min and x max are the minimum feature and the maximum feature in the original features, respectively. As another example, the multimodal vehicle safety starting device can use a logical function or a sigmoid function to normalize the facial features of the facial image and the fingerprint features of the fingerprint image. Since the face and the fingerprint are two different biological modalities, the feature vectors of the face and the fingerprint can be normalized to a consistent interval with a mean of 0 and a variance of 1 through normalization, so that the facial features and the fingerprint features can be converted into standard forms respectively, so that the feature vectors of the two are in the same range, thereby enabling the subsequent feature fusion of the facial features and the fingerprint features.

[0075] The multimodal vehicle safety starting device performs weighted fusion processing on the normalized facial features and the normalized fingerprint features to obtain a feature fusion vector. Specifically, the multimodal vehicle safety starting device uses a weighted fusion method based on the nearest neighbor distance to perform weighted fusion on the normalized facial features and the normalized fingerprint features.

[0076] In one embodiment, the weighted fusion method based on the nearest neighbor distance includes the following steps: the first step is to use the nearest neighbor distance to respectively calculate the distance between the normalized facial features and the normalized fingerprint features and each training sample in the corresponding preset feature database, and obtain the facial feature set d = [d1, d2, ... d N ], N represents the number of training samples, d N represents the distance between the normalized facial features and the Nth training sample, the fingerprint feature set P = [P1, P2, ... P N ], N represents the number of training samples, P N represents the distance between the normalized fingerprint feature and the Nth training sample. N (i=1,2,…,N) Arrange in descending order and determine the shortest distance d x , P j (j=1,2,…,N) are arranged in descending order and the shortest distance P is determined x The third step is to calculate the distance mean of facial features and fingerprint features respectively; the distance mean of facial features is The mean distance of fingerprint features is Step 4: Determine the ratio of the distance mean and the shortest distance. The ratio of facial features is t1, and the ratio of fingerprint features is t2. Step 5: Determine the weight w, w = t2 / t1. Step 6: Determine the feature fusion vector y = [(1-w)y1, w*y2]. y1 represents the normalized facial features, y2 represents the normalized fingerprint features, and w represents the weight corresponding to the facial features.

[0077] Among them, the mean distance of facial features satisfies the following formula:

[0078]

[0079] Among them, the distance mean of the fingerprint feature satisfies the following formula:

[0080]

[0081] Among them, the ratio of facial features satisfies the following formula:

[0082]

[0083] Among them, the ratio of fingerprint features satisfies the following formula:

[0084]

[0085] The multi-modal vehicle safety starting device uses the nearest neighbor distance classification technology to perform distance classification processing on the feature fusion vector to obtain the distance score.

[0086] In one embodiment, the nearest neighbor distance classification technology includes: based on the shortest Euclidean distance as a judgment criterion, the distance between the feature fusion vector and each vector sample in the fusion vector training set is calculated in the search space, that is, d(y3,y4)=‖y3-y4‖2, ‖*‖2 is expressed as the L2 norm, y3 represents the vector sample, and y4 represents the feature fusion vector. According to the calculated distance, the K vector samples with the smallest and closest distance in the fusion vector training set are determined as the nearest neighbor sample set. From the M categories to which the K nearest neighbor sample sets belong, the category with the largest number of nearest neighbors to the feature fusion vector is determined by the majority voting decision rule, and the category with the largest number of nearest neighbors to the feature fusion vector is the category to which the feature fusion vector belongs, that is, the object to which the feature fusion vector belongs.

[0087] For example, the fusion vector training set includes vector samples corresponding to 10 facial images of different states of multiple objects. The Euclidean distance between the feature fusion vector and the vector samples corresponding to the 10 facial images of different states is calculated, and the K vector samples with the smallest and closest distance are used as the nearest neighbor sample set. For example, if the Euclidean distance between the feature fusion vector and the vector samples corresponding to the 4 facial images of the first object, the Euclidean distance between the vector samples corresponding to the 3 facial images of the second object, and the Euclidean distance between the vector samples corresponding to the 2 facial images of the third object are equal or the difference between them is within the error range, then the set of vector samples corresponding to these facial images is the nearest neighbor sample set. According to the majority voting method, the category to which the largest number of vector samples in the nearest neighbor sample set belongs is determined as the category to which the feature fusion vector belongs, that is, the object to which the feature fusion vector belongs.

[0088] In one embodiment, the facial features of the facial image include a feature vector of the facial image. Before the steps of normalizing the facial features of the facial image and the fingerprint features of the fingerprint image respectively to obtain the normalized facial features and the normalized fingerprint features, the method includes: denoising the facial image to obtain the denoised facial image; performing two-dimensional principal component analysis on the denoised facial image to obtain the feature vector of the facial image.

[0089] The multimodal vehicle safety starting device performs denoising on the facial image to obtain the denoised facial image. Specifically, the multimodal vehicle safety starting device uses the AdaBoost algorithm based on Haar-like features to process the facial image to obtain the denoised facial image. For example, the multimodal vehicle safety starting device performs preprocessing such as size normalization, filtering denoising, and cropping on the facial image; the preprocessed facial image is input into a preset classifier to obtain the denoised facial image.

[0090] Among them, the preset classifier includes a cascade classifier formed by cascading multiple weak classifiers and multiple strong classifiers. The cascade classifier is formed by cascading multiple strong classifiers, thereby improving the accuracy and efficiency of detection. In one embodiment, the preset classifier scans the facial image through a sliding window, and applies a strong classifier to each window position for classification judgment to obtain a classification result; the face area in the facial image is determined according to the classification result and the window position, and a denoised facial image is obtained. Therefore, the facial image is processed by the classifier, and background factors such as the environment in the facial image are removed, so as to obtain a clearer and more stable facial image, so that the face of the object can be recognized in the facial image under a complex background, reducing later interference.

[0091] The training method of the preset classifier includes: constructing a positive sample set and a negative sample set, the positive sample set includes a variety of facial image samples containing faces, and the negative sample set includes a plurality of environmental image samples that do not contain faces; the image samples in the positive sample set and the negative sample set are preprocessed by size normalization, filtering and denoising, cropping, etc. respectively to meet the input requirements of the classifier. The multimodal vehicle safety starting device uses the Haar-like features of the image samples as the input of the weak classifier to train the weak classifier. The multimodal vehicle safety starting device trains the strong classifier through the AdaBoost algorithm. Specifically, during the training process, each weak classifier is weighted for the samples that were misclassified in the previous round, and the performance of the classifier is gradually improved through multiple rounds of iterations.

[0092] Among them, Haar-like features refer to simple rectangular feature templates, which are used to describe local grayscale changes in images. Haar-like features can effectively capture the brightness differences of parts such as eyes, nose, mouth, etc. in the image. By calculating the feature value of each feature of the image sample and training the weak classifier based on the feature value, the accuracy of classification can be improved.

[0093] The multimodal vehicle safety starting device performs a two-dimensional principal component analysis on the denoised facial image to obtain a feature vector of the facial image. Specifically, the multimodal vehicle safety starting device uses a 2DPCA dimensionality reduction technology to perform a dimensionality reduction analysis on the denoised facial image to obtain a feature vector of the facial image.

[0094] In one embodiment, the 2DPCA dimensionality reduction technology can extract the most representative low-dimensional feature vectors from the original high-dimensional image data to facilitate subsequent feature fusion, classification, recognition and other tasks. These low-dimensional feature vectors can retain the main information of the original image while removing noise and unimportant features, thereby improving the performance and robustness of the recognition system.

[0095] The 2DPCA dimensionality reduction technology performs dimensionality reduction analysis on images in the following manner: the first step is to construct the training image samples in the preset training set into a two-dimensional image matrix, wherein the number of rows and columns of the two-dimensional image matrix respectively corresponds to the sum of the height and the sum of the width of the spliced ​​multiple training image samples. The second step is to calculate the mean matrix according to the training image samples in the preset training set, and each element corresponds to the average value of the pixel value of a single training image sample at the position. The third step is to calculate the two-dimensional covariance matrix based on the training image samples in the preset training set and the mean matrix, and the two-dimensional covariance matrix represents the covariance relationship between the pixel values ​​at different positions in the training image samples. The fourth step is to perform SVD (Single Value Decomposition) decomposition on the covariance matrix to obtain the eigenvalue and the corresponding eigenvector. Among them, the size of the eigenvalue reflects the importance of the corresponding eigenvector. The eigenvector corresponding to the eigenvalue greater than the preset feature threshold is used as the principal component to form the feature space. As a result, the feature space contains the main features of the image. The fifth step is to project the training image samples in the preset training set onto the feature space to obtain the projected feature matrix. The projection feature matrix retains the key feature information of each image, reduces the computational complexity and feature dimension, and facilitates subsequent feature fusion, classification and recognition. Finally, the facial image after denoising or the fingerprint image after enhancement is input into the corresponding projection feature matrix to obtain the corresponding feature vector. Therefore, by extracting the principal component dimension reduction features of the facial image and the fingerprint image, the extracted facial features and fingerprint features are both representative. The facial features and fingerprint features extracted by 2DPCA have complementary characteristics. When the facial features and fingerprint features are fused at the feature layer, more information can be obtained to improve the accuracy and stability of biometric recognition.

[0096] In one embodiment, the fingerprint features of the fingerprint image include a feature vector of the fingerprint image. Before the steps of normalizing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain the normalized facial features and the normalized fingerprint features, the method includes: enhancing the fingerprint image to obtain the enhanced fingerprint image; performing two-dimensional principal component analysis on the enhanced fingerprint image to obtain the feature vector of the fingerprint image.

[0097] The multi-modal vehicle safety starting device performs enhancement processing on the fingerprint image to obtain the enhanced fingerprint image. Specifically, the multi-modal vehicle safety starting device can use a 2D Gabor filter to enhance the fingerprint image to obtain the enhanced fingerprint image.

[0098] In one embodiment, first, a multimodal vehicle safety start device can obtain multiple fingerprint images in different states through a fingerprint reading device on an integrated screen of the vehicle, and preprocess the fingerprint image to obtain a preprocessed fingerprint image. Wherein, the preprocessing includes steps such as grayscale equalization, denoising, and binarization, thereby adjusting the contrast and grayscale of the preprocessed fingerprint image to a suitable range to reduce the image quality difference. Then, by analyzing the direction field of the preprocessed fingerprint image, the severity of the directional change of the directional entropy is used to determine the sub-block corresponding to the maximum directional entropy; the center of the sub-block corresponding to the maximum directional entropy is used as the center point of the fingerprint feature effective area to obtain the fingerprint feature effective area. Thus, the extraction of the fingerprint feature effective area (ROC, Receiver Operating Characteristic, subject working characteristic) is achieved. Then the effective area of ​​fingerprint features is divided into blocks, and the local spectrum information of each block is extracted by windowed Fourier transform. The transformed spectrum in the local spectrum information is filtered by Garbor filter, and the filtered local spectrum information is restored to spatial image information by inverse Fourier transform to obtain a preliminary enhanced fingerprint image, thereby extracting ridges from the filtered block images and combining them to obtain an enhanced image. Finally, the preliminary enhanced fingerprint image is subjected to median filtering to eliminate speckle noise and obtain an enhanced fingerprint image. As a result, the fingerprint ridge texture in the enhanced fingerprint image is clearer, thereby providing a high-quality fingerprint image for subsequent effective extraction of fingerprint features.

[0099] Since fingerprints contain rich texture information, noise is usually introduced during the fingerprint image acquisition process and the finger placement position may also be offset. In this embodiment, the fingerprint image is preprocessed to ensure the reliability of fingerprint feature extraction. The preprocessed fingerprint image is filtered using a 2D Gabor filter, which can take into account the direction and frequency selection characteristics, thereby enhancing the local texture features of the fingerprint image from multiple scales and directions.

[0100] The multimodal vehicle safety starting device performs a two-dimensional principal component analysis on the enhanced fingerprint image to obtain a feature vector of the fingerprint image. Specifically, the multimodal vehicle safety starting device uses the above-mentioned 2DPCA dimensionality reduction technology to perform a two-dimensional principal component analysis on the enhanced fingerprint image to obtain a feature vector of the fingerprint image.

[0101] The multi-modal vehicle safety starting device performs matching processing based on the heartbeat signal to obtain the matching result, including: performing feature processing on the heartbeat signal to obtain the heartbeat feature; inputting the heartbeat feature into a preset matching model to obtain the matching result.

[0102] The vehicle safety starting device based on multi-modality performs feature processing on the heartbeat signal to obtain the heartbeat feature. Specifically, the vehicle safety starting device based on multi-modality performs fast Fourier transform on the heartbeat signal to obtain multiple phases in the heartbeat signal; the multiple phases are differentiated frame by frame to obtain a phase difference signal; the phase difference signal is filtered to obtain a filtered phase difference signal, thereby eliminating the interference of human body movement; the filtered phase difference signal is processed using a preset cross-correlation algorithm to obtain an accurate heartbeat frequency; the heartbeat frequency is averaged using a wavelet low-pass filter, and the high-frequency signal is discarded to obtain the processed heartbeat frequency; the processed heartbeat frequency is subjected to continuous wavelet transform to calculate different scattering coefficients; the scattering coefficient is down-sampled to obtain the heartbeat feature. Therefore, the computational complexity can be reduced by down-sampling the scattering coefficient.

[0103] In some embodiments, the method for obtaining the preset matching model includes: first, constructing a heartbeat data set, which includes multiple heartbeat data, and the heartbeat data set is composed of an M*100 matrix and an M*1 cell array. The matrix is ​​used to store heartbeat data, and each row of data corresponds to a periodic signal of a heartbeat. The cell array is a label of the data, corresponding to each row of data in the matrix, and is used to identify the heartbeat data of different objects. Then, the heartbeat data set is divided into a training data set and a test data set; the learning rate is set to 0.01 and the number of epoch iterations is 200; the training data set is used to train the LSTM (Long Short-Term Memory) network model; the test data set is used to test the trained LSTM network model to obtain the accuracy of the trained LSTM network model to evaluate the model performance; through continuous parameter adjustment iterations, the best performing LSTM network model is used as the final matching model.

[0104] like Figure 2 As shown, the network framework of the preset matching model includes an input layer, a long short-term memory network layer (LSTM), a fully connected layer, a Dorpout (random inactivation) layer, and a classification output layer. The input layer inputs the heartbeat features, and inputs the heartbeat features into the long short-term memory network layer; the long short-term memory network layer learns the time series features between high-dimensional features, extracts the heartbeat time series of the heartbeat features, and sends the heartbeat time series to the fully connected layer; the fully connected layer adjusts the weights based on the heartbeat time series to achieve feature classification, and inputs the classified features into the Dorpout layer; the Dorpout layer performs regularization on the classified features to obtain the regularized features, and inputs the regularized features into the output layer; the output layer converts the regularized features into probability distributions through the Softmax activation function, and outputs the similarity between the heartbeat features and the preset heartbeat signals in the preset heartbeat signal library, that is, the matching score.

[0105] In this embodiment, the heartbeat features are input through the input layer, calculated through the long short-term memory network layer and the fully connected layer, overfitting is prevented through the Dropout layer, and finally output through the output layer. Since the heartbeat signal has time periodicity and waveform characteristics, this embodiment uses a long short-term memory network (LSTM) framework. On the one hand, by introducing a gating mechanism, the long-term dependencies in the heartbeat signal can be effectively captured, which is conducive to the analysis and identification of heartbeat features. On the other hand, LSTM can memorize and forget information, avoiding the problems of gradient vanishing and gradient exploding in conventional RNN (Recurrent Neural Network), thereby improving the accuracy and stability of matching.

[0106] The multimodal vehicle safety start device determines whether the target object to which the heartbeat signal belongs is the vehicle's safety start object based on the feature fusion result and the matching result, including: normalizing the feature fusion result and the matching result respectively to obtain the normalized feature fusion result and the normalized matching result; performing identity discrimination processing based on the normalized feature fusion result and the normalized matching result to obtain a discrimination result; in response to the discrimination result being consistent with a preset discrimination threshold, determining that the target object to which the heartbeat signal belongs is the vehicle's safety start object.

[0107] The LG (Logistic) normalization model based on the multimodal vehicle safety starting device normalizes the feature fusion results and matching results respectively to obtain the normalized feature fusion results and the normalized matching results. Therefore, by using the normalization method with unified physical meaning, the feature fusion results and matching results are transformed into the same physical meaning, so that the scores of different features can be fused.

[0108] In one embodiment, the LG model normalization function may be as follows:

[0109]

[0110] In the above formula, NOR(N Min-Max ) represents the logarithm N Min-Max The result after normalization, score N Min-Max It can be a feature fusion result or a matching result, A represents the first coefficient, and B represents the second coefficient.

[0111] Among them, the first coefficient satisfies the following formula:

[0112]

[0113] In the above formula, Δ is a preset minimum value.

[0114] Among them, the second coefficient satisfies the following formula:

[0115]

[0116] In the above formula, c is the center of the overlapping area of ​​the score distribution.

[0117] Since the error rate of biometric recognition is mainly caused by the overlapping area of ​​the intra-class score distribution (from the same category or the same individual) and the inter-class score distribution (from different categories or different individuals), the LG (Logistic) normalization model of this embodiment is based on the logarithmic function and the Min-Max normalized score, which can separate the intra-class score distribution and the inter-class score distribution to increase the distinguishability and thus improve the recognition accuracy.

[0118] The multimodal vehicle safety starting device performs identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain the discrimination result, including: obtaining the first probability density of the normalized feature fusion result; obtaining the second probability density of the normalized matching result; fitting the first probability density and the second probability density respectively to obtain the fitted first probability density and the fitted second probability density; using Bayesian decision technology to discriminate the fitted first probability density and the fitted second probability density to obtain the discrimination result.

[0119] In one embodiment, the identity recognition problem in the authentication mode can be regarded as a binary classification problem. When the target object to which the heartbeat signal belongs is an authorized security boot object (real identity authentication), H=1; when the target object to which the heartbeat signal belongs is not an authorized security boot object (impersonation identity authentication), H=0.

[0120] First, let the prior probability g = P(H = 1). Since the two categories in the binary classification problem of identity recognition in the authentication mode are mutually exclusive events, P(H = 0) = 1-g. According to Bayesian theory, the posterior identity authentication probability is:

[0121]

[0122] Where s is the normalized matching score or distance score.

[0123] The feature fusion result includes a distance score, and the discrete matching score probability density of the normalized feature fusion result is calculated to obtain a first probability density; the matching result includes a matching score, and the discrete matching score probability density of the normalized matching result is calculated to obtain a second probability density.

[0124] The formula for the discrete matching score probability density is as follows:

[0125]

[0126] In the above formula, p e (s e |H) represents the conditional probability density corresponding to the score e, and p(s|H) represents the probability density corresponding to the score e. It should be noted that when the score e is the normalized distance score, the corresponding p(s|H) represents the first probability density, and when the score e is the normalized matching score, the corresponding p(s|H) represents the second probability density.

[0127] The multimodal vehicle safety starting device uses a cubic spline function to fit the first probability density and the second probability density, and uses the Bayesian decision technology to discriminate the fitted first probability density and the fitted second probability density to obtain a discrimination result. In one embodiment, the Bayesian decision technology discriminates the first probability density and the second probability density after fitting as shown below:

[0128]

[0129] When When , the discrimination result D is 1, otherwise the discrimination result D is 0.

[0130] It should be noted that g=P(H=1)=P(H=0)=1 / 2.

[0131] In response to the discrimination result being consistent with the preset discrimination threshold, the multi-modal vehicle safety starting device determines that the target object to which the heartbeat signal belongs is the vehicle's safety starting object. In one embodiment, in response to the discrimination result being consistent with 1, the multi-modal vehicle safety starting device determines that the target object to which the heartbeat signal belongs is the vehicle's safety starting object; in response to the discrimination result being consistent with 0, the multi-modal vehicle safety starting device determines that the target object to which the heartbeat signal belongs is not the vehicle's safety starting object.

[0132] Furthermore, after the multi-modal vehicle safety start device determines that the target object to which the heartbeat signal belongs is the vehicle's safety start object, the vehicle is controlled to start. For example, the door is automatically unlocked and disarmed, and after the main driving door is opened, the vehicle is automatically powered on with high voltage, the steering wheel is unlocked, and the vehicle remains in a normal driving state.

[0133] Furthermore, after the multi-modal vehicle safety start device determines that the target object to which the heartbeat signal belongs is not the vehicle's safety start object and fails to determine three times in a row, password authentication or remote APP (Application) authentication is performed.

[0134] Figure 3 FIG. 1 is a flow chart of an exemplary embodiment of a multi-modal vehicle safety start method shown in the present application. Figure 3As shown, the multi-modal vehicle safety starting method includes the following steps:

[0135] Step S310: Acquire the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located.

[0136] Step S320: Process the facial image using the AdaBoost algorithm based on Haar-like features to obtain a denoised facial image; perform a two-dimensional principal component analysis on the denoised facial image to obtain a feature vector of the facial image.

[0137] Step S330: Process the fingerprint image based on a 2D Gabor filter to obtain an enhanced fingerprint image, and perform a two-dimensional principal component analysis on the enhanced fingerprint image to obtain a feature vector of the fingerprint image.

[0138] Step S340, the feature vector of the facial image and the feature vector of the fingerprint image are fused to obtain a feature fusion vector; the feature fusion vector is distance classified by using the nearest neighbor distance classification technology to obtain a distance score; and the distance score is normalized.

[0139] Step S350: perform feature processing on the heartbeat signal to obtain heartbeat features; input the heartbeat features into a preset matching model to obtain a matching score, and normalize the matching score.

[0140] Step S360: The normalized distance score and the normalized matching score are fused at the score matching layer and converted into a conditional probability density.

[0141] Step S370: Perform discrimination processing on the conditional probability density based on the Bayesian decision technology to obtain a discrimination result.

[0142] Step S380: Determine whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the discrimination result.

[0143] It can be seen that this embodiment flexibly exploits the complementarity between the three biometric features of face, fingerprint and heartbeat to improve the accuracy, robustness and security of determining the safe start object of the vehicle in complex scenarios. While improving the security level, it greatly simplifies the process of unlocking and starting the vehicle, redefining the integrated safe start solution for smart cars.

[0144] In one embodiment, if Figure 4As shown, determine whether the target object to which the heartbeat signal belongs is the safe start object of the vehicle. If so, open the door and start the vehicle normally; otherwise, select other authentication methods. When the object chooses password authentication, enter the unlock password through the identification screen; determine whether the unlock password is successful, if so, open the door and start the vehicle normally; otherwise, determine whether the number of unlocks is greater than 3 times, if so, lock the vehicle and start the alarm system, and send the alarm information to the preset mobile phone. When the alarm is touched, the process ends. When the object chooses remote APP authentication, determine whether the authorized object of the vehicle chooses to pass the authentication. If so, open the door and start the vehicle normally; otherwise, lock the vehicle and start the alarm system, and send the alarm information to the preset mobile phone. When the alarm is touched, the process ends.

[0145] In one application scenario, when the authentication fails three times in a row, the integrated identity recognition screen on the B-pillar of the vehicle will prompt the subject to select password authentication. When the subject selects the password authentication method, the recognition screen enters the password authentication interface. According to the prompt information on the password authentication interface, the subject enters a 6-digit password (the password is case-sensitive), clicks the confirmation button and submits. The multi-modal vehicle safety start device will immediately verify the entered password. If the password matches the preset password, the recognition screen will immediately display the message "Authentication Successful", accompanied by a prompt sound, giving the subject clear feedback. At this time, the door can be opened normally and the vehicle can enter the normal power-on start state. If the password does not match the preset password, the recognition screen will immediately give feedback and prompt the subject to re-enter. When the number of incorrect password inputs exceeds 3 times, the vehicle will be locked and enter the alarm state. In the alarm state, the left and right turn signals flash at a frequency of 250ms-ON / 250ms-OFF for 5 minutes, and the horn continues to alarm for 28s, with a cycle of 1s and a duty cycle of 50%.

[0146] In another application scenario, when the identity authentication fails three times in a row, the integrated identity recognition screen on the B-pillar of the vehicle will prompt the subject to choose remote APP authentication. When the subject chooses the remote APP authentication method, the unlock request and the subject's identity information, such as facial image, will be sent to the preset vehicle authorization authentication end, which will authenticate the subject's identity and return the authentication result; if the authentication result received is that the identity authentication is passed, the recognition screen will display that the authentication is successful, the door can be opened normally, and the vehicle can enter the normal power-on start state. If the authentication result received is that the identity authentication fails, the vehicle will be locked and enter the alarm state. In the alarm state, the left and right turn signals will flash at a frequency of 250ms-ON / 250ms-OFF for 5 minutes, and the horn will continue to alarm for 28s, with a period of 1s and a duty cycle of 50%.

[0147] Furthermore, after the vehicle enters the alarm state, the authorized party of the vehicle can remotely cancel the alarm through the mobile phone APP. After the alarm is canceled, the vehicle's turn signal and horn stop working and the vehicle returns to normal state. After the alarm state is canceled, an alarm report is sent to the authorized party of the vehicle and the vehicle service center. The alarm report includes information such as the time and location of the alarm and the reason for triggering the alarm. This can help the authorized party of the vehicle and the vehicle service center understand the safety status of the vehicle and take necessary measures in time to prevent similar incidents from happening again.

[0148] It can be seen that this embodiment provides the subject with a more flexible, convenient and safe vehicle use experience through multi-modal fusion identity recognition authentication, password authentication, remote APP authentication, vehicle authentication failure alarm, alarm state release and other functions. At the same time, these functions also further enhance the safety and manageability of the vehicle when it is just started, allowing the subject to better control the vehicle and ensure its safety.

[0149] Figure 5 FIG. 1 is a block diagram of a multi-modal vehicle safety starting device according to an exemplary embodiment of the present application. Figure 5 As shown, the exemplary multi-modal based vehicle safety startup device 500 includes: an acquisition module 510, a fusion module 520, a matching module 530 and a safety startup object determination module 540. Specifically:

[0150] The acquisition module 510 is used to acquire the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located.

[0151] The fusion module 520 is used to fuse the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result.

[0152] The matching module 530 is used to perform matching processing based on the heartbeat signal to obtain a matching result.

[0153] The safe start object determination module 540 is used to determine whether the target object to which the heartbeat signal belongs is a safe start object for the vehicle based on the feature fusion result and the matching result.

[0154] In this exemplary multi-modal vehicle safety start device, the heartbeat signal, facial image and fingerprint image in the preset area where the vehicle is located are obtained; then the facial features of the facial image and the fingerprint features of the fingerprint image are fused to obtain a feature fusion result; matching is performed based on the heartbeat signal to obtain a matching result; finally, based on the feature fusion result and the matching result, it is determined whether the target object to which the heartbeat signal belongs is the vehicle's safety start object. Thus, the vehicle's safety start object is determined by the feature fusion result between the facial image and the fingerprint image and the matching result of the heartbeat signal, while taking into account the facial image, fingerprint image and heartbeat signal, achieving multiple identity authentications for the target object, and improving the safety of vehicle start.

[0155] Among them, the functions of each module can be found in the embodiment of the vehicle safety starting method based on multi-modality, and will not be repeated here.

[0156] In order to implement the multi-modal vehicle safety starting method of the above embodiment, the present application proposes another electronic device, for details, please refer to Figure 6 , Figure 6 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application.

[0157] The electronic device 600 includes a memory 601 and a processor 602 , wherein the memory 601 and the processor 602 are coupled.

[0158] The memory 601 is used to store program data, and the processor 602 is used to execute the program data to implement the multi-modal based vehicle safety starting method of the above embodiment.

[0159] In this embodiment, the processor 602 may also be referred to as a CPU (Central Processing Unit). The processor 602 may be an integrated circuit chip having signal processing capabilities. The processor 602 may also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. A general-purpose processor may be a microprocessor or the processor 602 may also be any conventional processor, etc.

[0160] The present application also provides a computer-readable storage medium, such as Figure 7 As shown, the computer-readable storage medium 700 is used to store program data 701. When the program data 701 is executed by the processor, it is used to implement the multi-modal vehicle safety startup method in the method embodiment of the present application.

[0161] The method involved in the embodiment of the vehicle safety start method based on multimodality of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0162] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A vehicle safety starting method based on multi-modality, characterized in that: The method is applied to a vehicle, and comprises: Acquire a heartbeat signal, a facial image, and a fingerprint image within a preset area where the vehicle is located; Performing fusion processing on the facial features of the facial image and the fingerprint features of the fingerprint image to obtain a feature fusion result; Perform matching processing based on the heartbeat signal to obtain a matching result; Based on the feature fusion result and the matching result, it is determined whether the target object to which the heartbeat signal belongs is a safe starting object for the vehicle.

2. The method according to claim 1, characterized in that The feature fusion result includes a distance score between the facial feature and / or fingerprint feature and a corresponding preset feature in a preset feature database, and the step of fusing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain the feature fusion result includes: Normalizing the facial features of the facial image and the fingerprint features of the fingerprint image respectively to obtain normalized facial features and normalized fingerprint features; Performing weighted fusion processing on the normalized facial features and the normalized fingerprint features to obtain a feature fusion vector; The feature fusion vector is subjected to distance classification processing by using the nearest neighbor distance classification technology to obtain a distance score.

3. The method according to claim 2, characterized in that The facial features of the facial image include a feature vector of the facial image. Before the step of normalizing the facial features of the facial image and the fingerprint features of the fingerprint image to obtain the normalized facial features and the normalized fingerprint features, the method includes: Performing denoising on the facial image to obtain a denoised facial image; A two-dimensional principal component analysis is performed on the facial image after the denoising process to obtain a feature vector of the facial image.

4. The method according to claim 1, characterized in that The step of determining whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the feature fusion result and the matching result includes: Normalizing the feature fusion result and the matching result respectively to obtain a normalized feature fusion result and a normalized matching result; Performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result; In response to the discrimination result being consistent with a preset discrimination threshold, it is determined that the target object to which the heartbeat signal belongs is a safe startup object for the vehicle.

5. The method according to claim 4, characterized in that The step of performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain the discrimination result comprises: Obtaining a first probability density of the feature fusion result after the normalization processing; Obtaining a second probability density of the normalized matching result; Fitting the first probability density and the second probability density respectively to obtain a fitted first probability density and a fitted second probability density; The Bayesian decision technology is used to discriminate the fitted first probability density and the fitted second probability density to obtain a discrimination result.

6. The method according to claim 1, characterized in that The vehicle includes an image acquisition device and a fingerprint acquisition device, and the step of acquiring a heartbeat signal, a facial image, and a fingerprint image in a preset area where the vehicle is located includes: Continuously detecting the heartbeat signal within the preset area where the vehicle is located; In response to detecting the presence of a heartbeat signal in the preset area where the vehicle is located, controlling the image acquisition device and the fingerprint acquisition device to turn on, and controlling the image acquisition device and the fingerprint acquisition device to perform continuous acquisition; In response to the fingerprint collection device collecting a fingerprint image, a facial image of a target object corresponding to the fingerprint image is collected.

7. The method according to claim 1, characterized in that The matching result includes a matching score between the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library, and the step of performing matching processing based on the heartbeat signal to obtain the matching result includes: Calculating the similarity between the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library to obtain a similarity; The maximum similarity is determined as the matching score.

8. The method according to claim 1, characterized in that The step of performing matching processing based on the heartbeat signal to obtain a matching result includes: Performing feature processing on the heartbeat signal to obtain heartbeat features; The heartbeat feature is input into a preset matching model to obtain a matching result.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: include: Program data is stored, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 8.

Images