Multi-modal based vehicle safe start method, electronic device, and storage medium

Abstract

The application discloses a multi-modal-based vehicle safety starting method, an electronic device and a storage medium. The application obtains a heartbeat signal, a face image and a fingerprint image in a preset area where a vehicle is located; then, face features of the face image and fingerprint features of the fingerprint image are fused to obtain a feature fusion result; matching processing is performed based on the heartbeat signal to obtain a matching result; and finally, it is determined whether a target object to which the heartbeat signal belongs is a safety starting object of the vehicle based on the feature fusion result and the matching result. Thus, the safety of vehicle starting is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, and in particular to a vehicle safe starting method based on multi-modal, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, with the rapid development of the automobile industry, intelligent vehicles have become a new trend of social digital transformation. One of the biggest concerns of intelligent vehicles is whether the vehicle is safe enough. The safety of the vehicle is reflected in two aspects. On the one hand, it is reflected in driving safety. Ensuring driving safety during the driving of an intelligent vehicle can protect life and property. On the other hand, it is reflected in the safety of the vehicle starting. Only the authorized object of the vehicle can start the vehicle, which can better protect the property of the vehicle.

[0003] At present, the starting method of the vehicle usually uses a wireless remote key, an NFC (Near Field Communication) card, an RFID (Radio Frequency IDentification) card or a mechanical key to open the door and start the vehicle. However, an unauthorized object can also start the vehicle after obtaining the key or other unlocking tools. Therefore, the safety of this vehicle starting method is low. SUMMARY

[0004] The technical problem solved by the present application is to provide a vehicle safe starting method based on multi-modal, an electronic device and a storage medium, which can improve the safety of vehicle starting.

[0005] To solve the above technical problem, the present application provides a vehicle safe starting method based on multi-modal.

[0006] In an embodiment, a vehicle safe starting method based on multi-modal is applied to a vehicle, and the method comprises:

[0007] obtaining a heartbeat signal, a face image and a fingerprint image in a preset area where the vehicle is located;

[0008] fusing the face features of the face image and the fingerprint features of the fingerprint image to obtain a feature fusion result;

[0009] performing matching processing based on the heartbeat signal to obtain a matching result;

[0010] determining whether a target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the feature fusion result and the matching result.

[0011] In an embodiment, the feature fusion result comprises a distance score between the face feature and / or the fingerprint feature and a corresponding preset feature in a preset feature database, the step of fusing the face feature of the face image and the fingerprint feature of the fingerprint image to obtain a feature fusion result comprises:

[0012] normalizing the face feature of the face image and the fingerprint feature of the fingerprint image respectively to obtain a normalized face feature and a normalized fingerprint feature;

[0013] performing weighted fusion processing on the normalized face feature and the normalized fingerprint feature to obtain a feature fusion vector;

[0014] performing distance classification processing on the feature fusion vector using a nearest neighbor distance classification technique to obtain a distance score.

[0015] In an embodiment, the face feature of the face image comprises a feature vector of the face image, and before the step of normalizing the face feature of the face image and the fingerprint feature of the fingerprint image to obtain a normalized face feature and a normalized fingerprint feature, the method comprises:

[0016] performing denoising processing on the face image to obtain a denoised face image;

[0017] performing two-dimensional principal component analysis processing on the denoised face image to obtain a feature vector of the face image.

[0018] In an embodiment, the step of determining whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the feature fusion result and the matching result comprises:

[0019] normalizing the feature fusion result and the matching result respectively to obtain a normalized feature fusion result and a normalized matching result;

[0020] performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result;

[0021] in response to the discrimination result being consistent with a preset discrimination threshold, determining that the target object to which the heartbeat signal belongs is a safe starting object of the vehicle.

[0022] In an embodiment, the step of performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result comprises:

[0023] obtaining a first probability density of the normalized feature fusion result;

[0024] obtaining a second probability density of the normalized matching result;

[0025] fitting the first probability density and the second probability density respectively to obtain a fitted first probability density and a fitted second probability density;

[0026] discriminating the fitted first probability density and the fitted second probability density by using a Bayesian decision technique to obtain a discrimination result.

[0027] In an embodiment, the vehicle comprises an image acquisition device and a fingerprint acquisition device, and the step of obtaining the heartbeat signal, the face image and the fingerprint image in the preset area where the vehicle is located comprises:

[0028] continuously detecting the heartbeat signal in the preset area where the vehicle is located;

[0029] in response to detecting that the heartbeat signal exists in the preset area where the vehicle is located, controlling the image acquisition device and the fingerprint acquisition device to be turned on and continuously acquiring the image acquisition device and the fingerprint acquisition device;

[0030] in response to the fingerprint acquisition device acquiring the fingerprint image, acquiring the face image of the target object corresponding to the fingerprint image.

[0031] In an embodiment, the matching result comprises a matching score of the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library, and the step of performing matching processing based on the heartbeat signal to obtain a matching result comprises:

[0032] performing similarity calculation on the heartbeat signal and the preset heartbeat signal in the preset heartbeat signal library to obtain a similarity;

[0033] determining the maximum similarity as the matching score.

[0034] In an embodiment, the step of performing matching processing based on the heartbeat signal to obtain a matching result comprises:

[0035] performing feature processing on the heartbeat signal to obtain a heartbeat feature;

[0036] inputting the heartbeat feature into a preset matching model to obtain a matching result.

[0037] To solve the above technical problems, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above-mentioned vehicle safety starting method based on multi-modal.

[0038] To solve the above technical problems, the application provides a computer readable storage medium, comprising: program data stored, the program data is executed by a processor to implement the above-mentioned multi-modal based vehicle safety starting method.

[0039] The above scheme, by acquiring the heartbeat signal, the face image and the fingerprint image in the preset area where the vehicle is located; then the face features of the face image and the fingerprint features of the fingerprint image are fused to obtain the feature fusion result; based on the heartbeat signal, the matching processing is carried out to obtain the matching result; finally, based on the feature fusion result and the matching result, it is determined whether the target object to which the heartbeat signal belongs is the safety starting object of the vehicle. Therefore, by the feature fusion result between the face image and the fingerprint image and the matching result of the heartbeat signal, the safety starting object of the vehicle is determined, and the face image, the fingerprint image and the heartbeat signal are taken into account, the multi-item identity verification of the target object is realized, and the safety of the vehicle starting is improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings:

[0041] Figure 1 is a flowchart of an exemplary embodiment of the multi-modal based vehicle safety starting method shown by the present application;

[0042] Figure 2 is a network structure diagram of a preset matching model shown by the present application;

[0043] Figure 3 is a flowchart of another exemplary embodiment of the multi-modal based vehicle safety starting method shown by the present application;

[0044] Figure 4 is a flowchart of still another exemplary embodiment of the multi-modal based vehicle safety starting method shown by the present application;

[0045] Figure 5 is a block diagram of the multi-modal based vehicle safety starting device shown by an exemplary embodiment of the present application;

[0046] Figure 6 is a structural diagram of an embodiment of the electronic device provided by the present application;

[0047] Figure 7 is a structural diagram of an embodiment of the computer readable storage medium provided by the present application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0049] First, it's important to note that with the rapid development of the automotive industry in recent years, intelligent vehicles have become a new wave in society's digital transformation. One of the biggest concerns regarding intelligent vehicles is their safety. Vehicle safety is reflected in two aspects: firstly, driving safety, ensuring safe driving during intelligent vehicle operation protects lives and property; and secondly, the security of vehicle access, ensuring that only authorized individuals can open the vehicle to better protect property.

[0050] Currently, vehicles are typically unlocked using wireless remote keys, NFC (Near Field Communication) cards, RFID (Radio Frequency Identification) cards, or mechanical keys. However, unauthorized individuals can also start the vehicle after obtaining the key or other unlocking tools. Therefore, this method of vehicle starting offers relatively low security.

[0051] Based on this, this application provides a multimodal vehicle safe starting method, electronic device, and storage medium. For details, please refer to [reference needed]. Figure 1 , Figure 1 This is a schematic flowchart of an exemplary embodiment of the multimodal vehicle safe start method shown in this application.

[0052] The execution entity of the multimodal vehicle safe start method can be a terminal device, a server, or other processing device. The terminal device can be a computer, mobile device, terminal, computing device, in-vehicle device, etc. The execution entity of the multimodal vehicle safe start method can also be a multimodal vehicle safe start device. In some possible implementations, this multimodal vehicle safe start method can be implemented by a processor calling computer-readable instructions stored in memory.

[0053] Specifically, this embodiment of a multimodal vehicle safe starting method is applied to a vehicle, and the multimodal vehicle safe starting method includes the following steps:

[0054] In step S110, the heartbeat signal, the face image and the fingerprint image in the preset area where the vehicle is located are acquired.

[0055] The preset area where the vehicle is located can be a circular area with the vehicle as the center and a preset length as the radius, and can also be an area with the vehicle as the center and the farthest distance at which the radar monitoring device, the image acquisition device and the fingerprint acquisition device on the vehicle can collect information as the boundary.

[0056] The heartbeat signal is a signal representing the beating of the heart in the object. Specifically, the multi-modal based vehicle safety starting device includes a radar monitoring device, and the multi-modal based vehicle safety starting device emits a signal outwardly by using the radar monitoring device and receives a return wave signal fed back by the heart in the object, mixes the emitted signal and the return wave signal to obtain the heartbeat signal. The multi-modal based vehicle safety starting device can use a mixer to mix the emitted signal and the return wave signal to obtain the heartbeat signal. The radar monitoring device can be a millimeter wave radar.

[0057] In some embodiments, a millimeter wave radar is installed on the B-pillar of the vehicle, and the multi-modal based vehicle safety starting device emits a signal outwardly in real time by using the millimeter wave radar. When an object appears in the preset area where the vehicle is located, the signal emitted by the millimeter wave radar is reflected by the heart of the object to form a return wave signal. After the millimeter wave radar receives the return wave signal, a mixer is used to mix the emitted signal and the return wave signal to obtain the heartbeat signal.

[0058] The face image refers to an image containing the face of the object. Specifically, the multi-modal based vehicle safety starting device includes an image acquisition device, and the multi-modal based vehicle safety starting device can use the image acquisition device to acquire the face image of the object in the preset area where the vehicle is located. The image acquisition device can be a camera.

[0059] The fingerprint image refers to an image containing the fingerprint of the object. Specifically, the multi-modal based vehicle safety starting device includes a fingerprint acquisition device, and the multi-modal based vehicle safety starting device can use the fingerprint acquisition device to acquire the fingerprint of the object in the preset area where the vehicle is located to obtain the fingerprint image. The fingerprint acquisition device can be a screen fingerprint detection and recognition device. It should be noted that the fingerprint acquisition device can be installed on the handle of the vehicle door.

[0060] The multi-modal based vehicle safety starting device acquires a heartbeat signal, a face image and a fingerprint image in a preset area where the vehicle is located. As an example, the multi-modal based vehicle safety starting device continuously detects a heartbeat signal in a preset area where the vehicle is located; in response to detecting that there is a heartbeat signal in the preset area where the vehicle is located, controls the image acquisition device and the fingerprint acquisition device to start; and acquires a face image and a fingerprint image through the image acquisition device and the fingerprint acquisition device respectively.

[0061] As another example, the vehicle includes an image acquisition device and a fingerprint acquisition device. The multi-modal based vehicle safety starting device acquires a heartbeat signal, a face image and a fingerprint image in a preset area where the vehicle is located, including: continuously detecting a heartbeat signal in a preset area where the vehicle is located; in response to detecting that there is a heartbeat signal in the preset area where the vehicle is located, controlling the image acquisition device and the fingerprint acquisition device to start, and controlling the image acquisition device and the fingerprint acquisition device to continuously acquire; and in response to the fingerprint acquisition device acquiring a fingerprint image, acquiring a face image of a target object corresponding to the fingerprint image.

[0062] In step S120, the face feature of the face image and the fingerprint feature of the fingerprint image are fused to obtain a feature fusion result.

[0063] The face feature is a feature representing face information in the face image. Specifically, the multi-modal based vehicle safety starting device extracts the face feature in the face image by using a 2DPCA (Principal Components Analysis) dimension reduction technology to obtain the face feature.

[0064] The fingerprint feature is a feature representing fingerprint information in the fingerprint image. Specifically, the multi-modal based vehicle safety starting device extracts the fingerprint feature in the fingerprint image by using a 2DPCA (Principal Components Analysis) dimension reduction technology to obtain the fingerprint feature.

[0065] The fusion processing refers to the processing of integrating the face feature and the fingerprint feature into one feature.

[0066] The multi-modal based vehicle safety starting device fuses the face feature of the face image and the fingerprint feature of the fingerprint image to obtain a feature fusion result. Specifically, the multi-modal based vehicle safety starting device fuses the face feature and the fingerprint feature by using a preset weight to obtain the feature fusion result.

[0067] In step S130, a matching process is performed based on the heartbeat signal to obtain a matching result.

[0068] The multi-modal based vehicle safety starting device performs matching processing based on the heartbeat signal to obtain a matching result. Specifically, the matching result includes a matching score of the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library. The multi-modal based vehicle safety starting device performs similarity calculation on the heartbeat signal and the preset heartbeat signal in the preset heartbeat signal library to obtain a similarity. The maximum similarity is determined as the matching score.

[0069] In step S140, it is determined whether the target object to which the heartbeat signal belongs is a safety starting object of the vehicle based on the feature fusion result and the matching result.

[0070] The safety starting object refers to an object authorized to start the vehicle.

[0071] The multi-modal based vehicle safety starting device determines whether the target object to which the heartbeat signal belongs is a safety starting object of the vehicle based on the feature fusion result and the matching result. Specifically, the multi-modal based vehicle safety starting device compares the feature fusion result with a first preset score threshold to obtain a first comparison result. The multi-modal based vehicle safety starting device compares the matching result with a second preset score threshold to obtain a second comparison result. In response to the first comparison result representing that the feature fusion result is greater than the first preset score threshold and the second comparison result representing that the matching result is greater than the second preset score threshold, it is determined that the target object to which the heartbeat signal belongs is a safety starting object of the vehicle. Otherwise, it is determined that the target object to which the heartbeat signal belongs is not a safety starting object of the vehicle.

[0072] It can be seen that the heartbeat signal, the face image, and the fingerprint image in a preset area where the vehicle is located are obtained. Then, the face features of the face image and the fingerprint features of the fingerprint image are fused to obtain a feature fusion result. Matching processing is performed based on the heartbeat signal to obtain a matching result. Finally, it is determined whether the target object to which the heartbeat signal belongs is a safety starting object of the vehicle based on the feature fusion result and the matching result. Thus, the safety starting object of the vehicle is determined by the feature fusion result between the face image and the fingerprint image and the matching result of the heartbeat signal, while the face image, the fingerprint image, and the heartbeat signal are taken into account. Multi-item identity verification of the target object is achieved, and the safety of vehicle starting is improved.

[0073] In an embodiment, the feature fusion result comprises a distance score between the face feature and / or the fingerprint feature and a corresponding preset feature in a preset feature database, the step of fusing the face feature of the face image and the fingerprint feature of the fingerprint image to obtain the feature fusion result comprises: performing normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image respectively to obtain normalized face feature and normalized fingerprint feature; performing weighted fusion processing on the normalized face feature and the normalized fingerprint feature to obtain a feature fusion vector; and performing distance classification processing on the feature fusion vector by using a nearest neighbor distance classification technique to obtain the distance score.

[0074] Specifically, the multi-modal based vehicle safety starting device performs normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image respectively to obtain normalized face feature and normalized fingerprint feature. As an example, the multi-modal based vehicle safety starting device can perform normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image by using a Min-Max normalization technique. The Min-Max normalization technique comprises converting the face feature or the fingerprint feature into a range of [0, 1] by using a formula (y` = (x-x min ) / (x max -x min )), where y` is the normalized feature, which can be the normalized face feature or the normalized fingerprint feature, x is the original feature, which can be the face feature or the fingerprint feature, x min and x max are the minimum feature and the maximum feature in the original feature respectively. As another example, the multi-modal based vehicle safety starting device can perform normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image by using a logistic function or a sigmoid function. Since the face and the fingerprint are two different biological modalities, the feature vectors of the face and the fingerprint can be normalized to a consistent interval with a mean of 0 and a variance of 1 by normalization processing, so as to convert the face feature and the fingerprint feature into standard forms respectively, so that the feature vectors of the face and the fingerprint are in the same range, thereby enabling subsequent feature fusion of the face feature and the fingerprint feature.

[0075] The multi-modal based vehicle safety starting device performs weighted fusion processing on the normalized face feature and the normalized fingerprint feature to obtain a feature fusion vector. Specifically, the multi-modal based vehicle safety starting device performs weighted fusion on the normalized face feature and the normalized fingerprint feature by using a nearest neighbor distance based weight fusion method.

[0076] In an embodiment, the weight fusion method based on the nearest neighbor distance comprises the following steps: first, the distance between the normalized facial features and the normalized fingerprint features and each training sample in the corresponding preset feature database is calculated respectively by using the nearest neighbor distance, to obtain a facial feature set d = [d1, d2, … dN], wherein N represents the number of training samples, dN represents the distance between the normalized facial features and the Nth training sample, a fingerprint feature set P = [P1, P2, … PN], wherein N represents the number of training samples, and PN represents the distance between the normalized fingerprint features and the Nth training sample. Second, d (i = 1, 2, …, N) is arranged in descending order, and the shortest distance d is determined. N N ] and P (j = 1, 2, …, N) is arranged in descending order, and the shortest distance P is determined. N N N x j x Third, the distance average of the facial features and the distance average of the fingerprint features are calculated respectively; the distance average of the facial features is and the distance average of the fingerprint features is Fourth, the ratio of the distance average and the shortest distance is determined, the ratio of the facial features is t1, and the ratio of the fingerprint features is t2. Fifth, the weight w is determined, w = t2 / t1. Sixth, the feature fusion vector y = [(1-w)y1, w*y2] is determined. y1 represents the normalized facial features, y2 represents the normalized fingerprint features, and w represents the weight corresponding to the facial features.

[0077] wherein the distance average of the facial features satisfies the following formula:

[0078]

[0079] wherein the distance average of the fingerprint features satisfies the following formula:

[0080]

[0081] wherein the ratio of the facial features satisfies the following formula:

[0082]

[0083] wherein the ratio of the fingerprint features satisfies the following formula:

[0084]

[0085] The vehicle safety starting device based on multi-modal adopts the nearest neighbor distance classification technology to perform distance classification processing on the feature fusion vector, to obtain a distance score.

[0086] ​​​​​​In an embodiment, the nearest neighbor distance classification technique comprises: calculating the distance between the feature fusion vector and each vector sample in the fusion vector training set in the search space according to the shortest Euclidean distance as the judgment criterion, that is, d (y3, y4) = ‖y3-y4‖2, ‖*‖2 represents the L2 norm, y3 represents the vector sample, and y4 represents the feature fusion vector. According to the calculated distance, the K vector samples with the smallest distance in the fusion vector training set are determined as the nearest neighbor sample set. From the M categories to which the K nearest neighbor sample set belongs, the category with the most number of nearest neighbors of the feature fusion vector is determined by the majority voting decision rule. The category with the most number of nearest neighbors of the feature fusion vector is the category to which the feature fusion vector belongs, that is, the object to which the feature fusion vector belongs.

[0087] For example, the fusion vector training set includes vector samples corresponding to 10 different state face images of multiple objects. The Euclidean distances between the feature fusion vector and the vector samples corresponding to the 10 different state face images are calculated. The K vector samples with the smallest distance are taken as the nearest neighbor sample set. For example, the Euclidean distances between the feature fusion vector and the vector samples corresponding to the 4 face images of the first object, the Euclidean distances between the vector samples corresponding to the 3 face images of the second object, and the Euclidean distances between the vector samples corresponding to the 2 face images of the third object are equal or the difference between them is within the error range. Therefore, the set of vector samples corresponding to these face images is the nearest neighbor sample set. According to the majority voting method, the category to which the largest number of vector samples in the nearest neighbor sample set belongs is determined as the category to which the feature fusion vector belongs, that is, the object to which the feature fusion vector belongs.

[0088] In an embodiment, the face feature of the face image comprises a feature vector of the face image. Before the steps of respectively performing normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image to obtain the normalized face feature and the normalized fingerprint feature, the method comprises: performing denoising processing on the face image to obtain a denoised face image; and performing two-dimensional principal component analysis processing on the denoised face image to obtain the feature vector of the face image.

[0089] The multi-modal based vehicle safety starting device performs denoising processing on the face image to obtain a denoised face image. Specifically, the multi-modal based vehicle safety starting device uses an AdaBoost algorithm based on Haar-like features to process the face image to obtain a denoised face image. For example, the multi-modal based vehicle safety starting device performs size normalization, filter denoising, and cropping on the face image; and inputs the preprocessed face image into a preset classifier to obtain a denoised face image.

[0090] The preset classifier includes a plurality of weak classifiers and a cascade classifier formed by cascading a plurality of strong classifiers. The plurality of strong classifiers are cascaded to form the cascade classifier, thereby improving the accuracy and efficiency of detection. In an embodiment, the preset classifier scans a face image through a sliding window, applies a strong classifier at each window position to make a classification judgment, and obtains a classification result. A face region in the face image is determined according to the classification result and the window position, and a denoised face image is obtained. In this way, the face image is processed by the classifier, the environmental background factors in the face image are removed, a clearer and more stable face image is obtained, and the face of the object can be recognized in the face image in a complex background, thereby reducing the interference in the later stage.

[0091] The training method of the preset classifier includes: constructing a positive sample set and a negative sample set, the positive sample set including a plurality of face image samples containing faces, and the negative sample set including a plurality of environmental image samples not containing faces; and performing preprocessing such as size normalization, filtering denoising, and cropping on the image samples in the positive sample set and the negative sample set respectively to adapt to the input requirements of the classifier. The vehicle safety starting device based on multiple modalities uses Haar-like features of the image samples as inputs of the weak classifier to train the weak classifier. The vehicle safety starting device based on multiple modalities trains the strong classifier through an AdaBoost algorithm. Specifically, in the training process, each weak classifier is weighted for the samples incorrectly classified in the previous round, and the performance of the classifier is gradually improved through multiple rounds of iteration.

[0092] The Haar-like feature refers to a simple rectangular feature template used to describe local gray level changes in an image. The Haar-like feature can effectively capture the brightness difference of parts such as eyes, nose, and mouth in the image. The feature values of each feature of the image sample are calculated, and the weak classifier is trained based on the feature values to improve the accuracy of classification.

[0093] The vehicle safety starting device based on multiple modalities performs two-dimensional principal component analysis processing on the denoised face image to obtain a feature vector of the face image. Specifically, the vehicle safety starting device based on multiple modalities uses a 2DPCA dimension reduction technique to perform dimension reduction analysis processing on the denoised face image to obtain a feature vector of the face image.

[0094] In an embodiment, the 2DPCA dimension reduction technique can extract the most representative low-dimensional feature vectors from the original high-dimensional image data to facilitate subsequent feature fusion, classification, recognition, and other tasks. These low-dimensional feature vectors can retain the main information of the original image while removing noise and unimportant features, thereby improving the performance and robustness of the recognition system.

[0095] The 2DPCA dimension reduction technology includes the following modes for dimension reduction analysis and processing of images: in the first step, the training image samples in the preset training set are constructed into a two-dimensional image matrix, wherein the number of rows and the number of columns of the two-dimensional image matrix correspond to the sum of the heights and the sum of the widths of the plurality of training image samples spliced respectively. In the second step, the mean matrix is calculated according to the training image samples in the preset training set, and each element in the mean matrix corresponds to the average value of the pixel value of a single training image sample in the corresponding position. In the third step, the two-dimensional covariance matrix is calculated based on the training image samples in the preset training set and the mean matrix, and the two-dimensional covariance matrix represents the covariance relationship between the pixel values at different positions in the training image samples. In the fourth step, the covariance matrix is decomposed by SVD (Single Value Decomposition) to obtain the eigenvalues and the corresponding eigenvectors. The size of the eigenvalue reflects the importance of the corresponding eigenvector. The eigenvectors corresponding to the eigenvalues greater than the preset feature threshold are used as principal components to form a feature space. Thus, the feature space contains the main features of the image. In the fifth step, the training image samples in the preset training set are projected onto the feature space to obtain a projection feature matrix. The projection feature matrix retains the key feature information of each image, reduces the computational complexity and the feature dimension, and facilitates subsequent feature fusion, classification and recognition. Finally, the denoised face image or the enhanced fingerprint image is input into the corresponding projection feature matrix to obtain the corresponding feature vector. Thus, the principal component dimension reduction feature extraction is performed on the face image and the fingerprint image, so that the extracted face features and fingerprint features have certain representativeness. The face features and the fingerprint features extracted by the 2DPCA have complementary characteristics, and more information can be obtained when the face features and the fingerprint features are fused at the feature layer, so as to improve the accuracy and stability of the biometric feature recognition.

[0096] In an embodiment, the fingerprint features of the fingerprint image include the eigenvectors of the fingerprint image, and before the step of normalizing the face features of the face image and the fingerprint features of the fingerprint image to obtain the normalized face features and the normalized fingerprint features, the method includes: performing enhancement processing on the fingerprint image to obtain an enhanced fingerprint image; and performing two-dimensional principal component analysis processing on the enhanced fingerprint image to obtain the eigenvectors of the fingerprint image.

[0097] The multi-modal based vehicle safety starting device performs enhancement processing on the fingerprint image to obtain an enhanced fingerprint image. Specifically, the multi-modal based vehicle safety starting device can use a 2DGabor filter to perform enhancement processing on the fingerprint image to obtain an enhanced fingerprint image.

[0098] In an embodiment, first, the multi-modal based vehicle safety starting device can obtain multiple fingerprint images in different states through a fingerprint reading device on an integrated screen of a vehicle, pre-process the fingerprint images to obtain pre-processed fingerprint images. The pre-processing includes steps such as equalization of grayscale, denoising, and binarization, thereby adjusting the contrast and grayscale of the pre-processed fingerprint images to a suitable range to reduce image quality differences. Then, by analyzing the direction field of the pre-processed fingerprint images, the severity of the direction change of the direction entropy is used to determine the sub-block corresponding to the maximum direction entropy; the center of the sub-block corresponding to the maximum direction entropy is taken as the center point of the fingerprint feature effective area to obtain the fingerprint feature effective area. In this way, the extraction of the fingerprint feature effective area (Receiver Operating Characteristic, ROC) is realized. The fingerprint feature effective area is then divided into blocks, the local spectral information of each block is extracted by windowed Fourier transform, the transformed spectrum in the local spectral information is filtered by a Garbor filter, and the filtered local spectral information is restored to spatial image information by inverse Fourier transform to obtain a preliminarily enhanced fingerprint image, thereby realizing the extraction of ridge lines from the filtered block image and the combination of the enhanced image. Finally, the preliminarily enhanced fingerprint image is subjected to median filtering to eliminate speckle noise to obtain an enhanced processed fingerprint image. In this way, the fingerprint ridge texture in the enhanced processed fingerprint image is clearer, thereby providing a high-quality fingerprint image for subsequent effective extraction of fingerprint features.

[0099] Since the fingerprint contains rich texture information, noise is usually introduced during the fingerprint image acquisition process and the finger placement position may also be offset. The pre-processing of the fingerprint image in this embodiment can ensure the reliability of the fingerprint feature extraction, and the 2D Gabor filter is used to filter the pre-processed fingerprint image, which can take into account the direction and frequency selection characteristics, thereby realizing the enhancement of the local texture features of the fingerprint image from multiple scales and directions.

[0100] The multi-modal based vehicle safety starting device performs two-dimensional principal component analysis processing on the enhanced processed fingerprint image to obtain a feature vector of the fingerprint image. Specifically, the multi-modal based vehicle safety starting device uses the above-mentioned 2DPCA dimension reduction technology to perform two-dimensional principal component analysis processing on the enhanced processed fingerprint image to obtain a feature vector of the fingerprint image.

[0101] The step of performing matching processing based on the heartbeat signal by the multi-modal based vehicle safety starting device to obtain a matching result includes: performing feature processing on the heartbeat signal to obtain a heartbeat feature; inputting the heartbeat feature into a preset matching model to obtain a matching result.

[0102] The heartbeat signal is processed by the multi-modal vehicle safety starting device to obtain a heartbeat feature. Specifically, the multi-modal vehicle safety starting device performs fast Fourier transform on the heartbeat signal to obtain a plurality of phases in the heartbeat signal; the plurality of phases are frame-by-frame differenced to obtain a phase difference signal; the phase difference signal is filtered to obtain a filtered phase difference signal, thereby excluding the interference of human motion; a preset cross-correlation algorithm is used to process the filtered phase difference signal to obtain an accurate heartbeat frequency; a wavelet low-pass filter is used to average process the heartbeat frequency and discard high-frequency signals to obtain a processed heartbeat frequency; the processed heartbeat frequency is continuously wavelet transformed to calculate different scattering coefficients; and the scattering coefficients are down-sampled to obtain the heartbeat feature. Thus, the scattering coefficients are down-sampled to reduce the computational complexity.

[0103] In some embodiments, the preset matching model is obtained in the following manner. First, a heartbeat dataset is constructed, the heartbeat dataset including a plurality of heartbeat data, the heartbeat dataset being composed of an M*100 matrix and an M*1 cell array. The matrix is used to store the heartbeat data, and each row of data corresponds to a heartbeat period signal. The cell array is a data label corresponding to each row of data in the matrix and is used to identify the heartbeat data of different objects. Then, the heartbeat dataset is divided into a training dataset and a test dataset; a learning rate of 0.01 and 200 epoch iterations are set; the LSTM (Long Short-Term Memory) network model is trained using the training dataset; the trained LSTM network model is tested using the test dataset to obtain the accuracy of the trained LSTM network model, thereby evaluating the performance of the model; and the best-performing LSTM model is selected as the final matching model through continuous parameter adjustment and iteration.

[0104] As shown in FIG. 1, Figure 2 The network framework of the preset matching model includes an input layer, a long short-term memory network layer (LSTM), a fully connected layer, a Dorpout layer, and a classification output layer. The input layer inputs the heartbeat feature, and the heartbeat feature is input into the long short-term memory network layer. The long short-term memory network layer learns the time sequence features between high-dimensional features and extracts heartbeat time sequences of the heartbeat feature, which are sent to the fully connected layer. The fully connected layer adjusts weights based on the heartbeat time sequences to realize feature classification, and the classified features are input into the Dorpout layer. The Dorpout layer performs regularization processing on the classified features to obtain regularized features, which are input into the output layer. The output layer converts the regularized features into a probability distribution through a Softmax activation function and outputs the similarity between the heartbeat feature and the preset heartbeat signal in the preset heartbeat signal library, i.e., the matching score.

[0105] The embodiment inputs heartbeat features through an input layer, performs calculation through a long short-term memory network layer and a full connection layer, prevents overfitting through a Dropout layer, and finally outputs through an output layer. Since the heartbeat signal has time periodicity and waveform characteristics, the embodiment uses a long short-term memory network (LSTM) framework. On the one hand, by introducing a gating mechanism, long-term dependencies in the heartbeat signal can be effectively captured, which is conducive to analyzing and identifying heartbeat features. On the other hand, LSTM can remember and forget information, avoiding the problems of gradient disappearance and gradient explosion in a conventional recurrent neural network (RNN), thereby improving the accuracy and stability of matching.

[0106] The step of determining whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the feature fusion result and the matching result includes: performing normalization processing on the feature fusion result and the matching result respectively to obtain a normalized feature fusion result and a normalized matching result; performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result; and in response to the discrimination result being consistent with a preset discrimination threshold, determining that the target object to which the heartbeat signal belongs is the safe starting object of the vehicle.

[0107] The LG (Logistic) normalization model performs normalization processing on the feature fusion result and the matching result respectively to obtain a normalized feature fusion result and a normalized matching result. In this way, the feature fusion result and the matching result are converted into having the same physical meaning by using a normalization method with unified physical meaning, so that scores of different features can be fused.

[0108] In an embodiment, the LG model normalization function can be as follows:

[0109]

[0110] In the above formula, NOR (N Min-Max ) represents the normalized result of the score N Min-Max , the score N Min-Max may be the feature fusion result or the matching result, A represents a first coefficient, and B represents a second coefficient.

[0111] wherein the first coefficient satisfies the following formula:

[0112]

[0113] In the above formula, Δ is a preset minimum value.

[0114] The second coefficient satisfies the following formula:

[0115]

[0116] In the above formula, c is the center of the overlap region of the score distribution.

[0117] Since the biometric recognition error rate is mainly caused by the overlap region of the intra-class (from the same class or the same individual) score distribution and the inter-class (from different classes or different individuals) score distribution, the LG (Logistic) normalization model of the embodiment can separate the intra-class score distribution and the inter-class score distribution based on the logarithmic function and the Min-Max normalized score, increase the distinguishability, and thus improve the correct rate of recognition.

[0118] The step of obtaining the discrimination result based on the normalized feature fusion result and the normalized matching result includes: obtaining a first probability density of the normalized feature fusion result; obtaining a second probability density of the normalized matching result; fitting the first probability density and the second probability density respectively to obtain a fitted first probability density and a fitted second probability density; and discriminating the fitted first probability density and the fitted second probability density by using the Bayesian decision technology to obtain the discrimination result.

[0119] In an embodiment, the identity recognition problem in the authentication mode can be regarded as a binary classification problem. When the target object to which the heartbeat signal belongs is an authorized safe starting object (real identity authentication), H = 1; when the target object to which the heartbeat signal belongs is not an authorized safe starting object (impersonation identity authentication), H = 0.

[0120] First, let the prior probability g = P(H = 1). Since the two classes in the binary classification problem of the identity recognition in the authentication mode are mutually exclusive events, P(H = 0) = 1-g. According to the Bayesian theory, the posterior identity authentication probability is:

[0121]

[0122] Where s is the normalized matching score or distance score.

[0123] The feature fusion result includes a distance score, the discrete matching score probability density of the normalized feature fusion result is calculated to obtain the first probability density; the matching result includes a matching score, the discrete matching score probability density of the normalized matching result is calculated to obtain the second probability density.

[0124] The formula of the discrete matching score probability density is as follows:

[0125]

[0126] In the above formula, p e (s e |H) represents the conditional probability density corresponding to the score e, and p(s|H) represents the probability density corresponding to the score e. It should be noted that when the score e is the normalized distance score, the corresponding p(s|H) represents the first probability density, and when the score e is the normalized matching score, the corresponding p(s|H) represents the second probability density.

[0127] The vehicle safety starting device based on multi-modal adopts a cubic spline function to fit the first probability density and the second probability density, adopts a Bayesian decision technology to discriminate the fitted first probability density and the fitted second probability density, and obtains a discrimination result. In an embodiment, the discrimination function of the Bayesian decision technology on the fitted first probability density and the fitted second probability density is as follows:

[0128]

[0129] That is, when the discrimination result D is 1, otherwise the discrimination result D is 0.

[0130] It should be noted that g = P(H = 1) = P(H = 0) = 1 / 2.

[0131] The vehicle safety starting device based on multi-modal determines that the target object to which the heartbeat signal belongs is the safety starting object of the vehicle in response to the discrimination result being consistent with a preset discrimination threshold. In an embodiment, the vehicle safety starting device based on multi-modal determines that the target object to which the heartbeat signal belongs is the safety starting object of the vehicle in response to the discrimination result being consistent with 1, and determines that the target object to which the heartbeat signal belongs is not the safety starting object of the vehicle in response to the discrimination result being consistent with 0.

[0132] Further, after the vehicle safety starting device based on multi-modal determines that the target object to which the heartbeat signal belongs is the safety starting object of the vehicle, the vehicle starting is controlled. For example, the vehicle door is automatically unlocked and the main driver door is opened, the vehicle is automatically powered on, the steering wheel is unlocked, and the vehicle remains in a normal driving state.

[0133] Further, after the vehicle safety starting device based on multi-modal determines that the target object to which the heartbeat signal belongs is not the safety starting object of the vehicle, and the continuous discrimination fails three times, the password authentication or the remote APP (Application, application program) authentication is performed.

[0134] Figure 3 A flowchart of an exemplary embodiment of the vehicle safety starting method based on multi-modal shown in the present application is shown in FIG. 6. As shown in FIG. 6, the vehicle safety starting method based on multi-modal includes the following steps. Figure 3As shown, the multi-modal based vehicle safe starting method comprises the following steps:

[0135] In step S310, the heartbeat signal, the face image and the fingerprint image in a preset area where the vehicle is located are acquired.

[0136] In step S320, the face image is processed by using an AdaBoost algorithm based on Haar-like features to obtain a denoised face image; and the denoised face image is subjected to two-dimensional principal component analysis to obtain a feature vector of the face image.

[0137] In step S330, the fingerprint image is processed based on a 2D Gabor filter to obtain an enhanced fingerprint image; and the enhanced fingerprint image is subjected to two-dimensional principal component analysis to obtain a feature vector of the fingerprint image.

[0138] In step S340, the feature vector of the face image and the feature vector of the fingerprint image are fused to obtain a feature fusion vector; the feature fusion vector is subjected to distance classification processing by using a nearest neighbor distance classification technique to obtain a distance score; and the distance score is subjected to normalization processing.

[0139] In step S350, the heartbeat signal is subjected to feature processing to obtain a heartbeat feature; the heartbeat feature is input into a preset matching model to obtain a matching score; and the matching score is subjected to normalization processing.

[0140] In step S360, the normalized distance score and the normalized matching score are fused in a score matching layer and are converted into a conditional probability density.

[0141] In step S370, the conditional probability density is subjected to discriminant processing based on a Bayesian decision technique to obtain a discriminant result.

[0142] In step S380, it is determined whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the discriminant result.

[0143] As can be seen, the embodiment flexibly mines the complementarity among the face, fingerprint and heartbeat three biological characteristics to improve the accuracy, robustness and safety of determining the safe starting object of the vehicle in a complex scene. While improving the safety level, the process of unlocking and starting the vehicle is greatly simplified, and an integrated safe starting scheme of the intelligent automobile is redefined.

[0144] In an embodiment, as Figure 4As shown, it is judged whether the target object to which the heartbeat signal belongs is a safe starting object of the vehicle, and if so, the vehicle door is opened and the vehicle is normally started; if not, other authentication methods are selected. When the object selects password authentication, the unlocking password is input through the identification screen; it is judged whether the unlocking password is successful, if so, the vehicle door is opened and the vehicle is normally started; if not, it is judged whether the unlocking times are greater than 3 times, if so, the vehicle is locked and the alarm system is started, and the alarm information is sent to the preset mobile phone terminal, and when the alarm is contacted, the process is ended. When the object selects remote APP authentication, it is judged whether the authorized object of the vehicle selects authentication, if so, the vehicle door is opened and the vehicle is normally started; if not, the vehicle is locked and the alarm system is started, and the alarm information is sent to the preset mobile phone terminal, and when the alarm is contacted, the process is ended.

[0145] In an application scenario, when the authentication identity fails for three times in succession, the integrated identity recognition screen on the B pillar of the vehicle prompts the object to select password authentication. When the object selects the password authentication method, the recognition screen enters the password authentication interface, and the object inputs a 6-digit password (the password distinguishes between uppercase and lowercase letters) according to the prompt information of the password authentication interface and submits after clicking the confirmation button. The vehicle safety starting device based on multi-modal verifies the input password immediately, and if the password matches the preset password, the recognition screen displays the information “authentication success” immediately, accompanied by a prompt sound, giving the object clear feedback, at which time the vehicle door can be normally opened and the vehicle can enter the normal power-on starting state. If the password does not match the preset password, the recognition screen gives feedback immediately, prompting the object to re-input, and when the number of password input errors exceeds 3 times, the vehicle is locked and enters the alarm state. In the alarm state, the left and right turn signals flash at a frequency of 250ms-ON / 250ms-OFF and last for 5 minutes, and the horn continuously alarms for 28s with a cycle of 1s and a duty cycle of 50%.

[0146] In another application scenario, when the authentication identity fails for three times in succession, the integrated identity recognition screen on the B pillar of the vehicle prompts the object to select remote APP authentication. When the object selects the remote APP authentication method, the unlocking request and the identity information of the object, such as the face image, are sent to the preset vehicle authorized authentication terminal, the identity of the object is authenticated by the preset vehicle authorized authentication terminal, and the authentication result is returned; if the authentication result received is that the identity authentication is passed, the recognition screen displays that the authentication is successful at this time, the vehicle door can be normally opened, and the vehicle can also enter the normal power-on starting state. If the authentication result received is that the identity authentication is not passed, the vehicle is locked and enters the alarm state at this time. In the alarm state, the left and right turn signals flash at a frequency of 250ms-ON / 250ms-OFF and last for 5 minutes, and the horn also continuously alarms for 28s with a cycle of 1s and a duty cycle of 50%.

[0147] Further, after the vehicle enters the alarm state, the authorized object of the vehicle can remotely disarm the alarm through the mobile phone APP. After disarming the alarm, the turn signal and horn of the vehicle stop working, and the vehicle returns to the normal state. After the alarm state is disarmed, an alarm report is sent to the authorized object of the vehicle and the vehicle service center, and the alarm report includes information such as the time, location, and reason for triggering the alarm. This can help the authorized object of the vehicle and the vehicle service center understand the safety status of the vehicle and take necessary measures in a timely manner to prevent similar incidents from occurring again.

[0148] As can be seen, the present embodiment provides a more flexible, convenient and secure vehicle use experience for the object through multi-modal fusion identity recognition authentication, password authentication, remote APP authentication mode, vehicle authentication alarm, alarm state disarming and other functions. At the same time, these functions further enhance the safety and manageability of the vehicle when it is started, allowing the object to better control the vehicle and ensure its safety.

[0149] Figure 5 is a block diagram of a multi-modal based vehicle safety starting device according to an example embodiment of the present application. As shown in Figure 5 The example multi-modal based vehicle safety starting device 500 includes an acquisition module 510, a fusion module 520, a matching module 530, and a safety starting object determination module 540. Specifically:

[0150] The acquisition module 510 is configured to acquire a heartbeat signal, a face image, and a fingerprint image within a preset area where the vehicle is located.

[0151] The fusion module 520 is configured to perform fusion processing on the face features of the face image and the fingerprint features of the fingerprint image to obtain a feature fusion result.

[0152] The matching module 530 is configured to perform matching processing based on the heartbeat signal to obtain a matching result.

[0153] The safety starting object determination module 540 is configured to determine whether the target object to which the heartbeat signal belongs is a safety starting object of the vehicle based on the feature fusion result and the matching result.

[0154] In the example multi-modal based vehicle safety starting device, a heartbeat signal, a face image and a fingerprint image in a preset area where the vehicle is located are acquired; then, face features of the face image and fingerprint features of the fingerprint image are fused to obtain a feature fusion result; a matching process is performed based on the heartbeat signal to obtain a matching result; finally, whether the target object to which the heartbeat signal belongs is a safety starting object of the vehicle is determined based on the feature fusion result and the matching result. Thus, the safety starting object of the vehicle is determined through the feature fusion result between the face image and the fingerprint image and the matching result of the heartbeat signal, and meanwhile, the face image, the fingerprint image and the heartbeat signal are taken into account, multi-item identity verification of the target object is implemented, and the safety of vehicle starting is improved.

[0155] The functions of the modules can be referred to the multi-modal based vehicle safety starting method embodiments, which will not be repeated here.

[0156] To implement the multi-modal based vehicle safety starting method in the above embodiments, the present application provides another electronic device, which can be specifically referred to Figure 6 , Figure 6 is a structural schematic diagram of an embodiment of the electronic device provided by the present application.

[0157] The electronic device 600 includes a memory 601 and a processor 602, wherein the memory 601 and the processor 602 are coupled.

[0158] The memory 601 is configured to store program data, and the processor 602 is configured to execute the program data to implement the multi-modal based vehicle safety starting method in the above embodiments.

[0159] In the embodiment, the processor 602 can also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 602 can be an integrated circuit chip with a signal processing capability. The processor 602 can also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor 602 can also be any conventional processor.

[0160] The present application also provides a computer readable storage medium, as shown in Figure 7 The computer readable storage medium 700 is configured to store program data 701, and the program data 701 is used to implement the multi-modal based vehicle safety starting method in the method embodiments of the present application when executed by a processor.

[0161] In the implementation of the method involved in the multi-modal vehicle safety starting method embodiment, the software function unit exists in the form of a product and is sold or used independently. When stored in a device, such as a computer readable storage medium, the technical solution of the present application can be embodied in the form of a software product stored in a storage medium, including a plurality of instructions for executing all or part of the steps of the method described in the embodiments of the present application by a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0162] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A multi-modal based vehicle safety start method, characterized in that, The method is applied to a vehicle, and the method comprises: obtaining a heartbeat signal, a face image and a fingerprint image in a preset area where the vehicle is located; fusing a face feature of the face image and a fingerprint feature of the fingerprint image to obtain a feature fusion result; performing matching processing based on the heartbeat signal to obtain a matching result; determining whether a target object to which the heartbeat signal belongs is a safe starting object of the vehicle based on the feature fusion result and the matching result; the step of determining whether the target object to which the heartbeat signal belongs is the safe starting object of the vehicle based on the feature fusion result and the matching result comprises: performing normalization processing on the feature fusion result and the matching result respectively to obtain a normalized feature fusion result and a normalized matching result; performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result; and in response to the discrimination result being consistent with a preset discrimination threshold, determining that the target object to which the heartbeat signal belongs is the safe starting object of the vehicle.

2. The method of claim 1, wherein, The feature fusion result comprises a distance score between the face feature and / or the fingerprint feature and a corresponding preset feature in a preset feature database, and the step of fusing the face feature of the face image and the fingerprint feature of the fingerprint image to obtain a feature fusion result comprises: performing normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image respectively to obtain a normalized face feature and a normalized fingerprint feature; performing weighted fusion processing on the normalized face feature and the normalized fingerprint feature to obtain a feature fusion vector; performing distance classification processing on the feature fusion vector by using a nearest neighbor distance classification technique to obtain a distance score.

3. The method of claim 2, wherein, The face feature of the face image comprises a feature vector of the face image, and before the step of performing normalization processing on the face feature of the face image and the fingerprint feature of the fingerprint image respectively to obtain a normalized face feature and a normalized fingerprint feature, the method comprises: performing denoising processing on the face image to obtain a denoised face image; performing two-dimensional principal component analysis processing on the denoised face image to obtain the feature vector of the face image.

4. The method of claim 1, wherein, The step of performing identity discrimination processing according to the normalized feature fusion result and the normalized matching result to obtain a discrimination result comprises: obtaining a first probability density of the normalized feature fusion result; obtaining a second probability density of the normalized matching result; performing fitting on the first probability density and the second probability density respectively to obtain a fitted first probability density and a fitted second probability density; performing discrimination on the fitted first probability density and the fitted second probability density by using a Bayesian decision technique to obtain a discrimination result.

5. The method of claim 1, wherein, The vehicle comprises an image acquisition device and a fingerprint acquisition device, the step of acquiring the heartbeat signal, the face image and the fingerprint image in a preset area where the vehicle is located comprises: continuously detecting the heartbeat signal in the preset area where the vehicle is located; in response to detecting the presence of the heartbeat signal in the preset area where the vehicle is located, controlling the image acquisition device and the fingerprint acquisition device to be turned on, and controlling the image acquisition device and the fingerprint acquisition device to continuously acquire; in response to the fingerprint acquisition device acquiring a fingerprint image, acquiring a face image of a target object corresponding to the fingerprint image.

6. The method of claim 1, wherein, The matching result comprises a matching score of the heartbeat signal and a preset heartbeat signal in a preset heartbeat signal library, and the step of performing matching processing based on the heartbeat signal to obtain a matching result comprises: calculating the similarity of the heartbeat signal and the preset heartbeat signal in the preset heartbeat signal library to obtain a similarity; determining the maximum similarity as the matching score.

7. The method of claim 1, wherein, The step of performing matching processing based on the heartbeat signal to obtain a matching result comprises: performing feature processing on the heartbeat signal to obtain heartbeat features; inputting the heartbeat features into a preset matching model to obtain a matching result.

8. An electronic device, comprising: comprise: a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, comprise: program data stored therein, which, when executed by a processor, is used to implement the method of any one of claims 1-7.

Images