User identification method, device and charging cabinet for drone charging cabinet

Abstract

The present invention discloses a user identification method, device and charging cabinet for an unmanned aerial vehicle charging cabinet. The method comprises: collecting audio and video data of a user to be identified; extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector; extracting image data from the audio and video data, extracting dynamic features, static features and key point features of a face based on the image data, and obtaining mixed features based on a singular value decomposition method; performing face recognition based on the mixed features; and performing user authentication based on the speech recognition results and the face recognition results. The method can significantly shorten the recognition time, improve the recognition accuracy and reduce the complexity of the algorithm.

Description

Technical Field

[0001] The present invention relates to the field of drone intelligent charging technology, and in particular to a user identification method and device for a drone charging cabinet, and a charging cabinet. Background Art

[0002] Drones are now widely used in various fields, with power line inspection being a key application. Multiple drones are required to conduct inspections within an area, and as the number of drones increases, their management becomes increasingly difficult. Consequently, drone charging cabinets have emerged. For example, patent document CN214303105U discloses an intelligently managed drone warehouse. The warehouse's main body is equipped with a fuselage storage cabinet and a lithium battery charging cabinet. The fuselage storage cabinet has multiple separate fuselage compartments, each equipped with a door equipped with an electronic tag reader and a smart door lock. The lithium battery charging cabinet has multiple lithium battery slots, each with a charge / discharge interface at the bottom. The interface is electrically connected to a charge / discharge management circuit, which charges and discharges the lithium batteries. To prevent unauthorized access and monitor operators' work in real time, an intelligent access control switch and video camera are installed at the main warehouse entrance. Video cameras are installed on the tops of both the fuselage storage cabinet and the lithium battery charging cabinet. The intelligent access control switch and video camera are each connected to a central controller. Smart access switches can use facial recognition or fingerprint passwords to identify operators. Video cameras monitor and record operators entering the warehouse and send feedback to a central controller. However, access switches that rely solely on facial recognition or fingerprint passwords offer limited security. Summary of the Invention

[0003] The present invention provides a user identification method and device for a drone charging cabinet, and a charging cabinet, which can effectively improve the safety of the drone charging cabinet.

[0004] A method for identifying a user of a drone charging cabinet, comprising:

[0005] Collect audio and video data of the user to be identified;

[0006] Extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector;

[0007] Extracting image data from the audio and video data, extracting dynamic features, static features, and key point features of a face based on the image data, and obtaining mixed features based on a singular value decomposition method;

[0008] performing face recognition according to the mixed features;

[0009] User authentication is performed based on voice recognition and face recognition results.

[0010] Furthermore, the sound signal is processed to form a speech vector, including:

[0011] Dividing the sound signal into multiple frames and performing discrete Fourier transform to extract spectrum information;

[0012] Calculating Mel cepstral coefficients according to the spectrum information;

[0013] Perform discrete cosine transform on the Mel cepstral coefficients to obtain the speech vector.

[0014] Furthermore, extracting dynamic features and static features of the face based on the image data includes:

[0015] Extracting Haar-Like features from the image data using contrast values between adjacent rectangular pixel groups;

[0016] Add the pixel intensities of each Haar-Like feature area and calculate the difference between the sums of the pixel intensities of each Haar-Like feature area;

[0017] According to the pre-trained classifier, the difference between pixel intensities and pixels is used for feature classification to obtain the dynamic features of lips, the static features of eyes and the static features of nose.

[0018] Furthermore, extracting key features of the face based on the image data includes:

[0019] identifying scale- and direction-invariant points in the image data using a difference of Gaussian functions to obtain key points;

[0020] Taking the key point as the center of the circle, the pixels within a preset range are calculated for gradient, a histogram is created, and a direction is assigned to the key point according to the histogram;

[0021] A descriptor regarding position, scale, and orientation is established for each key point to obtain the key point feature.

[0022] Furthermore, hybrid features are obtained based on the singular value decomposition method, including:

[0023] Expressing the dynamic features, static features and key point features in a matrix;

[0024] The dynamic feature matrix and the static feature matrix are taken as the left singular feature matrix, and the key point feature matrix is taken as the right singular feature matrix to calculate and obtain the mixed feature matrix.

[0025] Furthermore, performing speech recognition according to the speech vector includes:

[0026] The speech vector is compared with a pre-established registered user speech model to determine whether the similarity between the speech vector and the registered user speech model exceeds a speech recognition threshold.

[0027] Furthermore, performing face recognition based on the mixed features includes:

[0028] The mixed feature is compared with a pre-established registered user face model to determine whether the similarity between the mixed feature and the registered user face model exceeds a facial recognition threshold.

[0029] Furthermore, user authentication is performed based on the voice recognition results and the face recognition results, including:

[0030] When the similarity between the voice vector and the registered user voice model exceeds a voice recognition threshold, and the similarity between the mixed feature and the registered user face model exceeds a face recognition threshold, the user to be identified is determined to be a registered user.

[0031] A user identification device for a drone charging cabinet, comprising:

[0032] An acquisition module, used to collect audio and video data of the user to be identified;

[0033] A speech recognition module, configured to extract a sound signal from the audio and video data, process the sound signal to form a speech vector, and perform speech recognition based on the speech vector;

[0034] An image feature extraction module is used to extract image data from the audio and video data, extract dynamic features, static features and key point features of the face based on the image data, and obtain mixed features based on the singular value decomposition method;

[0035] A recognition module, configured to perform face recognition based on the mixed features;

[0036] The authentication module is used to perform user authentication based on voice recognition results and face recognition results.

[0037] A charging cabinet includes a processor and a storage device, wherein the storage device stores a plurality of instructions, and the processor is configured to read and execute the instructions:

[0038] Collect audio and video data of the user to be identified;

[0039] Extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector;

[0040] Extracting image data from the audio and video data, extracting dynamic features, static features, and key point features of a face based on the image data, and obtaining mixed features based on a singular value decomposition method;

[0041] performing face recognition according to the mixed features;

[0042] User authentication is performed based on voice recognition and face recognition results.

[0043] The user identification method, device, and charging cabinet of the drone charging cabinet provided by the present invention have at least the following beneficial effects:

[0044] (1) User identity authentication based on multimodal biometric recognition technology of voice and facial features has high recognition rate and robustness, intelligently verifies user identity, significantly shortens recognition time, improves recognition accuracy, and reduces algorithm complexity;

[0045] (2) By extracting and fusing static features, dynamic features and key point features, hybrid features are obtained based on the singular value decomposition method for facial recognition, which can effectively reduce the dimension of the vector of the hybrid features and improve the recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of an embodiment of the user identification method for the drone charging cabinet provided by the present invention.

[0047] Figure 2 This is a flowchart of an embodiment of feature recognition in the user identification method of the drone charging cabinet provided by the present invention.

[0048] Figure 3 A schematic structural diagram of an embodiment of a user identification device for a drone charging cabinet provided by the present invention.

[0049] Figure 4 A schematic structural diagram of an embodiment of a charging cabinet provided by the present invention. DETAILED DESCRIPTION

[0050] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0051] refer to Figure 1 In some embodiments, a method for identifying a user of a drone charging cabinet is provided, comprising:

[0052] S1. Collect audio and video data of the user to be identified;

[0053] S2. extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector;

[0054] S3. Extracting image data from the audio and video data, extracting dynamic features, static features, and key point features of the face based on the image data, and obtaining mixed features based on a singular value decomposition method;

[0055] S4. performing face recognition based on the mixed features;

[0056] S5. Perform user authentication based on the voice recognition results and face recognition results.

[0057] Specifically, in step S1, a video device can be set up in the drone charging cabinet to obtain audio and video data of the user to be identified.

[0058] Furthermore, in step S2, the sound signal is processed to form a speech vector, including:

[0059] S21, dividing the sound signal into multiple frames and performing discrete Fourier transform to extract spectrum information;

[0060] S22, calculating Mel cepstral coefficients according to the spectrum information;

[0061] S23. Perform discrete cosine transform according to the Mel cepstral coefficient to obtain the speech vector.

[0062] In this embodiment, the speech vector is extracted by MFCC (Mel-Frequency Cepstral Coefficients).

[0063] In step S21, the sound signal is divided into multiple frames, thereby analyzing it into short time periods, and the spectrum information of discrete frequency bands is extracted using Fourier transform. Fourier transform converts the signal from the time domain to the frequency domain to prepare for MEL frequency warping.

[0064] Furthermore, in step S22, for a given frequency, the Mel cepstral coefficient is calculated using the following formula:

[0065]

[0066] Where f is the actual frequency, m f is the Mel cepstral coefficient.

[0067] In step S3, discrete cosine transform is performed based on the Mel cepstral coefficients. After discrete cosine transform, the set of all coefficients is a speech vector, which is used for speech recognition.

[0068] Further, refer to Figure 2 In step S3, the dynamic features and static features of the face are extracted according to the image data, including:

[0069] S31, extracting Haar-Like features from the image data using contrast values between adjacent rectangular pixel groups;

[0070] S32, adding the pixel intensities of each Haar-Like feature region, and calculating the difference between the sums of the pixel intensities of each Haar-Like feature region;

[0071] S33. Based on the pre-trained classifier, feature classification is performed using the difference between pixel intensities and to obtain dynamic features of the lips, static features of the eyes, and static features of the nose.

[0072] In this embodiment, Viola–Jones is used for facial recognition. First, Haar-Like features are extracted from the image data using the contrast values between adjacent rectangular pixel groups. The contrast differences between the pixel groups are used to determine the light and dark areas of the face detection target. The pixel intensities of each Haar-Like feature area are added, and the difference between the sum of the pixel intensities of each Haar-Like feature area is calculated. The eye area in the face image is darker than the cheek area. Based on a pre-trained classifier, the difference between the sum of the pixel intensities is used to perform feature classification to obtain dynamic features of the lips, static features of the eyes, and static features of the nose.

[0073] Further, refer to Figure 2 In step S3, the key features of the face are extracted according to the image data, including:

[0074] S34, using a difference of Gaussian functions to identify points in the image data whose scale and direction are invariant, to obtain key points;

[0075] S35, taking the key point as the center of the circle, performing gradient calculation on pixels within a preset range around the key point, establishing a histogram, and assigning a direction to the key point according to the histogram;

[0076] S36: Create a descriptor about the position, scale and direction for each key point to obtain the key point feature.

[0077] In this embodiment, the scale-invariant feature transform (SIFT) is used for feature extraction. The difference of Gaussian functions is used to identify scale- and direction-invariant points in the image data, obtaining key pixels. These key pixels are then located, retaining only stable and significant features. Gradients are calculated for pixels within a preset range around the key point, creating a histogram. Directions are assigned to the key points based on the histogram. Specifically, a histogram with 36 bins spanning 360 is constructed. The highest peak in the histogram and peaks greater than 80% are considered as direction assignments. Key points with 16 neighborhoods are divided into 16 sub-blocks of 4×4 size. Eight directional histograms are generated for each sub-block, resulting in 128 key point feature vectors.

[0078] Furthermore, in step S3, the hybrid features are obtained based on the singular value decomposition method, including:

[0079] S37, expressing the dynamic features, static features and key point features in a matrix;

[0080] S38. The dynamic feature matrix and the static feature matrix are used as the left singular feature matrix, and the key point feature matrix is used as the right singular feature matrix to calculate and obtain a mixed feature matrix.

[0081] Specifically, the calculation formula is as follows:

[0082]

[0083] Among them, A n×p is the n×p mixed feature matrix, U n×n is the left singular characteristic matrix, S n×p is the singular value matrix, is the right singular eigenvalue matrix.

[0084] The hybrid feature combines the dynamic features of lips, static features of eyes, and static features of the nose extracted by the Viola–Jones algorithm with key point features extracted by the SIFT algorithm. Singular value decomposition (SVD) is used as a fusion method to integrate all features, making the recognition system more accurate. SVD reduces the dimensionality of the hybrid feature and generates a compact feature vector.

[0085] Furthermore, in step S2, speech recognition is performed according to the speech vector, including:

[0086] The speech vector is compared with a pre-established registered user speech model to determine whether the similarity between the speech vector and the registered user speech model exceeds a speech recognition threshold.

[0087] Furthermore, in step S4, face recognition is performed based on the mixed features, including:

[0088] The mixed feature is compared with a pre-established registered user face model to determine whether the similarity between the mixed feature and the registered user face model exceeds a facial recognition threshold.

[0089] Furthermore, in step S5, user authentication is performed based on the voice recognition results and the face recognition results, including:

[0090] When the similarity between the voice vector and the registered user voice model exceeds a voice recognition threshold, and the similarity between the mixed feature and the registered user face model exceeds a face recognition threshold, the user to be identified is determined to be a registered user.

[0091] That is, when both the face and voice match, the user is confirmed to be a registered user, further improving the security of the smart cabinet.

[0092] Furthermore, when a user registers, pre-collected audio and video data of the registered user is used to establish a multimodal biometric recognition model based on voice and facial features. This multimodal biometric recognition model is trained based on the registered user's voice and facial image to obtain a registered user face model and a registered user voice model. Multiple registered user face models and registered user voice models are stored to form a model library of registered user face models and registered user voice models.

[0093] The method provided in the above embodiment has at least the following beneficial effects:

[0094] (1) User identity authentication based on multimodal biometric recognition technology of voice and facial features has high recognition rate and robustness, intelligently verifies user identity, significantly shortens recognition time, improves recognition accuracy, and reduces algorithm complexity;

[0095] (2) By extracting and fusing static features, dynamic features and key point features, hybrid features are obtained based on the singular value decomposition method for facial recognition, which can effectively reduce the dimension of the vector of the hybrid features and improve the recognition efficiency.

[0096] refer to Figure 3 In some embodiments, a user identification device for a drone charging cabinet is provided, comprising:

[0097] The acquisition module 201 is used to collect audio and video data of the user to be identified;

[0098] A speech recognition module 202 is configured to extract a sound signal from the audio and video data, process the sound signal to form a speech vector, and perform speech recognition based on the speech vector;

[0099] An image feature extraction module 203 is configured to extract image data from the audio and video data, extract dynamic features, static features, and key point features of a face based on the image data, and obtain mixed features based on a singular value decomposition method;

[0100] Recognition module 204, configured to perform face recognition based on the mixed features;

[0101] The authentication module 205 is used to perform user authentication based on the voice recognition results and the face recognition results.

[0102] Furthermore, the speech recognition module 202 is also used to divide the sound signal into multiple frames and perform discrete Fourier transform to extract spectrum information; calculate Mel cepstral coefficients based on the spectrum information; and perform discrete cosine transform based on the Mel cepstral coefficients to obtain the speech vector.

[0103] The image feature extraction module 203 is further configured to extract Haar-Like features from the image data using contrast values between adjacent rectangular pixel groups; add the pixel intensities of each Haar-Like feature region and calculate the difference between the sums of the pixel intensities of each Haar-Like feature region; and perform feature classification based on the difference between the sums of the pixel intensities according to a pre-trained classifier to obtain dynamic features of the lips, static features of the eyes, and static features of the nose.

[0104] Furthermore, the image feature extraction module 203 is also used to use the difference of Gaussian functions to identify points in the image data whose scale and direction remain unchanged to obtain key points; with the key point as the center of the circle, the pixels within a preset range around it are calculated with gradients, a histogram is established, and directions are assigned to the key points according to the histogram; a descriptor about position, scale and direction is established for each key point to obtain the key point features.

[0105] Furthermore, the image feature extraction module 203 is also used to express the dynamic features, static features and key point features in a matrix; the dynamic feature matrix and the static feature matrix are used as the left singular feature matrix, and the key point feature matrix is used as the right singular feature matrix to calculate and obtain a mixed feature matrix.

[0106] Furthermore, the speech recognition module 202 is further configured to compare the speech vector with a pre-established registered user speech model to determine whether the similarity between the speech vector and the registered user speech model exceeds a speech recognition threshold.

[0107] Furthermore, the recognition module 204 is further configured to compare the mixed feature with a pre-established registered user face model to determine whether the similarity between the mixed feature and the registered user face model exceeds a facial recognition threshold.

[0108] The authentication module 205 is also used to determine that the user to be identified is a registered user when the similarity between the voice vector and the registered user voice model exceeds the voice recognition threshold and the similarity between the mixed feature and the registered user face model exceeds the face recognition threshold.

[0109] The device provided by the above embodiment has at least the following beneficial effects:

[0110] (1) User identity authentication based on multimodal biometric recognition technology of voice and facial features has high recognition rate and robustness, intelligently verifies user identity, significantly shortens recognition time, improves recognition accuracy, and reduces algorithm complexity;

[0111] (2) By extracting and fusing static features, dynamic features and key point features, hybrid features are obtained based on the singular value decomposition method for facial recognition, which can effectively reduce the dimension of the vector of the hybrid features and improve the recognition efficiency.

[0112] refer to Figure 4 In some embodiments, a charging cabinet is further provided, including a processor 301 and a storage device 302, wherein the storage device 302 stores a plurality of instructions, and the processor 301 is configured to read the instructions and execute:

[0113] Collect audio and video data of the user to be identified;

[0114] Extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector;

[0115] Extracting image data from the audio and video data, extracting dynamic features, static features, and key point features of a face based on the image data, and obtaining mixed features based on a singular value decomposition method;

[0116] performing face recognition according to the mixed features;

[0117] User authentication is performed based on voice recognition and face recognition results.

[0118] In addition, the charging cabinet also includes a video acquisition device 303 for capturing audio and video data of the user to be identified.

[0119] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the invention. Thus, the present invention is intended to include such changes and modifications as fall within the scope of the claims and their equivalents.

Claims

1. A user identification method for a drone charging cabinet, characterized in that: include: Collect audio and video data of the user to be identified; Extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector; Extracting image data from the audio and video data, extracting dynamic features, static features, and key point features of a face based on the image data, and obtaining mixed features based on a singular value decomposition method; performing face recognition according to the mixed features; Perform user authentication based on voice recognition and face recognition results; Extracting key features of the face according to the image data includes: identifying scale- and direction-invariant points in the image data using a difference of Gaussian functions to obtain key points; Taking the key point as the center of the circle, the pixels within a preset range are calculated for gradient, a histogram is created, and a direction is assigned to the key point according to the histogram; Establishing a descriptor about position, scale and direction for each key point to obtain the key point feature; The hybrid features are obtained based on the singular value decomposition method, including: Expressing the dynamic features, static features and key point features in a matrix; The dynamic feature matrix and the static feature matrix are taken as the left singular feature matrix, and the key point feature matrix is taken as the right singular feature matrix to calculate and obtain the mixed feature matrix.

2. The method according to claim 1, characterized in that Processing the sound signal to form a speech vector includes: Dividing the sound signal into multiple frames and performing discrete Fourier transform to extract spectrum information; Calculating Mel cepstral coefficients according to the spectrum information; Perform discrete cosine transform on the Mel cepstral coefficients to obtain the speech vector.

3. The method according to claim 1, characterized in that Extracting dynamic features and static features of the face based on the image data includes: Extracting Haar-Like features from the image data using contrast values between adjacent rectangular pixel groups; Add up the pixel intensities of each Haar-Like feature area and calculate the difference between the sums of the pixel intensities of each Haar-Like feature area; According to the pre-trained classifier, the difference between pixel intensities and pixels is used for feature classification to obtain the dynamic features of lips, the static features of eyes and the static features of nose.

4. The method according to claim 1, wherein Performing speech recognition according to the speech vector includes: The speech vector is compared with a pre-established registered user speech model to determine whether the similarity between the speech vector and the registered user speech model exceeds a speech recognition threshold.

5. The method according to claim 4, characterized in that Performing face recognition based on the mixed features includes: The mixed feature is compared with a pre-established registered user face model to determine whether the similarity between the mixed feature and the registered user face model exceeds a facial recognition threshold.

6. The method according to claim 5, characterized in that User authentication is performed based on voice recognition and face recognition results, including: When the similarity between the voice vector and the registered user voice model exceeds a voice recognition threshold, and the similarity between the mixed feature and the registered user face model exceeds a face recognition threshold, the user to be identified is determined to be a registered user.

7. A user identification device for a drone charging cabinet, characterized in that: include: An acquisition module, used to collect audio and video data of the user to be identified; A speech recognition module, configured to extract a sound signal from the audio and video data, process the sound signal to form a speech vector, and perform speech recognition based on the speech vector; An image feature extraction module is used to extract image data from the audio and video data, extract dynamic features, static features and key point features of the face based on the image data, and obtain mixed features based on the singular value decomposition method; A recognition module, configured to perform face recognition based on the mixed features; Authentication module, used to perform user authentication based on voice recognition results and face recognition results; The image feature extraction module is further configured to use a difference of Gaussian functions to identify points in the image data whose scale and direction remain unchanged, thereby obtaining key points; performing gradient calculations on pixels within a preset range around the key point, creating a histogram, and assigning directions to the key points based on the histogram; Establishing a descriptor about position, scale and direction for each key point to obtain the key point feature; The image feature extraction module is also used to express the dynamic features, static features and key point features in a matrix; the dynamic feature matrix and the static feature matrix are used as the left singular feature matrix, and the key point feature matrix is used as the right singular feature matrix to calculate and obtain a mixed feature matrix.

8. A charging cabinet, characterized in that: The system comprises a processor and a storage device, wherein the storage device stores a plurality of instructions, and the processor is configured to read the instructions and execute: Collect audio and video data of the user to be identified; Extracting a sound signal from the audio and video data, processing the sound signal to form a speech vector, and performing speech recognition based on the speech vector; Extracting image data from the audio and video data, extracting dynamic features, static features, and key point features of a face based on the image data, and obtaining mixed features based on a singular value decomposition method; performing face recognition according to the mixed features; Perform user authentication based on voice recognition and face recognition results; Extracting key features of the face according to the image data includes: identifying scale- and direction-invariant points in the image data using a difference of Gaussian functions to obtain key points; Taking the key point as the center of the circle, the pixels within a preset range are calculated for gradient, a histogram is created, and a direction is assigned to the key point according to the histogram; Establishing a descriptor about position, scale and direction for each key point to obtain the key point feature; The hybrid features are obtained based on the singular value decomposition method, including: Expressing the dynamic features, static features and key point features in a matrix; The dynamic feature matrix and the static feature matrix are taken as the left singular feature matrix, and the key point feature matrix is taken as the right singular feature matrix to calculate and obtain the mixed feature matrix.

Images