Authority level authentication method and device and authority level setting method and device

Abstract

The invention discloses a permission level authentication method and device and a permission level setting method and device, and particularly relates to the technical field of vehicle voice control, and the method comprises the steps: extracting audio vector features from audio data through a depth model; face vector features are extracted from the face recognition features through a convolutional network; splicing the audio vector features and the face vector features to form comprehensive features; comparing the comprehensive features with an identity feature library, calculating the similarity, and judging the identity; and granting a corresponding authority level according to the judged identity. According to the permission level authentication method and device and the permission level setting method and device provided by the invention, the security problem in the vehicle voice interaction process is effectively solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle voice control technology, and in particular to an authority level authentication method, an authority level authentication device, an authority level setting method, and an authority level setting device. Background Art

[0002] In-vehicle voice interaction systems are widely used in smart vehicles, especially new energy vehicles. Currently, various manufacturers' systems generally pursue recognition accuracy, recall rate and functional support. However, unlike home voice interaction systems such as smart speakers, cars need to be responsible for safety, and there are also security issues in the voice interaction process.

[0003] At present, manufacturers are improving voice interaction functions, such as: CN202211478532.7 In-vehicle interactive voice game method, system and computer-readable storage medium, which supports the expansion of game functions. The voice interaction function is expanded to provide an in-vehicle interactive voice game method for game control.

[0004] But safety is also necessary in car use scenarios, such as:

[0005] Scenario 1: A user meets a stranger while parking in the wild. The stranger asks the user to open the car door and window. Without permission control, the user is likely to open the vehicle door and window, which could put the user in danger.

[0006] Scenario 2: When the user is driving at high speed, a child in the car suddenly asks to open the window. Opening the window may cause danger.

[0007] Scenario 3: Car owner A wants to lend his car to friend B, but wants to give B certain voice interaction permissions, such as music on demand. However, permissions that consume a lot of data, such as video playback, need to be prohibited.

[0008] In summary, in addition to some restrictions on the front-end execution, the voice interaction system also needs to be graded and different permissions are granted to different users. However, no other manufacturers have voice security control solutions. This solution innovatively proposes security graded control for voice interaction. Summary of the invention

[0009] The purpose of the present invention is to provide an authority level authentication method and device, an authority level setting method and device, which effectively solve the security problems in the vehicle voice interaction process.

[0010] The present invention provides the following scheme:

[0011] According to one aspect of the present invention, a permission level authentication method is provided, the permission level authentication method comprising:

[0012] Extract audio vector features from audio data through a deep model;

[0013] The face recognition features are extracted through the convolutional network to extract the face vector features;

[0014] Concatenate the audio vector features with the face vector features to form a comprehensive feature;

[0015] Compare the comprehensive features with the identity feature database, calculate the similarity, and determine the identity;

[0016] According to the determined identity, the corresponding permission level is granted.

[0017] Optionally, extract audio vector features from the audio data through a deep model, including:

[0018] Audio vector features are extracted through the ECAPA-TDNN network.

[0019] Optionally, the ECAPA-TDNN network includes: one-dimensional squeeze-excitation residual block, channel- and context-dependent statistical pooling, multi-layer feature aggregation and summation, and AAM-Softmax classifier.

[0020] Optionally, the training process of the ECAPA-TDNN network includes:

[0021] The voice data is converted into channels, sampled, and intercepted to construct the voice data heard in a real indoor scene;

[0022] The processed speech data is frequency-masked and time-masked to enhance the speech signal features, the audio data is translated to increase the amount of speech data, and the Mel-spectrogram features and label processing are extracted from the processed speech signal to construct a data set.

[0023] The training set in the obtained data set is input into the ECAPA-TDNN network for classification.

[0024] Optionally, the face recognition features are used to extract face vector features through a convolutional network, including:

[0025] ResNet is used to extract face vector features.

[0026] According to two aspects of the present invention, a method for setting an authority level is provided, the method comprising:

[0027] Extract audio vector features from audio data through a deep model;

[0028] The face recognition features are extracted through the convolutional network to extract the face vector features;

[0029] Concatenate the audio vector features with the face vector features to form a comprehensive feature;

[0030] Bind the comprehensive features to the identity and store them in the identity feature library;

[0031] In the identity feature library, users are granted corresponding permission levels.

[0032] According to three aspects of the present invention, there is provided a permission level authentication device, the permission level authentication device comprising:

[0033] A first audio feature extraction unit, configured to extract audio vector features from audio data through a deep model;

[0034] A first face feature extraction unit, used to extract face vector features from face recognition features through a convolutional network;

[0035] A first concatenation unit, used for concatenating the audio vector feature with the face vector feature to form a comprehensive feature;

[0036] An identity determination unit is used to compare the comprehensive features with the identity feature library, calculate the similarity, and determine the identity;

[0037] The first authorization unit is used to grant a corresponding authority level according to the determined identity.

[0038] According to four aspects of the present invention, there is provided a device for setting a permission level, the device comprising:

[0039] A second audio feature extraction unit, used to extract audio vector features from the audio data through a deep model;

[0040] A second face feature extraction unit, used for extracting face vector features from face recognition features through a convolutional network;

[0041] A second concatenation unit is used to concatenate the audio vector feature with the face vector feature to form a comprehensive feature;

[0042] An identity storage unit is used to bind the comprehensive features to the identity and store them in the identity feature library;

[0043] The second authorization unit is used to grant the corresponding authority level to the user in the identity feature library.

[0044] According to five aspects of the present invention, there is provided an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0045] A computer program is stored in the memory, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method.

[0046] According to a sixth aspect of the present invention, there is provided a computer-readable storage medium, comprising: a computer program executable by an electronic device is stored therein, and when the computer program runs on the electronic device, the electronic device executes the steps of the method.

[0047] Through the above scheme, the following beneficial technical effects are obtained:

[0048] At present, voice interaction systems are widely installed in vehicles, but the focus of various manufacturers on voice interaction systems is the recognition function, and a lot of effect and performance optimization work is carried out. This system solution is to improve the voice interaction system at the security level. It is completely different from other current solutions. This solution divides voice interaction control into three levels. The first level is full authority, including all control permissions such as doors, trunks, in-car equipment, entertainment, etc.; the second level is the use permission of in-car equipment, entertainment, etc.; the third level is entertainment permission, which is the lowest permission and will not cause security risks; the fourth level is no voice control permission; through permission classification measures, users can obtain security benefits, as follows:

[0049] 1. Safety. Car owners can set permissions at different levels to conveniently control the permissions of vehicle users and avoid arbitrary control by people not approved by the car owner. This is necessary during the use of the car. For example, children can be given level 3 permissions and cannot open and close doors by voice control, which may cause dangerous operations, but can control entertainment on demand. Family members can be given level 2 permissions to control air conditioning, seats and other in-car equipment while riding to improve the riding experience. Other people have level 4 permissions and cannot control any equipment by voice. This prevents people outside the car from randomly controlling the vehicle through voice interaction during the use of the car, which may cause danger to the vehicle or the owner.

[0050] 2. Convenience. This system supports different levels of passwords provided by the car owner or authorized face photos transmitted through the mobile phone, which can conveniently control the vehicle. For example, when remotely lending a vehicle, the user can conveniently use the different levels of permissions set by the car owner to control the vehicle equipment. It does not require the car owner and the vehicle user to set permissions on the vehicle at the same time, reducing the difficulty of operation.

[0051] 3. Accuracy: User authority identification is performed through multimodal methods, including voiceprint, key, and face photo, to improve the recognition success rate and enhance the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1It is a flowchart of a method for authenticating authority levels provided by one or more embodiments of the present invention.

[0053] Figure 2 It is a flowchart of a method for authenticating authority levels provided by one or more embodiments of the present invention.

[0054] Figure 3 It is a flowchart of the training process of the ECAPA-TDNN network provided by one or more embodiments of the present invention.

[0055] Figure 4 It is a flowchart of the first facial feature extraction operation in the permission level authentication method provided by one or more embodiments of the present invention.

[0056] Figure 5 It is a flowchart of a method for setting a permission level provided by one or more embodiments of the present invention.

[0057] Figure 6 is a flowchart of a permission level setting operation provided by one or more embodiments of the present invention.

[0058] Figure 7 It is a flow chart of vehicle-user voice interaction provided by one or more embodiments of the present invention.

[0059] Figure 8 is a flow chart of an identity recognition process provided by one or more embodiments of the present invention.

[0060] Fig. 9 It is a structural diagram of an authority level authentication device provided by one or more embodiments of the present invention.

[0061] Fig.10 It is a structural diagram of a permission level setting device provided by one or more embodiments of the present invention.

[0062] Fig.11 is a structural diagram of an electronic device provided by one or more embodiments of the present invention. DETAILED DESCRIPTION

[0063] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Figure 1 is a flowchart of a method for setting a permission level provided by one or more embodiments of the present invention. Figure 1 , the permission level setting method includes the following steps:

[0065] S11, extracting audio vector features from audio data through a deep model.

[0066] S12, extracting face vector features from face recognition features through a convolutional network.

[0067] S13, concatenating the audio vector feature with the face vector feature to form a comprehensive feature.

[0068] S14, comparing the comprehensive features with the identity feature library, calculating the similarity, and determining the identity.

[0069] S15, granting a corresponding authority level according to the determined identity.

[0070] Commands are issued to the vehicle through voice, and the vehicle recognizes the received voice commands and then executes the corresponding commands based on the results of voice recognition, enabling the owner to control various functions of the vehicle. This is now a function that ordinary car owners use every day.

[0071] However, in the process of implementing the above functions, there is an important issue: how to accurately determine the identity of the car owner. The reason why it is important is that it involves the data security of the car owner and even the car company. Imagine that if the identity of the car owner is stolen, the criminals can forge the identity of the car owner and pretend to be the car owner to log in to the system installed on the vehicle, then he can use all kinds of data of the car owner stored in the vehicle without restriction. What's more, he can also use all kinds of data tailored for the vehicle by the car company. Obviously, this will pose a threat to the personal data of the car owner and the corporate data of the car company.

[0072] Most of the data security controls currently used require the car owner to log in to the system using the account password before executing the voice command receiving process. Once the car owner completes such a login operation, the system no longer has any security protection measures. Obviously, the security of this security control method is not enough.

[0073] Different from the previous identity authentication method, this application uses the audio vector features contained in the voice command to complete the user identity verification during the process of receiving and parsing each voice command. In order to further ensure security, in addition to using audio vector features, face vector features are further used. When actually performing the identity authentication of the car owner, audio vector features and face vector features are used in combination.

[0074] It should be understood that the audio vector feature is obtained by analyzing the voice data in the voice command transmitted by the user in real time. The face vector feature is obtained by analyzing the face image collected in real time.

[0075] Since both features are obtained by analyzing real-time collected data, there is no need for prior login operations or prior security settings. The owner's identity authentication can be completed solely by relying on real-time interactive data, which is more convenient.

[0076] Moreover, since the above-mentioned identity authentication uses two different data sources as the basis for the identity authentication operation, that is, in the actual identity authentication operation, it is necessary to comprehensively consider the audio features and image features, thereby further improving the security of the identity authentication operation.

[0077] The above-mentioned identity authentication and permission granting operations need to be based on an identity feature library. The identity feature library can be configured locally on the vehicle computer or in the cloud that maintains a communication connection with the vehicle computer.

[0078] The identity feature database stores the feature data of different car owners. For each car owner, the feature data stored in the identity feature database includes both the audio vector features and the face vector features of the car owner. When performing identity authentication operations, you only need to compare the collected identity feature data with the feature data stored in the identity feature database one by one to complete the identity authentication of the car owner who issued the voice command.

[0079] After completing the identity verification and determination, a certain level of authority can be granted to the verified car owner. In this way, the car owner's voice control of the vehicle is completed based on the authorized authority level, which greatly improves the security of the car owner's voice control process of the vehicle.

[0080] In the above feature collection process, the extraction of audio vector features is completed by the deep model. The extraction of face vector features is completed by the convolutional network. After completing the above audio vector feature and face vector feature extraction operations, the audio vector features and face vector features are spliced ​​to form the comprehensive features of the car owner.

[0081] In the identity authentication operation, the formed comprehensive features are compared with the features stored in the identity feature library. By calculating the similarity, the identity of the car owner is determined, and the identity authentication operation is completed.

[0082] Figure 2 It is a flowchart of a permission level authentication method provided by one or more embodiments of the present invention.

[0083] See also Figure 2 , the permission level authentication method includes the following steps:

[0084] S21, extract audio vector features through the ECAPA-TDNN network.

[0085] S22, ResNet is used to extract face vector features.

[0086] S23, concatenates the audio vector feature and the face vector feature to form a comprehensive feature.

[0087] S24, comparing the comprehensive features with the identity feature library, calculating the similarity, and determining the identity.

[0088] S25, granting a corresponding authority level according to the determined identity.

[0089] In by Figure 2 In the illustrated embodiment, the deep model refers to the ECAPA-TDNN network. In addition, the convolutional network refers to the ResNet network model.

[0090] The improved time-delay neural network (TDNN) architecture can be used for various acoustic scene classifications. We use the Emphasized Channel Attention, Propagation and Aggregation in TDNN (ECAPA-TDNN) as the backbone model for speech recognition. The speech of different voices is input into ECAPA-TDNN to extract the deep embedding features of the speech of different car owners, and the speech of different car owners is classified through the AAM-Softmax classifier.

[0091] The ECAPA-TDNN backbone network is mainly composed of the following four parts: one-dimensional squeeze-excitation residual blocks (Squeeze-Excitation Res2Blocks, SE-Res2Blocks), channel- and context-dependent statistical pooling, multi-layer feature aggregation and summation, and AAM-Softmax classifier.

[0092] As a one-dimensional squeezed excitation module in the field of computer vision, the one-dimensional squeezed excitation residual block can effectively model the correlation of global channels. The first component of the one-dimensional squeezed excitation residual block is the squeezing operation, which involves calculating the mean vector of frame-level features across the time domain.

[0093] z is then used in the activation operation to calculate the weights for each channel:

[0094] s=σ(W2f(W1z+b1)+b2).

[0095] Let σ be the sigmoid function, f be the nonlinear transformation function, W1∈R R×C and W2∈R C×RThis operation acts as a bottleneck layer, where C and R refer to the number of input channels and the reduced dimensionality, respectively. The resulting vector s contains weights sc between 0 and 1, with each channel multiplied by the weights separately.

[0096] The backbone network of the convolutional network usually uses ResNet-50 or ResNet-101 integrated with Feature Pyramid Network (FPN). The ResNet backbone network can be divided into three parts: input, intermediate convolution part, and output. The input layer consists of a convolution layer with a stride of 2 and a convolution kernel size of 7×7 and a maximum pooling layer with a stride of 2 and a size of 3×3. The output layer is a classic average pooling and fully connected layer. The intermediate convolution part includes 4 stages, each of which consists of multiple residual blocks and a downsampling block.

[0097] Although convolution operations can effectively obtain local information, and models based on convolutional neural networks have achieved remarkable results in a variety of visual tasks. However, for tasks such as instance segmentation, if scene information can be obtained and associated from a larger domain, it will be very helpful for the model to learn the relationship between targets. However, if you want to obtain global information, the network architecture based on convolution operations needs to stack multiple convolution layers to expand the receptive field, making the model more complex.

[0098] Figure 3 is a flow chart of the training process of the ECAPA-TDNN network provided by one or more embodiments of the present invention. Figure 3 ,The training process of ECAPA-TDNN includes the following steps:

[0099] S31, performing channel conversion, sampling, and interception processing on the voice data to construct the voice data heard in a real indoor scene.

[0100] S32, frequency masking and time masking are performed on the processed speech data to enhance the characteristics of the speech signal, the strategy of translating audio data is used to increase the amount of speech data, and the Mel-spectrogram features and label processing are extracted from the processed speech signal to construct a data set.

[0101] S33, inputting the training set in the obtained data set into the ECAPA-TDNN classification network for classification.

[0102] For the training process of ECAPA-TDNN, it is necessary to construct training data first.

[0103] In order to ensure the accuracy of the training model, during the training process, it is hoped that the training data used to train the model is massive.

[0104] In addition, because the trained model needs to process the real voice data of the car owner in the actual application scenario, the massive training data should be based on the real voice data of the car owner as much as possible.

[0105] Real speech data is not completely without any drawbacks. Real speech data often has the problem of not having prominent signal features. In order to obtain a more accurate training model, frequency masking and time masking are required.

[0106] Before performing frequency masking and time masking, the original collected voice data needs to be channel converted, sampled, and intercepted. The reason for such channel conversion, sampling, and interception is to make the original collected voice data more standardized and more convenient to apply to the model training process.

[0107] After frequency masking and time masking are performed, in order to obtain a large amount of training data, the data volume needs to be expanded. In the embodiment of the present application, the strategy of time domain shift is adopted to expand the training speech data. After performing time domain shift, it is also necessary to perform Mel-spectrogram feature extraction and label processing.

[0108] Figure 4 FIG. 1 is a flowchart of the first facial feature extraction operation in the permission level authentication method provided by one or more embodiments of the present invention. Figure 4 , using ResNet to extract face vector features includes the following steps:

[0109] S41,uses front-end face detection, image cropping, and frame rate reduction to reduce the amount of calculation.

[0110] S42, the embedding layer extracts 128-dimensional feature data.

[0111] The embedding layer mentioned here refers to the embedding layer of the ResNet network.

[0112] It should be understood that if the original captured image is input into the ResNet network intact without any initial processing, it will cause unnecessary calculation in the ResNet network during the feature data extraction process, making it difficult to extract the feature data.

[0113] In order to reduce the computational complexity of the ResNet network during feature extraction, a series of preprocessing is required before the original face image is input into the ResNet network. Such preprocessing includes: front-end face detection, image cropping, and frame rate reduction.

[0114] The so-called front-end face detection is to detect the area corresponding to the face image in the original image that is originally collected and contains the face image, so as to facilitate further processing operations on these detected images.

[0115] In the daily process of face image acquisition, the face often only occupies a small part of the whole image. For this type of original acquired image, if the original acquired image is fully input into the ResNet network, it will cause unnecessary waste of computing resources in the ResNet network.

[0116] Therefore, in the preprocessing process, the front-end face recognition is first performed on the area occupied by the face image in the image, also known as front-end face detection, and then the original image is further cropped according to the area occupied by the detected face image, and then the cropped image is input into the ResNet network. This can avoid the following problem, that is, the waste of computing resources of the ResNet network caused by the large amount of non-face areas contained in the input image.

[0117] It should also be understood that during the facial image acquisition process, facial images are acquired according to a determined acquisition frame rate. If the frame rate of image acquisition is very high, the frequency of data input to the ResNet network will inevitably be very high. Therefore, in order to save ResNet computing resources, the acquisition frame rate of the original image should be appropriately reduced, and the frequency of data input to the ResNet network should be reduced.

[0118] Figure 5 This is a flowchart of a method for setting permission levels provided by one or more embodiments of the present invention. Figure 5 , the permission level setting method includes the following steps:

[0119] S51, extracting audio vector features from audio data through a deep model.

[0120] S52, extracting face vector features from face recognition features through a convolutional network.

[0121] S53, concatenate the audio vector feature and the face vector feature to form a comprehensive feature.

[0122] S54, binding the comprehensive feature with the identity and storing it in the identity feature library.

[0123] S55, granting the user a corresponding authority level in the identity feature library.

[0124] Depend on Figure 5 The embodiment shown in FIG. 1 shows the process of forming the identity feature library. Figure 5 The illustrated embodiments provide an embodiment of the present application. Figure 1The construction process of the identity feature library used in the embodiment shown is Figure 1 The process of generating background data of the security control method applied in the process.

[0125] The core of the so-called background data is the identity feature library. Figure 5 The identity feature library in the embodiment shown is Figure 1 The identity feature library shown is the same. In the identity feature library, not only the comprehensive feature data of each car owner is stored, that is, the comprehensive feature data that belongs to the car owner alone and corresponds to the identity of the car owner after the audio vector feature and the face vector feature are spliced, but also the permission level assigned to each car owner is stored.

[0126] Only after the permission level is assigned to each car owner in the identity feature library, can we know for sure what permission level each car owner should have during the identity verification and permission granting process.

[0127] In the process of generating the identity feature library, the initial stage is also the process of extracting audio vector features and face vector features. Figure 1 The process of applying the identity feature library is the same as that of the previous one. The extraction of audio vector features is also completed by using the deep model. The extraction of face vector features is also completed by the convolutional network.

[0128] More specifically, the extraction of audio vector features is completed through the ECAPA-TDNN network.

[0129] The extraction of face vector features is completed through the ResNet network.

[0130] After completing the extraction of audio vector features and face vector features, the audio vector features and face vector features are concatenated to form the comprehensive features of the car owner.

[0131] After the comprehensive characteristics of the car owner are formed, the formed comprehensive characteristics are bound to the identity of the car owner, and the comprehensive characteristics after the binding operation are stored in the identity characteristic library.

[0132] Finally, in the identity feature library, the corresponding permission level is granted to the user, thus completing the construction process of the identity feature library.

[0133] After completing the construction of the identity feature library, the owner's identity can be verified while the owner is issuing and parsing voice commands, and the owner can be assigned the corresponding authority level, which greatly improves the security of the owner's voice control of the vehicle.

[0134] Figure 6is a flowchart of the permission level setting operation provided by one or more embodiments of the present invention. Figure 6 , the permission level setting operation includes the following sub-operation steps:

[0135] The first part is the permission level setting.

[0136] 1. Key setting

[0137] 1) The car owner presets the permission level and password in advance. After obtaining the password, the vehicle user interacts with the vehicle by voice. The vehicle service system recognizes the permission password and the vehicle user's voice print and performs the corresponding specified level of permission.

[0138] 2. Face recognition settings

[0139] 1) The car owner uploads a facial photo of the vehicle user through the mobile app and specifies the permission level.

[0140] 2) The vehicle service system recognizes facial images, extracts facial features, matches facial features with permission levels, and implements image permission grading.

[0141] 3. Voiceprint settings

[0142] 1) The car owner uploads a piece of voice data of the vehicle user through the mobile phone program and specifies the permission level.

[0143] 2) The vehicle service system extracts voiceprint features from voice data, matches the voiceprint features with permission levels, and implements voiceprint permission grading.

[0144] The above three permission setting methods can be used separately or in combination. Combining them can improve accuracy and recall rate and enhance user experience.

[0145] Figure 7 is a flow chart of vehicle-user voice interaction provided by one or more embodiments of the present invention. Figure 7 , the vehicle-user voice interaction includes the following steps:

[0146] After obtaining voice control permission, vehicle users can interact with the vehicle through voice. The specific process is as follows:

[0147] 1. The user issues a voice command, such as: "Open the window."

[0148] The vehicle user's identity is recognized through the identity authentication module. The module uses facial images and voice data for identity recognition. Voice data can be used alone or in combination with facial images.

[0149] 1) The vehicle collects the user's facial image through the camera for identity verification. In order to reduce the amount of calculation, this function uses front-end face detection, image cropping, and lowering the frame rate to reduce the amount of calculation.

[0150] 2) The user's voice command data is transmitted to the identity authentication module for identity verification. This part is a basic module, because the voice data can use the user's command data, no additional collection is required, and the functional cost is reduced.

[0151] 3. Extract the user's permission level based on the user identity.

[0152] 4. Recognize the user's voice commands through the voice recognition function.

[0153] 5. Verify command control based on authority levels.

[0154] 6. The vehicle executes instructions that meet the authority.

[0155] Figure 8 is a flow chart of the identity recognition process provided by one or more attempts of the present invention. Figure 8 The identification process includes the following steps:

[0156] The identity authentication module adopts a multimodal approach. In the absence of image data, voice data can also be used alone. The recognition model is as follows Figure 8 shown.

[0157] In the first part, the audio data is extracted with a deep model to extract vector features, and the embedding layer extracts 128-dimensional features to give you data.

[0158] In the second part, the face recognition features are extracted through the convolutional network to extract vector features, and the embedding layer extracts 128-dimensional feature data.

[0159] The two features are concatenated to form a comprehensive feature. If there is no camera to collect images, only the voiceprint feature is used. After obtaining the feature, it is compared with the identity feature library, the similarity is calculated, and the identity is determined.

[0160] When adding identity permissions to someone, you can do so through the Internet or mobile phone, through desktop software, or directly in the vehicle.

[0161] like Fig. 9 As shown, the present invention also provides an authority level authentication device, comprising:

[0162] The first audio feature extraction unit 91 is used to extract audio vector features from audio data through a deep model.

[0163] The first face feature extraction unit 92 is used to extract face vector features from face recognition features through a convolutional network.

[0164] The first concatenation unit 93 is used to concatenate the audio vector feature with the face vector feature to form a comprehensive feature.

[0165] The identity determination unit 94 is used to compare the comprehensive features with the identity feature library, calculate the similarity, and determine the identity.

[0166] The first authorization unit 95 is used to grant a corresponding authority level according to the determined identity.

[0167] like Fig.10 As shown, the present invention also provides a permission level setting device, comprising:

[0168] The second audio feature extraction unit 101 is used to extract audio vector features from audio data through a deep model.

[0169] The second face feature extraction unit 102 is used to extract face vector features from face recognition features through a convolutional network.

[0170] The second concatenation unit 103 is used to concatenate the audio vector feature with the face vector feature to form a comprehensive feature.

[0171] The identity storage unit 104 is used to bind the comprehensive feature with the identity and store it in the identity feature library.

[0172] The second authorization unit 105 is used to grant the corresponding authority level to the user in the identity feature database.

[0173] It is worth noting that although only some basic functional modules are disclosed in the embodiments of the present invention, it does not mean that the composition of the present system is limited to the above basic functional modules. On the contrary, what the present embodiment wants to express is that on the basis of the above basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with the prior art to form an infinite number of embodiments or technical solutions. In other words, the present system is open rather than closed. Just because the present embodiment only discloses individual basic functional modules, it cannot be considered that the protection scope of the claims of the present invention is limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above devices are described in terms of functions and are described separately in various units and modules. Of course, when implementing the present invention, the functions of each unit and module can be implemented in the same or one or more software and / or hardware.

[0174] like Fig.11As shown, the present invention also provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the method.

[0175] Fig.11 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Fig.11 The electronic device provided in the embodiment of the present invention includes: one or more processors 1110 and a memory 1120; the processor 1110 in the electronic device may be one or more, Fig.11 A processor 1110 is taken as an example; the memory 1120 is used to store one or more programs; the one or more programs are executed by the one or more processors 1110, so that the one or more processors 1110 implement a method as described in any one of the embodiments of the present invention.

[0176] The electronic device may further include: an input device 1130 and an output device 1140 .

[0177] The processor 1110, memory 1120, input device 1130 and output device 1140 in the electronic device can be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0178] The memory 1120 in the electronic device is a computer-readable storage medium that can be used to store one or more programs, which can be software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the security control method provided in the embodiment of the present invention. The processor 1110 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 1120, that is, implementing the method in the above method embodiment.

[0179] The memory 1120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 1120 may further include a memory remotely arranged relative to the processor 1110, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0180] The input device 1130 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. The output device 1140 may include a display device such as a display screen.

[0181] The present invention also provides a computer-readable storage medium storing a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the method.

[0182] Specifically, the computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0183] The present invention also provides a vehicle, wherein the vehicle is provided with the safety control device as described above.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for authentication of authority level, characterized in that: The permission level authentication method comprises: Extract audio vector features from audio data through a deep model; The face recognition features are extracted through the convolutional network to extract the face vector features; Concatenate the audio vector features with the face vector features to form a comprehensive feature; Compare the comprehensive features with the identity feature database, calculate the similarity, and determine the identity; According to the determined identity, the corresponding permission level is granted.

2. The method according to claim 1, characterized in that The audio data is passed through a deep model to extract audio vector features, including: Audio vector features are extracted through the ECAPA-TDNN network.

3. The method according to claim 2, characterized in that The ECAPA-TDNN network includes: one-dimensional squeeze-excitation residual block, channel- and context-dependent statistical pooling, multi-layer feature aggregation and summation, and AAM-Softmax classifier.

4. The method according to claim 2, characterized in that: The training process of the ECAPA-TDNN network includes: The voice data is converted into channels, sampled, and intercepted to construct the voice data heard in a real indoor scene; The processed speech data is frequency-masked and time-masked to enhance the speech signal features, the audio data is translated to increase the amount of speech data, and the Mel-spectrogram features and label processing are extracted from the processed speech signal to construct a data set. The training set in the obtained data set is input into the ECAPA-TDNN network for classification.

5. The method according to claim 1, characterized in that The face recognition features are extracted through the convolutional network to extract the face vector features, including: ResNet is used to extract face vector features.

6. A method for setting permission levels, characterized in that: The permission level setting method includes: Extract audio vector features from audio data through a deep model; The face recognition features are extracted through the convolutional network to extract the face vector features; Concatenate the audio vector features with the face vector features to form a comprehensive feature; Bind the comprehensive features to the identity and store them in the identity feature library; In the identity feature library, users are granted corresponding permission levels.

7. A permission level authentication device, characterized in that: The authority level authentication device comprises: A first audio feature extraction unit, configured to extract audio vector features from audio data through a deep model; A first face feature extraction unit, used to extract face vector features from face recognition features through a convolutional network; A first concatenation unit, used for concatenating the audio vector feature with the face vector feature to form a comprehensive feature; An identity determination unit is used to compare the comprehensive features with the identity feature library, calculate the similarity, and determine the identity; The first authorization unit is used to grant a corresponding authority level according to the determined identity.

8. A device for setting authority level, characterized in that: The authority level setting device comprises: A second audio feature extraction unit, used to extract audio vector features from the audio data through a deep model; A second face feature extraction unit, used for extracting face vector features from face recognition features through a convolutional network; A second concatenation unit is used to concatenate the audio vector feature with the face vector feature to form a comprehensive feature; An identity storage unit is used to bind the comprehensive features to the identity and store them in the identity feature library; The second authorization unit is used to grant the corresponding authority level to the user in the identity feature library.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of any one of the methods described in claims 1 to 6.

10. A computer-readable storage medium, characterized in that: It stores a computer program executable by an electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the method described in any one of claims 1 to 6.

Images