Unlocking method, device, equipment, medium, program product and vehicle

Abstract

The invention relates to an unlocking method and device, equipment, a medium, a program product and a vehicle, and the method comprises the steps: extracting voiceprint information and auxiliary authentication information based on an unlocking audio under the condition that the unlocking audio input by a user is received; and executing an unlocking operation on the vehicle according to the voiceprint information and the auxiliary authentication information. The invention aims to improve the unlocking safety of the vehicle.

Description

Technical Field

[0001] The present application relates to the field of unlocking, and in particular to an unlocking method, device, equipment, medium, program product and vehicle. Background Art

[0002] The emergence of voiceprint recognition technology has greatly improved the convenience of people's lives. In the automotive field, voiceprint recognition technology is used in vehicle unlocking and equipment control, allowing users to operate vehicles more conveniently. Voiceprint recognition technology provides a safer and more convenient unlocking method by identifying personal voiceprint information in audio and matching it with authorized pre-stored voiceprints. However, with the replacement of various technologies, relying solely on voiceprint recognition can no longer provide sufficient security, resulting in low unlocking security. Summary of the invention

[0003] The embodiments of the present application provide an unlocking method, device, equipment, medium, program product and vehicle, which improve the security of vehicle unlocking and at least partially solve the above-mentioned technical problems.

[0004] In order to achieve the above object, according to a first aspect of the present application, an unlocking method is provided, the unlocking method comprising:

[0005] When receiving an unlocking audio input by a user, extracting voiceprint information and auxiliary authentication information based on the unlocking audio;

[0006] An unlocking operation is performed on the vehicle according to the voiceprint information and the auxiliary authentication information.

[0007] According to a second aspect of the present application, there is provided an electronic device, including:

[0008] A receiving module, receiving unlocking audio input by a user;

[0009] An extraction module, used to extract voiceprint information and auxiliary authentication information based on the unlocking audio;

[0010] A control module is used to perform an unlocking operation according to the voiceprint information and the auxiliary authentication information.

[0011] According to a third aspect of the present application, an electronic device is also provided, comprising a processor, wherein the processor is connected to a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any of the above-mentioned unlocking methods.

[0012] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned unlocking methods is performed.

[0013] According to a fifth aspect of the present application, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to implement any of the above-mentioned unlocking methods.

[0014] According to a sixth aspect of the present application, a vehicle is provided, which executes the unlocking method as described above, or includes the electronic device or electronic equipment as described above.

[0015] The unlocking method provided in the embodiment of the present application, when receiving the unlocking audio input by the user, extracts the voiceprint information and the auxiliary authentication information based on the unlocking audio, and performs the unlocking operation on the vehicle according to the voiceprint information and the auxiliary authentication information. In this way, the unlocking operation is performed by extracting the voiceprint information and the auxiliary authentication information from the unlocking audio, which is more secure than unlocking based on voiceprint recognition alone, and can effectively improve the security of vehicle unlocking. In addition, obtaining the auxiliary authentication information from the unlocking audio does not require adding additional information sources, does not increase the difficulty of the process, and can reduce the cost of unlocking.

[0016] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0018] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same figure numbers represent the same parts in the following description.

[0019] Figure 1 is a flowchart of an embodiment of an unlocking method provided in an embodiment of the present invention;

[0020] Figure 2 is a schematic diagram of a first scenario process involved in the unlocking method provided in an embodiment of the present invention;

[0021] Figure 3 is a schematic diagram of a second scenario process involved in the unlocking method provided in an embodiment of the present invention;

[0022] Figure 4 is a flowchart of another embodiment of the unlocking method provided in the embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention;

[0024] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0026] The emergence of voiceprint recognition technology has greatly improved the convenience of people's lives. In the automotive field, voiceprint recognition technology is used in vehicle unlocking and equipment control, allowing users to operate vehicles more conveniently. Voiceprint recognition technology provides a safer and more convenient unlocking method by identifying personal voiceprint information in audio and matching it with authorized pre-stored voiceprints.

[0027] The inventor found that: with the replacement of various technologies, relying solely on voiceprint recognition can no longer provide sufficient security, resulting in low security of unlocking. If combined with other security verification methods, such as body recognition, iris recognition, etc., although it can also enhance the security of the system, this comprehensive application also increases the complexity of the system.

[0028] In order to solve the above problems, the embodiments of the present application propose an unlocking method, device, equipment, medium, program product and vehicle. The embodiments of the present application extract voiceprint information and auxiliary authentication information from the unlocking audio to perform the unlocking operation, which is more secure than unlocking based on voiceprint recognition alone and can effectively improve the security of vehicle unlocking. In addition, obtaining auxiliary authentication information from the unlocking audio does not require adding additional information sources, does not increase the difficulty of the process, and can reduce the unlocking cost.

[0029] Specifically, the unlocking method in the present application can be applied to an electronic device, the executor of the unlocking method can be the electronic device, and the electronic device can be a vehicle, such as a car, an electric car, a hybrid car, etc.

[0030] In the following, each embodiment is described in detail by taking the electronic device as an example in which the execution subject of the unlocking method is an electronic device.

[0031] Correspondingly, if Figure 1 As shown, the unlocking method may include the following steps:

[0032] S10. When receiving an unlocking audio input by a user, extracting voiceprint information and auxiliary authentication information based on the unlocking audio;

[0033] In this embodiment, the unlocking method is applied to a vehicle, and the vehicle includes a sound recording device, such as a microphone. When a user needs to unlock the vehicle, he or she can speak to the sound recording device, and the sound recording device inputs an unlocking audio containing the user's voice. The unlocking audio contains the voice of the user to be unlocked, and the voiceprint information of the voice can be further obtained. Voiceprint information refers to the voice features extracted from the unlocking audio that can characterize and identify the speaker. After comparing the voiceprint information with the voiceprint information of a known identity, the identity of the user who made the corresponding sound can be determined, and then the identity of the user can be authenticated.

[0034] In addition to voiceprint information, auxiliary authentication information can also be extracted from the unlocking audio. The authenticity of the unlocking audio can be verified through the auxiliary authentication information. For example, the data required for liveness detection can be extracted from the unlocking audio as auxiliary authentication information, and liveness detection can be performed based on the auxiliary authentication information to verify the authenticity of the unlocking audio, proving that the unlocking audio is real-time collection or real-person sound, effectively preventing attacks on voiceprint locks using recorded or artificially synthesized audio.

[0035] The sound recording device on the vehicle can be in a standby state. When a preset trigger operation is received, the sound recording device is triggered to start recording the unlocking sound. The preset trigger operation can be at least one of recording a keyword voice, receiving a trigger instruction, etc. The sound recording device can be installed at a position corresponding to the unlocking operation to be able to completely record the user's voice and auxiliary authentication information. For example, if the unlocking operation is unlocking a car door, the sound recording device can be installed at a suitable position on the car door to be able to completely record the voice and auxiliary authentication information of the unlocking object.

[0036] S20, performing an unlocking operation on the vehicle according to the voiceprint information and the auxiliary authentication information;

[0037] In this embodiment, the voiceprint information can identify the identity of the unlocking object speaking in the unlocking audio, the auxiliary authentication information can identify the authenticity of the unlocking audio, and the unlocking audio is authenticated based on the voiceprint information and the auxiliary authentication information. After the authentication is passed, the corresponding unlocking operation can be performed.

[0038] In the technical solution disclosed in this embodiment, voiceprint information and auxiliary authentication information are extracted based on the unlocking audio; the unlocking operation is performed according to the voiceprint information and the auxiliary authentication information. In this way, the unlocking operation is performed by extracting voiceprint information and auxiliary authentication information from the unlocking audio, which is more secure than unlocking based on voiceprint recognition alone, and can effectively improve the security of vehicle unlocking. In addition, obtaining auxiliary authentication information from the unlocking audio does not require adding additional information sources, does not increase the difficulty of the process, and can reduce the cost of unlocking.

[0039] In one embodiment, performing the unlocking operation according to the voiceprint information and the auxiliary authentication information includes:

[0040] Verifying the voiceprint information;

[0041] Verifying the auxiliary authentication information;

[0042] When the voiceprint information verification is passed and the auxiliary authentication information verification is passed, the unlocking operation is performed.

[0043] In this embodiment, the voiceprint information can be used for identity recognition, and the auxiliary authentication information can be used to identify the authenticity of the unlocking audio. Therefore, the voiceprint information and the auxiliary authentication information are verified separately, and the voiceprint information is used for identity recognition to check whether the unlocking object corresponding to the voiceprint information is legal. If so, the verification is passed. The authenticity of the unlocking audio can be checked based on the auxiliary authentication information. If the surface unlocking audio is authentic, the auxiliary authentication information can be verified. If the voiceprint information passes the verification and the auxiliary authentication information passes the verification, the unlocking operation can be performed. If either the voiceprint information passes the verification or the auxiliary authentication information fails to pass the verification, the unlocking operation cannot be performed immediately, and the unlocking fails this time.

[0044] In this way, the voiceprint information and the auxiliary authentication information are verified separately, and the unlocking operation is performed only when both are verified separately, thereby further improving the unlocking security through independent verification.

[0045] In one embodiment, the auxiliary authentication information includes at least one of ambient audio and frequency characteristics.

[0046] In this embodiment, the unlocking audio that can be collected generally includes human voice and environmental sound. Then, environmental audio can be extracted from the unlocking audio, and the environmental audio can be the audio other than human voice in the unlocking audio. The environmental audio can be used to identify the environment where the unlocking object in the output unlocking audio is located, and then the authenticity of the unlocking audio can be determined by the environmental conditions.

[0047] In this embodiment, the unlocking audio is a kind of audio data, and the frequency characteristics of the unlocking audio can be extracted. The frequency characteristics are characteristics that can describe the frequency distribution of the unlocking audio, such as frequency distribution, frequency change rules, etc. There are differences in frequency between real voice and recorded or artificially synthesized voice. This method can be used to effectively prevent voiceprint spoofing attacks. The frequency characteristics can be used to verify whether the unlocking audio is synthesized audio, and then the authenticity of the unlocking audio can also be verified.

[0048] In some embodiments, the auxiliary authentication information includes the ambient audio and frequency characteristics of the unlocking audio. When the voiceprint information passes verification, the ambient audio passes verification, and the frequency characteristics pass verification, the unlocking operation is performed. This combination of sound characteristics, environmental characteristics, and frequency characteristics for verification can effectively improve the recognition and resistance to recording and artificially synthesized sound attacks, ensure that it is unlocked by a real person, and effectively improve the security of unlocking.

[0049] In some embodiments, verifying the auxiliary authentication information includes:

[0050] The ambient audio is matched with a pre-collected pre-stored ambient audio, and when the ambient audio matches the pre-stored ambient audio, it is determined that the ambient audio has passed the verification.

[0051] In this embodiment, before receiving the ambient audio, the pre-collected audio can be collected by the sound collection device as the pre-stored ambient audio. The pre-stored ambient audio is the audio collected when there is no voice such as the unlocking object, or the ambient audio extracted from the audio collected before collecting the unlocking audio. The pre-stored ambient audio can reflect the real environment where the vehicle is located, and further can reflect the real environment where the vehicle is located when unlocking. Therefore, the time difference between the collection time of the pre-stored ambient audio and the collection time of the unlocking audio does not exceed the preset time threshold, thereby ensuring that the pre-stored ambient audio is consistent with the real ambient audio when unlocking.

[0052] Since the pre-stored environmental audio reflects the real environment where the vehicle is located when unlocking, the environmental audio can be matched based on the pre-stored environmental audio to verify the accuracy of the unlocking audio. When the environmental audio extracted from the unlocking audio matches the pre-collected pre-stored environmental audio, it indicates that the environmental situation reflected by the environmental audio collected in the unlocking audio is consistent with the real environment where the vehicle is located when unlocking, and further indicates that the unlocking audio is collected from the real environment. If the environmental audio passes the verification, the authenticity of the unlocking audio can be further determined. If the voiceprint information in the unlocking audio comes from the recording, it is inevitable that there will be sounds from the recording environment, making the unlocking audio inconsistent with the real environment where the vehicle is located when unlocking. The environmental audio extracted from the unlocking audio does not match the pre-collected pre-stored environmental audio. When the environmental audio does not match the pre-collected pre-stored environmental audio, the environmental audio fails the verification, and the corresponding unlocking operation cannot be performed, thereby further improving the security of unlocking.

[0053] In one embodiment, the method further comprises:

[0054] Acquire a first ambient noise in the ambient audio;

[0055] If the first environmental noise is the same as the second environmental noise in the pre-stored environmental audio, it is determined that the environmental audio matches the pre-stored environmental audio.

[0056] In this embodiment, when the pre-stored environmental audio is collected in advance, the pre-stored environmental audio is also analyzed to obtain the first environmental noise in the real environment where the vehicle is located. The first environmental noise may be one or more, corresponding to different noise sources in the real environment where the vehicle is located. By analyzing the environmental audio in the unlocking audio, the second environmental noise contained in the environmental audio in the unlocking audio is obtained. The second environmental noise may be one or more, and the corresponding noise sources are different.

[0057] In some embodiments, the second ambient noise is compared with the first ambient noise in the pre-stored ambient audio. If the first ambient noise is the same as the second ambient noise in the pre-stored ambient audio, it indicates that there is no second ambient noise that is different from the first ambient noise in the pre-stored ambient audio, and no noise other than the real ambient noise is introduced into the unlocking audio, thereby reducing the possibility that the unlocking audio is a recording, and it can be determined that the ambient audio matches the pre-stored ambient audio.

[0058] In this way, by comparing the ambient audio with the noise in the pre-stored ambient audio, it is possible to more accurately determine whether the ambient audio matches the pre-stored ambient audio, thereby reducing the possibility that the unlocking audio is a recording, further improving the accuracy of vehicle unlocking.

[0059] In one embodiment, verifying the auxiliary authentication information further includes:

[0060] Determine whether the unlocked audio has been edited based on the frequency characteristics, where the editing includes splicing and / or resampling;

[0061] If the unlocking audio has not been edited, it is determined that the frequency feature has passed the verification.

[0062] In this embodiment, the frequency of the unlocking sound is analyzed to obtain the frequency characteristics. An algorithm or model can be designed to analyze the frequency characteristics of the unlocking audio in detail, including the frequency distribution of the sound, the frequency change law, etc. The frequency of the recorded or artificially synthesized audio is different from the frequency of real people speaking, and the corresponding frequency characteristics will also change. By analyzing the frequency characteristics, it can be noted that the real sound and the recorded or artificially synthesized sound have different frequencies, and then accurately determine whether the unlocking audio is a recorded or artificially synthesized sound, so as to realize the verification of the frequency characteristics, which can effectively prevent voiceprint spoofing attacks.

[0063] In this embodiment, based on the frequency characteristics, it can also be determined whether the unlocked audio has been edited, and the extracted frequency characteristics can be compared and analyzed with the frequency characteristics of normal audio. If the audio has been edited, its frequency characteristics will show abnormal changes, such as sudden changes in spectral flux, discontinuity in spectral slope, etc.

[0064] The editing process may include splicing, re-adopting, etc. By analyzing the changes in the short-time energy and short-time spectral entropy in the frequency features, it is possible to detect whether there are splicing points in the unlocked audio, because there are usually mutations in these frequency features at the splicing points. If there are splicing points, it indicates that the unlocked audio has been edited. It is also possible to detect whether there is a re-adopted part of the audio by analyzing the changes in the fundamental frequency in the frequency features. The re-adopted part usually has a repetitive pattern of the fundamental frequency. If there is a re-adopted part, it indicates that the unlocked audio has been edited. If the unlocked audio has been edited, the frequency feature has not passed the verification, otherwise it has passed the verification.

[0065] In this way, it is determined whether the unlocking audio has been edited through frequency characteristics to exclude the possibility that the unlocking audio is artificially synthesized audio, thereby further improving the accuracy of vehicle unlocking.

[0066] In one embodiment, the verifying the voiceprint information further includes:

[0067] Obtaining a similarity score between the voiceprint information and pre-stored voiceprint information;

[0068] If the similarity score is greater than the similarity threshold, it is determined that the voiceprint information verification has passed.

[0069] In this embodiment, the verification of the voiceprint information may be a comparative verification to determine the legitimacy of the unlocking audio. The comparative verification may use a matching algorithm of a Gaussian mixture model to compare the voiceprint information with pre-stored voiceprint information to obtain a similarity score. The pre-stored voiceprint information is pre-stored voiceprint information that is authorized to perform the unlocking operation. There may be one or more pre-stored voiceprint information. The voiceprint information extracted from the unlocking audio is matched with each pre-stored voiceprint information to obtain a similarity score between the voiceprint information and the pre-stored voiceprint information. The similarity score represents the possibility that the unlocking object corresponding to the voiceprint information is the authorized user corresponding to the pre-stored voiceprint information. The higher the similarity score, the greater the possibility. A similarity threshold is preset. If the similarity score is greater than the similarity threshold, it indicates that the unlocking object corresponding to the voiceprint information is the authorized user corresponding to the pre-stored voiceprint information, and the voiceprint information in the unlocking audio passes the verification. Otherwise, it fails the verification.

[0070] In this way, by comparing the similarity between the voiceprint information and the pre-stored voiceprint information, the voiceprint information can be verified, which can further improve the security of vehicle unlocking.

[0071] In some embodiments, in order to achieve a fast response effect for the verification of voiceprint information, a hardware structure capable of real-time response and fast calculation should be used, such as a 32-bit core microcontroller STM32 (STMicroelectronics 32-bit) and a field-programmable gate array (FPGA). The pre-stored voiceprint information for comparison can be extracted from the system and decrypted for use. The system here can be a vehicle-mounted system or a cloud system, etc. The specific decryption method and key location depend on the steps during encryption.

[0072] In one embodiment, reference Figure 4 , the method further comprises:

[0073] Step S30, upon receiving an authorization instruction triggered by an authorized user, obtaining the authorization sound audio corresponding to the authorized user;

[0074] Step S40: extracting the pre-stored voiceprint information according to the authorized voice audio.

[0075] In this embodiment, when a user uses a vehicle for the first time, he or she can enter his or her voiceprint information into the above system, making him or her an authorized user corresponding to the pre-stored voiceprint information that can control the unlocking of the vehicle. The authorized user can subsequently reset or add new authorized users as desired. Each time a new authorized user is added, the authorized user to be authorized can issue an authorization instruction to the application on the vehicle or the associated device of the vehicle. When receiving the authorization instruction triggered by the authorized user, it indicates that the voiceprint information of the new authorized user needs to be entered as the pre-stored voiceprint information. At this time, the authorized sound audio corresponding to the authorized user can be collected by a sound recording device. The sound recording device for collecting the authorized sound audio may not be the sound recording device for collecting the unlocking audio. When collecting the authorized sound audio, the authorized user needs to speak so that the authorized sound audio contains the voice of the authorized user, and then the voiceprint information of the authorized user can be extracted from the authorized sound audio and stored in the system as the pre-stored voiceprint information that can be unlocked.

[0076] In this way, when the authorization instruction is obtained, the authorized voice audio of the authorized user is collected, and the pre-stored voiceprint information corresponding to the authorized user is extracted based on the authorized voice audio, so as to facilitate the subsequent voiceprint information verification of the authorized user before the unlocking operation.

[0077] In one embodiment, upon receiving an authorization instruction triggered by an authorized user, obtaining the authorization sound audio corresponding to the authorized user includes:

[0078] When receiving an authorization instruction triggered by an authorized user, outputting a preset sentence through a speaker and / or a display;

[0079] The audio corresponding to the preset sentence is collected, and the audio corresponding to the preset sentence is set as the authorized sound audio.

[0080] In this embodiment, when the authorization instruction is obtained, a preset sentence can also be output through a speaker and / or a display in the form of voice, text, image, etc. to assist the authorized user to say the preset sentence required to collect the voiceprint information, so that the audio corresponding to the preset sentence can be collected and set as the authorized voice audio. The preset voiceprint information is extracted through the authorized voice audio containing the preset sentence. A neural network can be used to extract the voiceprint features of the authorized user from the authorized voice audio as the preset voiceprint information, such as a feature extraction method based on Mel-scale Frequency Cepstral Coefficients (MFCC).

[0081] When obtaining the unlocking audio, the authorized user can also speak based on the preset sentence, so that the unlocking audio also contains the preset sentence, and extract the voiceprint features from it as the voiceprint information, which can make the voiceprint information more consistent with the pre-stored voiceprint information, and facilitate comparison and verification.

[0082] In this way, fixed preset sentences are output through the speaker and / or display, which reduces the difficulty of voiceprint information extraction, reduces the difficulty of model learning, and improves the accuracy of voiceprint information verification.

[0083] In one embodiment, extracting the pre-stored voiceprint information according to the authorized voice audio includes:

[0084] Extracting initial voiceprint information based on the authorized voice audio;

[0085] Selecting part of the voiceprint information from the initial voiceprint information to perform frequency domain change processing to obtain watermark data;

[0086] The watermark data is embedded into the initial voiceprint information to obtain the pre-stored voiceprint information.

[0087] In this embodiment, the voiceprint information directly extracted from the authorized sound audio can be used as the initial voiceprint information. In this embodiment, the initial voiceprint information can also be encrypted to ensure the security of the pre-stored voiceprint information and prevent it from being stolen. A small part of the voiceprint information in the initial information is selected for frequency domain transformation, such as fast Fourier transform, to obtain watermark data. The watermark data is small enough and will not affect the voiceprint verification, and it should also be complex enough to ensure the uniqueness and security of the watermark data. The processed watermark data is embedded in the initial voiceprint information extracted from the authorized sound audio to obtain the pre-stored voiceprint information. After that, an embedding check can be performed and the relevant parameters and position of the watermark embedding can be recorded, which is helpful for the extraction of watermark data and the verification of the legitimacy of the voiceprint information in the future.

[0088] In this way, by extracting the initial voiceprint information from the authorized sound audio and adding watermark data, the pre-stored voiceprint information is obtained. The watermark data is superimposed on the pre-stored voiceprint information, which is equivalent to adding noise but will not affect the voiceprint verification, thereby avoiding the pre-stored voiceprint information from being stolen and improving the security of the pre-stored voiceprint information.

[0089] In this embodiment, the pre-stored voiceprint information may also be encrypted and entered into a system related to the vehicle, which may be a local storage device or a cloud device.

[0090] For better understanding, refer to Figure 2 The process of obtaining pre-stored voiceprint information can be as follows: first initialize the vehicle, and when receiving the authorization instruction triggered by the authorized user, the authorized voice audio of the authorized user can be collected through the sound recording device in the car to obtain the initial voiceprint information of the authorized user, embed watermark data into the voiceprint information to obtain the pre-stored voiceprint information, and seal it and record it into the vehicle's local storage or cloud device.

[0091] In one embodiment, the method further comprises:

[0092] Obtaining the real-time voice audio of the authorized user corresponding to the pre-stored voiceprint information;

[0093] Based on the real-time voiceprint information extracted from the real-time sound audio, the pre-stored voiceprint information corresponding to the authorized user is updated.

[0094] In this embodiment, considering that the voice of the same user may change due to illness, age, etc., the corresponding voiceprint information may also change. Therefore, after collecting the preset voiceprint information, this embodiment can also obtain the real-time sound audio of the authorized user corresponding to the pre-stored voiceprint information. The real-time voiceprint information is extracted from the real-time sound audio again to update the pre-stored voiceprint information of the authorized user according to the real-time voiceprint information. In some embodiments, the pre-stored voiceprint information can be updated to the real-time voiceprint information. In some embodiments, the voiceprint features in the real-time voiceprint information can be added to the pre-stored voiceprint information, so that the pre-stored voiceprint information of the authorized user has more information about the authorized corresponding voice, which is conducive to voiceprint recognition.

[0095] In this way, the pre-stored voiceprint information is updated in real time through real-time sound audio, which can improve the accuracy of voiceprint recognition and further improve the security of vehicle unlocking.

[0096] In one embodiment, obtaining the real-time voice audio of the authorized user corresponding to the pre-stored voiceprint information includes:

[0097] Sending a voiceprint update request to the authorized user at a preset interval;

[0098] The audio corresponding to the voiceprint update request is set as the real-time sound audio.

[0099] In this embodiment, a voiceprint update request can be sent to the authorized user at preset intervals. The voiceprint update request can be output through the speaker or display of the vehicle when the authorized user uses the vehicle in the form of voice, text, pictures, etc. The voiceprint update request can also be sent to a terminal device associated with the authorized user, such as a mobile phone, a smart wearable device, etc. After the authorized user agrees to the voiceprint update request, the audio corresponding to the voiceprint update request can be collected and set as real-time audio. The method of collecting real-time audio can be consistent with the method of collecting authorized audio. When the confirmation instruction corresponding to the voiceprint update request is obtained, a preset sentence can be output through the speaker and / or display, and the audio corresponding to the preset sentence can be collected and set as real-time audio.

[0100] By sending a voiceprint update request to the authorized user, real-time audio that accurately contains the authorized user's voice can be obtained, thereby improving the accuracy of updating pre-stored voiceprint information and voiceprint verification, and further improving unlocking security.

[0101] In one embodiment, when the unlocking result of the unlocking operation is an unlocking failure, and the number of consecutive unlocking failures exceeds a preset threshold, an unlocking failure message is sent to an associated device, wherein the unlocking failure message includes an unlocking failure prompt and / or an unlocking failure reason, and the associated device includes a cloud device and / or a terminal device.

[0102] In this embodiment, the purpose of the current unlocking audio input is to perform an unlocking operation, and a variety of reasons may lead to failure in performing the unlocking operation. The reasons for the unlocking failure may include failure of the unlocking audio verification, vehicle lock failure, etc. Among them, the failure of the unlocking audio verification includes failure of at least one of the voiceprint information and auxiliary authentication information in the unlocking audio, more specifically, failure of at least one of the voiceprint information, ambient audio, and frequency characteristics in the unlocking audio. When the unlocking audio verification fails, in order to ensure the safety of the vehicle, the unlocking operation cannot be performed, which will result in a failed unlocking result of the unlocking operation. If the unlocking audio verification passes, the unlocking operation can be performed, but when the vehicle lock (door lock, application lock, etc.) itself fails, the unlocking operation cannot be completed normally, which will result in a failed unlocking result of the unlocking operation.

[0103] In the event of a failure in the unlock operation, check the current number of consecutive unlock failures. If the number of consecutive unlock failures exceeds a preset threshold, it indicates that multiple attempts to input the unlock audio have failed. It is necessary to output unlock failure information to the authorized user or other relevant users to remind the user that an unauthorized user has tried to unlock the vehicle multiple times, or that the vehicle lock is faulty, etc. Specifically, the unlock failure information can be sent to the associated device of the vehicle to notify the relevant user. The unlock failure information may include an unlock failure prompt, a reason for the unlock failure, etc. The associated device may be a cloud device and / or a preset terminal device, so that the authorized user and other relevant users can receive the unlock failure information. The unlock failure information may be sent to the associated device via Bluetooth, wireless local area network (WiFi), text messages, vehicle cloud, etc., to remind the user that the vehicle is currently being illegally unlocked or the vehicle lock is faulty, etc., so as to intervene in time.

[0104] In one embodiment, the method further comprises:

[0105] receiving indication information sent by the associated device according to the unlocking failure information;

[0106] At least one of a warning operation, an unlocking operation, and shielding other unlocking audios associated with the unlocking audio is performed according to the indication information.

[0107] In this embodiment, after the authorized user receives the unlocking failure information, the associated device can issue an instruction message to instruct the vehicle to perform a corresponding operation according to the instruction message. The operation may include warning operations such as horn operation and double flash operation, which can warn the unauthorized user who is currently trying to unlock; it can also include unlocking operation, which can facilitate remote unlocking for trusted unauthorized users; it can also be to shield other unlocking audios associated with the unlocking audio of the unlocking failure to shield other unlocking requests from the unauthorized user; it can also be a troubleshooting operation to self-check or self-repair the vehicle lock fault, etc.

[0108] In this way, instruction information can be issued and corresponding operations can be performed through the associated device to deal with various situations of multiple unlocking failures and improve the user experience.

[0109] In one embodiment, the unlocking operation may be an unlocking operation performed on a vehicle, including door unlocking of a vehicle door and / or control unlocking of a vehicle control system, so that after unlocking, the unlocking object can open the door or perform control operations on the vehicle.

[0110] In one embodiment, when the unlocking result of the unlocking operation is successful unlocking, obtaining the in-vehicle audio of the vehicle;

[0111] The vehicle is controlled to perform a door closing operation and / or a locking operation according to the in-vehicle audio.

[0112] In this embodiment, the unlocking operation is the door unlocking, which sends a signal to the controller controlling the door to control the door to electrically drive the door to unlock, and can further open automatically. After executing the door unlocking operation, if the door is successfully unlocked, the unlocking result of the unlocking operation is unlocking success, then the vehicle can be opened and the user can enter the vehicle. Then, the in-car audio of the vehicle can be obtained, and by detecting effective sounds in the in-car audio, such as talking in the car, seat squeezing sounds, etc., it can be determined that the unlocking object has entered the car, and a signal can be sent to the controller controlling the door to control the door to electrically drive the door to close and automatically lock. This integrated automatic door opening and closing locking solution can provide users with convenient door operation, while also ensuring the safety of the interior of the vehicle.

[0113] For a better understanding, another application scenario is provided below. Figure 3 The car door waits for the input of the unlocking audio. When the key language or other triggering unlocking instructions is received, it can be determined whether the number of consecutive unlocking failures exceeds N times. If so, the unlocking instruction can be blocked. If not, recording is started to obtain the unlocking audio, from which the voiceprint information, ambient audio, and frequency characteristics are extracted for analysis and verification. If any one of the verifications fails, it is recorded as the unlocking failure of the unlocking audio, and then it is determined whether the number of consecutive unlocking failures exceeds N times. If so, the vehicle can be locked and the authorized user can be notified, so that the authorized user can choose the subsequent operation. If all three items are verified, the door unlocking operation can be performed, the electric drive door is unlocked and opened, and automatic locking is achieved according to the audio in the car.

[0114] This embodiment also provides an electronic device, which can be integrated into a vehicle, for example Figure 5 As shown, the electronic device may include:

[0115] Receiving module 1001, receiving unlocking audio input by a user;

[0116] An extraction module 1002, configured to extract voiceprint information and auxiliary authentication information based on the unlocking audio;

[0117] The control module 1003 is used to perform an unlocking operation according to the voiceprint information and the auxiliary authentication information.

[0118] Optionally, the extraction module 1002 is further configured to:

[0119] Verifying the voiceprint information;

[0120] Verifying the auxiliary authentication information;

[0121] When the voiceprint information verification is passed and the auxiliary authentication information verification is passed, the unlocking operation is performed.

[0122] Optionally, the auxiliary authentication information includes at least one of ambient audio and frequency characteristics.

[0123] Optionally, the extraction module 1002 is further used for:

[0124] Verifying the voiceprint information;

[0125] Verifying the auxiliary authentication information;

[0126] When the voiceprint information verification is passed and the auxiliary authentication information verification is passed, the unlocking operation is performed.

[0127] Optionally, the extraction module 1002 is further used for:

[0128] Acquire a first ambient noise in the ambient audio;

[0129] If the first environmental noise is the same as the second environmental noise in the pre-stored environmental audio, it is determined that the environmental audio matches the pre-stored environmental audio.

[0130] Optionally, the extraction module 1002 is further used for:

[0131] Determine whether the unlocked audio has been edited based on the frequency characteristics, where the editing includes splicing and / or resampling;

[0132] If the unlocking audio has not been edited, it is determined that the frequency feature has passed the verification.

[0133] Optionally, the extraction module 1002 is further configured to:

[0134] Obtaining a similarity score between the voiceprint information and pre-stored voiceprint information;

[0135] If the similarity score is greater than the similarity threshold, it is determined that the voiceprint information verification has passed.

[0136] Optionally, the extraction module 1002 is further configured to:

[0137] Upon receiving an authorization instruction triggered by an authorized user, obtaining the authorized voice audio corresponding to the authorized user;

[0138] The pre-stored voiceprint information is extracted according to the authorized sound audio.

[0139] Optionally, the receiving module 1001 is further configured to:

[0140] When receiving an authorization instruction triggered by an authorized user, outputting a preset sentence through a speaker and / or a display;

[0141] The audio corresponding to the preset sentence is collected, and the audio corresponding to the preset sentence is set as the authorized sound audio.

[0142] Optionally, the extraction module 1002 is further configured to:

[0143] Extracting initial voiceprint information based on the authorized voice audio;

[0144] Selecting part of the voiceprint information from the initial voiceprint information to perform frequency domain change processing to obtain watermark data;

[0145] The watermark data is embedded into the initial voiceprint information to obtain the pre-stored voiceprint information.

[0146] Optionally, the extraction module 1002 is further configured to:

[0147] Obtaining the real-time voice audio of the authorized user corresponding to the pre-stored voiceprint information;

[0148] Based on the real-time voiceprint information extracted from the real-time sound audio, the pre-stored voiceprint information corresponding to the authorized user is updated.

[0149] Optionally, the extraction module 1002 is further configured to:

[0150] Sending a voiceprint update request to the authorized user at a preset interval;

[0151] The audio corresponding to the voiceprint update request is set as the real-time sound audio.

[0152] Optionally, the control module 1003 is further configured to:

[0153] When the unlocking result of the unlocking operation is unlocking failure, and the number of consecutive unlocking failures exceeds a preset threshold, unlocking failure information is sent to an associated device, wherein the unlocking failure information includes an unlocking failure prompt and / or an unlocking failure reason, and the associated device includes a cloud device and / or a terminal device.

[0154] Optionally, the control module 1003 is further configured to:

[0155] receiving indication information sent by the associated device according to the unlocking failure information;

[0156] At least one of a warning operation, an unlocking operation, and shielding other unlocking audios associated with the unlocking audio is performed according to the indication information.

[0157] Optionally, the unlocking operation includes door unlocking and / or control unlocking.

[0158] Optionally, the control module 1003 is further configured to:

[0159] When the unlocking result of the unlocking operation is successful, obtaining the in-vehicle audio of the vehicle;

[0160] The vehicle is controlled to perform a door closing operation and / or a locking operation according to the in-vehicle audio.

[0161] In this embodiment, when the user inputs the unlocking audio, the voiceprint information and auxiliary authentication information are extracted based on the unlocking audio; the vehicle is unlocked according to the voiceprint information and the auxiliary authentication information. In this way, the unlocking operation is performed by extracting the voiceprint information and auxiliary authentication information from the unlocking audio, which is more secure than unlocking based on voiceprint recognition alone, and can effectively improve the security of vehicle unlocking. In addition, obtaining auxiliary authentication information from the unlocking audio does not require adding additional information sources, does not increase the difficulty of the process, and can reduce the cost of unlocking.

[0162] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0163] Accordingly, an embodiment of the present application further provides an electronic device, such as Figure 5 As shown, Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1100 also includes a processor 1101 having one or more processing cores, a memory 1102 having one or more computer-readable storage media, and a computer program stored in the memory 1102 and executable on the processor. The processor 1101 is electrically connected to the memory 1102. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0164] The processor 1101 is the control center of the electronic device 1100, and uses various interfaces and lines to connect various parts of the entire electronic device 1100. By running or loading software programs and / or units stored in the memory 1102, and calling data stored in the memory 1102, the processor 1101 executes various functions of the electronic device 1100 and processes data, thereby monitoring the electronic device 1100 as a whole. The processor 1101 can be a processor (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a network processor (Network Processor, NP), etc., and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0165] In the embodiment of the present application, the processor 1101 in the electronic device 1100 will load instructions corresponding to the processes of one or more application programs into the memory 1102 according to the following steps, and the processor 1101 will run the application programs stored in the memory 1102 to implement various functions, such as:

[0166] When receiving the unlocking audio input by the user, extract the voiceprint information and the auxiliary authentication information based on the unlocking audio, and perform the unlocking operation on the vehicle according to the voiceprint information and the auxiliary authentication information

[0167] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0168] Optional, such as Figure 5 As shown, the electronic device 1100 further includes: a touch screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107, respectively. Those skilled in the art can understand that Figure 5The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0169] The touch display screen 1103 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 1103 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. The touch panel can be used to collect the touch operation of the user on or near it (such as the user using any suitable object or accessory such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 1101, and can receive the command sent by the processor 1101 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1101 to determine the type of touch event, and then the processor 1101 provides corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present invention, the touch panel and the display panel can be integrated into the touch display screen 1103 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 1103 can also be used as a part of the input unit 1106 to realize the input function.

[0170] The radio frequency circuit 1104 may be used to send and receive radio frequency signals, so as to establish wireless communication with network medical devices or other electronic devices through wireless communication, and to send and receive signals between network medical devices or other electronic devices.

[0171] The audio circuit 1105 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 1105 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 1105 and converted into audio data, and then the audio data is output to the processor 1101 for processing, and then sent to another electronic device through the radio frequency circuit 1104, or the audio data is output to the memory 1102 for further processing. The audio circuit 1105 may also include an earplug jack to provide communication between an external headset and an electronic device.

[0172] The input unit 1106 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0173] The power supply 1107 is used to supply power to various components of the electronic device 1100. Optionally, the power supply 1107 can be logically connected to the processor 1101 through a power management device, so that the power management device can manage charging, discharging, power consumption and other functions. The power supply 1107 can also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0174] although Figure 5 Not shown, the electronic device 1100 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0175] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0176] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0177] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored. The computer program can be loaded by a processor to execute any unlocking method provided in the embodiment of the present application. The computer program can execute the following steps of the unlocking method:

[0178] When an unlocking audio input by a user is received, voiceprint information and auxiliary authentication information are extracted based on the unlocking audio, and an unlocking operation is performed on the vehicle according to the voiceprint information and the auxiliary authentication information.

[0179] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0180] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0181] Since the computer-readable storage medium can implement the beneficially stored computer program that can be implemented by any unlocking method provided in the embodiments of the present application, any unlocking method provided in the embodiments of the present application can be executed, the effect is detailed in the previous embodiments and will not be repeated here.

[0182] Optionally, an embodiment of the present application also provides a vehicle, which includes any one of the above electronic devices, electronic devices, computer-readable storage media, and computer program products.

[0183] In the above-mentioned unlocking method, electronic device, electronic device, vehicle, computer-readable storage medium, computer program product, etc., the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the above-mentioned electronic device, vehicle, computer-readable storage medium, computer program product and its corresponding units can refer to the description of the unlocking method in the above embodiment, and will not be repeated here.

[0184] The above is a detailed introduction to an unlocking method, an electronic device, an electronic device, a vehicle, a computer-readable storage medium, and a computer program product provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An unlocking method, characterized in that: include: When receiving an unlocking audio input by a user, extracting voiceprint information and auxiliary authentication information based on the unlocking audio; An unlocking operation is performed on the vehicle according to the voiceprint information and the auxiliary authentication information.

2. The unlocking method according to claim 1, characterized in that: The performing the unlocking operation according to the voiceprint information and the auxiliary authentication information includes: Verifying the voiceprint information; Verifying the auxiliary authentication information; When the voiceprint information verification is passed and the auxiliary authentication information verification is passed, the unlocking operation is performed.

3. The unlocking method according to claim 2, characterized in that: The auxiliary authentication information includes at least one of ambient audio and frequency characteristics.

4. The unlocking method according to claim 3, characterized in that: Verifying the auxiliary authentication information includes: The ambient audio is matched with a pre-collected pre-stored ambient audio, and when the ambient audio matches the pre-stored ambient audio, it is determined that the ambient audio has passed the verification.

5. The unlocking method according to claim 4, characterized in that: Matching the ambient audio with pre-collected pre-stored ambient audio includes: Acquire a first ambient noise in the ambient audio; If the first environmental noise is the same as the second environmental noise in the pre-stored environmental audio, it is determined that the environmental audio matches the pre-stored environmental audio.

6. The unlocking method according to claim 3, characterized in that: The verifying the auxiliary authentication information further includes: Determine whether the unlocked audio has been edited based on the frequency characteristics, where the editing includes splicing and / or resampling; If the unlocking audio has not been edited, it is determined that the frequency feature has passed the verification.

7. The unlocking method according to claim 2, characterized in that: The verifying the voiceprint information further includes: Obtaining a similarity score between the voiceprint information and pre-stored voiceprint information; If the similarity score is greater than the similarity threshold, it is determined that the voiceprint information verification has passed.

8. The unlocking method according to claim 7, characterized in that: Before obtaining the similarity score between the voiceprint information and the pre-stored voiceprint information, the method includes: Upon receiving an authorization instruction triggered by an authorized user, obtaining the authorized voice audio corresponding to the authorized user; The pre-stored voiceprint information is extracted according to the authorized sound audio.

9. The unlocking method according to claim 8, characterized in that: The obtaining of the authorized audio corresponding to the authorized user includes: outputting a preset sentence through a speaker and / or a display; The audio corresponding to the preset sentence is collected, and the audio corresponding to the preset sentence is set as the authorized sound audio.

10. The unlocking method according to claim 8, characterized in that: The extracting the pre-stored voiceprint information according to the authorized sound audio includes: Extracting initial voiceprint information based on the authorized voice audio; Selecting part of the voiceprint information from the initial voiceprint information to perform frequency domain change processing to obtain watermark data; The watermark data is embedded into the initial voiceprint information to obtain the pre-stored voiceprint information.

11. The unlocking method according to claim 7, characterized in that: The method further comprises: Obtaining the real-time voice audio of the authorized user corresponding to the pre-stored voiceprint information; Based on the real-time voiceprint information extracted from the real-time sound audio, the pre-stored voiceprint information corresponding to the authorized user is updated.

12. The unlocking method according to claim 11, characterized in that: The obtaining of the real-time voice audio of the authorized user corresponding to the pre-stored voiceprint information includes: Sending a voiceprint update request to the authorized user at a preset interval; The audio corresponding to the voiceprint update request is set as the real-time sound audio.

13. The unlocking method according to claim 1, characterized in that: The method further comprises: When the unlocking result of the unlocking operation is unlocking failure, and the number of consecutive unlocking failures exceeds a preset threshold, unlocking failure information is sent to an associated device, wherein the unlocking failure information includes an unlocking failure prompt and / or an unlocking failure reason, and the associated device includes a cloud device and / or a terminal device.

14. The unlocking method according to claim 13, characterized in that: The method further comprises: receiving indication information sent by the associated device according to the unlocking failure information; At least one of a warning operation, an unlocking operation, and shielding other unlocking audios associated with the unlocking audio is performed according to the indication information.

15. The unlocking method according to any one of claims 1 to 14, characterized in that: The unlocking operation includes door unlocking and / or control unlocking.

16. The unlocking method according to claim 15, characterized in that: include: When the unlocking result of the unlocking operation is successful, obtaining the in-vehicle audio of the vehicle; The vehicle is controlled to perform a door closing operation and / or a locking operation according to the in-vehicle audio.

17. An electronic device, characterized in that: include: A receiving module, receiving unlocking audio input by a user; An extraction module, used to extract voiceprint information and auxiliary authentication information based on the unlocking audio; A control module is used to perform an unlocking operation according to the voiceprint information and the auxiliary authentication information.

18. An electronic device, characterized in that: It includes a processor, the processor is connected to a memory, the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the unlocking method according to any one of claims 1 to 16.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the unlocking method according to any one of claims 1 to 16 is implemented.

20. A computer program product, characterized in that It includes a computer program, which is executed by a processor to implement the unlocking method described in any one of claims 1 to 16.

21. A vehicle, characterized in that: The vehicle executes the unlocking method as described in any one of claims 1 to 16, or includes the electronic device as described in claim 17 or the electronic device as described in claim 18.

Images