Biometric information verification method and equipment

Abstract

Biometric information verification method and device. In this method, the device can collect the user's voiceprint information and call a nearby face collection device to collect the user's face information. When the device receives a user operation requesting payment for paid audio resources, the device can verify the user's identity by combining voiceprint verification and face verification. When the verification is successful, the device can request the payment server to deduct money from the payment account to pay for the paid audio resources. The above method of combining multiple biometric information to verify the user's identity can improve the security of identity authentication. In addition, devices that are not equipped with a face collection device can also combine voiceprint verification and face verification to complete identity authentication during the payment process, which can simplify the user's payment operation.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a biometric information verification method and device. Background Art

[0002] With the development of smart home devices such as speakers, users can easily listen to various audio resources through speakers, such as music, audio books, etc. Paying for audio resources has also become a trend.

[0003] Completing identity authentication during the payment process by means of biometric information verification can improve the convenience of the payment process. However, general speakers only contain a voice collection device for collecting voiceprint information, and are not equipped with devices for collecting other types of biometric information, such as face collection devices, fingerprint collection devices, etc. Completing identity authentication during the payment process by only using a single biometric information verification method (such as voiceprint verification) is not conducive to ensuring identity authentication security. Summary of the invention

[0004] The present application provides a biometric information verification method and device. The method can be used for identity verification during the payment process. Among them, a device (such as a speaker) can collect the user's first biometric information, and call a nearby device with the ability to collect the second biometric information to collect the user's second biometric information. The first biometric information and the second biometric information are different types of biometric information. The device can verify the user's identity in the payment process by combining the first biometric information verification and the second biometric information verification. In this way, the user can complete the identity verification during the payment process through his or her first biometric information and the second biometric information. The above method can simplify the user's payment operation. In addition, a device that does not have the ability to collect the second biometric information can also verify the user's identity by combining the first biometric information verification and the second biometric information verification to improve the security of identity verification.

[0005] In a first aspect, the present application provides a biometric information verification method. The method includes: a first device can collect first biometric information. The first device can discover a second device with a second biometric information collection capability through a short-range wireless communication connection. The first device can receive second biometric information collected from the second device, and the second biometric information is different from the first biometric information. Further, the first device can determine that the first biometric information matches the third biometric information, and the second biometric information matches the fourth biometric information. The above-mentioned third biometric information can be stored in the first device or in a cloud storage space accessible to the first device. The third biometric information is the biometric information of the first user pre-collected by the first device. The fourth biometric information can be stored in the first device or in a cloud storage space accessible to the first device. The fourth biometric information is the biometric information of the first user pre-collected by the second device.

[0006] Through the biometric information verification method provided by the present application, the first device can complete the identity verification during the payment process. The first device that does not have the second biometric information collection capability can also verify the user's identity by combining the first biometric information verification and the second biometric information verification to improve the security of identity verification. In addition, the user can quickly complete the identity verification during the payment process through his or her first biometric information and the second biometric information, thereby improving the convenience of the payment process.

[0007] The third biometric information and the fourth biometric information can be used as reference biometric information of the first user, and are used to determine whether the first biometric information obtained by the first device is the biometric information of the first user and whether the second biometric information obtained by the first device is the biometric information of the first user.

[0008] The above short-range wireless communication connection may be one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

[0009] In a possible implementation, when the identity authentication succeeds, the first device may send a first message to the first server. The first message may be used to instruct the first server to deduct money from the first payment account. The first payment account is the payment account of the first user. The first server may be a payment server.

[0010] In combination with the first aspect, in some embodiments, the first device is a device equipped with a voice collection device, such as a speaker. The second device is a device equipped with a face collection device, such as a television. The first biometric information and the third biometric information can both be voiceprint information. The second biometric information and the fourth biometric information can both be face information.

[0011] In this way, the speaker can verify the user's identity by combining voiceprint verification and face verification. That is, the user can complete the identity verification through his or her voice and face, so as to make payment conveniently.

[0012] In combination with the first aspect, in other embodiments, the first device is a device equipped with a face acquisition device, such as a television. The second device can be a device equipped with a voice acquisition device, such as a speaker. The first biometric information and the third biometric information can both be face information. The second biometric information and the fourth biometric information can both be voiceprint information.

[0013] In combination with the first aspect, in some embodiments, before the first device collects the first biometric information, the method further includes: the first device may log in to a first account on the second server. The first account is an account of the first user on the second server. The second server may be used to send the content purchased by the first user to the first device after the first server successfully deducts money from the first payment account. The first device may send a request to obtain the first content to the second server. The first device may receive price information of the first content sent by the second server.

[0014] The first server may be a content server. Specifically, the content server may be, for example, a music server, a video server, etc. The first content may be, for example, audio data, video data, etc.

[0015] It should be noted that the first device can be bound to the first account. Exemplarily, a communication connection is established between the first device and the third device. An application for controlling the first device is installed in the third device. The third device can be, for example, a mobile phone, a tablet, etc. The third device can log in to the first account in the application. Furthermore, the third device can send relevant information of the first account to the first device. In this way, the first device can establish a binding relationship with the first account.

[0016] Before the first device collects the third biometric information and the fourth biometric information of the first user, the third device may authenticate the first payment account. Among them, the third device may receive the payment password of the first payment account entered by the user. Then the third device may compare the received payment password with the stored payment password of the first payment account. If they are the same, the third device successfully authenticates the first payment account. The first payment account may be associated with the first account. Further, the third device may send an instruction to the first device for collecting the third biometric information and the fourth biometric information. In this way, the first device may collect the third biometric information and the fourth biometric information, and associate the remaining first accounts. During the payment process, if the first biometric information matches the third biometric information, and the second biometric information matches the fourth biometric information, the first device may request the payment server to deduct the payment account associated with the first account, that is, the first payment account, to purchase the above-mentioned first content.

[0017] That is to say, the first user can first complete the identity authentication of the first payment account on the third device, and then instruct the first device to enter the third biometric information and the fourth biometric information through the third device.

[0018] In addition, the content server may store identifiers of multiple accounts. The multiple accounts include the first account. The content server may determine the paid resources purchased by each account based on the identifiers of the multiple accounts. When the first device logs in to the first account on the content server and obtains the first content from the content server, the content server may determine that the first account has not purchased the first content. Furthermore, the content server may send a message to the first device to prompt the first device that payment is required to obtain the first content. The content server may also send price information of the first content to the first device.

[0019] The following specifically takes the example of the first device being a speaker equipped with a voice collection device and the second device being a TV equipped with a face collection device to illustrate how the speaker completes identity authentication through biometric information and requests the payment server to deduct money from the first payment account.

[0020] For example, the first user wakes up the speaker and instructs the speaker to play music A. The music A is a paid resource. The speaker can receive information sent by the music server to indicate that music A can only be obtained after payment. The speaker can voice broadcast "You can listen to the complete music A after payment, do you want to pay?" to remind the first user that the music A that has not been purchased cannot be played at present. Furthermore, the speaker can receive a voice command "Please help me complete the payment." That is, the first user requests to purchase music A.

[0021] The speaker can start voiceprint verification and face verification. Specifically, the speaker can voice broadcast "OK, start voiceprint verification. Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the first user to enter the voiceprint information. The first user can say the verification word "Xiaoyi Xiaoyi" according to the prompt of the speaker. The speaker can extract the voiceprint information from the received voice input and compare it with the reference voiceprint information of the first user. If it matches, the voiceprint verification is successful.

[0022] Then, the speaker can find and call a nearby TV equipped with a camera to collect facial information. The speaker can voice broadcast "Voiceprint verification is successful, and facial verification begins. Please point your face to the camera of Li Ming's TV and blink" to prompt the first user to enter facial information through the TV named "Li Ming". Among them, the speaker can prompt the user to adjust the face through voice broadcast so that the face is aligned with the camera. When receiving facial information collected from the TV, the speaker can compare the facial information with the reference facial information of the first user. If they match, the facial verification is successful. In this way, the speaker can complete the identity authentication of the user.

[0023] Furthermore, the speaker can request the payment server to deduct money from the first payment account. The payment server can perform trusted identity authentication on the speaker to determine that the speaker is a trusted device. The speaker can send the identification information of the first payment account and the order information of the paid content (i.e., the above-mentioned music A) that the first user needs to purchase to the payment server. In this way, the payment server can deduct money from the first payment account based on the above-mentioned order information. When the deduction is successful, the payment server can send a message to the music server to indicate that the payment is successful. Thereby, the music server can send the resources of music A to the speaker. The speaker can play the successfully purchased music A for the user.

[0024] In combination with the first aspect, in some embodiments, the third biometric information and the fourth biometric information stored in the cloud storage space can be accessed by multiple devices. The multiple devices include the first device. The multiple devices may share the same account on the content server, or the multiple accounts of the multiple devices on the content server belong to the same account group; belonging to the same account group means that multiple accounts in the same account group share the content purchased by each account in the second server.

[0025] Among them, since the third biometric information and the fourth biometric information stored in the cloud storage space can be accessed by multiple devices sharing the same account, these multiple devices can obtain the third biometric information and the fourth biometric information from the cloud storage space for identity authentication. In this way, the user only needs to enter the reference biometric information once, and can pay for the paid resources requested by the first device through the first biometric information verification and the second biometric information verification on multiple speakers. The above implementation method can simplify the operation of the user entering the reference biometric information for payment and improve the user experience.

[0026] In combination with the first aspect, in some embodiments, the first user may authorize the second user to establish reference biometric information of the second user on the first device. The method may include: the first device may collect fifth biometric information of the second user. The fifth biometric information is the same type of biometric information as the first biometric information. The fifth biometric information may be used to determine whether the biometric information obtained by the first device is the biometric information of the second user. The fifth biometric information may be stored in the first device or in a cloud storage space accessible to the first device. The fifth biometric information is bound to the fourth biometric information.

[0027] If the first biometric information is voiceprint information, the fifth biometric information is also voiceprint information.

[0028] The first user may be a main user associated with the first account. The second user may be an authorized user associated with the first account. The main user associated with the first account may be a user corresponding to the reference biometric information established by the first device when the reference biometric information associated with the first account is not stored in the first device or the cloud storage space accessible to the first device. The authorized user associated with the first account may be a user corresponding to the reference biometric information established by the first device when the reference biometric information associated with the first account is stored in the first device or the cloud storage space accessible to the first device. The first account may be associated with one main user and one or more authorized users.

[0029] When the second user (i.e., authorized user) uses his or her own biometric information for verification, the main user's authorization is required to complete the identity authentication. Specifically, the first device can collect the sixth biometric information. The sixth biometric information and the first biometric information are the same type of biometric information. The first device can receive the second biometric information collected from the second device. The first device can determine that the sixth biometric information matches the fifth biometric information, and the third biometric information matches the fourth biometric information. The first device can send a second message to the first server, and the second message is used to instruct the first server to deduct money from the first payment account.

[0030] It can be seen that the above-mentioned main user authorization process can be a process of verifying the second biometric information of the main user. That is to say, when the voiceprint information of the authorized user is verified, combined with the verification of the main user's face information, the first device can request the payment server to deduct money from the main user's payment account (i.e., the first payment account) to purchase paid resources. In this way, it is not limited to the main user completing the payment through the first biometric information verification and the second biometric information verification. Authorized users, such as the main user's family and friends, can also complete the payment through the first biometric information verification and the second biometric information verification with the authorization of the main user. In addition, since the first payment account associated with the first account bound to the first device can be considered as the main user's payment account, the verification method combined with the authorized user's voiceprint information and the main user's face information can be confirmed by the main user before the first payment account makes a payment, thereby preventing the authorized user from abusing authority and over-consuming the main user's payment account.

[0031] In combination with the first aspect, in other embodiments, the first user may authorize the second user to establish reference biometric information of the second user on the first device. The method includes: the first device may collect the fifth biometric information of the second user. The fifth biometric information is the same type of biometric information as the first biometric information. The fifth biometric information can be used to determine whether the biometric information obtained by the first device is the biometric information of the second user. The fifth biometric information can be stored in the first device or in a cloud storage space accessible to the first device. The first device may receive the seventh biometric information of the second user collected from the second device. The seventh biometric information is the same type of biometric information as the second biometric information. The seventh biometric information can be used to determine whether the biometric information obtained by the first device is the biometric information of the second user. The seventh biometric information can be stored in the first device or in a cloud storage space accessible to the first device. Among them, the fifth biometric information is bound to the seventh biometric information.

[0032] When the second user (i.e., authorized user) uses his or her own biometric information for verification, the main user's authorization may not be required. That is, the second user can independently complete identity authentication using his or her own biometric information. The first device can request the payment server to deduct money from the first payment account after the second user's identity authentication is successful. Specifically, the first device can collect the sixth biometric information. The sixth biometric information is the same type of biometric information as the first biometric information. The first device can receive the eighth biometric information collected from the second device. The eighth biometric information is the same type of biometric information as the second biometric information. The first device can determine that the sixth biometric information matches the fifth biometric information, and the eighth biometric information matches the seventh biometric information. Furthermore, the first device can send a second message to the first server. The second message is used to instruct the first server to deduct money from the first payment account.

[0033] In the above method, the second user (ie, the authorized user) can complete identity verification independently. In this way, when the primary user is no longer near the first device, the second user can also complete identity verification on the first device and purchase paid resources using the primary user's payment account.

[0034] Optionally, the second user still needs the primary user's consent to purchase paid resources using the primary user's payment account.

[0035] Specifically, after the sixth biometric information verification and the eighth biometric information verification are both successful, the method further includes: the first device may send a third message to the third device, and receive a fourth message from the third device. The third device is a device installed with an application for controlling the first device, and the indication content of the third message may be displayed on the third device, and the indication content of the third message includes asking the third device whether it agrees with the first device to send the second message, and the indication content of the fourth message includes the third device agreeing with the first device to send the second message.

[0036] That is, after the second user completes identity authentication on the first device, the first device may send a third message to the third device. The third message may prompt the first user: the second user requests to use the first user's payment account to purchase paid resources. If the first user agrees that the second user uses his payment account to purchase paid resources, the third device may send a fourth message to the first device. Upon receiving the fourth message, the first device may request the payment server to deduct money from the first payment account to purchase paid resources.

[0037] It can be seen that the first user (i.e., the main user) can remotely authorize the first device to request payment of the paid resources from the payment server through the third device. The above-mentioned remote authorization by the main user can be: when the first device detects the biometric information of the authorized user successfully, the first device can send a message to the third device to confirm whether to pay. The third device can display the above-mentioned message for confirming whether to pay on the user interface. Furthermore, the main user can authorize the first device through the third device to request the payment server to deduct money from the first payment account.

[0038] In this way, even if the primary user is not near the first device, the second user (i.e., the authorized user) can also complete identity authentication and purchase paid resources. Combining the above-mentioned method of remote authorization by the primary user can ensure that the primary user's payment account is deducted only after the primary user agrees. This not only simplifies the payment operation when obtaining paid resources on the first device, but also improves the security of the payment account and effectively prevents the authorized user from overspending the primary user's payment account.

[0039] In some embodiments, in combination with the first aspect, the first device does not have the ability to collect the second biometric information. The first device can collect the second biometric information by searching for and calling a nearby device that has the ability to collect the second biometric information.

[0040] In this way, a device that does not have the second biometric information collection capability can also verify the user's identity by combining the first biometric information verification and the second biometric information verification, thereby improving the security of the first device's identity authentication.

[0041] In a second aspect, the present application provides a device, which is a first device. The first device may include: a first collection device, a communication device, a memory and a processor, wherein: the first collection device can be used to collect first biometric information. The communication device can be used for the first device to discover a second device with a second biometric information collection capability through a wireless communication connection. The second device includes a second collection device, and the second collection device can be used to collect the above-mentioned second biometric information. The communication device can also be used to receive the second biometric information collected from the second collection device. The memory can be used to store the first biometric information and the second biometric information, and can also be used to store a computer program. The memory can be used to call the above-mentioned computer program so that the first device executes any possible implementation method of the above-mentioned first aspect.

[0042] In a third aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on the device provided in the second aspect, enables the device to execute any possible implementation method of the first aspect.

[0043] In a fourth aspect, the present application provides a computer program product. When the above-mentioned computer program product is run on the device provided in the second aspect, the device executes any possible implementation method of the above-mentioned first aspect.

[0044] In a fifth aspect, the present application provides a chip, which is applied to the device provided in the second aspect, and the chip includes one or more processors, and the one or more processors are used to call computer instructions to enable the device provided in the second aspect to execute any possible implementation method of the above-mentioned first aspect.

[0045] It can be understood that the device provided in the second aspect, the computer-readable storage medium provided in the third aspect, the computer program product provided in the fourth aspect, and the chip provided in the fifth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a structural schematic diagram of a speaker provided in an embodiment of the present application;

[0047] Figure 2A to Figure 2E It is a schematic diagram of a series of user interfaces for authenticating a payment account provided in an embodiment of the present application;

[0048] Figure 2F to Figure 2J Schematic diagram of some scenarios of recording reference voiceprint information and reference face information by speakers provided in the embodiments of the present application;

[0049] Figure 3It is a flow chart of a method for a speaker to record reference voiceprint information and reference face information provided by an embodiment of the present application;

[0050] Figure 4A to Figure 4E This is a schematic diagram of a scenario of a speaker paying for paid audio resources provided in an embodiment of the present application;

[0051] Figure 5 This is a flow chart of a method for a speaker to pay for paid audio resources provided in an embodiment of the present application;

[0052] Figure 6A to Figure 6F Schematic diagram of other scenarios of recording reference voiceprint information and reference face information by other speakers provided in the embodiments of the present application;

[0053] Figure 7 It is a flow chart of another method for recording reference voiceprint information and reference face information in a speaker provided in an embodiment of the present application;

[0054] Figure 8A to Figure 8E It is a schematic diagram of another scenario of a speaker paying for paid audio resources provided in an embodiment of the present application;

[0055] Fig. 9 is a flow chart of another method for a speaker to pay for paid audio resources provided in an embodiment of the present application;

[0056] Figures 10A to 10E Schematic diagram of other scenarios of recording reference voiceprint information and reference face information by other speakers provided in the embodiments of the present application;

[0057] FIG. 11A to FIG. 11D This is a schematic diagram of another scenario of a speaker paying for paid audio resources provided in an embodiment of the present application;

[0058] FIG. 12A to FIG. 12D This is a schematic diagram of another scenario of speaker payment for paid audio resources provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0060] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0061] At present, when listening to paid audio resources through a speaker, the user can complete the payment through an electronic device installed with a speaker application (application, APP) associated with the speaker, so as to listen to the paid audio resources on the speaker. Specifically, an electronic device, such as a mobile phone, is installed with a speaker APP. In response to the user operation of pairing the speaker with the above-mentioned mobile phone, the speaker can be paired with the above-mentioned mobile phone. In this way, the speaker can establish a communication connection with the above-mentioned mobile phone. Further, in response to the user operation of logging into the first account of the speaker APP on the above-mentioned mobile phone, the speaker APP can log in to the above-mentioned first account. When the login is successful, the first account of the above-mentioned speaker APP can be bound to the speaker. Among them, the first account of the above-mentioned speaker APP can be associated with the payment account of the above-mentioned user (such as Huawei Wallet payment account, Alipay payment account, WeChat payment account). In this way, when the account bound to the speaker is the above-mentioned first account, in response to the user operation of paying for paid audio resources, the payment account associated with the above-mentioned first account can be paid.

[0062] When the audio resource being played needs to be paid, the speaker can prompt the user to pay on the above-mentioned speaker APP through voice broadcast. In other words, the user needs to open the speaker APP, determine the audio resource that needs to be paid, and enter the payment password of the payment account. Then, the electronic device with the speaker APP installed can complete the payment and send a message of successful payment to the speaker. In this way, the speaker can play paid audio resources.

[0063] It can be seen that the above payment process is cumbersome when using a speaker to listen to paid audio resources. Especially when it is inconvenient for the user to use an electronic device with a speaker APP installed, for example, when both hands are occupied, the user cannot complete the above payment process. As a result, the user cannot listen to the paid audio resources in time.

[0064] In addition, the speaker can also complete the identity verification in the payment process by means of biometric information verification. The speaker can call the payment account to pay the payment resources after the biometric information verification is successful. This can improve the convenience of the payment process. However, the speaker is often only equipped with a voice collection device for collecting voiceprint information. That is, the speaker is generally not equipped with a device for collecting other types of biometric information, such as a face collection device, a fingerprint collection device, a bone voiceprint collection device, and so on. Then, the speaker can only complete the identity verification in the payment process by means of voiceprint verification. The above-mentioned single biometric information verification method is not conducive to ensuring identity verification security.

[0065] The present application provides a biometric information verification method. The biometric information verification method can be used for a speaker that is not equipped with a face collection device (such as a camera) to complete identity authentication during the payment process. Specifically, the speaker can realize payment by combining voiceprint verification and face verification. Among them, the speaker can extract voiceprint information from the user's voice input and store it as reference voiceprint information. Further, the speaker can call a nearby device equipped with a face collection device, such as a mobile phone, a tablet, a laptop, a TV, a surveillance camera, etc., to collect the user's face image. When the face image is obtained, the above-mentioned device equipped with a face collection device can send the face image to the speaker. The speaker can extract face information from the above-mentioned face image and store it as reference face information, and bind the reference face information with the reference voiceprint information. When the audio resource played needs to be paid, the speaker can prompt the user to enter the voiceprint information and face information by voice broadcast. Furthermore, the speaker can compare the above-mentioned voiceprint information and face information obtained by entering with the stored reference voiceprint information and reference face information respectively. If the above-mentioned voiceprint information and face information obtained by entering match, the speaker can call the payment account for payment. Among them, when obtaining facial information during the above payment process, the speaker can still call a nearby device equipped with a facial collection device to collect facial images.

[0066] In this way, users only need to say the preset verification words and use their own facial information to complete the payment and then listen to the paid audio resources. This greatly simplifies the payment process when using the speaker to listen to paid audio resources. Speakers that are not equipped with face collection devices can also pay for paid audio resources by combining voiceprint verification and face verification. In addition, the combination of voiceprint verification and face verification can improve the security of the payment process.

[0067] Figure 1 The structural diagram of a speaker 100 provided in an embodiment of the present application is exemplarily shown.

[0068] The following is a detailed description of the embodiments of the present application using the speaker 100 as an example. It should be understood that: Figure 1The speaker 100 shown is only an example, and the speaker 100 may have more Figure 1 More or fewer components may be shown, two or more components may be combined, or there may be a different configuration of components. Figure 1 The various components shown in the EMBODIMENTS 2000 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0069] The speaker 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 2, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a sensor module 180, a button 190, a motor 191 and an indicator 192, etc.

[0070] It is understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the speaker 100. In other embodiments of the present application, the speaker 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0071] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0072] In some embodiments, the processor 110 may include a voice wake-up module and a voice command recognition module. Among them, the voice wake-up module and the voice command recognition module can be integrated in different processor chips and executed by different chips. For example, the voice wake-up module can be integrated in a coprocessor or DSP chip with lower power consumption, and the voice command recognition module can be integrated in an AP or NPU or other chip. In this way, after the voice wake-up module recognizes the preset voice wake-up word, the chip where the voice command recognition module is located can be started to trigger the voice command recognition function, thereby saving power consumption of the electronic device. Alternatively, the voice wake-up module and the voice command recognition module can be integrated in the same processor chip, and the related functions are performed by the same chip. For example, both the voice wake-up module and the voice command recognition module can be integrated in an AP chip or an NPU or other chip.

[0073] The processor 110 may further include a voice instruction execution module, that is, after recognizing a voice instruction, executing an operation corresponding to the voice instruction.

[0074] In some embodiments, the processor 110 may further include a security module. The security module may be a module integrated in the AP chip. Alternatively, the security module may also be integrated on an independent security chip. The security module may be used to store reference voiceprint information and reference facial information, and complete local verification during the payment process. Exemplarily, during the payment process, the security module may compare the voiceprint information and facial information to be verified with the reference voiceprint information and reference facial information, respectively. If the voiceprint information to be verified matches the reference voiceprint information, and the facial information to be verified matches the reference facial information, the local verification is passed. The speaker 200 can call the payment account for payment.

[0075] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0076] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the speaker 100. While the charging management module 140 is charging the battery 142, it may also power the speaker through the power management module 141.

[0077] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0078] The wireless communication function of the speaker 100 can be realized through the antenna 2, the wireless communication module 160, the modem processor and the baseband processor.

[0079] The antenna 2 is used to transmit and receive electromagnetic wave signals. Each antenna in the speaker 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization rate of the antennas.

[0080] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the speaker 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module.

[0081] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the speaker 100 can be realized, such as: face verification, voiceprint verification, text understanding, speech synthesis, etc.

[0082] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the speaker 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0083] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the speaker 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function), etc. The data storage area may store data (such as audio data) created during the use of the speaker 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0084] The sound box 100 can realize audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C and the application processor.

[0085] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0086] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The speaker 100 can listen to music or answer a call through the speaker 170A.

[0087] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.

[0088] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. Exemplarily, the speaker 100 can collect the user's voice input through the microphone 170C and extract voiceprint information from the voice input. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The speaker 100 can be provided with at least one microphone 170C. In other embodiments, the speaker 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the speaker 100 can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, and realization of directional recording function, etc.

[0089] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button or a touch button. The speaker 100 can receive the button input and generate a key signal input related to the user setting and function control of the speaker 100.

[0090] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as audio playback) can correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0091] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0092] In the present application, the speaker 100 needs to obtain the user's reference voiceprint information and reference face information before making a payment through voiceprint verification and face verification. In one possible implementation, the speaker 100 can receive the user's voice input and face image after receiving an instruction to instruct the creation of new biometric information. Further, the speaker 100 can extract the voiceprint information from the above voice input, and store the voiceprint information in the security module as reference voiceprint information. The speaker can extract face information from the above face image, and store the face information in the security module as reference face information.

[0093] The biometric information includes voiceprint information and face information. The instruction for instructing to create new biometric information may come from the electronic device 200. The electronic device 200 may be a device installed with a speaker APP associated with the speaker 100. For example, a mobile phone, a tablet, etc. The schematic diagram of the structure of the electronic device 200 can refer to Figure 1 The structure diagram of the speaker 100 is shown in FIG. Figure 1In addition to the components shown, the electronic device 200 may also include more or fewer components. For example, the electronic device 200 may also include components such as a display screen and a camera.

[0094] The above-mentioned speaker APP may be an application program named “speaker.” The embodiment of the present application does not specifically limit the above-mentioned speaker APP.

[0095] The embodiment of the present application does not specifically limit the type of the electronic device 200 .

[0096] In the present application, the speaker 100 may be a device that is not equipped with a face collection device (such as a camera). When face verification is required, the speaker 100 may call a face collection device 300 that is equipped with a face collection device nearby. When a face image is collected, the face collection device 300 may send the face image to the speaker 300. Furthermore, the speaker 100 may extract face information from the face image.

[0097] In a possible implementation, after the face acquisition device 300 acquires the face image, the face information can be extracted from the face image and sent to the speaker 100. In this way, the speaker 100 can directly store the received face information in the security module as reference face information. The embodiment of the present application does not limit the device for extracting face information from the face image.

[0098] The structural diagram of the face acquisition device 300 can be referred to Figure 1 The structure diagram of the speaker 100 is shown in FIG. The face acquisition device 300 may include a face acquisition device, such as a camera. Figure 1 In addition to the components shown, the face acquisition device 300 may also include more or fewer components.

[0099] The face acquisition device 300 may be, for example, a mobile phone, a tablet, a laptop, a television, a surveillance camera, etc. In some embodiments, the face acquisition device 300 and the electronic device 200 may be the same device.

[0100] The embodiment of the present application does not limit the type of the face acquisition device 300.

[0101] The following is a schematic diagram of a scenario in which the speaker 100 provided in an embodiment of the present application obtains reference voiceprint information and reference face information.

[0102] In a possible implementation, the speaker 100 may obtain the reference voiceprint information and reference face information for payment after receiving an instruction from the electronic device 200 to obtain the reference voiceprint information and reference face information. The electronic device 200 may send the instruction to obtain the reference voiceprint information and reference face information after the payment password of the payment account is successfully verified. In this way, the electronic device 100 can authenticate the user who enters the reference voiceprint information and reference face information to ensure that the user is the user corresponding to the above payment account.

[0103] Figure 2A to Figure 2E The user interface for the electronic device 200 to perform identity authentication for a payment account is exemplarily shown.

[0104] like Figure 2A As shown, the electronic device 200 can display a user interface 210 of the speaker APP. The user interface 210 may include a login account 211, a speaker status 212, and setting options 213. Among them:

[0105] The login account 211 can be used to display the name of the account currently logged in to the speaker APP. Exemplarily, the name of the account currently logged in to the speaker APP can be "Li Ming". In response to the user's operation on the login account 211, the electronic device 200 can display content such as switching account options and logout options. The above-mentioned switching account option can be used to switch the account currently logged in to the speaker APP. The above-mentioned logout option can be used to log out of the account currently logged in to the speaker APP.

[0106] The speaker status 212 can be used to display the status information of the speaker paired with the electronic device 200. Specifically, the electronic device 200 is successfully paired with the speaker 100. The speaker status 212 may include a Wi-Fi connection status 212A, a Bluetooth status 212B, a power level 212C, and a speaker setting 212D. The Wi-Fi connection status 212A may indicate whether the speaker is connected to the network, and the name of the Wi-Fi when connected to Wi-Fi. For example, the speaker 100 is connected to a Wi-Fi named "liming". The Bluetooth status 212B may indicate whether the Bluetooth of the speaker is turned on. For example, when the Bluetooth of the speaker 100 is turned on, the Bluetooth status 212B may display the prompt "turned on". The power level 212C may indicate the current remaining power of the speaker. For example, the current remaining power of the speaker 100 is 80% of the total power. The speaker setting 212D may be used to set relevant information of the speaker. For example, setting the name of the speaker, setting the speaker to use biometric information for payment, etc. In response to a user operation on the speaker setting 212D, the electronic device 200 may display the following information: Figure 2B User interface 220 is shown.

[0107] It should be noted that the electronic device 200 can be paired with the speaker 100 via a Bluetooth connection. Specifically, when connected for the first time, the speaker 100 turns on Bluetooth. The electronic device 200 turns on the speaker APP, turns on Bluetooth, and scans nearby Bluetooth. When the Bluetooth of the speaker 100 is scanned, the electronic device 200 can be paired and connected with the speaker 100. When the Bluetooth connection is disconnected and paired again, if both the electronic device 200 and the speaker 100 turn on Bluetooth, the electronic device 200 can automatically pair when it is close to the speaker 100. The specific process of establishing the Bluetooth connection above can refer to the implementation method of the prior art, which will not be repeated here.

[0108] The embodiment of the present application does not specifically limit the method for pairing the electronic device 200 and the speaker 100. In addition to pairing via Bluetooth, pairing can also be performed via other wired (such as data cable, optical fiber, etc.) or wireless (such as NFC, ZigBee, Wi-Fi, etc.) methods.

[0109] In addition, when the electronic device 200 is successfully paired with the speaker 100, the electronic device 200 can configure a network for the speaker 100. Exemplarily, after the electronic device 200 accesses a network (such as WiFi), the name and password of the WiFi can be sent to the speaker 100. After the speaker 100 receives the name and password of the WiFi, the speaker 100 can connect to the WiFi and then access the network. The speaker 100 accessing the network can search for audio resources on the network, and obtain the audio resources from a server that provides the audio resources, such as a music server, and play them.

[0110] When the speaker 100 is connected to the network, the speaker 100 can be bound to the account logged in on the speaker APP in the electronic device 100. For example, if there is an account named "Li Ming" logged in on the speaker APP, the speaker 100 can be bound to the account "Li Ming".

[0111] The above-mentioned speaker and account binding can mean that the speaker can obtain data associated with the account, such as the audio resources collected by the account, the audio resources purchased, the historical audio resources listened to, etc. For example, the speaker 100 is bound to the account named "Li Ming". For the paid audio resources that the account "Li Ming" has purchased, the speaker 100 can play them directly without the account "Li Ming" having to pay again. In addition, when it is necessary to pay for the paid audio resources, the speaker 100 can call the payment account associated with the account "Li Ming" to make the payment.

[0112] The embodiments of the present application do not limit the method of pairing the electronic device 200 with the speaker 100, the method of configuring the network for the speaker 100 by the electronic device 200, and the method of establishing a binding relationship between the speaker 100 and the account logged in to the speaker APP. The above-mentioned specific implementation methods can refer to the implementation methods in the prior art.

[0113] The setting options 213 may include setting options such as music collection, purchased programs, programs to be purchased, music preferences, audio subscriptions, APP message push, questions and suggestions. Exemplarily, the above-mentioned music collection can be used to display the music collected by the currently logged-in account. The above-mentioned purchased programs and programs to be purchased can be used to display the audio resources that have been purchased by the currently logged-in account and the audio resources that have been added to the shopping cart and are waiting to be purchased. The above-mentioned music preferences can be used to set the music style preferred by the currently logged-in account, such as rock music, classical music, light music, etc. The above-mentioned audio subscription can be used to display the audio programs subscribed by the currently logged-in account. The above-mentioned APP message push can be used to turn on or off the function of the speaker APP to push messages to electronic devices.

[0114] In response to the user operation on the above-mentioned speaker setting 212D, the electronic device 200 can display the following Figure 2B The user interface 220 shown. The user interface 220 may include the following setting options: speaker name 221, Bluetooth 222, wake-up response 223, alarm ringtone 224, home device authorization management 225, speaker information 226, unbinding 227 and biometric payment 228. Among them,

[0115] The speaker name 221 can be used to set the name of the speaker 100. For example, the current name of the speaker 100 is "Li Ming's speaker". In response to a user operation acting on the speaker name 221 and inputting a new name, the electronic device 200 can modify the name of the speaker 100.

[0116] The Bluetooth 222 may be used to turn on or off the Bluetooth of the speaker 100 .

[0117] The wake-up response 223 can be used to turn on or off the voice wake-up function of the speaker 100. For example, when the voice wake-up function of the speaker 100 is turned on, the speaker 100 can wake up the processor after receiving a preset wake-up word (such as Xiaoyi Xiaoyi), and then execute the voice command spoken by the user after the wake-up word.

[0118] The alarm tone 224 may be used to set an alarm for the speaker 100 .

[0119] The home device authorization management 225 can be used to set the home devices that the speaker 100 can control.

[0120] The speaker information 226 may be used to display configuration information of the speaker 100 , such as the model, version number, speaker identification, storage space size, etc.

[0121] Unbinding 227 can be used to unbind the account currently logged in to the speaker APP and the speaker 100.

[0122] Biometric payment 228 can be used to instruct the speaker 100 to create a new reference biometric information for paying for paid resources. The reference biometric information may include reference voiceprint information and reference face information. The reference biometric information can be used by the speaker 100 to verify the user's identity during the payment of paid resources. In other words, during the payment of paid resources, the speaker 100 collects the user's voiceprint information and face information, and compares the voiceprint information and face information with the above-mentioned reference voiceprint information and reference face information respectively. If both match, the user identity verification is successful.

[0123] For example, in response to a user operation on the biometric payment 228, the electronic device 200 may display the following Figure 2C The user interface 230 shown. The user interface 230 may include a biometric information function 231 and a new biometric information option 232. Among them:

[0124] The biometric information function 231 can be used to prompt the user of the function of biometric information. For example, the biometric information function 231 can include a prompt "biometric information is used to pay for paid resources". The biometric information function 231 can also include a switch 231A. The switch 231A can be used to turn on or off the function of using biometric information to pay for paid resources.

[0125] Not limited to the above setting options, the user interface 220 may also include more or fewer setting options.

[0126] like Figure 2C As shown, no biometric information for payment is currently stored in the speaker 100. When no biometric information for payment is stored in the speaker 100, the switch 231A is in a closed state by default.

[0127] The New Biometric option 232 may be used to begin establishing reference biometric information for payment.

[0128] In a possible implementation, the process of establishing reference biometric information for payment may include: authenticating the payment account, and entering reference voiceprint information and reference facial information after the authentication is passed.

[0129] In response to the user operation on the new biometric information option 232, the electronic device 200 may display the following Figure 2D The user interface 240 is shown. The user interface 240 may include a password input box 241 and a keyboard 243.

[0130] The password input box 241 can be used to input the payment password of the payment account associated with the account currently logged in to the speaker APP.

[0131] The keyboard 243 can be used to input a payment password in the password input box 241 .

[0132] When the password in the password input box 241 matches the payment password of the payment account stored in the electronic device 200, the identity authentication is successful. Figure 2E As shown, when the identity authentication is successful, the electronic device 200 can display a prompt box 242 on the user interface 240. The prompt box 242 can be used to prompt the user that the identity authentication is successful, and to perform operations to record voiceprint information and face information. The prompt box 242 may include a prompt "Authentication successful! Please continue to complete voice recording and face recording." The embodiment of the present application does not limit the specific content of the prompt in the prompt box 242.

[0133] That is, the user can instruct the speaker to create new biometric information through the speaker APP on the electronic device 200. After the user completes the identity authentication by entering the payment password of the payment account on the speaker APP, the reference voiceprint information and reference face information for payment can be further recorded on the speaker 100.

[0134] Above Figure 2A to Figure 2E The user interface shown may also include more or less content, which is not limited in the embodiments of the present application.

[0135] When the identity authentication of the payment account is completed, the electronic device 200 may send an instruction for obtaining reference voiceprint information and reference face information to the speaker 100. Further, the speaker 100 may prompt the user to enter the voiceprint information and face information.

[0136] Figure 2F to Figure 2I The schematic diagram exemplarily shows a scenario in which a first user enters reference voiceprint information and reference face information.

[0137] like Figure 2F As shown, the speaker 100 can prompt the first user to enter the voiceprint information by voice broadcast. For example, the speaker 100 can voice broadcast "Please follow me and say the following verification word: Xiaoyi Xiaoyi". After hearing the prompt of the speaker 100, the first user can say "Xiaoyi Xiaoyi". Furthermore, the speaker 100 can collect the voice input of the first user saying the above verification word, and extract the voiceprint information from the voice input.

[0138] When the voiceprint information of the first user is obtained, the speaker 100 can compare the voiceprint information with the reference voiceprint information stored in the security module. If the voiceprint information matches one of the reference voiceprint information stored, the speaker 100 can prompt the first user through voice broadcast that the voiceprint information has been entered.

[0139] If the voiceprint information does not match any of the stored reference voiceprint information, the speaker 100 may store the voiceprint information in the security module as the reference voiceprint information of the first user.

[0140] Furthermore, the speaker 100 can prompt the first user to enter facial information through voice broadcast.

[0141] Among them, since the speaker 100 is not equipped with a face collection device and cannot collect face images, the speaker 100 can search and call nearby available face collection devices to collect face images. Specifically, the speaker 100 can send a broadcast signal to inquire whether nearby devices are equipped with face collection devices. For example, the devices near the speaker 100 include electronic devices 200, face collection devices (such as TVs, mobile phones, tablets, laptops) 300, etc. When receiving the broadcast signal from the speaker 100, the device equipped with a face collection device near the speaker 100 can send a response message to the speaker 100. The response message may include the configuration of the face collection device (such as a 2D camera, a 3D camera, an infrared camera).

[0142] If multiple response messages are received from devices equipped with face collection devices, the speaker 100 can select one of the multiple devices to collect face images based on the order of the received response messages, the configuration of the face collection device, and other factors. For example, compared with 2D face authentication, 3D face authentication is more reliable. The speaker 100 can select a better configured device, such as a device equipped with a 3D camera.

[0143] The order in which the response messages are received can reflect the communication delay between the speaker 100 and the device equipped with a face collection device and the response speed of the device. The speaker 100 can select the device corresponding to the response message received first. The embodiment of the present application does not limit the manner in which the speaker 100 selects the device equipped with a face collection device nearby.

[0144] The device near the speaker 100 may be a device in the same local area network as the speaker, or may be a device within a preset distance range of the speaker 100. The preset distance may be determined by the maximum communication distance that can be achieved by the communication method of the speaker 100 sending the broadcast signal. The embodiment of the present application does not limit the electronic device near the speaker 100.

[0145] The speaker 100 may send the broadcast signal via a short-distance communication channel such as a near field communication channel, a Bluetooth communication channel, a WLAN direct communication channel, etc. The implementation method of the speaker 100 sending the broadcast signal may refer to the prior art, and the present application embodiment does not limit this.

[0146] Exemplarily, the speaker 100 can determine to call the face acquisition device 300, such as a television, to collect facial images according to the aforementioned method of searching and selecting a device with a face acquisition device nearby. The face acquisition device 300 is configured with a face acquisition device 301. In the subsequent embodiments of this application, the face acquisition device 300 is specifically described as a television. The face acquisition device 301 configured on the television can be a 3D camera. Not limited to televisions, the face acquisition device 300 can also be other electronic devices configured with a face acquisition state, such as mobile phones, tablets, laptops, etc.

[0147] The speaker 100 may send a command to the face acquisition device 300 to instruct the face acquisition device 301 to be turned on, such as a camera, to acquire a face image. Figure 2G As shown, the speaker 100 can make a voice announcement “Voiceprint entry is successful. Please complete face entry on Li Ming’s TV.” In the voice announcement, “Li Ming’s TV” can be the name of the face acquisition device 300 .

[0148] When receiving an instruction from the speaker 100 for instructing to turn on the face acquisition device 301 to collect a face image, the face acquisition device 300 can turn on the camera and send a message to the speaker 100 for indicating that the camera has been turned on.

[0149] like Figure 2H As shown, when receiving the above message indicating that the camera is turned on, the speaker 100 can voice broadcast "Please point your face to the camera of Li Ming's TV and blink." In this way, the speaker 100 can prompt the first user to record the face on the face acquisition device 300 (i.e., Li Ming's TV).

[0150] In a possible implementation, the face acquisition device 300 includes a display screen. When acquiring a face image, the face acquisition device 300 can also light up the display screen to display the image acquired by the camera and a text prompt "Focus on the camera, blink". In this way, the first user can adjust his position according to the image acquired by the camera displayed on the display screen so that his face is facing the camera.

[0151] In another possible implementation, the face acquisition device 300 does not include a display screen. Or the face acquisition device 300 includes a display screen, but the display screen is not lit when the face image is acquired. Then, during the process of the face acquisition device 300 acquiring the face image, the speaker 100 can prompt the first user to align his face with the face acquisition device by voice broadcast. For example, the voice broadcast is "The face is not aligned with the camera, please move a little to the left." In this way, when the face acquisition device 300 does not include a display screen or includes a display screen but the display screen is not lit, the first user can align his face with the face acquisition device according to the voice prompt of the speaker.

[0152] When the face image is obtained, the face acquisition device 300 can encrypt the face image and send it to the speaker 100. The face acquisition device 300 can encrypt the face image according to encryption methods such as symmetric encryption algorithm and asymmetric encryption algorithm. The specific implementation process of the above encryption can refer to the existing technology and will not be repeated here.

[0153] When receiving the encrypted face image, the speaker 100 can decrypt it according to the encryption method negotiated with the face acquisition device 300 to obtain the face image of the first user. Further, the speaker 100 can extract the face information from the face image and store it in the security module as reference face information. The speaker 100 can bind the reference voiceprint information and the reference face information.

[0154] That is to say, in the process of using biometric information for payment, the speaker 100 needs to verify whether the voiceprint information and face information match the reference voiceprint information and reference face information bound above. Only when the voiceprint information and face information are both verified, the speaker 100 can call the payment account associated with the account bound to the speaker 100 for payment. Compared with separate voiceprint verification and separate face authentication, the method of combining voiceprint verification and face authentication requires that the voiceprint information and face information entered by the user are both correct, which can improve the security of using biometric information for payment.

[0155] like Fig.2I As shown, when the reference voiceprint information and the reference face information are successfully bound, the speaker 100 can announce in voice “Face entry is successful, you have enabled biometric payment”. In this way, the speaker 100 can prompt the first user that the biometric payment has been successfully enabled. When the first user needs to pay for the paid audio resource later, he can complete the payment by entering his voiceprint information and face information.

[0156] In addition, the speaker 100 may also send a message to the electronic device 200 indicating that the biometric information is successfully established. Figure 2J As shown, when receiving the above message indicating that the biometric information is successfully established, the electronic device 200 can display a biometric information list 233 on the user interface 230. The biometric information list 233 can be used to display the reference biometric information established on the speaker 100.

[0157] For example, after the above Figure 2A to Figure 2IAccording to the operation shown, the biometric information of the first user has been established on the speaker 100. The biometric information list 233 of the electronic device 200 may include "User 1" 233A indicating the biometric information of the first user. In response to an operation acting on "User 1" 233A, such as a touch operation, the electronic device 200 may display options for changing the name of the biometric information and options for deleting the biometric information, etc. That is, the user can modify the name of the biometric information, such as changing the name "User 1" to "Li Ming". And, the user can delete the biometric information. In response to the user operation of deleting the biometric information, the electronic device 200 may send an instruction to the speaker 100 for deleting the biometric information named "User 1". In response to the above instruction, the speaker 100 may delete the biometric information named "User 1" (including reference voiceprint information and reference face information) stored in the security module.

[0158] The embodiment of the present application does not limit the specific content of the voice broadcast by the above-mentioned speaker 100.

[0159] The above-mentioned methods for extracting voiceprint information from speech input and extracting facial information from facial images can refer to the prior art, and the embodiments of the present application do not make specific limitations thereto.

[0160] The embodiment of the present application does not limit the above process of entering the user's voiceprint information. Among them, the speaker 100 can prompt the user to say a preset verification word multiple times. The above verification word is not limited to "Xiaoyi Xiaoyi", but can also be other words or sentences. Alternatively, the speaker 100 may not contain a preset verification word. That is, the user can say any word or sentence. The speaker 100 can extract the user's voiceprint information from the user's voice input.

[0161] In some embodiments, the speaker 100 is configured with a face collection device, such as a camera. When the above-mentioned voiceprint information is successfully recorded and the face information is further recorded, the speaker 100 can turn on its own face collection device to collect the face image and extract the face information in the face image. Exemplarily, the speaker 100 is configured with a camera. When recording the face information, the speaker 100 can voice broadcast "Please align your face with my camera and blink". In addition, the speaker 100 can also prompt the user to adjust the position by voice broadcast, so that the user's face is aligned with the camera of the speaker 100. For example, when the user's face is biased to the right of the camera's image collection area, the speaker 100 can voice broadcast "The face is not aligned with the camera, please move a little to the left".

[0162] When the speaker 100 acquires a face image through its own camera, the speaker 100 can extract face information from the face image and store the face information in the security module as reference face information. In this way, the speaker equipped with a voice acquisition device and a face acquisition device can independently complete the entry of reference biometric information, and use the reference biometric information to verify the user's identity during the payment of paid resources.

[0163] The embodiment of the present application does not limit the storage space of the reference biometric information. In addition to the security module of the speaker 100, the reference biometric information can also be stored in a cloud storage space. The cloud storage space can be a storage space accessible to the speaker 100 and the face acquisition device 300.

[0164] In some embodiments, when the speaker 100 obtains the voiceprint information of the first user, the speaker 100 can store the voiceprint information in the above-mentioned cloud storage space. Further, the speaker 100 can find and call the face acquisition device 300 to collect the face image. Among them, the face acquisition device 300 can perform feature extraction on the collected face image of the first user to obtain the face information of the first user.

[0165] Further, the face acquisition device 300 can store the face information of the first user in a cloud storage space. In the cloud storage space, the voiceprint information and face information of the first user can be associated. Alternatively, the face acquisition device 300 can send the collected face image of the first user to the speaker 100. The speaker 100 can perform feature extraction on the face image to obtain the face information of the first user. Further, the speaker 100 can associate the face information of the first user with the voiceprint information of the first user and store them in the cloud storage space.

[0166] Optionally, the above-mentioned reference biometric information includes reference voiceprint information and reference facial information. One of the reference voiceprint information and the reference facial information is stored in the above-mentioned cloud storage space. Exemplarily, the speaker 100 can collect the voice input of the first user and obtain the voiceprint information. The speaker 100 can store the voiceprint information in a local security module. Further, the speaker 100 can call a nearby facial acquisition device to collect a facial image and obtain facial information. Among them, the facial acquisition device can extract facial information from the facial image and store the facial information in the cloud storage space.

[0167] Depend on Figure 2A to Figure 2IAs can be seen from the scenario diagram shown, a speaker that is not equipped with a face collection device can record face information by calling a nearby face collection device. In this way, the speaker can record the user's reference voiceprint information and reference face information, and verify the user's identity during the payment process of paid resources, thereby simplifying the user's payment operation when listening to paid resources on the speaker.

[0168] In combination with the above scenario diagram, a method for a speaker to record reference voiceprint information and reference face information provided by an embodiment of the present application is introduced below.

[0169] Figure 3 The flowchart of the method for recording reference voiceprint information and reference face information by a speaker is shown as an example. The method may include steps S101 to S109. Among them:

[0170] S101 . The electronic device 200 receives a user operation for opening a speaker APP, pairing the electronic device 200 with the speaker 100 , and establishing a binding relationship between a first account of the speaker APP and the speaker 100 .

[0171] In some embodiments, the speaker 100 and the electronic device 200 may be paired when they are connected for the first time. Exemplarily, the Bluetooth of both the electronic device 200 and the speaker 100 are turned on. The electronic device 200 turns on the speaker APP and scans the nearby Bluetooth after receiving a user operation for searching for nearby devices. When the Bluetooth of the speaker 100 is scanned, the electronic device 200 can be paired and connected with the speaker 100. The embodiment of the present application does not limit the way in which the electronic device 200 and the speaker 100 are paired.

[0172] When the electronic device 200 is successfully paired with the speaker 100, the electronic device 200 can configure a network for the speaker 100. The speaker 100 can access the network. The manner in which the electronic device 200 configures a network for the speaker 100 can refer to the aforementioned embodiment and will not be described in detail here.

[0173] Further, in response to the user operation of logging in the first account on the speaker APP, the electronic device 200 can log in the first account on the speaker APP. The first account can be the aforementioned Figure 2A The account name shown is "Li Ming". The user interface of the first account logged in on the speaker APP can refer to the above Figure 2A User interface 210 is shown.

[0174] When a first account is logged in to the speaker APP, the speaker 100 can establish a binding relationship with the first account. The implementation process of establishing a binding relationship between the speaker 100 and the first account logged in to the speaker APP can refer to the existing technology, and the embodiment of the present application is not limited to this.

[0175] S102 , the electronic device 200 is successfully paired with the speaker 100 , and a binding relationship is established between the first account of the speaker APP and the speaker 100 .

[0176] S103 : The electronic device 200 receives a user operation for requesting to establish first biometric information for payment, and sends a request for establishing first biometric information for payment to the speaker 100 .

[0177] The user operation for requesting to establish the first biometric information for payment may be performed on the aforementioned Figure 2C The electronic device 200 can send a request to the speaker 100 to establish the first biometric information for payment after authenticating the payment account associated with the first account. The implementation method of the above-mentioned authentication of the payment account can refer to the above-mentioned Figure 2C to Figure 2E The embodiments shown will not be described in detail here.

[0178] S104. The speaker 100 records the first voiceprint information of the first user.

[0179] The first voiceprint information may be the reference voiceprint information. When the first voiceprint information is obtained, the speaker 100 may store the first voiceprint information in the security module. The first voiceprint information may be used for voiceprint verification in subsequent payment processes.

[0180] S105. The speaker 100 finds and calls an available face acquisition device 300.

[0181] S106. The face collection device 300 records the first face information of the first user.

[0182] The first face information may be the aforementioned reference face information. The face acquisition device 300 may use a face acquisition device to acquire a face image, and extract the first face information from the face image.

[0183] Alternatively, the face acquisition device 300 may send the acquired face image to the speaker 100, and the speaker 100 may extract the face image to obtain the first face information.

[0184] S107. The face collection device 300 sends the encrypted first face information to the speaker.

[0185] S108. The speaker 100 decrypts the encrypted first facial information, binds and stores the first voiceprint information and the first facial information.

[0186] When the first face information is obtained, the speaker 100 can store the first face information in the security module. The first face information can be used for face verification in the subsequent payment process.

[0187] In addition, the speaker 100 can bind the first voiceprint information and the first face information as the first biometric information of the first user.

[0188] S109 , the speaker 100 sends a message to the electronic device 200 indicating that the first biometric information is successfully established.

[0189] The specific implementation process of the above steps S104 to S109 can refer to the above Figure 2F to Figure 2J The description of the illustrated embodiment will not be repeated here.

[0190] The following is a schematic diagram of a scenario in which the speaker 100 provided in an embodiment of the present application pays for paid audio resources by combining voiceprint verification and face verification.

[0191] When the reference voiceprint information and reference face information of the first user are stored in the security module of the speaker 100, the speaker 100 can verify the identity of the user requesting payment by combining voiceprint verification and face verification.

[0192] like Figure 4A As shown, the first user wakes up the speaker 100 through a preset wake-up word and issues a voice command to play music A. Specifically, the first user says "Xiaoyi Xiaoyi, I want to listen to music A" near the speaker 100.

[0193] When the preset wake-up word is detected, the speaker 100 can wake up the application processor, identify and execute the received voice command. In response to the first user's voice command of "listen to music A", the speaker 100 can obtain the resources of music A from the music server and play it. In addition, the speaker 100 can also voice broadcast "OK, play music A for you" to respond to the first user's voice command.

[0194] If music A is a paid audio resource, and the first account (the account bound to the speaker 100) has not purchased music A, the music server may send a message to the speaker 100 indicating that music A is paid music. Figure 4B As shown, when receiving a message indicating that music A is paid music, the speaker 100 can voice broadcast "You can listen to the complete music A after payment, do you want to pay?" to prompt the first user that he needs to pay to listen to music A.

[0195] When hearing the prompt from the speaker 100, the first user can say "Please help me complete the payment" near the speaker 100. When the voice command "Please help me complete the payment" is detected, the speaker 100 can start voiceprint verification and face verification. Figure 4CAs shown, the speaker 100 can voice broadcast "OK, start voiceprint verification. Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the first user to enter the voiceprint information. The first user can say the verification word "Xiaoyi Xiaoyi" according to the prompt of the speaker 100.

[0196] The speaker 100 may receive the voice input of the first user saying the verification word, and extract the voiceprint information from the voice input. Further, the speaker 100 may compare the voiceprint information with the stored reference voiceprint information of the first user. If the voiceprint information matches the reference voiceprint information of the first user, the voiceprint verification is successful.

[0197] like Figure 4D As shown, the speaker 100 can voice broadcast "Voiceprint verification is successful, start face verification. Please face your face to the camera of Li Ming's TV and blink." Among them, since the speaker 100 is not equipped with a face collection device and cannot collect face images, the speaker 100 can search and call nearby available face collection devices to collect face images after the voiceprint verification is successful.

[0198] The way in which the speaker 100 finds and calls the face acquisition device 300 (i.e., Li Ming's TV) to obtain face information can refer to the aforementioned embodiment, which will not be repeated here. When the above face information is obtained, the speaker 100 can compare the above face information with the stored reference face information of the first user. If the above face information matches the reference face information of the first user, the face verification is successful.

[0199] When both the voiceprint verification and the face verification are successful, the speaker 100 can complete the local verification of the payment account. Further, the speaker 100 can verify with the payment server of the payment account. When the verification between the speaker 100 and the above payment server is successful, the payment server can deduct the payment account associated with the account bound to the speaker 100.

[0200] The verification between the speaker 100 and the payment server can be used by the payment server to confirm that the speaker 100 is a trusted device. In this way, the payment server can complete the deduction of the payment account according to the instruction of the speaker 100 to deduct the payment account. Specifically, the speaker 100 and the payment server can perform verification based on fast identity online alliance (FIDO).

[0201] Among them, the security module of the speaker 100 can generate an asymmetric key pair. The asymmetric key pair may include a private key and a public key. The security module of the speaker 100 can store the private key and send the public key to the payment server. The payment server can associate the payment account associated with the account bound to the speaker 100 with the public key.

[0202] When the above voiceprint verification and face verification are both successful, the speaker 100 can use the above private key to sign the verification request sent by the payment server, and send the verification request containing the above signature to the payment server. Then the payment server can use the above public key to verify the verification request containing the above signature. If the verification is successful, the payment server can confirm that the speaker 100 is a trusted device.

[0203] In addition to the above-mentioned FIDO verification, other trusted identity authentication methods can also be used for verification between the speaker 100 and the payment server. For example, trusted identity authentication methods such as Internet Finance Authentication Alliance (IFAA) and Tencent User Identity Authentication (TUSI). The verification between the above-mentioned speaker 100 and the payment server can refer to the implementation method of the prior art, which will not be repeated here.

[0204] In a possible implementation, when the payment server completes the deduction of the payment account, the payment server may send a message indicating successful payment to the speaker 100. The speaker 100 may send the message indicating successful payment to the music server. Upon receiving the message indicating successful payment, the music server may send the resource of music A to the speaker 100. In this way, the speaker 100 can play the complete music A.

[0205] In another possible implementation, a communication connection is established between the payment server and the music server. When the payment server completes the deduction of the payment account, the payment server may send a message to the music server indicating that the deduction of the payment account associated with the account bound to the speaker 100 is successful. Further, the music server may send the resource of the music A to the speaker 100. In this way, the speaker 100 can play the complete music A. The embodiment of the present application does not limit the method by which the music server determines that the account bound to the speaker 100 has purchased the above-mentioned music A.

[0206] like Figure 4E As shown, when receiving a message sent by the payment server indicating successful payment, the speaker 100 can voice broadcast "Face verification is successful, you have successfully purchased music A, and play the complete music A for you" to prompt the first user that the payment has been successful. Then, the speaker 100 can play the resource of music A received from the music server.

[0207] It should be noted that the music server may store music resources and account related information (such as paid audio resources purchased by the account, etc.). It is not limited to a music server that provides music, but may also be a server that provides other types of audio resources for the speaker 100. The embodiment of the present application does not limit the type of the music server and the content contained therein.

[0208] In the present application, the implementation process of the speaker 100 waking up the application processor after detecting the wake-up word, recognizing and executing the voice command can refer to the existing technology and will not be repeated here.

[0209] Depend on Figure 4A to Figure 4E As can be seen from the scenario shown, when the first user enters the reference voiceprint information and reference face information in the speaker, the first user can complete the payment by voiceprint verification and face verification when paying for paid audio resources. In this way, the first user does not need to open the electronic device with the speaker APP installed, and perform a series of cumbersome operations such as selecting paid audio resources and entering the payment password. The above-mentioned payment method of voiceprint verification combined with face verification can simplify user operations and make the payment process more convenient.

[0210] In some embodiments, the speaker 100 may receive an instruction for instructing the speaker 100 to perform user identity authentication. The instruction may be, for example, Figure 4B The first user's voice input is shown as "Please help me complete the payment". Further, the speaker 100 can first record the user's voiceprint information, and call the face acquisition device 300 to obtain the user's face information. Then, the speaker 100 can compare the above voiceprint information and the above face information with the reference voiceprint information and reference face information of the first user respectively. If both match, the voiceprint verification and face verification are successful.

[0211] In this method, the speaker 100 can first obtain the voiceprint information and face information that need to be verified. Then, the speaker 100 can verify the voiceprint information and face information that need to be verified. By performing voiceprint verification and face verification at the same time, the speaker 100 can improve the efficiency of verifying the user's identity, thereby improving the efficiency of the above-mentioned combination of voiceprint verification and face verification for payment.

[0212] In some embodiments, the reference biometric information of the first user is stored in a cloud storage space. The speaker 100 can obtain the reference biometric information of the first user from the cloud storage space to verify the user's identity during the process of paying for paid resources.

[0213] Specifically, Figure 4BAs shown, the first user requests the speaker 100 to pay for the paid resources. The speaker 100 can obtain the reference biometric information of the first user from the above-mentioned cloud storage space. The speaker 100 can collect the voice input of the first user and obtain the voiceprint information, and compare the voiceprint information with the reference voiceprint information in the reference biometric information of the above-mentioned first user. If the voiceprint information matches the reference voiceprint information, the speaker 100 can call the face acquisition device 300 to collect the face image of the first user and obtain the face information. The speaker 100 can compare the face information with the reference face information in the reference biometric information of the above-mentioned first user. When the speaker 100 completes the above-mentioned voiceprint verification and face verification, the speaker 100 can perform a trusted identity authentication with the payment server. The implementation method of the above-mentioned trusted identity authentication can refer to the above-mentioned embodiment, which will not be repeated here. If the above-mentioned trusted identity authentication is successful, the payment server can deduct the payment account of the first user. The payment account of the above-mentioned first user can be a payment account associated with the account bound to the speaker 100.

[0214] In other embodiments, the reference biometric information of the first user is stored in a cloud storage space. The cloud storage space can be accessed by the payment server. In the process of paying for the paid resources, the speaker 100 can send the obtained voiceprint information and face information of the first user to the payment server. The payment server can obtain the reference biometric information of the first user from the cloud storage space, and use the biometric information to verify the voiceprint information and face information from the speaker 100.

[0215] Specifically, the speaker 100 may obtain the voiceprint information of the first user according to the method in the aforementioned embodiment. The speaker 100 may send the voiceprint information to the payment server and request the payment server to perform voiceprint verification. The payment server may obtain the reference biometric information of the first user from the cloud storage space and compare the reference voiceprint information of the first user with the voiceprint information from the speaker 100. If they match, the payment server may send a message to the speaker 100 indicating that the voiceprint information verification is successful.

[0216] Further, the speaker 100 may obtain the facial information of the first user according to the method in the aforementioned embodiment. The speaker 100 may send the facial information to the payment server and request the payment server to perform facial verification. The payment server may compare the reference facial information of the first user with the facial information from the speaker 100. If they match, the payment server may deduct money from the payment account of the first user.

[0217] The embodiment of the present application does not limit the manner in which the speaker 100 calls a payment account for payment. For details, reference may be made to the implementation manner in which an electronic device calls a payment account for payment in the prior art.

[0218] In the scenario where the first user has multiple speakers, the reference biometric information stored in the cloud storage space can be shared by the multiple speakers. Exemplarily, the first user has multiple speakers. The multiple speakers may include the speaker 100. The multiple speakers may be located in a bedroom, a living room, etc. The multiple speakers may be bound to an account logged in by the speaker APP in the electronic device 200.

[0219] When one of the multiple speakers records the reference biometric information of the first user, the speaker can store the reference biometric information of the first user in the cloud storage space. All of the multiple speakers can access the above cloud storage space. In this way, the user only needs to enter the reference biometric information once, and can pay for the paid resources played by the speaker through voiceprint verification and face verification on multiple speakers. The above implementation method can simplify the operation of users entering reference biometric information for payment and improve the user experience.

[0220] It is not limited to the scenario of multiple speakers mentioned above, but can also be a distributed scenario including speakers, TVs, mobile phones, story machines and other electronic devices. In the above distributed scenario, multiple electronic devices bound to the same account can share the reference biometric information stored in the cloud storage space associated with the account. That is, the user only needs to enter the reference biometric information once, and these multiple electronic devices can obtain the reference biometric information from the cloud storage space for verification, so as to call the payment account to pay for paid resources, such as paid audio resources, paid video resources, etc.

[0221] In some embodiments, the accounts bound to the multiple electronic devices in the above distributed scenario may be different accounts. These different accounts may be accounts belonging to the same account group. Among them, multiple accounts belonging to the same account group can share the content purchased by each account in the content server. The above content server may be, for example, a music server, a video server, etc.

[0222] Exemplarily, a distributed scenario of a family includes speaker A, speaker B and speaker C. The accounts bound to speaker A, speaker B and speaker C are account A, account B and account C respectively. Account A, account B and account C can be respectively associated with reference biometric information of different family members in this family. Among them, account A, account B and account C are accounts belonging to a family group. When speaker A is bound to account A, music A can be purchased according to the payment method in the aforementioned embodiment. Since speaker B is bound to account B and speaker B is bound to account C, speaker B and speaker C can play music A without purchasing it again.

[0223] Among them, the above-mentioned accounts belonging to the same family group can be associated with the same payment account. That is to say, when purchasing paid resources using the above-mentioned account A, account B and account C as identity identifiers, speakers A, speaker B and speaker C can all instruct the payment server to deduct money from the same payment account associated with account A, account B and account B. When purchasing paid resources using the above-mentioned account A as an identity identifier, speaker A can perform user identity verification based on the reference biometric information associated with account A. When purchasing paid resources using account B as an identity identifier, speaker B can perform user identity verification based on the reference biometric information associated with account B. When purchasing paid resources using account C as an identity identifier, speaker C can perform user identity verification based on the reference biometric information associated with account C.

[0224] The embodiment of the present application does not limit the method of setting different accounts to belong to the same family group. For details, please refer to the implementation method in the prior art.

[0225] The following is combined with the above Figure 4A to Figure 4E The scenario shown introduces a payment method provided by an embodiment of the present application for verifying user identity through biometric information verification.

[0226] Figure 5 The flowchart of a payment method is shown as an example. The method may include steps S201 to S209. Among them:

[0227] The security module of the speaker 100 may store the first biometric information of the first user. The first biometric information may include the first voiceprint information and the first face information. The first voiceprint information may be the reference voiceprint information in the aforementioned embodiment. The first face information may be the reference face information in the aforementioned embodiment.

[0228] S201. The speaker 100 establishes a binding relationship with a first account.

[0229] The first account may be an account logged in by the first user on the speaker APP of the electronic device 200, such as an account named "Li Ming". The first account may be associated with a payment account. When the account logged in to the speaker APP is the first account, the payment server may deduct money from the payment account associated with the first account when paying for paid audio resources.

[0230] The method for establishing a binding relationship between the speaker 100 and the first account can refer to the aforementioned Figure 3 Step S101 and step S102 in the method shown are not described in detail here.

[0231] S202 , the speaker 100 receives a user operation to wake up the speaker 100 , play a paid resource, and request payment.

[0232] The above-mentioned user operation of receiving the wake-up speaker 100 and playing the paid resource can be, for example, Figure 4A The first user shown speaks a voice command "Xiaoyi Xiaoyi, I want to listen to music A" near the speaker 100. Music A is a paid resource.

[0233] The user operation for requesting payment may be, for example, Figure 4B The first user shown speaks a voice command "Please help me complete the payment" near the speaker 100.

[0234] S203: The speaker 100 records the voiceprint information of the first user.

[0235] S204. The speaker 100 verifies the voiceprint information recorded in step S203, and determines that the voiceprint information matches the first voiceprint information.

[0236] S205. The speaker 100 finds and calls an available face acquisition device 300.

[0237] S206. The face collection device 300 records the face information of the first user.

[0238] S207 , the face collection device 300 encrypts the face information entered in step S206 and sends it to the speaker 100 .

[0239] S208. The speaker 100 decrypts the encrypted facial information and performs verification to confirm that the facial information matches the first facial information.

[0240] S209: The speaker 100 calls the payment account associated with the first account to pay the payment resource.

[0241] The specific process of the speaker 100 verifying the voiceprint information and the face information, and calling the payment account associated with the first account to make payment can refer to the introduction of the aforementioned embodiment, which will not be repeated here.

[0242] Depend on Figure 5 It can be seen from the method shown that when paying for paid audio resources, the speaker can collect the user's voice input and extract voiceprint information from it for verification. In addition, a speaker that is not equipped with a face collection device can call a nearby device that is equipped with a face collection device to collect a face image. Furthermore, the speaker can verify the face information in the above face image. After the above-mentioned verification of the voiceprint information and the face information are passed, the speaker can call the payment account to pay for the paid audio resources. In this way, a speaker that is not equipped with a face collection device can also combine the voiceprint verification and face verification methods to complete the payment, which simplifies the user's payment operation when using the speaker to listen to paid audio resources.

[0243] In some embodiments, the speaker 100 and the first user's account, such as the first account ( Figure 2A In the case of binding with the first user's ID card (the account name is "Li Ming" shown in the figure), the first user can authorize his family or friends to complete the payment by combining voiceprint verification and face verification when listening to paid audio resources using the speaker 100. Among them, the account for payment can be the payment account associated with the first account. The payment account associated with the first account can be the payment account of the first user. That is, after the user authorized by the first user successfully verifies using biometric information, the first user's payment account can pay for the paid audio resources.

[0244] The security module of the speaker 100 stores the reference biometric information of the main user. The speaker 100 can also establish the reference biometric information of the authorized user after the main user authorizes. The reference biometric information can include reference voiceprint information and reference face information. The reference biometric information can be used to compare whether the biometric information obtained by the speaker 100 matches the reference biometric information during the process of the speaker 100 paying for paid resources, so as to verify the identity of the user who instructs the speaker 100 to pay.

[0245] The speaker 100 may establish reference biometric information of the primary user and reference biometric information of the authorized user for the first account. The reference biometric information of the primary user associated with the first account may be: the reference biometric information established by the speaker 100 when the speaker 100 does not store the reference biometric information associated with the first account. In other words, the primary user associated with the first account may be: the user corresponding to the reference biometric information established by the speaker 100 when the speaker 100 does not store the reference biometric information associated with the first account.

[0246] The authorized user associated with the first account may be: when the speaker 100 stores reference biometric information associated with the first account, the user corresponding to the reference biometric information established by the speaker 100. An account bound to the speaker 100 may have a main user and one or more authorized users. Exemplarily, the speaker 100 is bound to the first account. When the speaker 100 does not store reference biometric information associated with the first account, the speaker 100 enters the biometric information of the first user as the reference biometric information. Then the first user is the main user associated with the first account. When the speaker 100 stores the reference biometric information of the first user, the speaker 100 enters the biometric information of the second user as the reference biometric information. Then the second user is the authorized user associated with the first account. Among them, in the process of paying for paid resources, if the above-mentioned first user or second user is verified by the speaker 100 using the reference biometric information, the speaker 100 can call the payment account associated with the first account to pay for the paid resources. That is, the speaker 100 can call the payment account of the main user (i.e., the first user) to pay.

[0247] In a possible implementation, the reference biometric information of the primary user may include reference voiceprint information and reference face information. The reference biometric information of the authorized user may only include reference voiceprint information. The speaker 100 may bind the reference voiceprint information of the authorized user with the reference face information of the primary user.

[0248] Among them, when the main user uses biometric information for verification, the verification can be completed through the main user's voiceprint information and facial information. When the authorized user uses biometric information for verification, in addition to verification through the authorized user's voiceprint information, it is also necessary to complete the verification through the main user's facial information. That is to say, if the speaker 100 detects that the recorded voiceprint information matches the reference voiceprint information of the authorized user, the speaker 100 can further prompt that the main user's facial information needs to be entered to complete the verification. In this way, with the authorization of the main user, the authorized user can use his or her own voiceprint information for verification, so as to listen to paid audio resources on the speaker 100. In addition, the main user authorization can also effectively prevent the authorized user from abusing authority and over-consuming the payment account associated with the account bound to the speaker 100.

[0249] The following is a schematic diagram of a scenario in which a first user authorizes a second user to establish reference biometric information on the speaker 100, provided in an embodiment of the present application.

[0250] Figure 6A to Figure 6F The schematic diagram exemplarily shows a scenario in which a first user authorizes a second user to establish reference biometric information on the speaker 100 .

[0251] Among them, a first account, namely an account named "Li Ming", is logged in to the speaker APP of the electronic device 200. The speaker 100 is bound to the first account. The first user may be the main user associated with the first account in the aforementioned embodiment. The second user may be the authorized user associated with the first account in the aforementioned embodiment.

[0252] The implementation process of the first user entering reference biometric information can refer to the aforementioned Figure 2A to Figure 2I When the speaker 100 successfully records the reference biometric information of the first user, the electronic device 200 may display the following Fig. 6AThe user interface 230 shown. The user interface 230 may include a biometric information list 233. The biometric information list 233 may be used to indicate the reference biometric information entered into the speaker 100. For example, the reference biometric information 233A named "User 1". Among them, the reference biometric information named "User 1" is the reference biometric information of the first user. Since this reference biometric information is the reference biometric information associated with the first user established by the speaker 100 when the speaker 100 does not store the reference biometric information associated with the first user, the speaker 100 and the electronic device 200 may default this reference biometric information to the reference biometric information of the main user associated with the first account. The electronic device 200 may use the prompt text "Main User" in the biometric information list 233 to indicate which reference biometric information in the user list is the reference biometric information of the main user.

[0253] like Figure 6B As shown, in response to the action on Fig. 6A For the user operation of the new biometric information option 232 shown, the electronic device 200 can display a prompt box 234 on the user interface 230. The prompt box 234 can be used to prompt the user that when the reference biometric information of the main user already exists, the newly created biometric information needs the authorization of the main user when used for payment. Exemplarily, the prompt box 234 may include a prompt "The newly created biometric information will be bound to the biometric information of the main user. When using the newly created biometric information for payment, the main user's facial information needs to be verified. Do you want to create it?". The prompt box 234 may also include a confirmation button 234A and a cancel button 234B. The cancel button 234B can be used to cancel the establishment of the reference biometric information of the authorized user. The confirmation button 234A can be used to confirm the establishment of the reference biometric information of the authorized user.

[0254] In response to the user operation on the determined button 234A, the electronic device 200 can perform identity authentication of the payment account. The implementation process of identity authentication of the payment account can refer to the aforementioned Figure 2D and Figure 2E When the identity authentication of the payment account is completed, the electronic device 200 may display the following Figure 6C The user interface 240 shown. The user interface 240 may include a prompt box 244. The prompt box 244 may be used to prompt the user that the identity authentication is successful and further operations need to be performed. For example, the prompt box 244 may include a prompt "Authentication successful! Please continue to complete the voice recording."

[0255] Furthermore, when the identity authentication of the above-mentioned payment account is successful, the electronic device 200 may send an instruction to the speaker 100 for instructing the recording of the reference voiceprint information of the authorized user.

[0256] like Fig.6DAs shown, when receiving an instruction from the electronic device 200 for instructing to record the reference voiceprint information of the authorized user, the speaker 100 can prompt the user to record the voiceprint information by voice broadcast. For example, the speaker 100 can voice broadcast "Please follow me and say the following verification word: Xiaoyi Xiaoyi". The second user (ie, the authorized user) can say the verification word "Xiaoyi Xiaoyi" according to the voice prompt of the speaker 100.

[0257] The speaker 100 can receive the voice input when the second user says the verification word, and extract the voiceprint information from the voice input. When the voiceprint information is obtained, the speaker 100 can compare the voiceprint information with the reference voiceprint information stored in the security module. If the voiceprint information matches one of the reference voiceprint information stored, the speaker 100 can prompt the user that the voiceprint information already exists by voice broadcast.

[0258] If the voiceprint information does not match the stored reference voiceprint information, the speaker 100 can store the voiceprint information in the security module as the reference voiceprint information of an authorized user. In addition, the speaker 100 can bind the voiceprint information with the reference face information of the main user. Furthermore, the speaker 100 can use voice to prompt the main user and the authorized user that the reference biometric information has been successfully created. For example, the speaker 100 can voice broadcast "Voiceprint entry is successful, you have created new biometric information."

[0259] like Fig. 6F As shown, when the reference biometric information is successfully created, the speaker 100 can send a message indicating that the reference biometric information is successfully created to the electronic device 200. Further, the electronic device 200 can display the name of the newly created reference biometric information in the biometric information list of the user interface 230, such as the reference biometric information 233B named "User 2".

[0260] That is to say, the first user can perform the following operations on the electronic device 200: Figure 6A to Figure 6C The operation shown. Furthermore, the speaker 100 can prompt to start recording the voiceprint information of the authorized user by voice broadcast. When it is determined that the recorded voiceprint information does not exist in the stored reference voiceprint information, the speaker 100 can use the voiceprint information as the reference voiceprint information of the new authorized user. In this way, in the process of using the speaker 100 to listen to paid resources and make payments, the second user can use his own voiceprint information for verification, and complete the payment on the speaker 100 if the main user's face information is authorized.

[0261] In a possible implementation, when creating the reference voiceprint information of the authorized user, the speaker 100 needs to verify the face information of the main user. When the face information of the main user is verified, the speaker 100 can determine that the recorded voiceprint information is authorized by the main user. In this way, the reliability of local verification during the payment process can be improved.

[0262] Specifically, the speaker 100 may first prompt the authorized user to enter the voiceprint information through voice broadcast. Fig.6D As shown. When the voiceprint information is successfully entered, the speaker 100 can prompt the main user to enter the facial information through voice broadcast. For example, the speaker 100 that is not equipped with a facial acquisition device can find and call the facial acquisition device 300 (i.e., Li Ming's TV) to collect facial images. Then, the speaker 100 can voice broadcast "The creation of new biometric information requires the consent of the main user. Please point your face at the camera of Li Ming's TV and blink." The facial acquisition device 300 can encrypt the collected facial image and send it to the speaker 100.

[0263] Furthermore, the speaker 100 can extract the face information from the obtained face image and compare it with the stored reference face information of the main user. If they match, the face information of the main user is verified. The speaker 100 can bind the reference voiceprint information of the authorized user with the reference face information of the main user.

[0264] The embodiment of the present application does not limit the storage location of the reference biometric information of the main user and the reference biometric information of the authorized user. In addition to the security module in the speaker 100, it can also be a cloud storage space accessible by the speaker 100.

[0265] Figure 7 The flowchart of a method provided by an embodiment of the present application for a first user to authorize a second user to establish reference biometric information on the speaker 100 is exemplarily shown.

[0266] The method may include steps S301 to S306. Among them:

[0267] The speaker 100 is bound to the first account. The first user may be the main user associated with the first account in the aforementioned embodiment. The second user may be the authorized user associated with the first account in the aforementioned embodiment. The security module of the speaker 100 stores the first biometric information (including the first voiceprint information and the first face information) of the first user. The first biometric information is the reference biometric information in the aforementioned embodiment.

[0268] S301. The electronic device 200 receives a user operation for opening a speaker APP, pairing the electronic device 200 with the speaker 100, and establishing a binding relationship between a first account of the speaker APP and the speaker.

[0269] S302 , the electronic device 200 is successfully paired with the speaker 100 , and a binding relationship is established between the first account of the speaker APP and the speaker 100 .

[0270] The above-mentioned step S301 and step S302 may refer to step S101 and step S102 in the aforementioned embodiment respectively, and will not be described in detail here.

[0271] S303 : The electronic device 200 receives a user operation for requesting to establish second biometric information for payment, and sends a request for establishing second biometric information for payment to the speaker 100 .

[0272] The user operation for requesting to establish the second biometric information for payment may be the aforementioned Fig. 6A The second biometric information may be the reference biometric information of the authorized user. The second biometric information may only include the reference voiceprint information.

[0273] S304: The speaker 100 records the second voiceprint information of the second user.

[0274] S305. The speaker 100 stores the second voiceprint information, and binds the second voiceprint information with the first face information.

[0275] S306: The speaker 100 sends a message to the electronic device 200 indicating that the second biometric information is successfully established.

[0276] The specific process of the above steps S304 to S306 can refer to the above Fig.6D and Fig. 6E The embodiments of the present invention will not be described in detail.

[0277] The following is a schematic diagram of a scenario in which another speaker 100 provided in an embodiment of the present application pays for paid audio resources by combining voiceprint verification and face verification.

[0278] The security module of the speaker 100 stores the reference voiceprint information and reference face information of the first user, and the reference voiceprint information of the second user, wherein the reference face information of the first user is bound to both the reference voiceprint information of the first user and the reference voiceprint information of the second user.

[0279] The first user can pay for the paid audio resource on the speaker 100 by voiceprint verification and face verification. The second user can pay for the paid audio resource on the speaker 100 by voiceprint verification and authorization by the first user using face verification.

[0280] Specifically, Fig. 8AAs shown, the second user wakes up the speaker 100 through a preset wake-up word and issues a voice command to play music B. Among them, the second user can say "Xiaoyi Xiaoyi, I want to listen to music B" near the speaker 100.

[0281] When the preset wake-up word is detected, the speaker 100 can wake up the application processor, identify and execute the received voice command. In response to the second user's voice command of "listen to music B", the speaker 100 can obtain the resources of music B from the music server and play it. In addition, the speaker 100 can also voice broadcast "OK, play music B for you" to respond to the second user's voice command.

[0282] If music B is a paid resource and the first account (the account bound to the speaker 100) has not purchased music B, the music server may send a message to the speaker 100 indicating that music B is paid music. Figure 8B As shown, when receiving a message indicating that music B is paid music, the speaker 100 can voice broadcast "You can listen to the complete music B after payment, do you want to pay?" to prompt the first user that he needs to pay to listen to music B.

[0283] Furthermore, the second user can say "Please help me complete the payment" near the speaker 100. When the voice command "Please help me complete the payment" is detected, the speaker 100 can start voiceprint verification and face verification. Figure 8C As shown, the speaker 100 can voice broadcast "OK, start voiceprint verification. Please follow me and say the following verification word: Xiaoyi Xiaoyi" to prompt the second user to enter the voiceprint information. The second user can say the verification word "Xiaoyi Xiaoyi" according to the prompt of the speaker 100.

[0284] The speaker 100 can receive the voice input of the second user saying the verification word, and extract the voiceprint information from the voice input. Further, the speaker 100 can compare the voiceprint information with the stored reference voiceprint information, and determine that the voiceprint information matches the reference voiceprint information of the second user. That is, the speaker 100 can determine that the voiceprint verification is successful and start the face verification.

[0285] like Fig.8D As shown, since the reference voiceprint information of the second user is bound to the reference face information of the main user, the speaker 100 can announce by voice “The voiceprint verification is successful. The main user’s face needs to be further verified. Please aim your face at the camera of Li Ming’s TV and blink”. Since the speaker 100 is not equipped with a face collection device, the speaker 100 can search and call nearby available face collection devices to collect face images after the voiceprint verification is successful.

[0286] The way in which the speaker 100 finds and calls the face acquisition device 300 (i.e., Li Ming's TV) to obtain face information can refer to the aforementioned embodiment, which will not be repeated here. When the above face information is obtained, the speaker 100 can compare the above face information with the reference face information of the main user (i.e., the first user). If the above face information matches the reference face information of the main user, the face verification is successful.

[0287] When both the voiceprint verification and the face verification are successful, the speaker 100 can complete the local verification of the payment account. Further, the speaker 100 can verify with the payment server of the payment account.

[0288] The verification process between the speaker 100 and the payment server, the payment server deducting money from the payment account, and the implementation process of the music server sending the resources of music B to the speaker 100 after the payment is successful can all be referred to the aforementioned embodiments and will not be repeated here.

[0289] like Fig. 8E As shown, the speaker 100 can receive a message sent by the payment server indicating that the payment is successful. Then, the speaker 100 can announce in voice that "face verification is successful, you have successfully purchased music B, and play the complete music B for you" to prompt the user that the payment is successful. Further, the speaker 100 can play the resources of music B received from the music server.

[0290] In some embodiments, when the account used to log in to the speaker APP is a first account named "Li Ming", the speaker 100 can be bound to the first account. Among them, the audio resources purchased by the first user (i.e., the main user) after the biometric information verification is successful, and the audio resources purchased by the second user and other authorized users after the biometric information verification is successful can all belong to the audio resources purchased by the above-mentioned first account. The server that provides the above-mentioned audio resources can record the audio resources purchased by the first account. In other words, any audio resource purchased when any reference biometric information created in the first account is verified can be played when the account bound to the speaker 100 is the first account.

[0291] Exemplarily, the account currently bound to the speaker 100 is the first account. Reference biometric information of the first user and reference biometric information of the second user have been established in the first account. The first user can complete the verification through his or her own biometric information, and enable the speaker 100 to call the payment account to purchase music A. The second user can complete the verification through his or her own biometric information under the authorization of the first user, and enable the speaker 100 to call the payment account to purchase music B. Then both music A and music B can belong to the music purchased by the above-mentioned first account. When the first user instructs the speaker 100 to play music B through voice commands, there is no need to pay again. Similarly, when the second user instructs the speaker 100 to play music A through voice commands, there is no need to pay again.

[0292] Depend on Figure 8A to Figure 8E As can be seen from the scenario shown, the speaker 100 can store the reference biometric information of the main user (including reference voiceprint information and reference facial information) and the reference biometric information of the authorized user (only including reference voiceprint information). When the voiceprint information of the authorized user is verified, combined with the verification of the facial information of the main user, the speaker 100 can call the payment account to pay for the paid audio. In this way, it is not limited to the main user completing the payment through voiceprint verification and facial verification. Authorized users, such as the main user's family and friends, can also complete the payment through voiceprint verification and facial verification with the authorization of the main user. In addition, since the payment account associated with the first account bound to the speaker 100 can be considered as the payment account of the main user, the verification method combined with the voiceprint information of the authorized user and the facial information of the main user can be confirmed by the main user before the payment account makes a payment, thereby preventing the authorized user from abusing authority and over-consuming the payment account of the main user.

[0293] The following is combined with the above Figure 8A to Figure 8E The scenario shown introduces a payment method provided by an embodiment of the present application for verifying user identity through biometric information verification.

[0294] Fig. 9 The flowchart of a payment method is shown as an example. The method may include steps S401 to S409. Among them:

[0295] The security module of the speaker 100 stores the first biometric information of the first user and the second biometric information of the second user. The first biometric information may include the first voiceprint information and the first face information. The first biometric information may be the reference biometric information of the first user in the aforementioned embodiment. The second biometric information may only include the second voiceprint information. The second biometric information may be the reference biometric information of the second user in the aforementioned embodiment. That is, the second voiceprint information is the reference voiceprint information of the second user in the aforementioned embodiment. Among them, the first user may be the main user in the aforementioned embodiment. The second user may be the authorized user in the aforementioned embodiment.

[0296] S401. The speaker 100 establishes a binding relationship with the first account.

[0297] S402 , the speaker 100 receives a user operation to wake up the speaker 100 , play a paid resource and request payment.

[0298] S403: The speaker 100 records the voiceprint information of the second user.

[0299] S404: The speaker 100 verifies the voiceprint information recorded in step S403, and determines that the voiceprint information matches the second voiceprint information.

[0300] S405, the speaker 100 finds and calls the available face acquisition device 300.

[0301] S406. The face collection device 300 records the face information of the first user.

[0302] S407 , the face collection device 300 encrypts the face information entered in step S406 and sends it to the speaker 100 .

[0303] S408: The speaker 100 decrypts the encrypted facial information and verifies that the facial information matches the first facial information.

[0304] S409: The speaker 100 calls the payment account associated with the first account to pay the payment resource.

[0305] The specific implementation process of the above steps S401 to S409 can all be referred to the above embodiments, which will not be repeated here.

[0306] In a possible implementation, the second biometric information may also include second facial information. The second facial information may be facial information of a second user. The second facial information may be used as reference facial information to verify whether the facial information acquired by the speaker 100 is the facial information of the second user. The second voiceprint information in the second biometric information may be bound to the second facial information. That is to say, in the process of the speaker 100 combining voiceprint verification and facial verification to pay for paid audio, if the voiceprint information and facial information of the second user (ie, the authorized user) are both verified, the speaker 100 may call the payment account of the main user to make payment. In this way, in a scenario where the main user is not near the speaker 100, the authorized user may also complete the verification by saying the verification word and using his or her own facial information, and then listen to the paid audio resources.

[0307] The following is a schematic diagram of another scenario in which a first user authorizes a second user to establish reference biometric information on the speaker 100, provided in an embodiment of the present application.

[0308] The first user may be the main user in the aforementioned embodiment. The second user may be the authorized user in the aforementioned embodiment.

[0309] like Fig. 10A As shown, the biometric information list includes reference biometric information named "User 1". The reference biometric information named "User 1" may represent the reference biometric information of the main user. In other words, the security module of the speaker 100 stores the reference biometric information of the main user.

[0310] In response to the user operation on the new biometric information option 232, the electronic device 200 may send an instruction to the speaker 100 for instructing to establish a new reference biometric information of the authorized user. Before the electronic device 200 sends the instruction for instructing to establish a new reference biometric information of the authorized user, the electronic device 200 may first perform identity authentication of the payment account. The first user (i.e., the primary user) is the user corresponding to the payment account. That is, the first user knows the password of the payment account. The first user can Fig. 10B The password of the payment account is entered in the user interface 240 shown to complete the identity authentication of the payment account. When the identity authentication of the payment account is successful, the electronic device 200 may deem that the first user agrees to establish the reference voiceprint information of the authorized user. Then, the speaker 100 may start recording the voiceprint information and face information and use them as the reference voiceprint information and reference face information of the new authorized user respectively.

[0311] The specific implementation process of the electronic device 200 performing the identity authentication of the payment account can refer to the aforementioned Figure 2C to Figure 2E The embodiments shown will not be described in detail here.

[0312] like Fig. 10C As shown, when receiving an instruction to establish reference biometric information of a new authorized user, the speaker 100 can start recording reference voiceprint information. Specifically, the speaker 100 can voice broadcast "Please say the following verification word after me: Xiaoyi Xiaoyi" to prompt the user to record the voiceprint.

[0313] When hearing the voice prompt of the speaker 100, the second user can say the verification word "Xiao Yi Xiao Yi". The speaker 100 can receive the voice input when the second user says the verification word, and extract the voiceprint information from the voice input. When the voiceprint information is obtained, the speaker 100 can compare the voiceprint information with the reference voiceprint information already stored in the security module. If the voiceprint information matches one of the reference voiceprint information already stored, the speaker 100 can prompt the user that the voiceprint information already exists by voice broadcast.

[0314] If the voiceprint information does not match any of the stored reference voiceprint information, the speaker 100 may store the voiceprint information in the security module as reference voiceprint information of an authorized user.

[0315] When the reference voiceprint information is successfully entered, the speaker 100 can search and call a nearby device equipped with a face acquisition device to obtain facial information. The specific method for the speaker 100 to search and call a device equipped with a face acquisition device can refer to the aforementioned embodiment. Exemplarily, the speaker 100 can find and call a face acquisition device 300 to obtain a facial image. The face acquisition device 300 can be a TV named "Li Ming's TV". The face acquisition device 300 can include a face acquisition device 301. The face acquisition device 301 can be, for example, a camera.

[0316] like Fig. 10D As shown, when the face acquisition device 300 is found and called, the speaker 100 can voice broadcast "Voiceprint entry is successful. Please complete the face entry on Li Ming's TV. Please face your face to the camera of Li Ming's TV and blink". The second user can complete the face entry on the face acquisition device 300 according to the prompt of the speaker 100. When the facial image of the second user is collected, the face acquisition device 300 can encrypt the facial image and send it to the speaker 100.

[0317] Furthermore, the speaker 100 can decrypt the encrypted face image and extract the face information from it. Then, the speaker 100 can store the face information in the security module as reference face information and store it in the above Fig. 10C In this way, the speaker 100 can complete the entry of the reference biometric information of the new authorized user. Fig.10E As shown, the speaker 100 can voice broadcast "Face entry is successful, you have created new biometric information" to prompt the user that the reference biometric information of the new authorized user has been successfully established.

[0318] In addition, the speaker 100 can also send a message to the electronic device 200 indicating that the biometric information is newly created successfully. The electronic device 200 can display the following information: Fig. 6F The user interface shown in FIG. 14 shows a user interface in which the name of the newly created biometric information is added to the biometric information list 233 , for example, “User 2”.

[0319] Depend on Figures 10A to 10EAs can be seen from the scenario shown, the speaker 100 can establish reference biometric information for multiple users. Each user's reference biometric information can include their own reference voiceprint information and reference face information. That is, when the speaker 100 prompts that the currently playing audio is paid audio, each of the multiple users can use a combination of voiceprint verification and face verification to allow the speaker 100 to call a payment account for payment, thereby listening to the paid audio. The payment account used for payment can be a payment account associated with the account bound to the speaker 100. That is, in the above Fig. 10B An authenticated payment account.

[0320] In some embodiments, after the authorized user enters the reference voiceprint information and reference face information in the speaker 100, the payment account can be called on the speaker 100 to pay for the paid audio resources in combination with the voiceprint verification and face verification. In this way, when the main user is not near the speaker, the authorized user can also use the main user's payment account to purchase paid audio resources on the speaker 100.

[0321] Another payment method for verifying user identity through biometric information verification provided by an embodiment of the present application is described in detail below.

[0322] like Fig.11A As shown, the reference biometric information of the first user (ie, the main user) and the reference biometric information of the second user (ie, the authorized user) may be stored in the speaker 100. The reference biometric information includes reference voiceprint information and reference face information.

[0323] The second user can wake up the speaker 100 with a preset wake-up word and issue a voice command to play music B. The second user can say "Xiaoyi Xiaoyi, I want to listen to music B" near the speaker 100. The music B is a paid resource, and the first account (the account bound to the speaker 100) has not purchased music B. The music server that provides the resource of music B can send a message to the speaker 100 to indicate that music B is paid music.

[0324] like Fig. 11B As shown, when receiving the above message indicating that music B is paid music, the speaker 100 can voice broadcast "You can listen to the complete music B after paying, do you want to pay", to prompt the first user that he needs to pay to listen to music B. Furthermore, the second user can say "Please help me complete the payment" near the speaker 100. When the voice command "Please help me complete the payment" is detected, the speaker 100 can start voiceprint verification and face verification. Specifically, as Fig. 11C and Fig.11D As shown, the speaker 100 can obtain the voiceprint information and face information of the second user. The method for the speaker 100 to obtain the voiceprint information and face information of the second user can refer to the above embodiment, which will not be repeated here.

[0325] When the voiceprint information and face information are obtained, the speaker 100 can compare the voiceprint information and face information with the stored reference voiceprint information and reference face information respectively. The speaker 100 can determine that the voiceprint information and face information are the voiceprint information and face information of the second user. That is, both the voiceprint verification and the face verification are successful. Further, the speaker 100 can call the payment account associated with the above-mentioned first account to purchase music B. In this way, the second user can listen to music B on the speaker 100.

[0326] In other embodiments, after the authorized user enters reference voiceprint information and reference face information in the speaker 100, the payment account can be called on the speaker 100 to pay for the paid audio resources in a manner combining voiceprint verification and face verification as well as remote authorization by the main user.

[0327] The above-mentioned remote authorization of the main user can be: when the speaker 100 detects that the voiceprint verification and face verification of the authorized user are successful, the speaker 100 can send a message to the electronic device 200 to confirm whether to pay. The electronic device 200 can display the above-mentioned message for confirming whether to pay on the user interface. Furthermore, the main user can authorize the speaker 100 to call the payment account to pay for the paid audio resources through the electronic device 200. In this way, in the scenario where the main user is not near the speaker, the authorized user can also complete the voiceprint verification and face verification to request the speaker 100 to call the payment account to pay for the paid audio resources. Combining the above-mentioned method of remote authorization of the main user can ensure that the payment account of the main user is deducted after the consent of the main user. This not only simplifies the payment operation when listening to paid audio resources on the speaker 100, but also improves the security of the payment account, and effectively avoids the authorized user from over-consuming the payment account of the main user.

[0328] Another payment method for verifying user identity through biometric information verification provided by an embodiment of the present application is described in detail below.

[0329] The speaker 100 can be configured according to the aforementioned FIG. 11A to FIG. 11D The process shown completes the voiceprint verification and face verification of the second user (ie, the authorized user). When the voiceprint verification and face verification of the authorized user are both successful, the speaker 100 can send a message to the electronic device 200 for confirming the payment.

[0330] like Fig. 12A As shown, the speaker 100 can also broadcast "face verification successful, waiting for mobile phone confirmation" to remind the authorized user that the payment account needs to be confirmed by the main user. When receiving the payment confirmation message from the speaker 100, the electronic device 200 can display the following Fig. 12BThe user interface 250 shown. The user interface 250 may include a message notification box 251. The message notification box 251 may be used to confirm with the user whether to agree that the speaker calls the payment account to pay for the paid audio resource, such as purchasing music B.

[0331] In response to the user operation on the message notification box 251, the electronic device 200 can open the speaker APP and display the following information: Fig. 12C The user interface 260 may include a prompt 261, an OK button 262, and a cancel button 263.

[0332] Prompt 261 can be a text prompt "User 2 applies to purchase music B, do you want to buy it?" to prompt the user whose main user name is "User 2" (that is, the second user mentioned above) to apply to purchase music B.

[0333] The OK button 262 may be used to approve the user named "User 2" to purchase Music B using the primary user's payment account.

[0334] The cancel button 263 may be used to deny the user named "User 2" from purchasing Music B using the primary user's payment account.

[0335] In response to a user operation, such as a touch operation, on the confirmation button 262, the electronic device 200 may send a message indicating agreement to purchase the music B to the speaker 100.

[0336] Further, when receiving the message indicating the agreement to purchase music B, the speaker 100 can call the payment account of the primary user to purchase music B. When the payment is successful, the speaker 100 can receive the resources of music B. Fig.12D As shown, the speaker 100 can announce in voice that "the mobile phone has confirmed that you have successfully purchased music B and will play the complete music B for you". Then, the speaker 100 can play music B. The above-mentioned speaker 100 calls the payment account of the main user to purchase music B, which can be referred to the above embodiment and will not be repeated here.

[0337] It should be noted that the application is not limited to Figure 1 The biometric information verification method provided in the embodiment of the present application can also be applied to other electronic devices. For example, a television, a mobile phone, a tablet, a laptop computer, a storytelling machine, etc. The structure of these electronic devices can refer to Figure 1 The structural diagram of the speaker 100 is shown. Figure 1 The electronic devices may include more or fewer modules than the modules included in the structural schematic diagram.

[0338] For example, in a scenario where a user plays a paid video resource on a TV, a TV equipped with a voice collection device but not equipped with a face collection device can record the user's reference biometric information according to the method in the aforementioned embodiment and use the reference biometric information to verify the user's identity during the payment of the paid video resource. In this way, an electronic device that is not equipped with a face collection device can combine voiceprint verification and face verification to perform identity verification during the payment process.

[0339] Alternatively, the TV is equipped with a voice acquisition device and a face acquisition device. The TV can search for a face acquisition device with a better face acquisition device nearby. For example, the TV is equipped with a 2D camera. When recording facial information, if the TV finds a face acquisition device with a 3D camera nearby, the TV can call the face acquisition device with the 3D camera to obtain facial information. Using a better face acquisition device can improve the reliability of identity authentication during the payment process.

[0340] In some embodiments, the electronic device is configured with a face collection device but not with a voice collection device. That is, the electronic device cannot independently collect voice input to obtain voiceprint information. Similar to the aforementioned method of finding and calling nearby face collection devices, the electronic device can find and call nearby devices configured with voice collection devices to collect the user's voice input and obtain voiceprint information. Alternatively, the electronic device is not configured with a face collection device and a voice collection device. The electronic device can find and call nearby face collection devices to collect face images and obtain face information. In addition, the electronic device can also find and call nearby voice collection devices to collect voice input and obtain voiceprint information.

[0341] It can be seen from the above method that when an electronic device combines voiceprint verification and face verification to perform identity verification during the payment process, the electronic device is not limited to electronic devices that are equipped with both a voice collection device and a face collection device. Electronic devices that are not equipped with a voice collection device and / or a face collection device can also implement a method of combining voiceprint verification and face verification to authenticate the user.

[0342] In some embodiments, the device with a face collection device configured near the speaker 100 is a mobile phone. In the process of entering reference facial information and performing facial verification when paying for paid resources, the speaker 100 can call the face collection device on the mobile phone, such as a camera, to obtain facial information. Since mobile phones are used in many occasions, and mobile phones are generally equipped with face collection devices such as cameras (such as 2D cameras, 3D cameras, infrared cameras), electronic devices that need to play paid resources can call the mobile phone in various occasions to obtain facial information and then perform facial verification. For example, in the scenario of using a portable speaker outside, if the portable speaker plays paid resources, the portable speaker can easily find and call the user's mobile phone to obtain facial information.

[0343] The embodiment of the present application does not specifically limit the content contained in the above-mentioned reference biometric information. It is not limited to voiceprint information and face information, and the reference biometric information may also include other data. For example, fingerprint information, bone voiceprint information, iris information, etc. In other words, the speaker can also instruct the user to enter more information when entering the reference biometric information.

[0344] In the present application, the first device collects the first biometric information. The first device may be the speaker 100 in the aforementioned embodiment. The first device may also be other electronic devices that request to obtain paid resources, such as a television, a story machine, etc. Exemplarily, the first device is the speaker 100. The first device is equipped with a voice collection device. The first biometric information may be the voiceprint information in the aforementioned embodiment.

[0345] In the present application, the first device can discover the second device with the second biometric information collection capability through a short-range wireless communication connection. Take the first device as the speaker 100 in the aforementioned embodiment as an example for illustration. The second device can be the face collection device 300 in the aforementioned embodiment. The face collection device 300 is configured with a face collection device. The face collection device can be, for example, a 2D camera, a 3D camera, an infrared camera, a millimeter wave radar and other devices. The above-mentioned second biometric information can be the face information in the aforementioned embodiment. The above-mentioned short-range wireless communication connection can be, for example, a near field communication connection, a Bluetooth communication connection, a WiFi communication connection, a ZigBee communication connection, a WLAN direct communication connection, etc. In other words, the first device can find and call devices near itself to obtain face information.

[0346] In the present application, the first device can pre-collect the third biometric information of the first user, and call the second device to pre-collect the fourth biometric information of the first user. The above-mentioned third biometric information can be the reference voiceprint information of the first user in the aforementioned embodiment. The above-mentioned fourth biometric information can be the reference face information of the first user in the aforementioned embodiment. The third biometric information can be bound to the fourth biometric information and stored in the security module of the speaker 100 in the aforementioned embodiment, or stored in a cloud storage space accessible to the speaker 100. The specific method of pre-collecting the third biometric information and the fourth biometric information can refer to the aforementioned Figure 2A to Figure 2I The embodiment shown.

[0347] If the first device determines that the first biometric information matches the third biometric information, and the second biometric information matches the fourth biometric information, the identity authentication is successful. That is, the first device can determine that the user corresponding to the first biometric information and the second biometric information is the first user.

[0348] When the identity authentication is successful, the first device may send a first message to the first server. The first server may be the payment server in the aforementioned embodiment. The first message may include identification information of the first payment account and order information of the paid resources to be paid. The embodiment of the present application does not limit the content contained in the first message. The payment server may deduct money from the first payment account according to the first message. The first payment account may be a payment account associated with the account lock bound to the first device. For example, the user associated with the account named "Li Ming" in the aforementioned embodiment.

[0349] In addition, before the payment server deducts money from the first payment account, it can first perform a trusted identity verification on the first device to determine whether the first device is a trusted device. The above-mentioned trusted identity verification method can refer to the above-mentioned embodiment, which will not be repeated here.

[0350] When the payment server successfully deducts the first payment account, the payment server may send a payment success message to the second server. Furthermore, the second server may send the paid resources purchased using the account number bound to the first device as the identity identifier to the first device. The above-mentioned second server may be a content server. The content server may store the paid resources requested by the first device. The content server may be, for example, a music server, a video server, etc. The embodiment of the present application does not limit the type of the above-mentioned content server.

[0351] In this application, the first device logs in to the first account on the second server. The first account is the account bound to the first device in the above embodiment. For example, Figure 2AThe account name shown is "Li Ming". The method for binding the first device to the first account can refer to the aforementioned embodiment. The first device can log in to the first account on the content server. The identification of multiple accounts can be stored in the content server. The above multiple accounts include the first account. The content server can determine the paid resources purchased by each account based on the identification of the above multiple accounts. When the first device logs in to the first account on the second server and obtains the first content from the second server, the second server can determine that the first account has not purchased the first content. Furthermore, the second server can send a message to the first device to prompt the first device that it needs to pay to obtain the first content. The second server can also send the price information of the first content to the first device.

[0352] In the present application, the first device can collect the fifth biometric information of the second user. If the first device is the speaker 100 in the aforementioned embodiment, the fifth biometric information can be the reference voiceprint information of the second user in the aforementioned embodiment. The second user can enter the fifth biometric information with the authorization of the first user. The specific implementation process can refer to the aforementioned Figure 6A to Figure 6C The first user may be the main user in the above embodiment. The second user may be the authorized user in the above embodiment.

[0353] In the present application, the first device can collect the sixth biometric information. The sixth biometric information can be the voiceprint information in the aforementioned embodiment. The first device can also call the second to collect the second biometric information. If the first device determines that the sixth biometric information matches the fifth biometric information mentioned above, and the second biometric information matches the fourth biometric information, the identity authentication is successful. That is to say, if the voiceprint information of the authorized user is successfully verified, if the main user authorizes, the first device can request the payment server to deduct the first payment account. The above-mentioned main user authorization can indicate that the first device has successfully verified the facial information of the main user.

[0354] In this application, the first device can collect the fifth biometric information of the second user, and call the second device to collect the seventh biometric information of the second user. The fifth biometric information can refer to the introduction of the above embodiment. The seventh biometric information can be the reference face information of the second user. That is, the first device can obtain the reference voiceprint information and reference face information of the authorized user.

[0355] In the present application, the second user (i.e., the authorized user) can use his or her voiceprint information and facial information for identity authentication. Among them, the first device can collect the sixth biometric information and call the second device to collect the eighth biometric information. The above-mentioned sixth biometric information can refer to the introduction of the aforementioned embodiment. The above-mentioned eighth biometric information can be facial information. If the speaker 100 determines that the fifth biometric information matches the sixth biometric information, and the seventh biometric information matches the eighth biometric information, the identity authentication of the authorized user is successful.

[0356] It can be seen from the above embodiments that the biometric information verification method provided in the embodiments of the present application can be used for identity verification in the payment process. It is not limited to identity verification in the payment process, and the above biometric information verification method can also be used for identity verification in other scenarios. For example, logging in to an account, obtaining permissions, etc.

[0357] Among them, when payment is required for listening to paid audio resources on the speaker, the speaker can combine voiceprint verification and face verification to perform user identity authentication. After the voiceprint verification and face verification are successful, the speaker can request the payment server to deduct money from the payment account to pay for the paid audio resources. This simplifies the operation of users paying for listening to paid audio resources on the speaker. The above-mentioned combination of voiceprint verification and face verification can also improve the reliability of the payment process. In addition, a speaker that is not equipped with a face collection device can complete face verification by calling a nearby device that is equipped with a face collection device. That is, speakers that are equipped with or not equipped with a face collection device can implement the payment method provided in the embodiment of the present application to simplify the operation of users paying for paid audio resources.

[0358] In addition, after the main user has entered his or her reference voiceprint information and reference face information into the speaker, he or she can also authorize other users, such as his or her family and friends, to enter reference voiceprint information and reference face information. The above-mentioned authorized users can also perform user identity verification by combining voiceprint verification and face verification. After the authorized user's identity verification is successful, the speaker can request the payment server to deduct money from the main user's payment account to pay for the paid audio resources. In other words, not limited to the main user, users authorized by the main user can also conveniently complete the operation of paying for paid audio resources on the speaker.

[0359] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0360] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A biometric information verification method, characterized in that: include: The first device collects first biological information; If the first biometric information matches the third biometric information, the first device discovers the second device capable of collecting the second biometric information through a short-range wireless communication connection, and the first device does not have the capability of collecting the second biometric information; The first device receives the second biological information collected from the second device, where the second biological information is different from the first biological information; If the second biometric information matches the fourth biometric information, the first device determines that the first user's identity authentication is successful; wherein, the third biometric information is stored in the first device or in a cloud storage space accessible to the first device, and the third biometric information is the biometric information of the first user pre-collected by the first device; the fourth biometric information is stored in the first device or in a cloud storage space accessible to the first device, and the fourth biometric information is the biometric information of the first user pre-collected by the second device.

2. The method according to claim 1, characterized in that The third biometric information is used to determine whether the first biometric information obtained by the first device is the biometric information of the first user, and the fourth biometric information is used to determine whether the second biometric information obtained by the first device is the biometric information of the first user.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: The first device sends a first message to a first server; the first message is used to instruct the first server to deduct money from a first payment account, and the first payment account is the payment account of the first user.

4. The method according to claim 3, characterized in that Before the first device collects the first biological information, the method further includes: The first device logs in to a first account on a second server; the first account is an account of the first user on the second server, and the second server is used to send the content purchased by the first user to the first device after the first server successfully deducts money from the first payment account; The first device sends a request for obtaining the first content to the second server; The first device receives price information of the first content sent by the second server.

5. The method according to claim 4, characterized in that The third biometric information and the fourth biometric information stored in the cloud storage space can be accessed by multiple devices, the multiple devices include the first device, the multiple devices share the same account on the second server, or the multiple accounts of the multiple devices on the second server belong to the same account group; The belonging to the same account group means that multiple accounts in the same account group share the content purchased by each account in the second server.

6. The method according to any one of claims 1, 2, 4 and 5, characterized in that: The first device is a device equipped with a voice collection device, and the first biometric information and the third biometric information are both voiceprint information.

7. The method according to claim 3, characterized in that The first device is a device equipped with a voice collection device, and the first biometric information and the third biometric information are both voiceprint information.

8. The method according to claim 6, characterized in that The second device is a device equipped with a face acquisition device, and the second biological information and the fourth biological information are both face information.

9. The method according to claim 7, characterized in that: The second device is a device equipped with a face acquisition device, and the second biological information and the fourth biological information are both face information.

10. The method according to any one of claims 1, 2, 4 and 5, characterized in that: The first device is a device equipped with a face acquisition device, and the first biological information and the third biological information are both facial information.

11. The method according to claim 3, characterized in that The first device is a device equipped with a face acquisition device, and the first biological information and the third biological information are both facial information.

12. The method according to claim 10, characterized in that The second device is a device equipped with a voice collection device, and the second biometric information and the fourth biometric information are both voiceprint information.

13. The method according to claim 11, characterized in that The second device is a device equipped with a voice collection device, and the second biometric information and the fourth biometric information are both voiceprint information.

14. The method according to any one of claims 1, 2, 4, 5, 7-9, 11-13, characterized in that: The method further comprises: The first device collects the fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, the fifth biometric information is stored in the first device or in a cloud storage space accessible to the first device, and the fifth biometric information is bound to the fourth biometric information.

15. The method according to claim 3, characterized in that The method further comprises: The first device collects the fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, the fifth biometric information is stored in the first device or in a cloud storage space accessible to the first device, and the fifth biometric information is bound to the fourth biometric information.

16. The method according to claim 6, characterized in that The method further comprises: The first device collects the fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, the fifth biometric information is stored in the first device or in a cloud storage space accessible to the first device, and the fifth biometric information is bound to the fourth biometric information.

17. The method according to claim 10, characterized in that The method further comprises: The first device collects the fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, the fifth biometric information is stored in the first device or in a cloud storage space accessible to the first device, and the fifth biometric information is bound to the fourth biometric information.

18. The method according to claim 14, characterized in that The method further comprises: The first device collects sixth biological information, and the sixth biological information and the first biological information are biological information of the same type; The first device receives the second biological information collected from the second device; The first device determines that the sixth biometric information matches the fifth biometric information, and the second biometric information matches the fourth biometric information; The first device sends a second message to the first server, where the second message is used to instruct the first server to deduct money from the first payment account.

19. The method according to any one of claims 15 to 17, characterized in that: The method further comprises: The first device collects sixth biological information, and the sixth biological information and the first biological information are biological information of the same type; The first device receives the second biological information collected from the second device; The first device determines that the sixth biometric information matches the fifth biometric information, and the second biometric information matches the fourth biometric information; The first device sends a second message to the first server, where the second message is used to instruct the first server to deduct money from the first payment account.

20. The method according to any one of claims 1, 2, 4, 5, 7-9, 11-13, characterized in that: The method further comprises: The first device collects fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the fifth biometric information is stored in the first device or in a cloud storage space accessible by the first device; The first device receives seventh biometric information of the second user collected by the second device; the seventh biometric information and the second biometric information are of the same type of biometric information, the seventh biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the seventh biometric information is stored in the first device or in a cloud storage space accessible to the first device; Wherein, the fifth biological information is bound to the seventh biological information.

21. The method according to claim 3, characterized in that The method further comprises: The first device collects fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the fifth biometric information is stored in the first device or in a cloud storage space accessible by the first device; The first device receives seventh biometric information of the second user collected by the second device; the seventh biometric information and the second biometric information are of the same type of biometric information, the seventh biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the seventh biometric information is stored in the first device or in a cloud storage space accessible to the first device; Wherein, the fifth biological information is bound to the seventh biological information.

22. The method according to claim 6, characterized in that The method further comprises: The first device collects fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the fifth biometric information is stored in the first device or in a cloud storage space accessible by the first device; The first device receives seventh biometric information of the second user collected by the second device; the seventh biometric information and the second biometric information are of the same type of biometric information, the seventh biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the seventh biometric information is stored in the first device or in a cloud storage space accessible to the first device; Wherein, the fifth biological information is bound to the seventh biological information.

23. The method according to claim 10, characterized in that The method further comprises: The first device collects fifth biometric information of the second user; the fifth biometric information and the first biometric information are of the same type of biometric information, the fifth biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the fifth biometric information is stored in the first device or in a cloud storage space accessible by the first device; The first device receives seventh biometric information of the second user collected by the second device; the seventh biometric information and the second biometric information are of the same type of biometric information, the seventh biometric information is used to determine whether the biometric information obtained by the first device is the biometric information of the second user, and the seventh biometric information is stored in the first device or in a cloud storage space accessible to the first device; Wherein, the fifth biological information is bound to the seventh biological information.

24. The method according to claim 20, characterized in that The method further comprises: The first device collects sixth biological information, and the sixth biological information and the first biological information are biological information of the same type; The first device receives eighth biological information collected by the second device; the eighth biological information and the second biological information are biological information of the same type; The first device determines that the sixth biometric information matches the fifth biometric information, and the eighth biometric information matches the seventh biometric information; The first device sends a second message to the first server, where the second message is used to instruct the first server to deduct money from the first payment account.

25. The method according to any one of claims 15 to 17, characterized in that The method further comprises: The first device collects sixth biological information, and the sixth biological information and the first biological information are biological information of the same type; The first device receives eighth biological information collected by the second device; the eighth biological information and the second biological information are biological information of the same type; The first device determines that the sixth biometric information matches the fifth biometric information, and the eighth biometric information matches the seventh biometric information; The first device sends a second message to the first server, where the second message is used to instruct the first server to deduct money from the first payment account.

26. The method according to claim 24, characterized in that Before the first device sends the second message to the first server, the method further includes: The first device sends a third message to a third device and receives a fourth message from the third device; the third device is a device installed with an application for controlling the first device, and the indication content of the third message is displayed on the third device, and the indication content of the third message includes asking the third device whether it agrees with the first device sending the second message, and the indication content of the fourth message includes the third device agreeing with the first device sending the second message.

27. The method according to claim 25, characterized in that Before the first device sends the second message to the first server, the method further includes: The first device sends a third message to a third device and receives a fourth message from the third device; the third device is a device installed with an application for controlling the first device, and the indication content of the third message is displayed on the third device, and the indication content of the third message includes asking the third device whether it agrees with the first device sending the second message, and the indication content of the fourth message includes the third device agreeing with the first device sending the second message.

28. The method according to any one of claims 1, 2, 4, 5, 7-9, 11-13, 15-18, 21-24, 26, 27, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

29. The method according to claim 3, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

30. The method according to claim 6, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

31. The method according to claim 10, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

32. The method according to claim 14, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

33. The method according to claim 19, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

34. The method according to claim 20, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

35. The method according to claim 25, characterized in that The short-range wireless communication connection is one or more of the following: near field communication connection, Bluetooth communication connection, WLAN direct communication connection, and ZigBee communication connection.

36. A device, the device being a first device, characterized in that: The first device comprises: a first acquisition device, a communication device, a memory and a processor, wherein: The first collecting device is used to collect first biological information; The communication device is used for the first device to discover the second device having the second biological information collection capability through wireless communication connection; the second device includes a second collection device, and the second collection device is used to collect the second biological information; The communication device is further used to receive the second biological information collected by the second collection device; The memory is used to store the first biological information and the second biological information, and is also used to store a computer program; The processor is configured to call the computer program so that the first device executes the method according to any one of claims 1 to 35.

37. A computer-readable storage medium, characterized in that: include: Computer instructions, when the computer instructions are executed on a device, cause the device to perform the method according to any one of claims 1 to 35.

38. A computer program product, characterized in that When the computer program product is executed on a device, the device is caused to perform the method according to any one of claims 1 to 35.

Images