Method for launching application, electronic device and computer readable medium

Abstract

The embodiments of the present disclosure disclose a method for starting an application, an electronic device, and a computer-readable medium. A specific implementation of the method includes: receiving a request from a target user to start an application on a device, sending the request to a first terminal, wherein the first terminal collects sound information in the request, receives sound information returned by the first terminal, starts the device based on the sound information, determines whether a keyword instruction in the request is authenticated based on the request, and starts the application on the device in response to the keyword instruction being authenticated. This method can complete the tasks of starting the device and starting the application based on the request from the target user to start the application, has few operating steps and a simple process, is convenient for users to start applications in the device, and improves the user experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a human-computer interaction method, device, electronic device, and computer-readable medium. Background Art

[0002] The method of human-computer interaction is generally that users communicate with the system through a human-computer interaction interface, and the quality of the human-computer interaction interface directly affects the success or failure of software development. Identity authentication based on human biometric information can determine whether the user has access to and use rights to a certain resource, thereby enabling the access policy of the computer and network system to be reliably and effectively executed, ensuring the security of the system and data. Recognition of human-computer interaction instructions based on biometric information such as voice can enable the machine to automatically complete specific functions such as starting programs and loading applications. Summary of the invention

[0003] The embodiment of the present disclosure provides a method for starting an application.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for launching an application, the method comprising: receiving a request from a target user to launch an application on a device, wherein the request is a voice command and the request includes a keyword command; sending the request to a first terminal, wherein the first terminal collects user information; receiving the user information returned by the first terminal; launching the device based on the user information; determining whether the keyword command is authenticated based on the request; and launching the application on the device in response to the keyword command being authenticated.

[0005] In some embodiments, starting a device based on user information includes: in response to the user information including sound information and physiological information, determining whether the sound information authentication is passed based on the sound information, and, based on the physiological information, determining whether the physiological information authentication is passed; in response to the sound information authentication being passed and the physiological information authentication being passed, starting the device.

[0006] In some embodiments, determining whether the sound information authentication is passed based on the sound information includes: inputting the sound information into a pre-trained first convolutional neural network to obtain an output as a voice feature of the target user; and determining whether the sound information authentication is passed based on the voice feature of the target user.

[0007] In some embodiments, the first convolutional neural network is pre-trained through the following steps: determining the network structure of the first initial convolutional neural network and initializing the network parameters of the first initial convolutional neural network; obtaining a first training sample set, wherein the first training sample includes sample sound information and pre-obtained sample speech features; using a machine learning method, taking the sample sound information included in the first training sample in the first training sample set as the input of the first initial convolutional neural network, and taking the pre-obtained sample speech features corresponding to the input sample sound information as the expected output of the first initial convolutional neural network, to train the first convolutional neural network.

[0008] In some embodiments, the first convolutional neural network includes a first feature extraction network, a second feature extraction network and a first fully connected layer; and the sound information is input into the pre-trained first convolutional neural network to obtain an output as the voice feature of the target user, including: inputting the sound information into the pre-trained first feature extraction network to obtain a first feature vector; inputting the first feature vector into the pre-trained second feature extraction network to obtain a second feature vector; inputting the second feature vector into the pre-trained first fully connected layer to generate the voice feature of the target user.

[0009] In some embodiments, determining whether a keyword instruction is passed based on a request includes: determining phoneme data of the keyword instruction based on the keyword instruction; inputting the phoneme data into a pre-trained second convolutional neural network to obtain output as speech features of the phoneme data; and determining whether the keyword instruction is passed based on the speech features of the phoneme data.

[0010] In some embodiments, the second convolutional neural network is pre-trained through the following steps: determining the network structure of the initial second convolutional neural network and initializing the network parameters of the initial second convolutional neural network; obtaining a second training sample set, wherein the second training sample includes sample phoneme data and pre-obtained sample speech features; using a machine learning method, the sample phoneme data included in the second training sample in the second training sample set is used as the input of the second initial convolutional neural network, and the pre-obtained speech features corresponding to the input sample phoneme data are used as the expected output of the second initial convolutional neural network, to train the second convolutional neural network.

[0011] In some embodiments, the second convolutional neural network includes a third feature extraction network, a first pooling layer, and a second fully connected layer; and the phoneme data is input into a pre-trained second convolutional neural network to obtain speech features as output of the phoneme data, including: inputting the phoneme data into a pre-trained third feature extraction network to obtain a third feature vector; inputting the third feature vector into a pre-trained first pooling layer to obtain a fourth feature vector; and inputting the fourth feature vector into a pre-trained second fully connected layer to generate speech features of the phoneme data.

[0012] In some embodiments, the method includes: in response to the number of times that the target user's request to start the application fails exceeds a predetermined number, sending an alarm message to a device with a display function, and controlling the device to display the alarm message.

[0013] In a second aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method as described in any implementation manner in the first aspect.

[0014] An embodiment of the present disclosure provides a method for starting an application, receiving a request from a target user to start an application on a device, sending the request to a first terminal, wherein the first terminal collects sound information in the request, receives the sound information returned by the first terminal, starts the device based on the sound information, determines whether a keyword instruction in the request is authenticated based on the request, and starts the application on the device in response to the keyword instruction being authenticated.

[0015] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: based on the request input by the target user, the voice features of the target user are obtained by using the first convolutional neural network, and the device is started by recognizing the voice features. Based on the keyword instructions in the request, the phoneme data is determined, and the voice features of the phoneme data are obtained by using the second convolutional neural network, and the application is started by recognizing the voice features of the phoneme data. This method can complete the tasks of starting the device and starting the application according to the request to start the application issued by the target user. Compared with the implementation method of the application startup method of the prior art, which requires unlocking the device first and then starting the application on the device, the embodiment of the present disclosure realizes a one-step implementation method, that is, from the user's perspective, a single request can directly realize the unlocking of the device and the startup of the application. The embodiment of the present disclosure has few operating steps and a simple process, which is convenient for users to start applications in the device and significantly improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0017] Figure 1 is an architectural diagram of an exemplary system in which some embodiments of the present disclosure may be applied;

[0018] Figure 2 is a flow chart of some embodiments of a method for launching an application according to the present disclosure;

[0019] Figure 3is a flowchart of an embodiment of a first training step for training a first convolutional neural network according to the present application;

[0020] Figure 4 is a flow chart of other embodiments of a method for verifying whether a keyword instruction is passed according to the present disclosure;

[0021] Figure 5 is a flowchart of an embodiment of a second training step for training a second convolutional neural network according to the present application;

[0022] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0025] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0026] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0027] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the method of launching an application of the present disclosure can be applied.

[0028] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and an electronic device 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the electronic device 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0029] The user can use the terminal devices 101, 102, 103 to interact with the electronic device 105 through the network 104 to collect request instructions such as voice signals. Various identity authentication applications can be installed on the terminal devices 101, 102, 103, such as storage applications, payment applications, inquiry applications, etc.

[0030] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various terminal devices with sound signal collection, including but not limited to smart phones, tablet computers, laptop portable computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the terminal devices listed above. It can be implemented as multiple software or software modules (for example, for providing conversational voice input, text input, etc.), or it can be implemented as a single software or software module. No specific limitation is made here.

[0031] The electronic device 105 may be an electronic device that provides various services, such as an identity authentication electronic device that supports the identity authentication of the terminal devices 101, 102, and 103, or an electronic device that processes the voice signals input by the terminal devices 101, 102, and 103 and starts the application. The electronic device may process the received voice signal and feed back the processing result (such as starting the application) to other terminal devices. The above-mentioned electronic device 105 includes but is not limited to smart phones, tablet computers, laptops and desktop computers, servers, and the like.

[0032] It should be noted that the method for starting an application provided in the embodiment of the present disclosure is generally executed by the electronic device 105 , and accordingly, the device for pushing information is generally disposed in the electronic device 105 .

[0033] It should be noted that the electronic device 105 can also directly receive voice signal input locally, and the electronic device 105 can directly start the application according to the local voice signal. In this case, the exemplary system architecture 100 may not include the terminal devices 101, 102, 103 and the network 104.

[0034] It should be noted that the electronic device 105 can be hardware or software. When the electronic device 105 is hardware, it can be implemented as a distributed electronic device cluster consisting of multiple electronic devices, or it can be implemented as a single electronic device. When the electronic device is software, it can be implemented as multiple software or software modules, or it can be implemented as a single software or software module. No specific limitation is made here.

[0035] It should be understood that Figure 1The number of terminal devices, networks and electronic devices in the embodiment is only for illustration. Any number of terminal devices, networks and electronic devices may be provided according to implementation requirements.

[0036] Continue to refer Figure 2 , shows a process 200 of some embodiments of the method for starting an application according to the present disclosure. The method for starting an application includes the following steps:

[0037] Step 201: receiving a request from a target user to start an application on a device.

[0038] In some embodiments, the execution body (eg, Figure 1 The electronic device shown) can receive a request from the target user to start an application on the device, wherein the request is a voice command, and the request contains a keyword command. The above-mentioned keyword command is usually an instruction that the target user needs to start executing the application in the above-mentioned execution subject. For example, the target user directly says: "Xiao De classmate", then the device can be directly unlocked and the Amap application in the device can be opened, and the voice input function of the Amap application can be activated at the same time. The user can continue to use voice to perform functions such as location search and navigation. For another example, the target user directly says: "Xiao Wei classmate, pay money", which can directly unlock the device and open the WeChat software application in the device, and then automatically enter the payment code interface.

[0039] Step 202: Send a request to the first terminal, wherein the first terminal collects user information.

[0040] In some embodiments, the execution subject of the method for starting an application sends a request to start the application to the first terminal, and the first terminal collects user information according to the request. The user information includes at least one of the following: sound information collected in response to the request information, and physiological information collected in response to the request information. Here, the above physiological information usually includes but is not limited to at least one of the following: the target user's pulse, heartbeat, palm print, fingerprint features, skin dryness, facial features, and iris information. Sound information is a sound wave spectrum that carries speech information displayed by an electroacoustic instrument. Facial features include feature information of specific parts and overall feature information. Fingerprint features are white digitized multi-grayscale fingerprint images that can be processed by fingerprint device algorithms.

[0041] Step 203: receiving user information returned by the first terminal.

[0042] In some embodiments, the execution body (eg, Figure 1The electronic device shown) can receive user information returned by the first terminal through a wired connection or a wireless connection, wherein the user information includes at least one of the following: a sound signal collected in response to the request information, and physiological information collected in response to the request information.

[0043] Step 204: start the device based on the user information.

[0044] In some embodiments, the execution body (eg, Figure 1 The electronic device shown in the figure can compare the received user information with the pre-stored information or image. If the comparison is consistent, the identity authentication of the target user is passed and the device is started. Here, the above identity authentication can be matched only by the sound information in the user information, or it can be matched only by the physiological information in the user information. In response to the sound information authentication passing and the physiological information authentication passing, the device is started.

[0045] In some embodiments, the execution subject inputs the sound information into a pre-trained first convolutional neural network to obtain an output as the speech feature of the target user. The first convolutional neural network may be pre-trained through the first training step.

[0046] Please refer to Figure 3 , Figure 3 A process 300 of an embodiment of a first training step of pre-training a first convolutional neural network according to the present application is shown. The first training step may include the following steps:

[0047] Step 301, determine the network structure of a first initial convolutional neural network and initialize the network parameters of the first initial convolutional neural network.

[0048] In this embodiment, the execution subject of the first training step can be the same as the execution subject of the method for starting the application (for example Figure 1 The electronic device shown in the figure is the same or different. If they are the same, the execution subject of the first training step can store the network structure information of the trained first convolutional neural network and the parameter values ​​of the network parameters locally after the first convolutional neural network is trained. If they are different, the execution subject of the first training step can send the network structure information of the trained first convolutional neural network and the parameter values ​​of the network parameters to the execution subject of the method for starting the application after the first convolutional neural network is trained.

[0049] In this embodiment, the execution subject of the first training step may first determine the network structure of the initial first convolutional neural network. For example, it is necessary to determine which layers the initial first convolutional neural network includes, the connection order relationship between the layers, which neurons each layer includes, the weight and bias corresponding to each neuron, the activation function of each layer, etc.

[0050] In some optional implementations of this embodiment, since the convolutional neural network is a multi-layer neural network, each layer is composed of multiple two-dimensional planes, and each plane is composed of multiple independent neurons, it is necessary to determine which layers the initial neural network of the convolutional neural network type includes (for example, convolution layer, pooling layer, fully connected layer, classifier, etc.), the connection order relationship between layers, and which parameters each layer includes (for example, weights, bias terms, convolution step length), etc. Among them, the convolution layer can be used to extract speech features. For each convolution layer, it can be determined how many convolution kernels there are, the size of each convolution kernel, the weights of each neuron in each convolution kernel, the bias term corresponding to each convolution kernel, the step length between two adjacent convolutions, etc.

[0051] Then, the execution subject of the first training step can initialize the network parameters of the initial first convolutional neural network. In practice, the various network parameters (e.g., weight parameters and bias parameters) of the first convolutional neural network can be initialized with some different small random numbers. "Small random numbers" are used to ensure that the network will not enter a saturated state due to excessive weights, thereby causing training failure, and "different" is used to ensure that the network can learn normally.

[0052] Step 302: Obtain a first training sample set.

[0053] In this embodiment, the execution subject of the first training step can obtain the training sample set locally or remotely from other electronic devices connected to the execution subject through a network. The first training sample includes sample sound information and pre-obtained sample speech features. For example, the speech features of the sample sound can be manually determined.

[0054] Step 303, using a machine learning method, taking the sample sound information included in the first training sample in the first training sample set as the input of the first initial convolutional neural network, taking the pre-obtained sample speech features corresponding to the input sample sound information as the expected output of the first initial convolutional neural network, and training the first convolutional neural network.

[0055] In this embodiment, the executor of the first training step can use a machine learning method to take the sample sound information included in the first training sample in the first training sample set as the input of the first initial convolutional neural network, and take the pre-obtained sample speech features corresponding to the input sample sound information as the expected output of the first initial convolutional neural network, so as to train the first convolutional neural network.

[0056] Specifically, the difference between the speech features obtained by calculating the preset loss function and the sample speech features in the training sample can be first calculated. For example, the difference between the speech features obtained by calculating the loss function and the sample speech features in the training sample can be calculated using the L2 norm. Then, the network parameters of the initial first convolutional neural network can be adjusted based on the calculated difference, and the training can be terminated when the preset training termination condition is met. For example, the preset training termination condition here can include but is not limited to at least one of the following: the training time exceeds the preset duration; the number of training times exceeds the preset number of times; the calculated difference is less than the preset difference threshold.

[0057] Here, various implementations may be used to adjust the network parameters of the initial first convolutional neural network based on the difference between the generated speech features and the sample speech features in the training sample. For example, the network parameters of the initial neural network may be adjusted using a BP (Back Propagation) algorithm or an SGD (Stochastic Gradient Descent) algorithm.

[0058] In this embodiment, the execution entity of the first training step determines the initial first convolutional neural network obtained by training as the pre-trained first convolutional neural network.

[0059] In some optional implementations of this embodiment, the pre-trained first convolutional neural network may include a first feature extraction network, a second feature extraction network, and a first fully connected layer. Based on the above first convolutional neural network, step 204 may be performed as follows:

[0060] First, the target user's voice information can be input into a pre-trained first feature extraction network to generate a first feature vector. Here, the first feature extraction network is a convolutional neural network including one convolutional layer.

[0061] Then, the first feature vector is input into a pre-trained second feature extraction network to generate a second feature vector. Here, the second feature extraction network is a convolutional neural network including a convolutional layer.

[0062] Afterwards, the second feature vector is input into the pre-trained first fully connected layer to generate the speech features of the target user. Here, the first fully connected layer is used to characterize the correspondence between the sound information and the speech features.

[0063] It should be noted that the first feature extraction network, the second feature extraction network and the first fully connected layer can also be adopted as follows Figure 3 The first training step shown is obtained.

[0064] The method for starting an application provided by the above embodiment of the present application trains a first convolutional neural network by using sound information and corresponding voice features as training samples, so that the trained first convolutional neural network can learn how to generate voice features of sound information. When generating voice features, the voice features of the sound information can be generated by inputting the sound information into the pre-trained first convolutional neural network.

[0065] According to the voice features of the target user, the voice features of the pre-stored start device are matched to determine whether the voice information authentication is passed.

[0066] Step 205: Based on the request, determine whether the keyword instruction is authenticated.

[0067] In some embodiments, the execution body (eg, Figure 1 The electronic device shown in the figure) extracts the phoneme data according to the keyword instruction. In some optional implementations of some embodiments, the start and end points of the keyword instruction are accurately located from the request using voice endpoint detection technology to extract the phoneme data in the keyword instruction.

[0068] In some embodiments, the execution entity inputs the phoneme data into a pre-trained second convolutional neural network to obtain the final layer output as the speech feature of the phoneme data.

[0069] The second convolutional neural network is trained in advance through the following steps: obtaining a second training sample set, wherein the second training sample includes sample phoneme data and pre-obtained sample speech features. Determining the network structure of the initial second convolutional neural network and initializing the network parameters of the initial second convolutional neural network. Using a machine learning method, the sample phoneme data included in the second training sample in the second training sample set is used as the input of the second initial convolutional neural network, and the pre-obtained speech features corresponding to the input sample phoneme data are used as the expected output of the second initial convolutional neural network, and the second convolutional neural network is trained.

[0070] According to the voice features of the above-mentioned phoneme data, the voice features of the pre-stored keyword for starting the application are matched to determine whether the above-mentioned keyword instruction is passed.

[0071] Step 206 , in response to the keyword command authentication being successful, starting the application on the device.

[0072] In some embodiments, the execution body (eg, Figure 1 The electronic device shown in the figure responds to the keyword instruction authentication passing and starts the application on the device. Here, the keyword instruction authentication generally refers to the process of matching the voice features of the keyword instruction with the voice features of the pre-stored keyword instruction for starting the application.

[0073] In some optional implementations of some embodiments, the execution subject may send an alarm message to a device with a display function in response to the number of times that the target user's request to start the application fails exceeds a predetermined number, and control the device to display the alarm message. The predetermined number is usually set manually. The device with a display function includes but is not limited to a smart phone, a tablet computer, an e-book reader, a laptop computer, a desktop computer, etc. The alarm message may be a text or voice, etc.

[0074] Figure 2 An embodiment provided has the following beneficial effects: based on a request input by a target user, a first convolutional neural network is used to obtain the voice features of the target user, and the device is started by recognizing the voice features. Based on a keyword instruction in the request, phoneme data is determined, and the voice features of the phoneme data are obtained by a second convolutional neural network, and the application is started by recognizing the voice features of the phoneme data. This method can complete the tasks of starting a device and starting an application based on a request to start an application issued by the target user, with fewer operating steps and a simple process, so that the present disclosure facilitates users to start applications in devices and improves the user experience.

[0075] Continue to refer Figure 4 , Figure 4 A process 400 of another embodiment of a method for verifying whether a keyword instruction is passed according to the present disclosure is shown. The method for verifying whether a keyword instruction is passed may include the following steps:

[0076] Step 401, extracting the phoneme data of the keyword instruction.

[0077] In some embodiments, the execution body (eg, Figure 1The electronic device shown in the figure receives a keyword instruction issued by a target user, wherein the keyword instruction is a voice signal. Voice is a kind of sound, which is emitted by a person's vocal organs and has a certain grammar and meaning. The brain sends motor nerve instructions to the vocal organs to control the various muscle movements of the vocal organs to vibrate the air to form. Phonemes are the basis for humans to distinguish one word from another, and phonemes are the smallest unit of voice signals. Within the time period of 10 seconds to 25 seconds, the voice is static. Within the range of 25 seconds to 50 seconds, the output can be guaranteed to be within one phoneme. The voice signal in the keyword instruction is intercepted with a period of 25 seconds to 50 seconds, the Mel-frequency cepstral coefficients of the voice signal are calculated, and the phoneme data of the keyword instruction is generated.

[0078] Step 402: Generate speech features of phoneme data.

[0079] In some embodiments, the execution subject inputs the phoneme data of the keyword instruction into the second convolutional neural network. It can be understood that the network structure and network parameters of the second convolutional neural network are as follows.

[0080] Optionally, the second convolutional neural network includes a third feature extraction network, a first pooling layer, and a second fully connected layer. The third feature extraction network includes multiple convolution kernel filters that convolve the input phoneme data to extract multiple features, wherein each convolution kernel can extract one feature. The first pooling layer performs pooling processing on the results of the third feature extraction network to make the extracted features more compact and reduce the number of neurons. After the third feature extraction network and the first pooling layer, the extracted features are passed through the second fully connected layer to obtain a set of feature vectors, and the output feature vectors are used as the speech features of the phoneme data.

[0081] In some embodiments, the execution subject inputs the phoneme data into a pre-trained second convolutional neural network to obtain a speech feature output as the phoneme data. The second convolutional neural network may be pre-trained through the second training step.

[0082] Please refer to Figure 5 , Figure 5 A process 500 of an embodiment of a second training step of pre-training a second convolutional neural network according to the present application is shown. The second training step may include the following steps:

[0083] Step 501, determine the network structure of the second initial convolutional neural network and initialize the network parameters of the second initial convolutional neural network.

[0084] In this embodiment, the execution subject of the second training step can be the same as the execution subject of the method for starting the application (for example Figure 1The electronic device shown in the figure is the same or different. If they are the same, the execution subject of the second training step can store the network structure information of the trained second convolutional neural network and the parameter values ​​of the network parameters locally after the second convolutional neural network is trained. If they are different, the execution subject of the second training step can send the network structure information of the trained second convolutional neural network and the parameter values ​​of the network parameters to the execution subject of the method for starting the application after the second convolutional neural network is trained.

[0085] In this embodiment, the execution subject of the second training step may first determine the network structure of the initial second convolutional neural network. For example, it is necessary to determine which layers the initial second convolutional neural network includes, the connection order relationship between the layers, which neurons each layer includes, the weight and bias corresponding to each neuron, the activation function of each layer, etc.

[0086] In some optional implementations of this embodiment, since the convolutional neural network is a multi-layer neural network, each layer is composed of multiple two-dimensional planes, and each plane is composed of multiple independent neurons, it is necessary to determine which layers the initial neural network of the convolutional neural network type includes (for example, convolution layer, pooling layer, fully connected layer, classifier, etc.), the connection order relationship between layers, and which parameters each layer includes (for example, weights, bias terms, convolution step length), etc. Among them, the convolution layer can be used to extract speech features. For each convolution layer, it can be determined how many convolution kernels there are, the size of each convolution kernel, the weights of each neuron in each convolution kernel, the bias term corresponding to each convolution kernel, the step length between two adjacent convolutions, etc.

[0087] Then, the execution subject of the second training step can initialize the network parameters of the initial second convolutional neural network. In practice, the various network parameters of the second convolutional neural network (e.g., weight parameters and bias parameters) can be initialized with some different small random numbers. "Small random numbers" are used to ensure that the network will not enter a saturated state due to excessive weights, thereby causing training failure, and "different" is used to ensure that the network can learn normally.

[0088] Step 502: Obtain a second training sample set.

[0089] In this embodiment, the execution subject of the second training step can obtain the second training sample set locally or remotely from other electronic devices connected to the execution subject through a network. The second training sample includes sample phoneme data and pre-obtained sample speech features. For example, the speech features of the sample phoneme data can be manually determined.

[0090] Step 503, using a machine learning method, taking the sample phoneme data included in the second training sample in the second training sample set as the input of the second initial convolutional neural network, taking the obtained speech features corresponding to the input sample phoneme data as the expected output of the second initial convolutional neural network, and training the second convolutional neural network.

[0091] In this embodiment, the execution subject of the second training step can first use a preset loss function to calculate the difference between the speech features of the obtained phoneme data and the speech features of the phoneme data in the training sample. For example, the L2 norm can be used as the loss function to calculate the difference between the speech features of the obtained phoneme data and the speech features of the phoneme data in the training sample. Then, based on the calculated difference, the network parameters of the initial second convolutional neural network can be adjusted, and the training can be terminated when the preset training termination conditions are met. For example, the preset training termination conditions here may include but are not limited to at least one of the following: the training time exceeds the preset duration; the number of training times exceeds the preset number of times; the calculated difference is less than the preset difference threshold.

[0092] Here, various implementations may be used to adjust the network parameters of the initial first convolutional neural network based on the difference between the generated speech features and the sample speech features in the training sample. For example, the network parameters of the initial neural network may be adjusted using a BP (Back Propagation) algorithm or an SGD (Stochastic Gradient Descent) algorithm.

[0093] In this embodiment, the execution subject of the second training step determines the initial second convolutional neural network obtained by training as the pre-trained second convolutional neural network.

[0094] In some optional implementations of this embodiment, the pre-trained second convolutional neural network may include a third feature extraction network, a first pooling layer, and a second fully connected layer. Based on the above second convolutional neural network, step 402 may be performed as follows:

[0095] First, the phoneme data may be input into a pre-trained third feature extraction network to generate a third feature vector. Here, the third feature extraction network is a convolutional neural network including one convolutional layer.

[0096] Then, the third eigenvector is input into the pre-trained first pooling layer to generate a fourth eigenvector.

[0097] Afterwards, the fourth feature vector is input into the pre-trained second fully connected layer to generate the speech features of the phoneme data. Here, the second fully connected layer is used to characterize the corresponding relationship between the phoneme data and the speech features.

[0098] It should be noted that the third feature extraction network, the first pooling layer and the second fully connected layer can also be adopted as follows Figure 5 The first training step shown is obtained.

[0099] The method for starting an application provided by the above embodiment of the present application trains the second convolutional neural network by using the phoneme data and the corresponding voice features of the phoneme data as training samples, so that the trained second convolutional neural network can learn how to generate the voice features of the phoneme data. When generating the voice features, the phoneme data is input into the pre-trained second convolutional neural network to generate the voice features of the phoneme data.

[0100] Step 403: Determine whether the keyword instruction is verified based on the voice features of the phoneme data.

[0101] In some embodiments, the execution body (eg, Figure 1 The electronic device shown in the figure matches the voice feature of the keyword instruction with the voice feature of the keyword instruction pre-stored in the execution subject. If the voice feature of the keyword instruction input by the target user matches the voice feature of the keyword instruction pre-stored in the execution subject, the voice feature authentication of the keyword instruction is passed and the device is started.

[0102] Figure 4 An embodiment provided has the following beneficial effects: using a convolutional neural network to extract speech features of keyword instructions, ensuring the translation invariance of speech features in time and space, effectively dealing with the problem of speech signal diversity, improving the accuracy of feature matching, and meeting the requirements of fast and efficient recognition of keyword instructions.

[0103] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 suitable for implementing an electronic device of an embodiment of the present disclosure. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0104] like Figure 6As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. Various programs and data required for the operation of the system 600 are also stored in the RAM 603. The CPU 601, the ROM 602 and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0105] The following components are connected to the I / O interface 605: a storage section 606 including a hard disk, etc.; and a communication section 607 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 607 performs communication processing via a network such as the Internet. A drive 608 is also connected to the I / O interface 605 as needed. A removable medium 609, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 608 as needed, so that a computer program read therefrom is installed into the storage section 606 as needed.

[0106] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 607, and / or installed from the removable medium 609. When the computer program is executed by the central processing unit (CPU) 601, the above functions defined in the method of the present disclosure are executed. It should be noted that the computer-readable medium described in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0107] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0108] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A method for starting an application, comprising: receiving a request from a target user to start an application on a device, wherein the request is a voice command and the request includes a keyword command; Sending the request to a first terminal, wherein the first terminal collects user information; Receiving user information returned by the first terminal; Based on the user information, start the device; Based on the request, determining whether the keyword instruction is authenticated; In response to the keyword instruction being authenticated, starting the application on the device; Wherein, starting the device based on the user information includes: In response to the user information including voice information and physiological information, the voice information is input into a pre-trained first convolutional neural network to obtain an output as a voice feature of a target user; based on the voice feature of the target user, determining whether the voice information is authenticated; The first convolutional neural network is pre-trained by the following steps: Determine a network structure of a first initial convolutional neural network and initialize network parameters of the first initial convolutional neural network; Acquire a first training sample set, wherein the first training sample includes sample sound information and pre-obtained sample speech features; Using a machine learning method, taking the sample sound information included in the first training sample in the first training sample set as the input of the first initial convolutional neural network, taking the pre-obtained sample speech features corresponding to the input sample sound information as the expected output of the first initial convolutional neural network, and training the first convolutional neural network; The L2 norm is used as a loss function to calculate the difference between the speech features output by the first initial convolutional neural network and the sample speech features in the first training sample. Based on the calculated difference, the network parameters of the first initial convolutional neural network are adjusted, and the training is terminated when a preset training termination condition is met. The preset training termination condition includes at least one of the following: the training time exceeds a preset duration; the number of training times exceeds a preset number of times; the calculated difference is less than a preset difference threshold; Wherein, the first convolutional neural network includes a first feature extraction network, a second feature extraction network and a first fully connected layer; The step of inputting the sound information into a pre-trained first convolutional neural network to obtain an output as a speech feature of a target user includes: Inputting the sound information into a pre-trained first feature extraction network to obtain a first feature vector; Inputting the first feature vector into a pre-trained second feature extraction network to obtain a second feature vector; Inputting the second feature vector into a pre-trained first fully connected layer to generate a speech feature of the target user; Wherein, starting the device based on the user information further includes: In response to the user information including voice information and physiological information, determining whether the physiological information is authenticated based on the physiological information, the physiological information including at least one of the following: pulse, heartbeat, palm print, fingerprint feature, skin dryness, facial feature, and iris information of the target user; In response to the voice information authentication being passed and the physiological information authentication being passed, starting the device; Wherein, determining whether the keyword instruction is authenticated based on the request includes: Based on the keyword instruction, determining the phoneme data of the keyword instruction, wherein the start and end points of the keyword instruction are accurately located from the request using a voice endpoint detection technology to extract the phoneme data in the keyword instruction; Inputting the phoneme data into a pre-trained second convolutional neural network to obtain output as speech features of the phoneme data; Determining whether the keyword instruction is authenticated based on the voice features of the phoneme data; The second convolutional neural network is pre-trained by the following steps: Determine a network structure of a second initial convolutional neural network and initialize network parameters of the second initial convolutional neural network; Acquire a second training sample set, wherein the second training sample includes sample phoneme data and pre-obtained sample speech features; Using a machine learning method, taking the sample phoneme data included in the second training sample in the second training sample set as the input of the second initial convolutional neural network, taking the pre-obtained sample speech features corresponding to the input sample phoneme data as the expected output of the second initial convolutional neural network, and training the second convolutional neural network; The second convolutional neural network includes a third feature extraction network, a first pooling layer, and a second fully connected layer; and The step of inputting the phoneme data into a pre-trained second convolutional neural network to obtain output as speech features of the phoneme data includes: Inputting the phoneme data into a pre-trained third feature extraction network to obtain a third feature vector, wherein a plurality of convolution kernel filters included in the third feature extraction network convolve the input phoneme data to extract a plurality of features, wherein each convolution kernel can extract a feature; Inputting the third feature vector into a pre-trained first pooling layer to obtain a fourth feature vector, wherein the first pooling layer performs pooling processing on the result of the third feature extraction network to make the extracted features more compact and reduce the number of neurons; The fourth feature vector is input into the pre-trained second fully connected layer to generate speech features of the phoneme data, wherein the features extracted by the third feature extraction network and the first pooling layer are input into the second fully connected layer to obtain a set of feature vectors, and the output feature vectors are used as the speech features of the phoneme data.

2. The method according to claim 1, wherein: The method further comprises: In response to the target user's request to start the application failing to be authenticated a number of times exceeding a predetermined number, an alarm message is sent to a device with a display function, and the device with a display function is controlled to display the alarm message.

3. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-2.

Images