Voice data transmission method and device based on SIM card, equipment and storage medium
By using SIM cards to establish a transmission channel with the gateway in the terminal, and using the target encoder and decoder to process voice data, the problem of poor voice call quality in cross-border communications is solved, and high-quality voice data transmission and translation effects are achieved.
Patent Information
- Application Number
- CN202411276289.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-06
AI Technical Summary
In cross-border communication, due to different languages, users use translation software to communicate, the quality of wireless network voice calls in mobile state is poor, making it difficult to achieve high-quality data transmission.
By using a SIM card in the terminal, an identity authentication message is sent to the gateway and a transmission channel is established. After receiving the authentication pass message, an identification is sent to the counterpart device to establish a data interaction channel, and the voice data is used by the target encoder and decoder. Perform translation and prediction to ensure the quality of voice data transmission.
It improves the quality of voice calls, ensures the transmission quality of voice data, reduces translation errors, and improves user experience and data security.
Smart Images

Figure CN119095039A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of communication technology, and in particular, relates to a voice data transmission method, device, equipment and storage medium based on a SIM card. Background Art
[0002] Currently, in cross-border communications, since users use different languages, they need to use translation software to communicate.
[0003] When users use translation software for cross-border communication, in order to ensure that a communication link is established between communication devices and realize data transmission, the user's voice information is usually collected in real time through a wireless network. However, when the user is in a mobile state and makes a real-time voice call through a wireless network, the call quality is poor. Summary of the invention
[0004] The embodiments of the present application provide a SIM card-based voice data transmission method, apparatus, device, storage medium, and program product, which can ensure the transmission quality of voice data and thus improve the voice call quality.
[0005] In a first aspect, an embodiment of the present application provides a SIM card-based voice data transmission method, which is applied to a subscriber identity module SIM card in a terminal, and the method includes:
[0006] Sending an identity authentication message to the gateway, and establishing a transmission channel between the SIM card and the gateway when the gateway successfully authenticates the terminal identity;
[0007] Receiving the authentication success message sent by the gateway;
[0008] The peer identifier is sent to the gateway so that the gateway can use the peer identifier to verify the identity of the peer device. After the identity of the peer device is verified, a transmission channel between the gateway and the SIM card of the peer device is established.
[0009] In a possible implementation manner, before sending the identity authentication message to the gateway, the method further includes:
[0010] Sending a discovery message to the gateway, wherein the discovery message is used to instruct the gateway to determine a protocol version and an encryption suite for information interaction with the SIM card, wherein the discovery message includes information about the protocol version and encryption suite supported by the SIM card;
[0011] receiving a response message sent by the gateway, wherein the response message carries a target protocol version and a target encryption suite determined by the gateway from the discovery message;
[0012] In a possible implementation, the identity authentication message includes key information, and the identity authentication message is used to instruct the gateway to obtain a key corresponding to the key information from a key management platform, so that the gateway uses the key to encrypt and decrypt interactive data with the SIM card.
[0013] In a possible implementation manner, the SIM card includes a target encoder and a target decoder; after sending the peer identifier to the gateway, the method further includes:
[0014] Receiving the voice information to be processed sent by the SIM card of the opposite device;
[0015] When the language of the speech information to be processed is different from the preset language, obtaining, by the target encoder, an original speech vector corresponding to the speech information to be processed and a target speech vector of the preset language;
[0016] Extracting the features of the target speech vector using a preset algorithm to obtain a target speech representation;
[0017] Cascading the target speech representation and the original speech vector to obtain a target encoding vector;
[0018] The target encoding vector is input into the target decoder, and the target decoder predicts the missing speech information in the original speech vector using the target speech representation, and outputs the translated speech information corresponding to the preset language.
[0019] In a possible implementation, extracting the features of the target speech vector using a preset algorithm to obtain the target speech representation includes:
[0020] Normalizing the product of the original speech vector and the target speech vector to obtain a scalar score of the original speech vector, wherein the scalar score is used to represent the probability that each vector in the original speech vector belongs to each character;
[0021] The product of the scalar score and the target speech vector is summed to obtain the target speech representation.
[0022] In a possible implementation, the step of concatenating the target speech representation and the original speech vector to obtain a target encoding vector includes:
[0023] The target encoding vector is calculated according to the following formula:
[0024]
[0025] in, represents the target encoding vector, R represents a natural number, and h represents the number of neurons of the target encoder; represents the original speech vector, represents the target speech representation.
[0026] In a possible implementation, the inputting the target encoding vector into the target decoder, the target decoder predicting the missing speech information in the original speech vector using the target speech representation, and outputting the translated speech information corresponding to the preset language, includes:
[0027] Accumulating and summing the scalar scores corresponding to each vector in the target encoding vector to obtain the index of each vector;
[0028] Obtaining the index of each word vector in the target speech corresponding to the preset language;
[0029] According to the index of each vector in the target encoding vector and the index of each word vector in the target speech, the target encoding vector and each word vector in the target speech are matched to obtain a matching result;
[0030] The target decoder is used to predict the missing speech information in the original speech vector according to the matching result to obtain the translated speech information corresponding to the preset language.
[0031] In a possible implementation, matching the target encoding vector with each word vector in the target speech according to the index of each vector in the target encoding vector and the index of each word vector in the target speech to obtain a matching result includes:
[0032] The matching result is obtained according to the following formula:
[0033]
[0034] in, represents the index of each vector in the target encoding vector, W ik represents the index of each word vector in the target speech, d LA Indicates the number of vectors driven forward, d LB represents the number of vectors driven backward, and F represents the size of the matching window when matching the index of each vector in the target encoding vector.
[0035] In a possible implementation, before obtaining the original speech vector corresponding to the speech information to be processed and the target speech vector of the preset language through the target encoder, the method further includes:
[0036] Acquire sample voice information corresponding to the language of the voice information to be processed;
[0037] The first encoder is obtained by training using first training data, wherein the first training data includes the sample speech information and a label speech vector corresponding to the sample speech information, and the first encoder is used to obtain the original speech vector;
[0038] Acquire sample target speech information corresponding to the preset language;
[0039] The second encoder is obtained by training with second training data, wherein the second training data includes the sample target speech information and a label speech vector corresponding to the sample target speech information, and the second encoder is used to obtain the target speech vector.
[0040] In a possible implementation, before inputting the target encoding vector into a target decoder, the method further includes:
[0041] Inputting the sample character vector sequence of the preset language into a decoder;
[0042] Obtaining a target character vector output by the encoder that is predicted based on the character vector sequence;
[0043] The model parameters of the decoder are adjusted using the label character vector and the target character vector to obtain the target decoder.
[0044] In a second aspect, an embodiment of the present application provides a SIM card-based voice data transmission device, which is applied to a subscriber identity module SIM card in a terminal, and the device includes:
[0045] A sending module, used for sending an identity authentication message to a gateway, and establishing a transmission channel between the SIM card and the gateway when the gateway successfully authenticates the terminal identity;
[0046] A receiving module, used for receiving the authentication pass message sent by the gateway;
[0047] The sending module is further used to send a peer identifier to the gateway, so that the gateway uses the peer identifier to verify the identity of the peer device, and after the identity of the peer device is verified, a transmission channel between the gateway and the SIM card of the peer device is established.
[0048] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions;
[0049] When the processor executes the computer program instructions, the method according to any one of the first aspects is implemented.
[0050] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in any one of the first aspects is implemented.
[0051] In a fifth aspect, an embodiment of the present application provides a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes a method as described in any one of the first aspects.
[0052] In the SIM card-based voice data transmission method, apparatus, device, storage medium and program product of the embodiment of the present application, the SIM card sends an identity authentication message to the gateway, and when the gateway passes the identity authentication of the SIM card, a transmission channel between the SIM card and the gateway is established to realize data interaction between the SIM card and the gateway. Then, after receiving the authentication pass message sent by the gateway, the SIM card sends the peer identification to the gateway so that the gateway performs identity authentication on the peer device, and after the authentication passes, a transmission channel between the gateway and the SIM card of the peer device is established to realize data interaction between the gateway and the SIM card of the peer device. In this way, when the SIM card of the peer device and the SIM card are performing data interaction, the collected voice data can be transmitted through the established transmission channel, thereby ensuring the transmission quality of the voice data and thereby improving the quality of the voice call. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0054] Figure 1 It is a flowchart of a voice data transmission method based on a SIM card provided by an embodiment of the present application;
[0055] Figure 2 It is a flowchart of a method for translating speech data provided by an embodiment of the present application;
[0056] Figure 3 is a flowchart of a method for decoding a target code vector provided by an embodiment of the present application;
[0057] Figure 4 is an exemplary schematic diagram of a method for establishing a talk group provided by an embodiment of the present application;
[0058] Figure 5 is an exemplary schematic diagram of a method for joining a talk group provided by an embodiment of the present application;
[0059] Figure 6 is an exemplary schematic diagram of a voice information translation method provided by an embodiment of the present application;
[0060] Figure 7 is an exemplary schematic diagram of a model translation method provided by an embodiment of the present application;
[0061] Figure 8 is an exemplary schematic diagram of a vector matching method provided by an embodiment of the present application;
[0062] Fig. 9 is a structural diagram of a SIM card-based voice data transmission device provided by another embodiment of the present application;
[0063] Fig.10 It is a structural diagram of an electronic device provided by yet another embodiment of the present application. DETAILED DESCRIPTION
[0064] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0065] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0066] In order to solve the problems of the prior art, the embodiments of the present application provide a SIM card-based voice data transmission method, device, equipment, storage medium and program product. The following first introduces the SIM card-based voice data transmission method provided by the embodiments of the present application.
[0067] Figure 1The flowchart of a voice data transmission method based on a SIM card provided by an embodiment of the present application is shown. The method is applied to the subscriber identity module (SIM) in the terminal, and a call group of the calling end and the called end is established in the gateway so that members in the call group can make voice calls. Figure 1 As shown, the method includes:
[0068] S101, sending an identity authentication message to a gateway, and establishing a transmission channel between the SIM card and the gateway when the gateway successfully authenticates the terminal identity.
[0069] Before the terminal sends an authentication message to the gateway, the terminal sends a discovery message to the gateway. After the gateway feeds back a response message to the terminal, it indicates that the gateway can establish a connection with the terminal.
[0070] The terminal first sends a discovery message to the gateway, and after the gateway feeds back a response message to the terminal, the terminal can send an authentication message to the gateway. The authentication message includes the key information corresponding to the terminal, and the authentication message is used to instruct the gateway to obtain the key corresponding to the key information from the key management platform, so that the gateway can use the key to encrypt and decrypt the interactive data with the SIM card.
[0071] Specifically, the identity authentication message also includes the terminal identifier. The gateway uses the terminal identifier to search for the corresponding pre-stored key on the key management platform and uses the pre-stored key to verify the key information. If the verification is successful, the gateway successfully authenticates the terminal identity. The key management platform stores the corresponding relationship between the terminal identifier and the pre-stored key.
[0072] S102: Receive an authentication success message sent by the gateway.
[0073] S103: Send the peer identifier to the gateway, so that the gateway can use the peer identifier to verify the identity of the peer device. After the identity of the peer device is verified, a transmission channel between the gateway and the SIM card of the peer device is established.
[0074] The above-mentioned identity authentication message also includes the terminal identification, and the gateway can record the peer identification corresponding to the peer device, so that the gateway can establish a talk group including the peer identification and the terminal identification. After the talk group is established, each member in the talk group can have a real-time voice call.
[0075] In one implementation, after receiving the peer identifier, the gateway sends the peer identifier to a third-party verification platform, and the third-party verification platform sends verification data to the gateway. The gateway verifies the legitimacy of the peer identifier based on the verification data. The peer identifier is a mobile phone number, and the verification data includes whether the peer identifier is valid and whether the peer identifier is in a shutdown state.
[0076] In another implementation, after receiving the peer identifier, the gateway verifies whether the peer identifier is still in use to ensure that the peer identifier is in a non-shutdown state. The peer identifier is a mobile phone number.
[0077] After the gateway passes the identity verification of the peer device, it sends a link-building SMS encapsulated with the peer ID to the SMS gateway. The SMS gateway forwards the link-building SMS to the SIM card of the peer device. The link-building SMS encapsulates the public network address of the gateway. After receiving the link-building SMS, the SIM card of the peer device can establish a transmission channel with the gateway based on the public network address in the link-building SMS.
[0078] Using the above method, the SIM card sends an identity authentication message to the gateway, and when the gateway passes the identity authentication of the SIM card, a transmission channel between the SIM card and the gateway is established to realize data interaction between the SIM card and the gateway. Then, after receiving the authentication pass message sent by the gateway, the SIM card sends the peer ID to the gateway, so that the gateway performs identity authentication on the SIM card corresponding to the peer ID, and after the authentication passes, a transmission channel between the gateway and the SIM card of the peer device is established to realize data interaction between the gateway and the SIM card of the peer device. In this way, when the SIM card of the peer device and the SIM card are performing data interaction, the collected voice data can be transmitted through the established transmission channel, thereby ensuring the transmission quality of the voice data and improving the quality of the voice call.
[0079] In some embodiments of the present application, before the above S101, sending the identity authentication message to the gateway, the SIM card and the gateway negotiate the communication protocol and the encryption suite, so that the negotiated communication protocol and the encryption suite are used for data interaction in the subsequent interaction process between the SIM card and the gateway. Based on this, before the above S101, sending the identity authentication message to the gateway, the method also includes:
[0080] Send a discovery message to the gateway and receive a response message from the gateway.
[0081] The discovery message is used to instruct the gateway to determine the protocol version and encryption suite for information exchange with the SIM card, and the discovery message includes the protocol version information and encryption suite information supported by the SIM card. The response message carries the target protocol version and target encryption suite determined by the gateway from the discovery message.
[0082] According to the method provided in the embodiment of the present application, before the SIM card sends the identity authentication message to the gateway, the SIM card sends the discovery message to the gateway, so that the gateway and the SIM card negotiate the protocol version and encryption suite of the communication protocol for the subsequent message transmission. The discovery message includes the protocol version information and encryption suite information supported by the SIM card. The gateway can select the corresponding target protocol version and target encryption suite from the protocol version information and encryption suite information carried in the discovery message, thereby ensuring that both the gateway and the SIM card can obtain the protocol version and encryption suite for data interaction, thereby ensuring the security of data interaction.
[0083] It should be noted that after the gateway establishes a talk group, other terminals can also join the talk group. Specifically, the talk group can be joined in the following manner:
[0084] After the gateway receives the joining request sent by the terminal to be processed, if the identity authentication of the terminal to be processed is passed, the gateway sends a query request to the terminal. After the terminal receives the query request sent by the gateway, it queries the terminal identification of the terminal to be processed carried in the query request in the local storage space to obtain the query result.
[0085] The joining request includes the terminal identification of the pending terminal and the terminal identification of the talk group member; the joining request is forwarded to the gateway by the SMS gateway after receiving the joining request sent by the pending terminal. The query request is generated by the gateway after receiving the terminal identification sent by the pending terminal. The pending terminal is the terminal requesting to join the talk group.
[0086] Specifically, the gateway determines the object to receive the query request based on the terminal identifier of the talk group member in the join request.
[0087] The terminal searches the local address book for the terminal identifier of the terminal to be processed. The terminal identifier may be a mobile phone number.
[0088] The terminal feeds back the query result to the gateway, so that the gateway can judge whether to add the terminal identifier of the terminal to be processed into the talk group according to the query result.
[0089] It should be noted that, when the query result includes the terminal identification of the terminal to be processed, the gateway adds the terminal to be processed to the talk group; when the query result does not include the terminal identification of the terminal to be processed, the gateway refuses to add the terminal to the talk group.
[0090] On the basis of establishing the talk group as described above, when other terminals request to join the talk, in order to avoid joining the wrong talk group, there are the following two implementation methods:
[0091] Implementation method 1: After the call ends, the gateway can delete the locally stored talk group to avoid the situation where multiple talk groups in the gateway contain the same terminal ID, thereby preventing other terminals from joining the wrong talk group when joining the talk group.
[0092] Implementation method 2: When establishing a talk group, the gateway can record the establishment time of each talk group. When other terminals request to join the talk group, the gateway determines the latest talk group containing the terminal identifier stored locally based on the terminal identifier carried in the joining request, thereby preventing other terminals from joining the wrong talk group.
[0093] After the gateway establishes a talk group with terminal ID and peer ID, members in the talk group can have real-time voice calls. After the terminal receives the pending voice information sent by other members in the talk group, it can determine whether the pending voice information needs to be translated according to the language ID corresponding to the pending voice information.
[0094] During a voice call, for the terminal sending voice information, there is a dynamically expandable memory in the SIM card of the terminal for storing voice information. The SIM card monitors whether there is incremental information in the memory at a preset time. If there is, it means that the user is still outputting voice information. If not, it means that the user has stopped outputting voice information. The SIM card can send the voice information stored in the memory to the gateway, and then the gateway forwards the voice information to other members in the call group.
[0095] When the above gateway establishes a talk group, members in the talk group can make voice calls. That is, the above SIM card can receive the pending voice information sent by the SIM card of the other end device in the talk group, and translate the received pending voice information. Figure 2 The process of translating the speech information to be processed is introduced, wherein the SIM card includes a target encoder and a target decoder. Figure 2 As shown, after the above S103, sending the peer identifier to the gateway, the method further includes:
[0096] S201: Receive voice information to be processed sent by a SIM card of a peer device.
[0097] S202. When the language of the speech information to be processed is different from the preset language, an original speech vector corresponding to the speech information to be processed and a target speech vector of the preset language are obtained through a target encoder.
[0098] The voice information to be processed is the voice information sent by the peer device, the preset language is the language pre-set locally in the terminal, and the target voice information corresponding to the target voice vector is determined based on the language package of the preset language downloaded from the third-party platform.
[0099] Specifically, the target speech vector is obtained by encoding any sentence in a preset language using a target encoder.
[0100] Before the target encoder obtains the original speech vector corresponding to the speech information to be processed and the target speech vector of the preset language, the speech information to be processed and the target speech information are subjected to word segmentation processing. Then, the speech information to be processed after the word segmentation processing is encoded by the target encoder to obtain a vector sequence of the speech information to be processed, wherein each parameter in the vector sequence corresponds to a word segmentation processing result. Similarly, the target speech information after the word segmentation processing is encoded by the target encoder to obtain a vector sequence of the target speech information, wherein each parameter in the vector sequence corresponds to a word segmentation processing result.
[0101] S203: extracting features of the target speech vector using a preset algorithm to obtain a target speech representation.
[0102] S204: Cascade the target speech representation and the original speech vector to obtain a target encoding vector.
[0103] It can be understood that by cascading the target speech identifier and the original speech vector, the language features of the preset language can be added to the original speech vector, so that the subsequent target decoder can predict the missing speech information in the original speech vector with the target encoding vector, thereby completing the missing information in the speech information to be processed.
[0104] S205: Input the target encoding vector into a target decoder. The target decoder predicts the missing speech information in the original speech vector using the target speech representation, and outputs the translated speech information corresponding to the preset language.
[0105] Using the above method, after obtaining the original speech vector of the speech data to be processed and the target speech vector of the preset language, the terminal uses a preset algorithm to extract the features of the target speech vector to obtain the target speech representation. In this way, the target speech representation can carry the language features of the preset language. Then the target speech representation and the original speech vector are cascaded to obtain a target coding vector, which carries the complete language features of the preset language. After the target coding vector is subsequently input into the target decoder, the target decoder can predict the missing speech data in the original speech vector based on the complete language features of the preset language carried in the target speech representation, thereby completing the missing original speech vector. Finally, the target decoder can accurately output the complete translated speech data corresponding to the preset language, thereby improving the accuracy of the translated speech data.
[0106] After receiving the voice message, other members in the call group can compare the language of the received voice message to be processed with the local preset language to determine whether to translate the voice message to be processed. Specifically, after receiving the voice message to be processed sent by the SIM card of the opposite device in S201, it can be implemented as follows:
[0107] Step A: obtaining the language identifier corresponding to the voice information to be processed.
[0108] Step B: determining whether the language identifier is consistent with the language identifier corresponding to the preset language.
[0109] Step C: when the language identifier is inconsistent with the language identifier corresponding to the preset language, the speech information to be processed is input into the target encoder.
[0110] By adopting the method provided in the embodiment of the present application, after obtaining the voice information to be processed, the language identifier corresponding to the voice information to be processed is determined. Among them, the terminal is pre-set with a language representation of a preset language, and the preset language is the language used by the terminal user. Therefore, when the language representation corresponding to the voice information to be processed and the language identifier corresponding to the preset language are different, it means that the user using the terminal cannot understand the voice information to be processed. Therefore, it is necessary to translate the voice information to be processed into voice information corresponding to the preset language for the user to understand. In this way, by comparing the language identifiers, the translation of all voice information received by the terminal is avoided, thereby reducing the workload of the SIM card.
[0111] In some embodiments of the present application, before using the target encoder and the target decoder to process the speech information to be processed, the encoder and the decoder are pre-trained to obtain the target encoder and the target decoder. The following describes how to train and obtain the target encoder.
[0112] The target encoder includes a first encoder and a second encoder. The first encoder is used to process the speech information to be processed to obtain the original speech vector. The specific training process is as follows:
[0113] Sample speech information corresponding to the language of the speech information to be processed is obtained. A first encoder is obtained by training using the first training data.
[0114] The first training data includes sample speech information and a label speech vector corresponding to the sample speech information.
[0115] The second encoder is used to process the target speech information to obtain the target speech vector. The specific training process is as follows:
[0116] Get the sample target speech information corresponding to the preset language.
[0117] The second encoder is trained using the second training data.
[0118] The second training data includes sample target speech information and a label speech vector corresponding to the sample target speech information.
[0119] It should be noted that since the first encoder is used to encode the voice information to be processed and the second encoder is used to encode the target voice information, and the target voice information and the voice information to be processed are in different languages, the encoding rules for encoding the voice information to be processed and the target voice information are different, and the encoders are trained for voice information in different preset languages.
[0120] In this way, the first encoder is trained using sample speech information corresponding to the language of the speech information to be processed, and the second encoder is trained using sample target speech information corresponding to a preset language. This allows the target encoder to accurately encode speech information in different languages at the same time, thereby ensuring the accuracy of the encoder output results.
[0121] The target decoder is used to predict the missing speech information in the original speech vector using the target speech representation. The specific training process is as follows:
[0122] Step 1: Input the sample character vector sequence of the preset language into the decoder.
[0123] The sample character vector is obtained by performing word segmentation processing on the speech information of a preset language and using vectorized representation for the word segmentation processing.
[0124] Step 2: Obtain the target character vector output by the encoder based on the prediction of the sample character vector sequence.
[0125] The target decoder is a character-level autoregressive language model that can predict the next character in a vector sequence using all characters and audio context in the sample character vector sequence. Specifically, the following formula is used to predict the next character in the sequence:
[0126]
[0127] Among them, c k is the next character predicted, O AV is a vector of all the characters in the sample character vector sequence, represents the vector of all characters before the kth character, and F is the predicted character.
[0128] Step 3: Use the label character vector and the target character vector to adjust the model parameters of the decoder to obtain the target decoder.
[0129] Specifically, after the target decoder is trained using the above method, the audio output of the decoder can be expressed as:
[0130]
[0131] Among them, Y represents the input vector sequence, c k Represents the predicted character.
[0132] By adopting the method provided in the embodiment of the present application, the encoder is trained using a sequence of sample character vectors of a preset language, and then the model parameters of the decoder are adjusted using the label character vector. The obtained target decoder can predict the missing information in the speech information to be processed, thereby completing the speech information to be processed and obtaining the complete translated speech information. The accuracy of the translated speech information is improved.
[0133] In some embodiments of the present application, on the basis of training the first encoder and the second encoder, the original speech vector can be obtained through the first encoder, and the target speech vector can be obtained through the second encoder.
[0134] In one example, after the original voice sentence corresponding to the voice information to be processed is segmented, the segmentation result is converted into a vector form to obtain the input of the first encoder, specifically represented by A=[a 1 ,a 2 , ..., a N ], use the vector sequence A input by the first encoder to encode and obtain the original speech vector. Specifically, the encoding process is expressed according to the following formula:
[0135] O A =Encode(A,e LB ,e LA );
[0136] Among them, O A represents the original speech vector, e LB Indicates the number of rear-drive nodes, e LA Number of predecessor nodes.
[0137] The terminal obtains the target speech V of the preset language 1 ,v 2 , ..., v M ], the target speech is encoded by the second encoder to obtain the target speech vector. Specifically, the encoding process is expressed according to the following formula:
[0138] O V =Encode(V,e LB ,e LA );
[0139] Among them, O V Represents the target speech vector.
[0140] Thus, since the language of the target speech is different from the language of the speech information to be processed, the coding rules for encoding the target speech are different from the coding rules for encoding the speech information to be processed. Thus, using encoders corresponding to different languages to encode the speech information can improve the accuracy of encoding.
[0141] Based on the target encoder and target decoder obtained by the above training, feature extraction is performed on the target speech vector output by the target encoder to obtain the target speech representation, which can be specifically implemented as follows:
[0142] The product of the original speech vector and the target speech vector is normalized to obtain a scalar score of the original speech vector. The product of the scalar score and the target speech vector is summed to obtain the target speech representation.
[0143] Among them, the scalar score is used to represent the probability that each vector in the original speech vector belongs to each character.
[0144] Specifically, the scalar score of the original speech vector is calculated according to the following formula:
[0145]
[0146] Among them, a ij represents a scalar fraction, represents the original speech vector, Represents the target speech vector.
[0147] The target speech representation is calculated according to the following formula:
[0148]
[0149] in, represents the target speech representation.
[0150] By adopting the method provided in the embodiment of the present application, the original speech vector and the target speech vector are normalized, and a scalar score of the original speech vector can be calculated, wherein the scalar score can be used to measure the probability of the corresponding character in the original speech vector, thereby associating the characters in the original speech vector and the target speech vector, that is, the original speech vector is represented by the speech features of a preset language. Therefore, after summing the product of the scalar score and the target speech vector, the target speech representation is obtained, which can represent the language features of the preset language, so that the subsequent target decoder can use the language features of the preset language to predict and complete the speech information to be processed, thereby improving the accuracy of the translated speech information.
[0151] With respect to the above S204, the target speech representation and the original speech vector are concatenated to obtain a target coding vector. Specifically, the target speech representation and the original speech vector can be concatenated according to the following formula:
[0152]
[0153] in, represents the target encoding vector, R represents a natural number, and h represents the number of neurons in the original speech encoder; represents the original speech vector, represents the target speech representation.
[0154] In this way, the original speech vector and the target speech vector corresponding to each original speech vector are aligned.
[0155] After the target speech representation and the original speech vector are concatenated to obtain the target encoding vector, the target encoding vector is translated using the target decoder to obtain the translated speech data, such as Figure 3 As shown, it can be implemented as follows:
[0156] S301. Accumulate and sum the scalar scores corresponding to each vector in the target coding vector to obtain the index of each vector.
[0157] Specifically, the index of each vector is calculated according to the following formula:
[0158]
[0159] in, Represents the index of the vector, a ij Denotes the scalar score corresponding to each vector.
[0160] S302: Obtain the index of each word vector in the target speech corresponding to the preset language.
[0161] The index of each word vector in the target speech is the word index assigned to each word using the SPACE tag.
[0162] Specifically, the index of each word vector in the target speech can be expressed as:
[0163]
[0164] Among them, W k Represents the index of the word vector, y j Represents different word vectors.
[0165] S303. According to the index of each vector in the target coding vector and the index of each word vector in the target speech, the target coding vector and each word vector in the target speech are matched to obtain a matching result.
[0166] Among them, the matching window length of the target decoder is calculated through the preset F function, the contextual semantic features of the current vector to be matched are obtained according to the matching window, and matched with each word vector in the target speech to obtain the matching result.
[0167] Specifically, the F function is shown in the following formula:
[0168]
[0169] in, represents the index of each vector in the target encoding vector, W ik represents the index of each word vector in the target speech, d LA Indicates the number of vectors driven forward, d LB represents the number of vectors driven backward, and F represents the size of the matching window when matching the index of each vector in the target encoding vector.
[0170] In this way, when the terminal matches the target coding vector and each word vector in the target speech, it comprehensively considers the number of vectors of the predecessor frame and the successor frame of the vector currently being matched, that is, considers the contextual semantic features of the vector currently being matched, and can output the translated speech data more accurately.
[0171] S304: using the target decoder to predict the missing speech information in the original speech vector according to the matching result, and obtaining the translated speech information corresponding to the preset language.
[0172] By adopting the method provided in the embodiment of the present application, the index of each vector in the target coding vector is calculated, and the index of each vector calculated is matched with the index of each vector in the target speech, thereby obtaining a matching result, so that the speech information to be processed can be translated into speech information of a preset language. Moreover, the decoder can supplement the missing speech information in the original speech vector according to the matching result and the target speech information of the preset language, thereby obtaining complete translated speech information, thereby improving the accuracy of the translated speech information.
[0173] The following combination Figure 4 The voice data transmission method based on SIM card provided in the embodiment of the present application is introduced. Figure 4 As shown, Figure 4The process of establishing a talk group of SIM card 1 and SIM card 2 by the gateway is exemplified. The voice data processing system includes SIM card 1 in terminal 1, SIM card 2 in terminal 2, SIM card 3 in terminal 3, a gateway, a trusted service management platform (Trusted Service Manager, TSM), and a key management platform (Key Management Service, KMS). The specific interaction process includes:
[0174] S401, SIM card 1 requests to talk with SIM card 2.
[0175] The user initiates a voice call on the terminal 1, and the SIM card 1 initiates a call with the SIM card 2 in response to the mobile phone number corresponding to the SIM card 2 input by the user.
[0176] S402: SIM card 1 sends a link establishment request to the gateway.
[0177] The gateway may be a Bearer Independent Protocol (BIP) gateway.
[0178] S403: The gateway and SIM card 1 establish a TCP connection.
[0179] Wherein, based on the TCP connection, the gateway and the SIM card 1 can perform data exchange.
[0180] S404. SIM card 1 sends a client hello message to the gateway.
[0181] The client hello message is used for the terminal and the gateway to negotiate and establish a secure connection. The client hello message may include information such as the protocol version, random number, encryption suite, etc. The client hello message is the discovery message in the above embodiment.
[0182] S405 . The gateway sends a server hello message and a server hello down message to SIM card 1 .
[0183] The server hello message is used to carry the protocol version and encryption suite information selected by the gateway from the protocol version and encryption suite information provided by the client hello message. The server hello down message is used to indicate the end of the current interaction process so that the terminal can initiate the next interaction process. The server hello message is the response message in the above embodiment.
[0184] S406. SIM card 1 sends a client key exchange message to the gateway.
[0185] The client key exchange message is used to carry the encrypted key determined by the terminal. The client key exchange message is the identity authentication message in the above embodiment.
[0186] S407: The gateway queries key information.
[0187] S408: The gateway and the KMS platform establish an http key negotiation interface.
[0188] After the gateway and the KMS platform establish an http key negotiation interface, the gateway can query the key information from the KMS platform, and use the queried key information to encrypt and decrypt the data exchanged between the terminal and the gateway.
[0189] S409: Calculate the verify Data interface.
[0190] The verify Data interface is used to implement the KMS platform's verification of the integrity of the received information.
[0191] S410: Negotiate a data decryption interface.
[0192] Among them, the data decryption interface is used to ensure data security and accessibility during the data decryption process.
[0193] S411. Verify the MAC interface.
[0194] The client key exchange message also carries the Media Access Control (MAC) address of the SIM card 1 , and the gateway can verify the MAC address of the SIM card 1 based on the KMS platform.
[0195] S412: Calculate the verify Data interface.
[0196] The verify Data interface is used to implement the gateway's verification of the integrity of the received information.
[0197] S413: Generate a MAC interface.
[0198] When the KMS platform feeds back the query result to the gateway, the MAC address of the KMS is encapsulated in the query result.
[0199] S414: Negotiate a data encryption interface.
[0200] Among them, the data decryption interface is used to ensure data security and accessibility during the data encryption process.
[0201] S415. The gateway sends a finished message to SIM card 1.
[0202] The finished message is used to indicate that the gateway has successfully authenticated SIM card 1.
[0203] S416: SIM card 1 requests to talk with SIM card 2.
[0204] SIM card 1 sends a call request to the gateway, and the call request carries the mobile phone number corresponding to SIM card 1 and the mobile phone number corresponding to SIM card 2.
[0205] S417, query the information of SIM card 2 and verify its legitimacy.
[0206] The gateway obtains the information of SIM card 2 through the interface established by the above negotiation and verifies the legitimacy.
[0207] S418, encapsulate SIM card 2 and create a link SMS.
[0208] S419: The gateway sends a link establishment SMS to the SMS gateway.
[0209] S420: The SMS gateway forwards the link establishment SMS to SIM card 2.
[0210] S421, SIM card 2 processes the link establishment SMS.
[0211] S422. SIM card 2 feeds back a response to the SMS gateway.
[0212] S423: The SMS gateway forwards the feedback response to the gateway.
[0213] S424: The gateway and SIM card 2 establish a connection.
[0214] S425: The gateway establishes a talk group.
[0215] On the basis of the above gateway establishing a talk group including SIM card 1 and SIM card 2, other terminal devices can request to join the talk group. Taking SIM card 3 requesting to join the talk group as an example, the method for other terminal devices to join the talk group is introduced. Figure 5 As shown:
[0216] S501, SIM card 3 requests to join a talk group.
[0217] The user enters the mobile phone number corresponding to the SIM card 1 on the terminal 3, and the SIM card 3 sends a joining request to the SMS gateway in response to the information entered by the user.
[0218] S502. SIM card 3 sends a join request to the SMS gateway.
[0219] Among them, the joining request includes the mobile phone number corresponding to SIM card 1 and the mobile phone number corresponding to SIM card 3. The terminal device corresponding to SIM card 3 is the above-mentioned terminal to be processed, and the mobile phone number corresponding to SIM card 3 is the terminal identification of the above-mentioned terminal to be processed.
[0220] S503: The SMS gateway forwards the join request to the gateway.
[0221] S504: The gateway parses the joining request.
[0222] S505: The gateway verifies the legitimacy of SIM card 3.
[0223] Specifically, the method for verifying the legitimacy is described in the above embodiments and will not be repeated here.
[0224] S506: The gateway sends a query request to SIM card 1.
[0225] The query request includes the mobile phone number corresponding to SIM card 3.
[0226] S507, SIM card 1 queries the address book to determine whether there is a friend.
[0227] S508. SIM card 1 feeds back the query result to the gateway.
[0228] S509: If joining is not allowed, the gateway notifies SIM card 3.
[0229] S510, the gateway encapsulates the SIM card 3 and creates a link SMS.
[0230] It should be noted that, when the query result includes the mobile phone number corresponding to SIM card 3, the gateway encapsulates the link establishment SMS of SIM card 3.
[0231] S511. The gateway sends a link establishment SMS to the SMS gateway.
[0232] S512: The SMS gateway sends a link establishment SMS to SIM card 3.
[0233] S513: SIM card 3 establishes a connection with the gateway.
[0234] S514: The gateway adds SIM card 3 to the talk group.
[0235] Among them, after the gateway adds SIM card 3 to the talk group, the talk group includes the mobile phone number corresponding to SIM card 1, the mobile phone number corresponding to SIM card 2 and the mobile phone number corresponding to SIM card 3.
[0236] On the basis of establishing a talk group as described above, members in the talk group can realize real-time voice calls. Figure 6 Introduce the process of voice call, such as Figure 6As shown, the method includes:
[0237] S601, call out voice A.
[0238] S602: SIM card 1 reads voice information.
[0239] Specifically, the method for the SIM card to obtain voice information refers to the relevant description in the above embodiment, which will not be repeated here.
[0240] S603: SIM card 1 sends voice information to the gateway.
[0241] The voice information is the voice information to be processed.
[0242] S604: The gateway searches for other members of the talk group.
[0243] The members of the call group include the mobile phone number corresponding to SIM card 1, the mobile phone number corresponding to SIM card 2 and the mobile phone number corresponding to SIM card 3.
[0244] S605: The gateway sends voice information to SIM card 2.
[0245] S606: The gateway sends voice information to SIM card 3.
[0246] S607: SIM card 2 determines whether the language is the same as the language set by the user.
[0247] When the gateway sends voice information to the SIM card 2, the gateway carries the language identifier corresponding to the voice information.
[0248] S608: If they are different, perform voice translation.
[0249] Specifically, the SIM card 2 determines whether the languages are the same according to the language identifier of the voice information and the language identifier of the preset language locally preset by the user.
[0250] S609: SIM card 2 requests the gateway to download a voice package.
[0251] S610: The gateway requests the TSM platform to download a voice package.
[0252] The TSM platform stores voice packages corresponding to different language identifiers, and the gateway can download the corresponding voice package from the TSM platform according to the language identifier.
[0253] S611. The gateway downloads the voice package from the TSM platform.
[0254] S612: The gateway sends a voice packet to SIM card 2.
[0255] S613. SIM card 2 uses the voice package to translate and outputs the voice information in language B.
[0256] Specifically, SIM card 2 uses the built-in target encoder and target decoder to translate the voice information to obtain the voice information in language B.
[0257] S614. SIM card 3 determines whether the language is the same as the language set by the user.
[0258] S615. If they are different, perform voice translation.
[0259] S616: IM card 3 requests the gateway to download the voice package.
[0260] S617: The gateway requests the TSM platform to download the voice package.
[0261] S618. The gateway downloads the voice package from the TSM platform.
[0262] S619: The gateway sends a voice packet to SIM card 3.
[0263] S620. SIM card 3 uses the voice package to translate and outputs the voice information in C language.
[0264] The action performed by SIM card 3 after receiving the voice information is the same as that performed by SIM card 2, which will not be described in detail here.
[0265] The following combination Figure 7 The process of translating the speech information to be processed by using the target encoder and the target decoder in the above embodiment is introduced as follows: Figure 7 As shown, the SIM card includes a first encoder, a second encoder and a target decoder.
[0266] in, Figure 7 a 1 to a N The first encoder encodes the input speech information to be processed and outputs the original speech vector O A , v 1 to v M The vector representation form of the target speech information after word segmentation processing is represented. After the second encoder encodes the above input target speech information, it outputs the target speech vector. The specific encoding method refers to the relevant description in the above embodiment, which will not be repeated here.
[0267] The terminal performs cross-modal alignment on the target speech vector and the original speech vector. Specifically, after the scalar score of the original speech vector is obtained by the above calculation, the product of the scalar score and the target speech vector is calculated for each vector in the original speech vector to obtain a first product. The first product is accumulated to obtain the target speech representation. Specifically, the target speech representation can be calculated according to the following formula:
[0268]
[0269] in, represents the target speech representation, a ij represents a scalar fraction, Represents the target speech vector.
[0270] After the target speech vector and the original speech vector are obtained as described above, the target speech vector and the original speech vector are cascaded to obtain a target encoding vector.
[0271] y 1 To L-1 Represents the input of the target decoder, and the input of the decoder is the voice package downloaded by the terminal from the third-party platform. The target decoder calculates the index of the characters in the voice package and the index of the target coding vector. For the specific method of calculating the index, refer to the relevant description in the above embodiment. The target decoder matches the index of the target coding vector with the index of the characters in the voice package through a preset dynamic matching window, thereby matching the target coding vector with the characters. After the target coding vector is calculated, a linear connection is used to achieve soft alignment of the target coding vector and the character. The target decoder decodes and obtains y 1 To L .
[0272] The specific matching process is as follows: Figure 8 As shown:
[0273] Among them, the spoken sentence corresponds to the original voice vector of the voice information to be processed. a0 to a9 are the indexes of the calculated target encoding vector, and each index corresponds to a word vector of the voice information. The characters in the sentence "Can you hear my voice?" are determined based on the voice package downloaded by the terminal. They correspond to indexes b0 to b9 respectively.
[0274] In one example, the current matching segment is a4, the matched segments are a2 and a3, and the segments to be matched are a5 and a6. Figure 8 The following statements are matched to achieve soft alignment at the character level. The matched segment is the successor frame in the above embodiment, and the segment to be matched is the predecessor frame in the above embodiment.
[0275] By adopting the method provided in the embodiment of the present application, since the target encoder and the target decoder are built into the SIM card, the computing resources and storage resources in the SIM card can be used to translate the voice information to be processed, thereby improving translation efficiency and reducing power consumption. In addition, the target encoder and the target decoder in the SIM card are used to perform real-time reasoning on the voice information to be processed, thereby avoiding network transmission delays and improving response speed. Thanks to the security performance of the SIM card, the voice information to be processed is processed using the SIM card, and the voice information to be processed only runs locally on the SIM card, thereby improving user data security.
[0276] Based on the same concept, the embodiment of the present application provides a SIM card-based voice data transmission device, which is applied to a subscriber identification module SIM card in a terminal, such as Fig. 9 As shown, the device comprises:
[0277] The sending module 901 is used to send an identity authentication message to the gateway, and establish a transmission channel between the SIM card and the gateway when the gateway successfully authenticates the terminal identity;
[0278] The receiving module 902 is used to receive the authentication pass message sent by the gateway;
[0279] The sending module 901 is also used to send a peer identifier to the gateway, so that the gateway uses the peer identifier to verify the identity of the peer device, and after the identity of the peer device is verified, establish a transmission channel between the gateway and the SIM card of the peer device.
[0280] In a possible implementation, the sending module 901 is further configured to send a discovery message to the gateway before sending the identity authentication message to the gateway, wherein the discovery message is used to instruct the gateway to determine a protocol version and an encryption suite for information interaction with the SIM card, and the discovery message includes protocol version information and encryption suite information supported by the SIM card;
[0281] The receiving module 902 is further configured to receive a response message sent by the gateway, wherein the response message carries the target protocol version and the target encryption suite determined by the gateway from the discovery message.
[0282] In a possible implementation, the identity authentication message includes key information, and the identity authentication message is used to instruct the gateway to obtain a key corresponding to the key information from a key management platform, so that the gateway uses the key to encrypt and decrypt interactive data with the SIM card.
[0283] In a possible implementation, the SIM card includes a target encoder and a target decoder, and the device further includes:
[0284] The receiving module 902 is further configured to receive the voice information to be processed sent by the SIM card of the opposite device.
[0285] An acquisition module, configured to acquire, through the target encoder, an original speech vector corresponding to the speech information to be processed and a target speech vector of the preset language when the language of the speech information to be processed is different from the preset language;
[0286] An extraction module, used to extract the features of the target speech vector using a preset algorithm to obtain a target speech representation;
[0287] A cascading module, used for cascading the target speech representation and the original speech vector to obtain a target encoding vector;
[0288] The decoding module is used to input the target encoding vector into the target decoder, and the target decoder predicts the missing speech information in the original speech vector using the target speech representation, and outputs the translated speech information corresponding to the preset language.
[0289] In a possible implementation, the extraction module is specifically used to:
[0290] Normalizing the product of the original speech vector and the target speech vector to obtain a scalar score of the original speech vector, wherein the scalar score is used to represent the probability that each vector in the original speech vector belongs to each character;
[0291] The product of the scalar score and the target speech vector is summed to obtain the target speech representation.
[0292] In a possible implementation, the cascade module is specifically configured to:
[0293] The target encoding vector is calculated according to the following formula:
[0294]
[0295] in, represents the target encoding vector, R represents a natural number, and h represents the number of neurons of the original speech encoder; represents the original speech vector, represents the target speech representation.
[0296] In a possible implementation, the decoding module is specifically configured to:
[0297] Accumulating and summing the scalar scores corresponding to each vector in the target encoding vector to obtain the index of each vector;
[0298] Obtaining the index of each word vector in the target speech corresponding to the preset language;
[0299] According to the index of each vector in the target encoding vector and the index of each word vector in the target speech, the target encoding vector and each word vector in the target speech are matched to obtain a matching result;
[0300] The target decoder is used to predict the missing speech information in the original speech vector according to the matching result to obtain the translated speech information corresponding to the preset language.
[0301] In a possible implementation, the decoding module is specifically configured to:
[0302] The matching result is obtained according to the following formula:
[0303]
[0304] in, represents the index of each vector in the target encoding vector, W ik represents the index of each word vector in the target speech, d LA Indicates the number of vectors driven forward, d LB represents the number of vectors driven backward, and F represents the size of the matching window when matching the index of each vector in the target encoding vector.
[0305] In a possible implementation, the device further includes:
[0306] The acquisition module is further used to acquire sample voice information corresponding to the language of the voice information to be processed;
[0307] A training module, configured to train the first encoder using first training data, wherein the first training data includes the sample speech information and a label speech vector corresponding to the sample speech information, and the first encoder is configured to obtain the original speech vector;
[0308] The acquisition module is also used to acquire sample target speech information corresponding to the preset language;
[0309] The training module is also used to train the second encoder using second training data, where the second training data includes the sample target speech information and a label speech vector corresponding to the sample target speech information, and the second encoder is used to obtain the target speech vector.
[0310] In a possible implementation, the device further includes:
[0311] An input module, used for inputting a sample character vector sequence of the preset language into a decoder;
[0312] A prediction module, used to obtain a target character vector output by the encoder based on the prediction of the character vector sequence;
[0313] An adjustment module is used to adjust the model parameters of the decoder using the label character vector and the target character vector to obtain the target decoder.
[0314] It should be noted that the SIM card-based voice data transmission device is a device corresponding to the above-mentioned SIM card-based voice data transmission method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiment of the device and can achieve the same technical effect.
[0315] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.
[0316] The electronic device may include a processor 1001 and a memory 1002 storing computer program instructions.
[0317] Specifically, the processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0318] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 802 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1002 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 802 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1002 is a non-volatile solid-state memory.
[0319] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.
[0320] The processor 1001 implements any one of the SIM card-based voice data transmission methods in the above embodiments by reading and executing computer program instructions stored in the memory 1002 .
[0321] In one example, the electronic device may further include a communication interface 1003 and a bus 1004. Fig.10 As shown, the processor 1001, the memory 1002, and the communication interface 1003 are connected via a bus 1004 and communicate with each other.
[0322] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.
[0323] Bus 1004 includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1004 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.
[0324] In addition, in combination with the SIM card-based voice data transmission method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the SIM card-based voice data transmission methods in the above embodiment is implemented.
[0325] In combination with the SIM card-based voice data transmission method in the above embodiments, an embodiment of the present application may provide a computer program product, the computer product including a computer program, and when the computer program is executed by a processor, the SIM card-based voice data transmission method as in any one of the above embodiments is implemented.
[0326] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.
[0327] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0328] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.
[0329] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.
[0330] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. A voice data transmission method based on a SIM card, characterized in that: A user identification module SIM card is applied to a terminal, and the method comprises: Sending an identity authentication message to the gateway, and establishing a transmission channel between the SIM card and the gateway when the gateway successfully authenticates the terminal identity; Receiving the authentication success message sent by the gateway; The peer identifier is sent to the gateway so that the gateway can use the peer identifier to verify the identity of the peer device. After the identity of the peer device is verified, a transmission channel between the gateway and the SIM card of the peer device is established.
2. The method according to claim 1, characterized in that Before sending the identity authentication message to the gateway, the method further includes: Sending a discovery message to the gateway, wherein the discovery message is used to instruct the gateway to determine a protocol version and an encryption suite for information interaction with the SIM card, wherein the discovery message includes information about the protocol version and encryption suite supported by the SIM card; A response message sent by the gateway is received, wherein the response message carries the target protocol version and the target encryption suite determined by the gateway from the discovery message.
3. The method according to claim 1, characterized in that The identity authentication message includes key information, and the identity authentication message is used to instruct the gateway to obtain a key corresponding to the key information from a key management platform, so that the gateway uses the key to encrypt and decrypt interactive data with the SIM card.
4. The method according to claim 1, characterized in that: The SIM card includes a target encoder and a target decoder; after sending the peer identifier to the gateway, the method further includes: Receiving the voice information to be processed sent by the SIM card of the opposite device; When the language of the speech information to be processed is different from the preset language, obtaining, by the target encoder, an original speech vector corresponding to the speech information to be processed and a target speech vector of the preset language; Extracting the features of the target speech vector using a preset algorithm to obtain a target speech representation; Cascading the target speech representation and the original speech vector to obtain a target encoding vector; The target encoding vector is input into the target decoder, and the target decoder predicts the missing speech information in the original speech vector using the target speech representation, and outputs the translated speech information corresponding to the preset language.
5. The method according to claim 4, characterized in that The step of extracting the features of the target speech vector using a preset algorithm to obtain the target speech representation includes: Normalizing the product of the original speech vector and the target speech vector to obtain a scalar score of the original speech vector, wherein the scalar score is used to represent the probability that each vector in the original speech vector belongs to each character; The product of the scalar score and the target speech vector is summed to obtain the target speech representation.
6. The method according to claim 5, characterized in that The step of concatenating the target speech representation and the original speech vector to obtain a target encoding vector comprises: The target encoding vector is calculated according to the following formula: in, represents the target encoding vector, R represents a natural number, and h represents the number of neurons of the target encoder; represents the original speech vector, represents the target speech representation.
7. The method according to claim 4, characterized in that The step of inputting the target encoding vector into the target decoder, wherein the target decoder predicts the missing speech information in the original speech vector using the target speech representation and outputs the translated speech information corresponding to the preset language, comprises: Accumulating and summing the scalar scores corresponding to each vector in the target encoding vector to obtain the index of each vector; Obtaining the index of each word vector in the target speech corresponding to the preset language; According to the index of each vector in the target encoding vector and the index of each word vector in the target speech, the target encoding vector and each word vector in the target speech are matched to obtain a matching result; The target decoder is used to predict the missing speech information in the original speech vector according to the matching result to obtain the translated speech information corresponding to the preset language.
8. The method according to claim 7, characterized in that The step of matching the target encoding vector with each word vector in the target speech according to the index of each vector in the target encoding vector and the index of each word vector in the target speech to obtain a matching result includes: The matching result is obtained according to the following formula: in, represents the index of each vector in the target encoding vector, W ik represents the index of each word vector in the target speech, d LA Indicates the number of vectors driven forward, d LB represents the number of vectors driven backward, and F represents the size of the matching window when matching the index of each vector in the target encoding vector.
9. The method according to claim 4, characterized in that Before obtaining the original speech vector corresponding to the speech information to be processed and the target speech vector of the preset language through the target encoder, the method further includes: Acquire sample voice information corresponding to the language of the voice information to be processed; A first encoder is obtained by training using first training data, wherein the first training data includes the sample speech information and a label speech vector corresponding to the sample speech information, and the first encoder is used to obtain the original speech vector; Acquire sample target speech information corresponding to the preset language; A second encoder is obtained by training with second training data, wherein the second training data includes the sample target speech information and a label speech vector corresponding to the sample target speech information, and the second encoder is used to obtain the target speech vector.
10. The method according to claim 9, characterized in that: Before inputting the target encoding vector into the target decoder, the method further comprises: Inputting the sample character vector sequence of the preset language into a decoder; Obtaining a target character vector output by the encoder that is predicted based on the character vector sequence; The model parameters of the decoder are adjusted using the label character vector and the target character vector to obtain the target decoder.
