Data processing method and device, terminal, storage medium and computer program product

Abstract

The invention discloses a data processing method and device, a terminal, a storage medium and a computer program product. The method comprises the steps that in an application layer, a first terminal reads first data through a first interface between the application layer and a chip of the first terminal, and the first data comprises voice data of a first language type associated with a call between the first terminal and a second terminal; the chip at least has a data playing function and / or a data transmission function; performing language type conversion processing on the first data to obtain second data, the second data including voice data of a second language type associated with the call; and writing the second data into the chip through a second interface between the application layer and the chip.

Description

Technical Field

[0001] This application relates to the field of network transmission technology, and in particular to a data processing method, apparatus, terminal, storage medium, and computer program product. Background Technology

[0002] With the rapid development of large-scale artificial intelligence (AI) technology, real-time speech translation technology has become possible, further breaking down communication barriers between people worldwide and promoting communication and understanding between different language groups.

[0003] Currently, 5G calls made by operators are all based on the IP Multimedia Subsystem (IMS) network and use Voice Over New Radio (VoNR) / VoLTE (VoLTE) for voice calls, which results in a waste of network resources. Summary of the Invention

[0004] To address the related technical issues, embodiments of this application provide a data processing method, apparatus, terminal, storage medium, and computer program product.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a data processing method, including:

[0007] At the application layer, first data is read through the first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with the call between the first terminal and the second terminal. The chip has at least data playback function and / or data transmission function.

[0008] The first data is subjected to language type conversion processing to obtain the second data, which includes voice data of a second language type associated with the call;

[0009] The second data is written to the chip through the second interface between the application layer and the chip.

[0010] In the above scheme, reading the first data through the first interface between the application layer and the chip of the first terminal includes:

[0011] The first data is read through a third interface, the first data containing voice data of a first language type transmitted downlink in association with the call, the first interface including the third interface.

[0012] In the above scheme, reading the first data through the first interface between the application layer and the chip of the first terminal includes:

[0013] The first data is read through a third interface, the first data containing voice data of a first language type transmitted downlink in association with the call, the first interface including the third interface.

[0014] In the above scheme, writing the second data into the chip through the second interface between the application layer and the chip includes:

[0015] The second data is written to the chip via a fifth interface, the second data including voice data of a second language type in downlink transmission associated with the call, and the second interface including the fifth interface.

[0016] The method in the above scheme further includes:

[0017] The chip is used to process the second data for playback.

[0018] In the above scheme, writing the second data into the chip through the second interface between the application layer and the chip includes:

[0019] The second data is written to the chip via a sixth interface, the second data including voice data of a second language type transmitted uplink in association with the call, and the second interface including the sixth interface.

[0020] The method in the above scheme further includes:

[0021] The chip transmits the second data to the second terminal.

[0022] In the above scheme, the step of performing language type conversion on the first data to obtain the second data includes:

[0023] Using the first data, third data is obtained, the third data including text data of a first language type associated with the call;

[0024] The third data is subjected to language type conversion processing to obtain the fourth data, which contains text data in a second language type associated with the call;

[0025] The second data is obtained using the fourth data.

[0026] In the above scheme, the step of performing language type conversion on the first data to obtain the second data includes:

[0027] When the computing power of the first terminal meets the first condition, the first data is processed by language type conversion to obtain the second data, and the first condition includes conditions associated with the first data.

[0028] In the above scheme, the step of performing language type conversion on the first data to obtain the second data includes:

[0029] If the computing power of the first terminal does not meet the first condition, the first data is sent to the first device, which is used for language type conversion processing of the first data and is deployed in the cloud.

[0030] Receive the second data sent by the first device.

[0031] The method in the above scheme further includes:

[0032] At the application layer, the language type conversion function of the first terminal is initiated through the seventh interface;

[0033] And / or,

[0034] At the application layer, the language type conversion function of the first terminal is stopped through the eighth interface.

[0035] This application embodiment also provides a data processing apparatus, disposed in a first terminal, including:

[0036] The reading unit is used to read first data at the application layer through a first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with a call between the first terminal and the second terminal. The chip has at least a data playback function and / or a data transmission function.

[0037] A conversion unit is used to perform language type conversion processing on the first data to obtain second data, the second data containing voice data of a second language type associated with the call;

[0038] A writing unit is used to write the second data into the chip through a second interface between the application layer and the chip.

[0039] This application also provides a terminal, including: a processor and a communication interface; wherein,

[0040] The processor is configured to, at the application layer, read first data through a first interface between the application layer and the chip of the first terminal, the first data including voice data of a first language type associated with a call between the first terminal and the second terminal, the chip having at least data playback and / or data transmission functions; perform language type conversion processing on the first data to obtain second data, the second data including voice data of a second language type associated with the call; and write the second data into the chip through a second interface between the application layer and the chip.

[0041] This application also provides a terminal, including: a processor and a memory for storing computer programs capable of running on the processor.

[0042] When the processor runs the computer program, it executes the steps of any of the data processing methods described above.

[0043] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the data processing methods described above.

[0044] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the data processing methods described above.

[0045] The data processing method, apparatus, terminal, storage medium, and computer program product provided in this application embodiment include a first terminal at the application layer that reads first data through a first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with a call between the first terminal and a second terminal. The chip has at least data playback and / or data transmission functions. The first data undergoes language type conversion processing to obtain second data, which includes voice data of a second language type associated with the call. The second data is then written to the chip through a second interface between the application layer and the chip. The technical solution provided in this application embodiment, by setting a universal interface between the terminal's application layer and the chip, enables the application layer to obtain voice data from inter-terminal calls through the universal interface and perform real-time voice translation on the voice data. Thus, in scenarios where the terminal is making an IMS call, voice translation can be implemented through the terminal, thereby avoiding the increased network load caused by network-side voice translation and reducing the waste of network resources. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the terminal structure in an IMS call scenario;

[0047] Figure 2 This is a flowchart illustrating a data processing method according to an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of a downlink data processing flow according to an embodiment of this application;

[0049] Figure 4 This is a schematic diagram of an uplink data processing flow according to an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the terminal structure in an IMS call scenario, which is an application example of this application.

[0051] Figure 6 A flowchart illustrating the real-time translation process for the application example terminal in this application;

[0052] Figure 7 This is a schematic diagram of the data processing device structure according to an embodiment of this application;

[0053] Figure 8 This is a schematic diagram of the terminal structure according to an embodiment of this application. Detailed Implementation

[0054] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0055] In related technologies, during an IMS call on a terminal, such as Figure 1 As shown, the terminal's IMS protocol stack is implemented by the chip manufacturer at the bottom layer. This means that the voice data associated with IMS calls is processed by the chip's underlying software logic. As a result, the application layer (specifically, the user interface (UI) of the telephone application or the operating system) has no interface to obtain voice data, and therefore cannot process the voice data. Consequently, the terminal struggles to implement corresponding functions when encountering audio processing services, such as real-time voice translation.

[0056] To address the aforementioned issues, the real-time translation function for IMS calls is currently implemented on the network side. However, on the one hand, real-time translation on the network side requires modifications to the network architecture, resulting in high resource consumption, and the user demand for translation is usually small, easily leading to wasted network resources; on the other hand, the real-time translation function on the network side is usually for speech-to-text conversion, lacking a function for speech-to-speech conversion.

[0057] Based on this, in various embodiments of this application, a universal interface is designed between the application layer and the underlying chip, enabling the application layer to obtain call data between terminals through the universal interface, and to realize real-time voice translation of the call data by the application layer with the help of AI large model technology; that is, the voice translation function is realized through the terminal capability. In this way, in the IMS call scenario, the network load during the voice translation process can be reduced, and at the same time, there is no need to modify the network architecture, thereby reducing the consumption of network resources.

[0058] This application provides a data processing method, such as... Figure 2 As shown, applied to a first terminal, the method includes:

[0059] Step 201: At the application layer, first data is read through the first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with the call between the first terminal and the second terminal. The chip has at least data playback function and / or data transmission function.

[0060] Step 202: Perform language type conversion processing on the first data to obtain second data, the second data containing voice data of a second language type associated with the call;

[0061] Step 203: Write the second data into the chip through the second interface between the application layer and the chip.

[0062] In practical applications, the first terminal and the second terminal can be understood as two terminals with an established call connection in a call scenario (specifically, an IMS call). The first terminal has at least a language type conversion processing function (also known as a voice translation function or a real-time translation function) and / or a language type recognition function. A first model can be pre-configured on the first terminal to enable the first terminal to have a language type conversion processing function and / or a language type recognition function for voice data. The first model can be called a terminal translation mini-model, translation mini-model, etc. This application embodiment does not limit this, as long as its function is implemented.

[0063] In practical applications, before step 201, the first terminal can start the language type conversion function at the application layer.

[0064] Based on this, in one embodiment, the method may further include:

[0065] At the application layer, the language type conversion function of the first terminal is initiated through the seventh interface.

[0066] The seventh interface can be called the Start interface, which is used to initiate the language type conversion function at the application layer. Both the seventh interface and the chip are located at the bottom layer (also called the bottom protocol stack or bottom system). In this embodiment, the name of the seventh interface is not limited, as long as its function is implemented.

[0067] Here, during the call between the first terminal and the second terminal, the first terminal needs to distinguish between the uplink and downlink data in the voice data of the first language type so that the audio path will not be disordered when the language type conversion is performed at the application layer later; the downlink data can be read through the downlink interface.

[0068] Specifically, in one embodiment, reading the first data through the first interface between the application layer and the chip of the first terminal includes:

[0069] The first data is read through a third interface, the first data containing voice data of a first language type transmitted downlink in association with the call, the first interface including the third interface.

[0070] In practical applications, downlink data can be understood as voice data of the first language type transmitted from the second terminal to the first terminal via the network. Additionally, the third interface, which can be called the downlink voice data reading interface (ReadRxVoiceData), is used to read downlink data at the application layer. This third interface can be located at the lower layer. This application embodiment does not limit the name of the third interface, as long as its function is implemented.

[0071] In practical applications, after reading downlink data from the chip, the first terminal can cache the read downlink data; wherein, the downlink data can be stored in a buffer, which can be set at the underlying layer.

[0072] For example, the first terminal can read downlink data from the chip based on a preset period and data size, and store the read downlink data in a buffer.

[0073] In practical applications, before performing language type conversion on downlink data, the first terminal can assess its computing power to determine whether it is capable of performing language type conversion on downlink data.

[0074] Specifically, in one embodiment, the step of performing language type conversion processing on the first data to obtain the second data includes:

[0075] When the computing power of the first terminal meets the first condition, the first data is processed by language type conversion to obtain the second data, and the first condition includes conditions associated with the first data.

[0076] The computing power of the first terminal can be associated with the available resources of the first terminal (such as computing resources and storage resources) and / or the language types supported by the language type conversion function; that is, the computing power of the first terminal can be determined based on the available resources of the first terminal and / or the language types supported by the language type conversion function.

[0077] In practical applications, the first terminal can obtain cached downlink data and automatically identify (or analyze) the downlink data to determine the first language type and the second language type, without needing to manually set the language type. Based on the first language type and the second language type, it is determined whether the computing power of the first terminal meets the first condition. Here, for downlink data, the first language type can be understood as the language type supported by the second terminal (also known as the source language), such as English, and the second language type can be understood as the language type supported by the first terminal (also known as the target language), such as Chinese. The first language type and the second language type are different, and the specific types of the first language type and the second language type are not limited in this application embodiment.

[0078] For example, in the process of determining whether the computing power of the first terminal meets the first condition, if the language type conversion function of the first terminal does not support the first language type or the second language type, then the first terminal can determine that the computing power of the first terminal does not meet the first condition; if the available resources of the first terminal cannot support the language type conversion processing of downlink data, then the first terminal can determine that the computing power of the first terminal does not meet the first condition.

[0079] Here, if the computing power of the first terminal meets the first condition, it means that the first terminal can perform language type conversion processing; in this case, the first terminal can perform language type conversion processing on the downlink data.

[0080] Specifically, in one embodiment, the step of performing language type conversion processing on the first data to obtain the second data includes:

[0081] Using the first data, third data is obtained, the third data including text data of a first language type associated with the call;

[0082] The third data is subjected to language type conversion processing to obtain the fourth data, which contains text data in a second language type associated with the call;

[0083] The second data is obtained using the fourth data.

[0084] In practical applications, during the language type conversion process of downlink data, the first terminal can use Automatic Speech Recognition (ASR) technology to convert downlink data in speech form into downlink data in text form (i.e., the third data); by performing language type conversion on the downlink data in text form, the language type conversion downlink data in text form is obtained (i.e., the fourth data); then, based on Text to Speech (TTS) technology, the language type conversion downlink data in text form is converted into language type conversion downlink data in speech form. In this way, the language type conversion process of downlink data is realized.

[0085] It should be noted that the first terminal can perform language type conversion on the downlink data while caching it. In the above process, if the processing rate of the downlink data is less than the caching rate of the downlink data, the amount of cached data may reach the storage threshold of the buffer. In this case, the first terminal can stop reading downlink data at the application layer through the ninth interface. The ninth interface can be called a callback interface (which can be expressed as AudioBufferReady in English) and is used to stop reading downlink data at the application layer. The name of the ninth interface is not limited in this application embodiment, as long as its function is implemented.

[0086] In practical applications, if the computing power of the first terminal does not meet the first condition, the first terminal can adopt an end-to-cloud collaborative approach to perform language type conversion processing on the downlink data through the cloud.

[0087] Specifically, in one embodiment, the step of performing language type conversion processing on the first data to obtain the second data includes:

[0088] If the computing power of the first terminal does not meet the first condition, the first data is sent to the first device, which is used for language type conversion processing of the first data and is deployed in the cloud.

[0089] Receive the second data sent by the first device.

[0090] In practical applications, the first device may include a second model, which has language type conversion processing function and / or language type recognition function for voice data. The second model may be referred to as a cloud-based AI large model.

[0091] In practical applications, in order to communicate with the first device, the first terminal can establish a first channel and a second channel with the first device respectively, and send downlink data through the first channel and receive downlink data after language type conversion processing through the second channel; wherein, the first channel and the second channel may include long connection channels.

[0092] For example, assuming the computing power of the first terminal does not meet the first condition, the first terminal can send downlink data to the first device through the first channel (which can be represented as Rx1); after receiving the downlink data, the first device can automatically identify the downlink data to determine the first language type and the second language type respectively, and perform language type conversion processing on the downlink data based on the first language type and the second language type to obtain the language type converted downlink data; then, the first device can send the language type converted downlink data to the first terminal through the second channel (which can be represented as Tx1).

[0093] In practical applications, after completing the language type conversion processing of the downlink data, the first terminal can write the language type converted downlink data into the underlying layer.

[0094] Specifically, in one embodiment, writing the second data into the chip through the second interface between the application layer and the chip includes:

[0095] The second data is written to the chip via a fifth interface, the second data including voice data of a second language type in downlink transmission associated with the call, and the second interface including the fifth interface.

[0096] The fifth interface can be referred to as the downlink voice data writing interface (WriteRxVoiceData), which is used to write downlink data at the application layer. The fifth interface is located at the bottom layer. In this embodiment, the name of the fifth interface is not limited, as long as its function is implemented.

[0097] In practical applications, after the downlink data with language type conversion is written into the chip, the first terminal can play the written downlink data locally.

[0098] Based on this, in one embodiment, the method may further include:

[0099] The chip is used to process the second data for playback.

[0100] In practical applications, without changing the downlink data playback logic, the first terminal can use the chip to play the language-type converted downlink data locally through the speaker.

[0101] For example, for downlink data, such as Figure 3 As shown, assuming the audio interface includes a third interface and a fifth interface, after the chip's modem communication module receives downlink data transmitted from the network side, it transmits the downlink data to the chip's audio driver. At the application layer, the first terminal reads the downlink data from the audio driver through the third interface and performs translation processing on the downlink data based on the language type conversion function to obtain translated downlink data (specifically, it can include downlink data in text form and downlink data in audio form). Then, through the fifth interface, the translated downlink data in audio form is written to the underlying layer to achieve local playback of the downlink data. At the same time, the translated downlink data in text form is written to the underlying layer to achieve local display of the downlink data.

[0102] In practical applications, during a call between the first terminal and the second terminal, the first terminal can also read uplink data through the uplink interface so that the uplink data can be converted to different languages ​​later.

[0103] Specifically, in one embodiment, reading the first data through the first interface between the application layer and the chip of the first terminal includes:

[0104] The first data is read through a fourth interface, the first data containing voice data of a first language type transmitted uplink in association with the call, and the first interface includes the fourth interface.

[0105] In practical applications, uplink data can be understood as voice data of a first language type to be transmitted from the first terminal to the second terminal via the network. For uplink data, the first language type can be understood as the language type supported by the first terminal, and the second language type can be understood as the language type supported by the second terminal. Furthermore, the fourth interface can be called the uplink voice data reading interface (which can be expressed as ReadTxVoiceData in English), used to read uplink data at the application layer. The fourth interface can be located at the lower layer. This application embodiment does not limit the name of the fourth interface, as long as its function is implemented.

[0106] In practical applications, after reading uplink data from the chip, the first terminal can cache the read uplink data; wherein, the uplink data can be stored in the buffer.

[0107] In practical applications, before performing language type conversion processing on the uplink data, the first terminal can assess its computing power to determine whether it is capable of performing language type conversion processing on the uplink data. If the computing power of the first terminal meets the first condition, it means that the first terminal can perform language type conversion processing. In this case, the first terminal can perform language type conversion processing on the uplink data.

[0108] In practical applications, during the language type conversion process of uplink data, the first terminal can use ASR technology to convert the uplink data in voice form into uplink data in text form (i.e., the third data); by performing language type conversion on the uplink data in text form, the uplink data after language type conversion in text form is obtained (i.e., the fourth data); then, based on TTS technology, the uplink data after language type conversion in text form is converted into uplink data after language type conversion in voice form. In this way, the language type conversion process of uplink data is realized.

[0109] It should be noted that the first terminal can perform language type conversion on the uplink data while caching it. In the above process, if the processing rate of the uplink data is less than the caching rate of the uplink data, the amount of cached data may reach the storage threshold of the buffer. In this case, the first terminal can stop reading uplink data at the application layer through the ninth interface. The ninth interface is also used to stop reading uplink data at the application layer.

[0110] In addition, if the computing power of the first terminal does not meet the first condition, the first terminal can perform language type conversion processing on the uplink data through the first device. During this process, the first terminal can establish a third channel and a fourth channel with the first device respectively, and send uplink data through the third channel and receive uplink data after language type conversion processing through the fourth channel. The third channel and the fourth channel may include long connection channels.

[0111] For example, assuming the computing power of the first terminal does not meet the first condition, the first terminal can send uplink data to the first device through the third channel (which can be represented as Rx2); after receiving the uplink data, the first device can automatically identify the uplink data to determine the first language type and the second language type respectively, and perform language type conversion processing on the uplink data based on the first language type and the second language type to obtain the language type converted uplink data; then, the first device can send the language type converted uplink data to the first terminal through the fourth channel (which can be represented as Tx2).

[0112] It should be noted that, regarding the language type conversion processing of uplink data, if the second terminal does not have a language type conversion function, the first terminal can perform the language type conversion processing and transmit the language type-converted uplink data to the second terminal so that the second terminal can understand the content expressed by the uplink data; if the second terminal has a language type conversion function, the first terminal can also perform the language type conversion processing and transmit the language type-converted uplink data to the second terminal, so that the second terminal can directly play the second data.

[0113] In practical applications, after completing the language type conversion of the uplink data, the first terminal can write the language type converted uplink data into the underlying layer.

[0114] Specifically, in one embodiment, writing the second data into the chip through the second interface between the application layer and the chip includes:

[0115] The second data is written to the chip via a sixth interface, the second data including voice data of a second language type transmitted uplink in association with the call, and the second interface including the sixth interface.

[0116] The sixth interface, which can be called the uplink voice data writing interface (WriteTxVoiceData), is used to write uplink data at the application layer. The sixth interface is located at the bottom layer. In this embodiment, the name of the sixth interface is not limited, as long as its function is implemented.

[0117] In practical applications, after the uplink data with language type conversion is written into the chip, the first terminal can perform data transmission processing on the written uplink data.

[0118] Based on this, in one embodiment, the method may further include:

[0119] The chip transmits the second data to the second terminal.

[0120] In practical applications, without changing the uplink data playback logic, the first terminal can transmit the language-type converted uplink data to the second terminal based on the communication function of the chip.

[0121] For example, regarding upstream data, such as Figure 4 As shown, assuming the voice interface includes a fourth interface and a sixth interface, after the microphone device collects uplink data, it transmits downlink data to the voice driver of the chip. At the application layer, the first terminal reads uplink data from the voice driver through the fourth interface and performs translation processing on the uplink data based on the language type conversion function to obtain translated uplink data (which may include uplink data in text form and uplink data in voice form). Then, through the sixth interface, the translated uplink data in voice form is written into the modem communication module of the chip to realize the transmission of uplink data.

[0122] In practical applications, after step 203, the first terminal can disable the language type conversion function at the application layer.

[0123] Based on this, in one embodiment, the method may further include:

[0124] At the application layer, the language type conversion function of the first terminal is stopped through the eighth interface.

[0125] The eighth interface can be called the stop interface, which is used to stop the language type conversion function at the application layer. The eighth interface can be set at the bottom layer. In this embodiment of the application, the name of the eighth interface is not limited, as long as its function is implemented.

[0126] The data processing method provided in this application embodiment involves a first terminal at the application layer reading first data through a first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with a call between the first terminal and a second terminal. The chip has at least data playback and / or data transmission functions. The first data undergoes language type conversion processing to obtain second data, which includes voice data of a second language type associated with the call. The second data is then written to the chip through a second interface between the application layer and the chip. This technical solution, by setting a universal interface between the terminal's application layer and the chip, enables the application layer to obtain voice data from inter-terminal calls through the universal interface and perform real-time voice translation. Thus, in scenarios where the terminal is making an IMS call, voice translation can be implemented through the terminal, thereby avoiding the increased network load caused by network-side voice translation and reducing the waste of network resources.

[0127] The following section provides a more detailed description of this application with reference to application examples.

[0128] This application example proposes a solution for real-time translation of IMS calls on the terminal side, leveraging AI large-scale model technology and edge-cloud collaboration technology; specifically, such as... Figure 5 As shown, by adding an AI intelligent assistant module (also known as a real-time voice call translation module) and an audio buffer module, the terminal (also known as an AI terminal) can obtain the voice data of the IMS call from the underlying layer, and after translating the voice data in real time, write the translated voice data into the underlying IMS protocol stack. Among them, the AI ​​intelligent assistant module includes a small translation model, and the audio buffer module is used to cache the voice data of the IMS call and provides 7 audio interfaces to realize the control of the translation function and data reading.

[0129] Here, in the scenario where terminal A and terminal B are making an IMS call, such as Figure 6 As shown, the real-time translation process performed by terminal A includes the following steps:

[0130] Step 601: Terminal A (i.e., the first terminal mentioned above) and Terminal B (i.e., the second terminal mentioned above) establish an IMS voice call;

[0131] Among them, the language type supported by terminal A (i.e., the second language type mentioned above) is English, and the language type supported by terminal B (i.e., the first language type mentioned above) is Chinese.

[0132] Step 602: Terminal A starts the real-time telephone translation function (i.e., the language type conversion function) through the telephone application (i.e., the application layer mentioned above), and reads the uplink voice data and downlink voice data by calling the start interface (i.e., the seventh interface mentioned above) through the voice buffer module;

[0133] Here, terminal A will also set a callback interface (i.e., the ninth interface mentioned above) through the telephone application.

[0134] Step 603: Terminal A determines whether the translation small model capability and terminal capability can be used for local translation on the terminal;

[0135] The AI ​​intelligent assistant module of terminal A analyzes the uplink and downlink voice data, determines the source language and target language, and determines whether the terminal should perform real-time translation based on the terminal's computing power. If it is determined that the terminal should perform real-time translation, then step 608 is executed; if the AI ​​translation big model on the cloud side performs real-time translation, then step 604 is executed.

[0136] Step 604: Terminal A establishes four long connections with the AI ​​translation model (Rx1 channel, Tx1 channel, Rx2 channel, and Tx2 channel);

[0137] Step 605: Terminal A sends the downlink and uplink voice data to be translated to the AI ​​translation big model through the Rx1 channel and Tx1 channel, respectively;

[0138] Step 606: The AI ​​translation big model judges the language of the downlink and uplink voice data that need to be translated in order to perform real-time translation. Then, it returns the translated data to terminal A through the Rx2 channel and Tx2 channel.

[0139] Step 607: After receiving the translated data, terminal A writes the translated data into the underlying system;

[0140] Here, if the translated data is text data, terminal A can convert the text data into speech data and write the speech data into the underlying system.

[0141] Step 608: Terminal A obtains data (i.e., the first data mentioned above) from the Audio Buffer module through the Audio interface, and after local large model translation, the translated data (i.e., the second data mentioned above) is written to the underlying layer through the Audio interface;

[0142] Here, for uplink voice data, terminal A processes the uplink voice data based on PCM technology to obtain uplink voice PCM data; based on ASR technology, it converts the uplink voice PCM data into uplink text data (i.e., the third data mentioned above), and translates the uplink text data into uplink text data in the target language (i.e., the fourth data mentioned above); based on TTS technology, it converts the uplink text data in the target language into uplink voice PCM data in the target language; and writes the uplink voice PCM data in the target language into the underlying system so as to play downlink voice data without changing the relevant downlink voice data playback logic.

[0143] For downlink voice data, terminal A processes the downlink voice data based on PCM technology to obtain downlink voice PCM data; based on ASR technology, it converts the downlink voice PCM data into downlink text data and translates the downlink text data into downlink text data in the target language; based on TTS technology, it converts the downlink text data in the target language into downlink voice PCM data in the target language; and writes the downlink voice PCM data in the target language into the underlying system.

[0144] Step 609: Terminal A sends uplink voice data to Terminal B.

[0145] Here, terminal A reads the written uplink voice data through the audio interface and sends the uplink voice data to terminal B without changing the relevant uplink voice data transmission process.

[0146] It should be noted that if terminal B has real-time translation capabilities, the real-time translation process of terminal B is the same as that of terminal A.

[0147] In this application example, the terminal adopts an end-to-cloud collaborative approach, making full use of the terminal's computing power and AI large model capabilities for real-time voice translation. This reduces network load and thus reduces network resource consumption, while also eliminating the need to modify the IMS core network.

[0148] To implement the method of the embodiments of this application, the embodiments of this application also provide a data processing device, which is disposed on a first terminal, such as... Figure 7 As shown, the device includes:

[0149] The reading unit 701 is used to read first data at the application layer through a first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with a call between the first terminal and the second terminal. The chip has at least a data playback function and / or a data transmission function.

[0150] The conversion unit 702 is used to perform language type conversion processing on the first data to obtain second data, the second data including voice data of a second language type associated with the call;

[0151] The writing unit 703 is used to write the second data into the chip through the second interface between the application layer and the chip.

[0152] In one embodiment, the reading unit 701 is configured to read the first data via a third interface, the first data including voice data of a first language type transmitted downlink in association with the call, and the first interface including the third interface.

[0153] In one embodiment, the reading unit 701 is configured to read the first data via a fourth interface, the first data including voice data of a first language type transmitted uplink in association with the call, and the first interface including the fourth interface.

[0154] In one embodiment, the writing unit 703 is configured to write the second data into the chip via a fifth interface, the second data comprising voice data of a second language type transmitted downlink in association with the call, and the second interface comprising the fifth interface.

[0155] In one embodiment, the writing unit 703 is further configured to perform playback processing on the second data via the chip.

[0156] In one embodiment, the writing unit 703 is configured to write the second data into the chip via a sixth interface, the second data including uplink voice data of a second language type associated with the call, and the second interface including the sixth interface.

[0157] In one embodiment, the writing unit 703 is further configured to send the second data to the second terminal via the chip.

[0158] In one embodiment, the conversion unit 702 is configured to:

[0159] Using the first data, third data is obtained, the third data including text data of a first language type associated with the call;

[0160] The third data is subjected to language type conversion processing to obtain the fourth data, which contains text data in a second language type associated with the call;

[0161] The second data is obtained using the fourth data.

[0162] In one embodiment, the conversion unit 702 is used to perform language type conversion processing on the first data to obtain the second data when the computing power of the first terminal meets the first condition, wherein the first condition includes conditions associated with the first data.

[0163] In one embodiment, the conversion unit 702 is configured to:

[0164] If the computing power of the first terminal does not meet the first condition, the first data is sent to the first device, which is used for language type conversion processing of the first data and is deployed in the cloud.

[0165] Receive the second data sent by the first device.

[0166] In one embodiment, the conversion unit 702 is further configured to:

[0167] At the application layer, the language type conversion function of the first terminal is initiated through the seventh interface;

[0168] And / or,

[0169] At the application layer, the language type conversion function of the first terminal is stopped through the eighth interface.

[0170] In practical applications, the reading unit 701 can be implemented by a processor in the data processing device; the conversion unit 702 and the writing unit 703 can be implemented by a processor in the data processing device combined with a communication interface.

[0171] It should be noted that the data processing apparatus provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the apparatus can be divided into different program modules to complete all or part of the processing described above. In addition, the data processing apparatus and data processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0172] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide a terminal (i.e., the first terminal mentioned above), such as Figure 8 As shown, the terminal 800 includes:

[0173] The communication interface 801 enables information exchange with other devices;

[0174] The processor 802 is connected to the communication interface 801 to enable information interaction with other devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program;

[0175] The computer program is stored in memory 803.

[0176] Specifically, the processor 802 is configured to, at the application layer, read first data through a first interface between the application layer and the chip of the first terminal, the first data including voice data of a first language type associated with a call between the first terminal and the second terminal, the chip having at least data playback and / or data transmission functions; perform language type conversion processing on the first data to obtain second data, the second data including voice data of a second language type associated with the call; and write the second data into the chip through a second interface between the application layer and the chip.

[0177] In one embodiment, the processor 802 is configured to read the first data via a third interface, the first data comprising voice data of a first language type transmitted downlink in association with the call, the first interface including the third interface.

[0178] In one embodiment, the processor 802 is configured to read the first data via a fourth interface, the first data comprising voice data of a first language type transmitted uplink in association with the call, the first interface including the fourth interface.

[0179] In one embodiment, the processor 802 is configured to write the second data into the chip via a fifth interface, the second data comprising voice data of a second language type transmitted downlink in association with the call, the second interface comprising the fifth interface.

[0180] In one embodiment, the processor 802 is further configured to perform playback processing on the second data via the chip.

[0181] In one embodiment, the processor 802 is configured to write the second data into the chip via a sixth interface, the second data comprising uplink voice data of a second language type associated with the call, the second interface comprising the sixth interface.

[0182] In one embodiment, the communication interface 801 is used to send the second data to the second terminal through the chip.

[0183] In one embodiment, the processor 802 is configured to:

[0184] Using the first data, third data is obtained, the third data including text data of a first language type associated with the call;

[0185] The third data is subjected to language type conversion processing to obtain the fourth data, which contains text data in a second language type associated with the call;

[0186] The second data is obtained using the fourth data.

[0187] In one embodiment, the processor 802 is configured to perform language type conversion processing on the first data to obtain the second data when the computing power of the first terminal meets a first condition, wherein the first condition includes conditions associated with the first data.

[0188] In one embodiment, the communication interface 801 is used for:

[0189] If the computing power of the first terminal does not meet the first condition, the first data is sent to the first device, which is used for language type conversion processing of the first data and is deployed in the cloud.

[0190] Receive the second data sent by the first device.

[0191] In one embodiment, the processor 802 is configured to:

[0192] At the application layer, the language type conversion function of the first terminal is initiated through the seventh interface;

[0193] And / or,

[0194] At the application layer, the language type conversion function of the first terminal is stopped through the eighth interface.

[0195] It should be noted that the specific processing procedures of the processor 802 and the communication interface 801 can be understood by referring to the above method.

[0196] Of course, in practical applications, the various components in terminal 800 are coupled together through bus system 804. It can be understood that bus system 804 is used to implement communication between these components. In addition to a data bus, bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 8 The general labeled all buses as Bus System 804.

[0197] The memory 803 in this embodiment is used to store various types of data to support the operation of the terminal 800. Examples of such data include any computer program used to operate on the terminal 800.

[0198] The methods disclosed in the embodiments of this application can be applied to, or implemented by, the processor 802. The processor 802 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 802 or by instructions in software form. The processor 802 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 802 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, specifically in memory 803. The processor 802 reads information from memory 803 and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0199] In an exemplary embodiment, terminal 800 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0200] It is understood that the memory (memory 803) in this embodiment of the application can be volatile memory or non-volatile memory, or both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); the magnetic surface memory can be disk storage or magnetic tape storage. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0201] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 803 storing a computer program, which can be executed by the processor 802 of the terminal 800 to complete the steps described in the aforementioned first terminal-side method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0202] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 802 of a terminal 800 to complete the steps described in the aforementioned first terminal-side method.

[0203] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0204] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0205] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data processing method, characterized in that, Applied to the first terminal, including: At the application layer, first data is read through the first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with the call between the first terminal and the second terminal. The chip has at least data playback function and / or data transmission function. The first data is subjected to language type conversion processing to obtain the second data, which includes voice data of a second language type associated with the call; The second data is written to the chip through the second interface between the application layer and the chip.

2. The method according to claim 1, characterized in that, The step of reading the first data through the first interface between the application layer and the chip of the first terminal includes: The first data is read through a third interface, the first data containing voice data of a first language type transmitted downlink in association with the call, the first interface including the third interface.

3. The method according to claim 1, characterized in that, The step of reading the first data through the first interface between the application layer and the chip of the first terminal includes: The first data is read through a fourth interface, the first data containing voice data of a first language type transmitted uplink in association with the call, and the first interface includes the fourth interface.

4. The method according to claim 1, characterized in that, The step of writing the second data into the chip through the second interface between the application layer and the chip includes: The second data is written to the chip via a fifth interface, the second data including voice data of a second language type in downlink transmission associated with the call, and the second interface including the fifth interface.

5. The method according to claim 4, characterized in that, The method further includes: The chip is used to process the second data for playback.

6. The method according to claim 1, characterized in that, The step of writing the second data into the chip through the second interface between the application layer and the chip includes: The second data is written to the chip via a sixth interface, the second data including voice data of a second language type transmitted uplink in association with the call, and the second interface including the sixth interface.

7. The method according to claim 6, characterized in that, The method further includes: The chip transmits the second data to the second terminal.

8. The method according to claim 1, characterized in that, The step of performing language type conversion on the first data to obtain the second data includes: Using the first data, third data is obtained, the third data including text data of a first language type associated with the call; The third data is subjected to language type conversion processing to obtain the fourth data, which contains text data in a second language type associated with the call; The second data is obtained using the fourth data.

9. The method according to claim 1, characterized in that, The step of performing language type conversion on the first data to obtain the second data includes: When the computing power of the first terminal meets the first condition, the first data is processed by language type conversion to obtain the second data, and the first condition includes conditions associated with the first data.

10. The method according to claim 9, characterized in that, The step of performing language type conversion on the first data to obtain the second data includes: If the computing power of the first terminal does not meet the first condition, the first data is sent to the first device, which is used for language type conversion processing of the first data and is deployed in the cloud. Receive the second data sent by the first device.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: At the application layer, the language type conversion function of the first terminal is initiated through the seventh interface; And / or, At the application layer, the language type conversion function of the first terminal is stopped through the eighth interface.

12. A data processing apparatus, characterized in that, Applied to the first terminal, including: The reading unit is used to read first data at the application layer through a first interface between the application layer and the chip of the first terminal. The first data includes voice data of a first language type associated with a call between the first terminal and the second terminal. The chip has at least a data playback function and / or a data transmission function. A conversion unit is used to perform language type conversion processing on the first data to obtain second data, the second data containing voice data of a second language type associated with the call; A writing unit is used to write the second data into the chip through a second interface between the application layer and the chip.

13. A terminal, characterized in that, include: Processor and communication interface; among which, The processor is configured to, at the application layer, read first data through a first interface between the application layer and the chip of the first terminal, the first data including voice data of a first language type associated with a call between the first terminal and the second terminal, the chip having at least data playback and / or data transmission functions; perform language type conversion processing on the first data to obtain second data, the second data including voice data of a second language type associated with the call; and write the second data into the chip through a second interface between the application layer and the chip.

14. A terminal, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 11.

15. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 11.

Images