Data processing method and device and intelligent glasses

Abstract

The invention discloses a data processing method and device and intelligent glasses, and the method comprises the steps: obtaining first audio data and second audio data, wherein the first audio data and the second audio data correspond to different types of languages; encoding the first audio data and the second audio data into one path of audio encoding data; sending the audio coding data to a second device, so that the second device controls the first voice data and the second voice data in the audio coding data to be translated respectively; and receiving and outputting a second translated audio sent by the second device, the second translated audio corresponding to the second audio data and having the same language category as the first audio data.

Description

Technical Field

[0001] The present application relates to the field of real-time translation technology, and more specifically, to a data processing method, device and smart glasses. Background Art

[0002] Current real-time translation smart devices usually cannot translate the languages ​​of the two interacting parties synchronously in real time, but can only collect a sentence of user 1 first, and after this sentence is translated, collect the next sentence of user 2 and translate it. This translation method still has the problem of translation lag, and the smart translation device needs to move frequently between the two interacting parties during the process, affecting the interaction experience and efficiency of the two interacting parties. Summary of the invention

[0003] In view of this, this application provides the following technical solutions:

[0004] A data processing method, applied to a first device, comprising:

[0005] Obtaining first audio data and second audio data, wherein the first audio data and the second audio data correspond to different categories of languages;

[0006] Encoding the first audio data and the second audio data into one channel of audio encoding data;

[0007] Sending the audio coding data to a second device so that the second device controls the first voice data and the second voice data in the audio coding data to be translated respectively;

[0008] A second translated audio sent by the second device is received and output, where the second translated audio corresponds to the second audio data and has the same language category as the first audio data.

[0009] In a possible implementation, obtaining the first audio data and the second audio data includes:

[0010] Acquire original voice data by collecting through a sound pickup device array;

[0011] The original voice data is subjected to voice focusing and noise reduction processing to obtain first audio data and second audio data, wherein the first audio data and the second audio data correspond to different spatial positions.

[0012] In a possible implementation, encoding the first audio data and the second audio data into one channel of audio encoding data includes:

[0013] Orthogonal encoding is performed on the first audio data and the second audio data to obtain a channel of audio encoding data including the first audio data and the second audio data.

[0014] In a possible implementation, the audio encoding data includes an encoding protocol identifier, so that the second device determines a decoding protocol based on the encoding protocol identifier, and decodes the audio encoding data through the decoding protocol to obtain the first audio data and the second audio data.

[0015] A data processing method, applied to a second device, comprising:

[0016] Receiving audio encoding data sent by a first device, where the audio encoding data includes first audio data and second audio data, where the first audio data and the second audio data correspond to different categories of speech;

[0017] Decoding the audio encoding data to obtain the first audio data and the second audio data;

[0018] Controlling to translate the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data;

[0019] The second translated audio is sent to the first device.

[0020] In a possible implementation, the audio coding protocol includes a coding protocol identifier of the audio coding data, and the decoding process of the audio coding data includes:

[0021] A decoding protocol is determined based on the encoding protocol identifier, and the audio encoding data is decoded based on the decoding protocol to obtain the first audio data and the second audio data.

[0022] In a possible implementation, the control performs translation processing on the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data, including:

[0023] Controlling conversion of the first audio data into first text data and converting the second audio data into second text data;

[0024] Controlling translation of the first text data to obtain a first translated text, and translating the second text data to obtain a second translated text, wherein a language type of the first translated text is the same as a language type of the second audio data, and a language type of the second translated text is the same as a language type of the first audio data;

[0025] The first translated text is converted into a first translated audio, and the second translated text is converted into a second translated audio.

[0026] In a possible implementation, the method further includes: outputting the first translated audio, including any one of the following:

[0027] before or after the second translated audio is transmitted to the first device, outputting the first translated audio;

[0028] The first translated audio is sent to a third device, so that the third device outputs the first translated audio.

[0029] In a possible implementation, it also includes:

[0030] A first language type and a second language type are preset, the first language type corresponds to the first audio data, and the second language type corresponds to the second audio data.

[0031] A data processing device, applied to a first device, comprising:

[0032] An audio acquisition module, used to obtain first audio data and second audio data, wherein the first audio data and the second audio data correspond to different categories of languages;

[0033] An audio encoding module, used for encoding the first audio data and the second audio data into one channel of audio encoding data;

[0034] A data sending module, used for sending the audio coding data to a second device, so that the second device controls the first voice data and the second voice data in the audio coding data to be translated respectively;

[0035] The audio receiving module is used to receive and output a second translated audio sent by the second device, where the second translated audio corresponds to the second audio data and has the same language category as the first audio data.

[0036] A data processing device, applied to a second device, comprising:

[0037] A data receiving module, configured to receive audio encoding data sent by a first device, wherein the audio encoding data includes first audio data and second audio data, and the first audio data and the second audio data correspond to different categories of speech;

[0038] an audio decoding module, used for decoding the audio encoding data to obtain the first audio data and the second audio data;

[0039] an audio translation module, used for controlling the translation processing of the first audio data and the second audio data respectively, to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data;

[0040] An audio sending module is used to send the second translated audio to the first device.

[0041] A pair of smart glasses, comprising: a frame, lenses and temples, wherein the frame and / or the temples are provided with at least three pickups at different positions, and further comprising at least one processor and a memory connected to the processor, wherein:

[0042] The memory is used to store computer programs;

[0043] The processor is used to execute the computer program so that the electronic device can implement any one of the data processing methods applied to the first device. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0045] Figure 1 A flowchart of a data processing method disclosed in an embodiment of the present application;

[0046] Figure 2 A flowchart of another data processing method disclosed in an embodiment of the present application;

[0047] Figure 3 A flowchart of the translation of audio data disclosed in the embodiment of the present application;

[0048] Figure 4 A schematic diagram of the implementation principle of the data processing method disclosed in the embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of a data processing device disclosed in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of the structure of another data processing device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] Figure 1 This is a flow chart of a data processing method disclosed in an embodiment of the present application. Figure 1 The method shown is applied to a first device, which may be a smart wearable device, such as smart glasses, smart headphones, smart necklaces, etc. This application does not impose any fixed restrictions on its product form. Figure 1 As shown, the data processing method may include:

[0053] Step 101: Obtain first audio data and second audio data, where the first audio data and the second audio data correspond to different categories of languages.

[0054] The first device is provided with a microphone array, including at least 3 microphones arranged at different positions, which can locate sound sources at different positions in space. The first audio data may be audio corresponding to a first user using the first device, and the second audio data may be audio of a second user talking to the first user. The first user and the second user communicate in different languages, such as the first user speaks Chinese and the second user speaks English, then the first audio data corresponds to Chinese, and the second audio data corresponds to English.

[0055] Step 102: Encode the first audio data and the second audio data into one channel of audio encoding data.

[0056] In order to achieve a more timely and natural real-time interaction effect, it is necessary to translate and output the languages ​​of the two interacting parties in a timely manner. Since the current wireless communication protocol between the first device and the other device has only one communication channel and supports the transmission of one data channel, after obtaining the first audio data and the second audio data, in order to ensure that the two audio data can be transmitted and translated in time, the first audio data and the second audio data can be encoded into one audio encoded data channel. The encoding method used can be but is not limited to orthogonal encoding, such as storing the first audio data in odd bits such as 1, 3, 5, and storing the second audio data in even bits such as 2, 4, 6.

[0057] In one example, the first device can customize and develop a 2-way orthogonal audio coding protocol through the BT SPP (serial port profile) customized private audio transmission protocol, and use the limited bandwidth of Bluetooth to simultaneously transmit the 2-way audio after pickup and processing to the second device, such as a mobile phone. In this way, even if the first user and the second user speak at the same time, the voices of both users can be simultaneously transmitted to the second device for translation processing.

[0058] Step 103: Send the audio encoding data to the second device, so that the second device controls the translation processing of the first voice data and the second voice data in the audio encoding data respectively.

[0059] The second device is a device that can establish a wireless connection with the first device, can control the translation of audio data, and can realize wireless communication with the first device via Bluetooth, WiFi, etc. The second device can control the voice translation locally or in the cloud, and this application does not have a fixed restriction on this.

[0060] The specific processing of the audio encoding data by the second device will be described in detail in other embodiments below and will not be explained in detail here.

[0061] Step 104: receiving and outputting a second translated audio sent by the second device, wherein the second translated audio corresponds to the second audio data and has the same language category as the first audio data.

[0062] After the second device completes the translation of the first audio data and the second audio data in the audio encoding data, it transmits the second translated audio corresponding to the second audio data to the first device, so that the first device plays and outputs the second audio data, so that the first user can understand the speech content of the second user. For example, the first device is smart glasses, and the second device is a mobile phone. When the mobile phone completes the translation of the second audio data corresponding to the second user, it transmits the translated audio back to the smart glasses, and the smart glasses control the speakers set at the temples to output the second translated audio, so that the first user can hear the translated audio and understand the language content of the second user.

[0063] The data processing method described in this embodiment can collect the entire environmental sound in a real-time translation scenario, and obtain the audio data of each of the two parties in the language interaction from the collection results, that is, the first audio data and the second audio data, and encode the two into one audio encoding data to transmit to the second device, which is controlled by the second device to translate the first audio data and the second audio data respectively, and receive the second translated audio output corresponding to the second audio data returned by the second device, so that the user of the first device can timely and accurately understand the language content of the interactive object. During the implementation of this solution, the languages ​​of the two parties in the language interaction can be translated synchronously, and the translation work of the languages ​​of the two parties in the language interaction does not interfere with each other, which can achieve a more natural interactive experience similar to natural language dialogue.

[0064] In the above embodiment, obtaining the first audio data and the second audio data may include: obtaining original voice data by collecting through a sound pickup device array; performing voice focusing and noise reduction processing on the original voice data to obtain the first audio data and the second audio data, wherein the first audio data and the second audio data correspond to different spatial positions.

[0065] In the implementation, a sound pickup device array is installed on the first device, which includes at least 3 voice collection devices with different positions and / or sound pickup directions. When the first user and / or the second user speaks, the sound pickup device array collects and obtains original voice data, and then voice focusing can be achieved based on the known setting parameters (including setting position and sound pickup direction) of each voice collection device in the sound pickup device array, and the voice of the first user and the voice of the second user at different spatial positions can be separated from the original voice data.

[0066] For example, through voice focus, the voice within 30 cm is determined as the voice of the first user, and the original voice data is denoised and separated based on the acoustic algorithm to obtain the first audio data, that is, the near-field audio; at the same time, through voice focus, the voice within 5 meters in front of the first user is determined as the voice of the second user, and the original voice data is denoised and separated based on the acoustic algorithm to obtain the second audio data, that is, the far-field audio.

[0067] The encoding of the first audio data and the second audio data into one channel of audio coded data may include: performing orthogonal coding processing on the first audio data and the second audio data to obtain one channel of audio coded data including the first audio data and the second audio data.

[0068] For example, smart glasses use audio algorithms to simultaneously pick up near-field (wearer's voice) and far-field (interlocutor's voice), reduce noise, and separate them into two channels of audio, namely the first audio data and the second audio data. Then, the two channels of audio can be customized and developed through the BT SPP private protocol to perform orthogonal encoding to obtain one channel of audio encoding data, which can then be transmitted to the mobile phone APP through the BT SPP private protocol, and the audio encoding data can be processed by the APP installed on the mobile phone.

[0069] It is understandable that the wireless connection between the first device and the second device may also transmit other data in addition to the audio encoding data, and the transmission of other data, such as voice messages in instant messaging software, can be achieved through the original default transmission protocol of the Bluetooth protocol, such as the Bluetooth A2DP protocol, or when the second device transmits the second translated audio back to the first device, since the data to be transmitted is itself one channel of data, it can be transmitted based on the A2DP protocol. It can be seen that in the communication between the first device and the second device, it may be based on the A2DP protocol or the SPP protocol. Therefore, the audio encoding data may include a coding protocol identifier, so that the second device determines the decoding protocol based on the coding protocol identifier, and decodes the audio encoding data through the decoding protocol to obtain the first audio data and the second audio data.

[0070] Figure 2 This is a flowchart of another data processing method disclosed in an embodiment of the present application. Figure 2 The method shown should have a second device. Figure 2 As shown, the data processing method may include:

[0071] Step 201: receiving audio encoding data sent by a first device, wherein the audio encoding data includes first audio data and second audio data, and the first audio data and the second audio data correspond to different categories of speech.

[0072] The audio encoding data includes first audio data and second audio data, that is, the voice of the first user and the voice of the second user. Since the wireless communication channel between the first device and the second device can only transmit one channel of data, the first device encodes the first audio data and the second audio data into one channel of audio encoding data and transmits it to the second device.

[0073] Step 202: Decode the encoded audio data to obtain the first audio data and the second audio data.

[0074] After receiving the audio encoding data, the second device first needs to perform decoding processing on it because it is encoded data. The decoding protocol of the decoding processing corresponds to the encoding protocol used by the first device to encode the first audio data and the second audio data. For example, the first device encodes the first audio data and the second audio data using an orthogonal encoding method, and the second device needs to decode the audio encoding data based on the orthogonal decoding method to obtain independent first audio data and independent second audio data.

[0075] Step 203: Control the translation processing of the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data.

[0076] After obtaining the independent first audio data and the second audio data, the second device can control the translation processing of the first audio data and the second audio data respectively. The translation processing of the two audio data does not interfere with each other, and there is no logical order relationship, so that it can be ensured that whether it is the words spoken by the first user or the words spoken by the second user, they can be translated and output in time, so that the two parties of the interaction have a more natural and real interaction experience. The specific implementation of controlling the translation processing of the first audio data and the second audio data will be described in detail in the embodiment content below.

[0077] Step 204: Send the second translated audio to the first device.

[0078] After the second device translates the first translated audio and the second translated audio, since the wearer of the first device has a translation requirement for the second audio data, the second device will play the second translated audio after the second audio data is translated to the first device, so that the first device can output the second translated audio and understand the meaning of the language expressed by the second user. The first translated audio is the audio obtained by translating the first audio data of the first user, and its output needs to be received by the second user. There are different implementations for the output of the first translated audio, which will be introduced in the following embodiments.

[0079] According to the data processing method described in this embodiment, after receiving the audio encoding data, the second device can decode and separate the first audio data and the second audio data, and independently translate the two, and send the second translated audio obtained by translating the second audio data to the first device for output, so that the user of the first device can obtain the translation content of the second user's language in time and realize language communication with the second user.

[0080] In the above embodiment, the audio encoding protocol includes an encoding protocol identifier of the audio encoding data, and the decoding processing of the audio encoding data may include: determining a decoding protocol based on the encoding protocol identifier, decoding the audio encoding data based on the decoding protocol, and obtaining the first audio data and the second audio data.

[0081] As mentioned above, different transmission protocols may be used between the first device and the second device when transmitting different data. Therefore, the communication data between the first device and the second device can carry a coding protocol identifier so that the device receiving the communication data can determine which decoding protocol to use to decode the communication data based on the coding protocol identifier.

[0082] Figure 3 The following is a flowchart of the translation of audio data disclosed in the embodiment of the present application. Figure 3 As shown, the control performs translation processing on the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data, which may include:

[0083] Step 301: Control the conversion of the first audio data into first text data, and the conversion of the second audio data into second text data;

[0084] Since the translation process needs to be performed based on text data, in this implementation, the first audio data and the second audio data need to be converted into corresponding text data before the subsequent translation process can be performed. In the implementation, for the first audio data and the second audio data, the first audio data can be converted into the first text data and the second audio data can be converted into the second text data by calling the big model API. In order to ensure the timeliness of the translation process, two paths can be divided to call different big model APIs to convert the first audio data and the second audio data into text in parallel.

[0085] Step 302: Control the translation of the first text data to obtain a first translated text, and control the translation of the second text data to obtain a second translated text, wherein the language type of the first translated text is the same as the language type of the second audio data, and the language type of the second translated text is the same as the language type of the first audio data.

[0086] After the text data is converted, the first text data and the second text data can be translated and processed respectively through the cloud-based large language model, and the first text data can be translated into a first translated text of the second language type (corresponding to the language type of the second user), and the second text data can be translated into a second translated text of the first language type (corresponding to the language type of the first user).

[0087] Step 303: convert the first translated text into a first translated audio, and convert the second translated text into a second translated audio.

[0088] In order to facilitate language interaction and avoid the tediousness of the first user and the second user needing to watch the translation results through the screen, in the present application, the translated text needs to be converted into translated audio for output, so that the first user and the second user can hear the translation results only through their ears, simulating natural communication between people.

[0089] This embodiment describes in detail the implementation process of translating the first audio data and the second audio data, so as to facilitate technical personnel in the field to better understand and implement the technical solution of the present application.

[0090] In other implementations, the data processing method may further include a step of outputting the first translated audio.

[0091] In one implementation, the first translated audio is outputted before or after the second translated audio is transmitted to the first device. That is, the first translated audio is played and outputted locally on the second device, and the second translated audio is sent to the first device for playback and output. Since the second translated audio is on one path, the default communication protocol between the first device and the second device can be used, such as the Bluetooth A2DP protocol to transmit the second translated audio to the first device. When Bluetooth transmits data, the second device cannot output the audio source at the same time, such as a mobile phone cannot play sound when Bluetooth transmits data. The intelligent device can play and output the second translated audio before or after the transmission is completed, so as to realize the asynchronous output of the first translated audio and the second translated audio.

[0092] In another implementation, the first translated audio is output, and the first translated audio can be sent to a third device so that the third device outputs the first translated audio. That is, the first translated audio is played and output on the third device, and the second translated audio is played and output on the first device. The third device can be a smart bracelet, a smart necklace, etc., but it is not the same device as the first device. Since the second device has transmitted the second translated audio to the first device through the default communication protocol (such as the Bluetooth A2DP protocol), if the second device also needs to transmit the first translated audio to the third device, the first translated audio can be transmitted based on a custom protocol (such as the Bluetooth SPP protocol). In this way, the transmission of the first translated audio and the second translated audio can be carried out independently, and the output of the first translated audio on the third device and the output of the second translated audio on the first device do not interfere with each other, and the two may be output successively or simultaneously.

[0093] The above content introduces different implementation schemes for outputting the first translated audio to meet different usage scenarios and different usage requirements.

[0094] In other implementations, the data processing method may further include: presetting a first language type and a second language type, wherein the first language type corresponds to the first audio data, and the second language type corresponds to the second audio data. Presetting the first language type and the second language type means predetermining the translation direction, thereby ensuring that the translation processing tool can quickly and efficiently complete the translation processing work according to the pre-set content.

[0095] Figure 4 The following is a schematic diagram of the implementation principle of the data processing method disclosed in the embodiment of the present application. The interlocutor represents the second user, and the overall implementation architecture is shown by taking the glasses device as the first device, the mobile phone as the second device, and the bracelet and watch as the third device as an example. Figure 4 , in a specific example, includes the following:

[0096] Smart glasses are connected to the mobile phone via Bluetooth;

[0097] Smart glasses use a 3-mic-5-mic array to pick up the wearer's near-field voice and the interlocutor's far-field voice (within 3 meters in front) through an audio algorithm, while also reducing environmental noise and separating the far-field and near-field voices into two audio channels.

[0098] After the smart glasses complete audio pickup and the acoustic algorithm completes audio noise reduction and separation, a 2-way orthogonal audio encoding protocol is customized and developed using the SPP in the Bluetooth protocol. The limited bandwidth of Bluetooth is used to simultaneously transmit the 2-way audio after pickup and processing to the mobile phone. A 2-way orthogonal audio decoding protocol is customized and developed in the mobile phone app to decode the transmitted encoded audio into 2-way audio (near-field audio and far-field audio).

[0099] The mobile app can set the other party's language and the target language to be translated in advance;

[0100] The mobile app transmits the voice of the smart glasses wearer and the interlocutor to the speech-to-text API, which converts the voice into text and then transmits it to the cloud-based large language model for recognition and translation;

[0101] The mobile app controls the smart glasses wearer to translate the audio text into the interlocutor's language text, and the interlocutor's voice text into the wearer's language text; then the translated text is sent to the text-to-speech API to convert it into 2-way translated audio;

[0102] The translated speech of the interlocutor is transmitted back to the smart glasses via the Bluetooth A2DP protocol;

[0103] After the smart glasses wearer's voice is translated, it can be played through the mobile phone speaker and the device in a time-sharing manner, or it can be transmitted to other wristband devices through the Bluetooth SPP channel with a customized audio encoding method. At the same time, it is necessary to customize and develop an SPP audio decoding solution in the mobile device, so that the smart glasses and the wristband device can play the translated audio at the same time, so as to achieve a better and more efficient real-time translation experience.

[0104] In the real-time translation scenario, the technical solution of the present application does not require the translation and sound pickup device (first device) to move back and forth between the interlocutor and the wearer, nor does it require the processing of lag and pause such as collecting and translating one sentence at a time. It can achieve simultaneous pickup of near-field (wearer's voice) and far-field (interlocutor's voice), noise reduction, and separation into two-channel audio. Through the BT SPP private protocol, the two-channel audio is customized and developed for orthogonal encoding, and then the two-channel audio is decoded and separated on the second device side. With the help of the large model API, speech-to-text is realized, and after translation into the target language, the text-to-speech is controlled to generate the corresponding two-channel audio; by defining a private protocol through SPP on the second device side app and the downlink receiving device (first device) side, the device originally linked to the A2DP protocol and the device linked to the SPP communication private customized development protocol can be selected to play simultaneously in the second device; the second device side can choose to play the first user's translated audio from its own speaker at the same time, or through a customized audio streaming solution, the translated audio is transferred to other wearable device sides for simultaneous playback, which greatly improves the user's different interactive experiences and communication efficiency in diverse scenarios.

[0105] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0106] The method is described in detail in the embodiments disclosed in the above-mentioned application. The method of the application can be implemented by various forms of devices. Therefore, the application also discloses a device, and a specific embodiment is given below for detailed description.

[0107] Figure 5 A schematic diagram of the structure of a data processing device disclosed in an embodiment of the present application. Figure 5 The device shown is applied to the first device. Figure 5 As shown, the data processing device 50 may include:

[0108] The audio acquisition module 501 is used to obtain first audio data and second audio data, where the first audio data and the second audio data correspond to different categories of languages.

[0109] The audio encoding module 502 is used to encode the first audio data and the second audio data into one channel of audio encoding data.

[0110] The data sending module 503 is used to send the audio coding data to the second device, so that the second device controls the translation processing of the first voice data and the second voice data in the audio coding data respectively.

[0111] The audio receiving module 504 is configured to receive and output a second translated audio sent by the second device, wherein the second translated audio corresponds to the second audio data and has the same language category as the first audio data.

[0112] The data processing device described in this embodiment can collect the entire environmental sound in a real-time translation scenario, and obtain the audio data of each of the two parties in the language interaction from the collection results, that is, the first audio data and the second audio data, and encode the two into one audio encoding data to transmit to the second device, which is controlled by the second device to translate the first audio data and the second audio data respectively, and receive the second translated audio output corresponding to the second audio data returned by the second device, so that the user of the first device can timely and accurately understand the language content of the interactive object. During the implementation of this solution, the languages ​​of the two parties in the language interaction can be translated synchronously, and the translation work of the languages ​​of the two parties in the language interaction does not interfere with each other, which can achieve a more natural interactive experience similar to natural language dialogue.

[0113] Figure 6 A schematic diagram of the structure of another data processing device disclosed in an embodiment of the present application. Figure 6 The device shown is applied to a second device. Figure 6 As shown, the data processing device 60 may include:

[0114] The data receiving module 601 is used to receive audio encoding data sent by the first device, where the audio encoding data includes first audio data and second audio data, and the first audio data and the second audio data correspond to different categories of speech.

[0115] The audio decoding module 602 is used to decode the audio encoding data to obtain the first audio data and the second audio data.

[0116] The audio translation module 603 is used to control the translation processing of the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data.

[0117] The audio sending module 604 is configured to send the second translated audio to the first device.

[0118] The data processing apparatus described in this embodiment can decode and separate the first audio data and the second audio data after receiving the audio encoding data, and independently translate the two, and send the second translated audio obtained by translating the second audio data to the first device for output, so that the user of the first device can obtain the translation content of the second user's language in time and realize language communication with the second user.

[0119] The specific implementation of each module included in the above data processing device and other possible implementations can be found in the corresponding part of the method embodiment, which will not be repeated here.

[0120] Any one of the data processing devices described in the above embodiments includes a processor and a memory. The audio acquisition module, audio encoding module, data sending module, audio receiving module, data receiving module, audio decoding module, audio translation module, audio sending module, etc. in the above embodiments are all stored in the memory as program modules, and the processor executes the above program modules stored in the memory to realize corresponding functions.

[0121] The processor includes a kernel, which retrieves the corresponding program module from the memory. One or more kernels can be set, and the processing of the access data can be realized by adjusting the kernel parameters.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer and contains software code. After being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the above data processing method.

[0124] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer, wherein the computer program contains software codes, and after being loaded and executed by a computer, the computer program can implement the steps shown in any embodiment of the data processing method described above.

[0125] Furthermore, an embodiment of the present application provides a pair of smart glasses, comprising: a frame, lenses and temples, wherein at least three pickups at different positions are arranged on the frame and / or the temples, and at least one processor and a memory connected to the processor, wherein:

[0126] The memory is used to store computer programs;

[0127] The processor is used to execute the computer program so that the smart glasses can implement any data processing method in the aforementioned embodiments that can be applied to the first device.

[0128] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0129] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0130] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0131] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data processing method, applied to a first device, include: Obtaining first audio data and second audio data, wherein the first audio data and the second audio data correspond to different categories of languages; Encoding the first audio data and the second audio data into one channel of audio encoding data; Sending the audio coding data to a second device so that the second device controls the first voice data and the second voice data in the audio coding data to be translated respectively; A second translated audio sent by the second device is received and output, where the second translated audio corresponds to the second audio data and has the same language category as the first audio data.

2. The data processing method according to claim 1, wherein the first audio data and the second audio data are obtained. include: Acquire original voice data by collecting through a sound pickup device array; The original voice data is subjected to voice focusing and noise reduction processing to obtain first audio data and second audio data, wherein the first audio data and the second audio data correspond to different spatial positions.

3. The data processing method according to claim 1, wherein the first audio data and the second audio data are encoded into one channel of audio encoded data. include: Orthogonal encoding is performed on the first audio data and the second audio data to obtain a channel of audio encoding data including the first audio data and the second audio data.

4. According to the data processing method according to claim 1, the audio encoding data includes an encoding protocol identifier, so that the second device determines a decoding protocol based on the encoding protocol identifier, and decodes the audio encoding data through the decoding protocol to obtain the first audio data and the second audio data.

5. A data processing method, applied to a second device, include: Receiving audio encoding data sent by a first device, where the audio encoding data includes first audio data and second audio data, where the first audio data and the second audio data correspond to different categories of speech; Decoding the coded audio data to obtain the first audio data and the second audio data; Controlling to translate the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data; The second translated audio is sent to the first device.

6. The data processing method according to claim 5, wherein the audio coding protocol includes a coding protocol identifier of the audio coding data, and the decoding process is performed on the audio coding data. include: A decoding protocol is determined based on the encoding protocol identifier, and the audio encoding data is decoded based on the decoding protocol to obtain the first audio data and the second audio data.

7. The data processing method according to claim 5, wherein the control performs translation processing on the first audio data and the second audio data respectively to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data, include: Controlling conversion of the first audio data into first text data and converting the second audio data into second text data; Controlling translation of the first text data to obtain a first translated text, and translating the second text data to obtain a second translated text, wherein a language type of the first translated text is the same as a language type of the second audio data, and a language type of the second translated text is the same as a language type of the first audio data; The first translated text is converted into a first translated audio, and the second translated text is converted into a second translated audio.

8. The data processing method according to claim 5, further comprising: include: Outputting the first translated audio includes any one of the following: before or after the second translated audio is transmitted to the first device, outputting the first translated audio; The first translated audio is sent to a third device, so that the third device outputs the first translated audio.

9. The data processing method according to claim 5, further comprising: include: A first language type and a second language type are preset, the first language type corresponds to the first audio data, and the second language type corresponds to the second audio data.

10. A data processing device, applied to a first device, include: An audio acquisition module, used to obtain first audio data and second audio data, wherein the first audio data and the second audio data correspond to different categories of languages; An audio encoding module, used for encoding the first audio data and the second audio data into one channel of audio encoding data; A data sending module, used for sending the audio coding data to a second device, so that the second device controls the first voice data and the second voice data in the audio coding data to be translated respectively; The audio receiving module is used to receive and output a second translated audio sent by the second device, where the second translated audio corresponds to the second audio data and has the same language category as the first audio data.

11. A data processing device, applied to a second device, include: A data receiving module, configured to receive audio encoding data sent by a first device, wherein the audio encoding data includes first audio data and second audio data, and the first audio data and the second audio data correspond to different categories of speech; an audio decoding module, used for decoding the audio encoding data to obtain the first audio data and the second audio data; an audio translation module, used for controlling the translation processing of the first audio data and the second audio data respectively, to obtain a first translated audio corresponding to the first audio data and a second translated audio corresponding to the second audio data; An audio sending module is used to send the second translated audio to the first device.

12. A pair of smart glasses, include: A spectacles frame, a lens and a temple, wherein the spectacles frame and / or the temple are provided with at least three pickups at different positions, and further comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the data processing method as described in any one of claims 1 to 4.

Images