Translation method, device and storage medium

Abstract

This application provides a translation method, device, and storage medium. In this method, when communication is established between the electronic device and the translation headset, and the audio reception is continuous, the user can automatically switch the headset's usage mode as needed without interrupting the currently executing translation task, such as capturing source language audio. This allows the translation application's dialogue translation function to better suit the current usage scenario, thereby providing better service and ensuring a better user experience.

Description

Technical Field

[0001] This application relates to the field of terminal equipment technology, and in particular to a translation method, device and storage medium. Background Technology

[0002] In the context of globalization, cross-cultural communication has become an inevitable trend. Because different countries and regions use different languages, people are no longer limited to their own native languages ​​when communicating and cooperating.

[0003] However, not everyone possesses the ability to learn and communicate across languages. Thus, when users speaking different languages ​​converse, language barriers can hinder communication, impacting efficiency and even leading to the failure of exchanges and collaborations. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a translation method, device, and storage medium, aiming to resolve the difficulties of cross-language communication, help users of different languages ​​communicate, and improve dialogue efficiency.

[0005] In a first aspect, embodiments of this application provide a translation method applied to an electronic device. The method includes: displaying a first interface of a translation application, the first interface including a first control and a second control, the first control indicating that the translation application is in a non-receiving state for dialogue translation, and the second control indicating that a translation headset is in a first usage mode, wherein the electronic device establishes a communication connection with the first and / or second headsets of the translation headset in the first usage mode; upon receiving a user's operation on the first control or the translation headset, starting to collect source language sound and displaying a second interface, the second interface including a second control and a third control, the third control indicating that the translation application is in a receiving state for dialogue translation; upon receiving a user's operation on the second control, continuing to collect source language sound and displaying a third interface, the third interface including the second control, first source language text content that changes in real time following the collected source language sound, and a first window including a first option; upon receiving a user's operation on the first option, continuing to collect source language sound and displaying a fourth interface, the fourth interface including the third control, The system includes a first source language text content that changes in real time following the acquired source language audio, and a fourth control. The fourth control indicates that the translation headset is in a second usage mode, in which the electronic device establishes a communication connection with the first and second headsets. When the language type corresponding to the first source language text content is the second language, after the first source language text content is fully displayed, the first headset is used to play the first translated language text content in the first language, and a fifth interface is displayed. When the language type corresponding to the first source language text content is the first language, after the first source language text content is fully displayed, the second headset is used to play the first translated language text content in the second language, and a fifth interface is displayed. The fifth interface includes the fourth control, the first source language text content, the first translated language text content, and the fifth control. The fifth control indicates that the translation application is in the broadcast state of the dialogue translation function, and the first translated language text content is the translation result of the first source language text content.

[0006] The first interface, for example, is Figure 17 Interface 239 is shown in (1).

[0007] The first control, for example, is Figure 17 Control 2391 is shown in (1).

[0008] The second control, for example, is Figure 17 Icon 2031 is shown in (1).

[0009] In scenarios where the audio reception method is continuous, the non-receiving state refers to the state where the microphone of the electronic device is not picking up sound and is not playing the translated audio. That is, the state where translation services are not being performed.

[0010] In scenarios where the sound reception method is continuous sound reception, the sound reception state refers to the state in which the electronic device acquires the source language sound and obtains the source language audio through its own microphone.

[0011] The "broadcasting state" refers to the state in which the electronic device plays the translated audio through its own speaker or connected translation headphones.

[0012] The first usage mode is, for example, the exclusive mode described in the following embodiments.

[0013] In this embodiment of the application, the exclusive mode refers to a situation where, among users engaged in multilingual real-time dialogue, one party wears a translation headset to listen to the translated audio through the headset, while the other party listens to the translated audio through the speaker of an electronic device.

[0014] The second usage mode is, for example, the sharing mode described in the following embodiments.

[0015] In this embodiment of the application, the sharing mode refers to the situation where, among users engaging in multilingual real-time dialogue, each party wears a translation headset and listens to the corresponding translated audio using their respective translation headset.

[0016] The second interface, for example, is Figure 17 Interface 240 shown in (2) Figure 17 Interface 241, etc., shown in (3).

[0017] Among them, the third control, for example, is Figure 17 Control 2403 is shown in (2).

[0018] The first window, for example, is Figure 10 The window 2191 shown in (2) is shown in the middle.

[0019] The first option, for example, is Figure 10 Option 2191-3 is shown in (2).

[0020] The third interface may be, for example, interface 240 or interface 241 that displays window 2191. That is, the third interface may include window 2191 and all elements of interface 240 or interface 241.

[0021] Understandably, during the display of window 2191, the electronic device's speaker can continue to capture the source language sound. Therefore, following the captured source language sound, the content of the first source language text can change in real time.

[0022] The fourth interface, for example, is Figure 18 Interface 246 is shown in (2).

[0023] Among them, the fourth control is, for example, Figure 18 Icon 2201 is shown in (2).

[0024] Here, the first language and the second language are two different languages. For example, the first language is the language displayed in control 2025-1, such as English, and the second language is the language displayed in control 2025-3, such as Chinese.

[0025] The fifth interface, for example, is Figure 18 Interface 248 is shown in (4).

[0026] Among them, the fifth control, for example is Figure 18 Control 2432 is shown in (4).

[0027] The translation earphones can be either wired or wireless.

[0028] In the embodiments of this application, the translation headset refers to a pair of headsets that can establish a communication connection with an electronic device and work together with a translation application installed in the electronic device to perform translation services.

[0029] Taking wireless earbuds as an example, this pair of earbuds can be understood as two earbuds belonging to the same earbud case. In some implementations, one of these earbuds can be called the first earbud, and the other can be called the second earbud.

[0030] The first earphone can be the master earphone, and the second earphone can be the slave earphone.

[0031] The main earpiece can be either the left or right earpiece.

[0032] Accordingly, the earphone can be either the right earphone or the left earphone.

[0033] The first and second earpieces can each be equipped with independent control chips. That is, both the first and second earpieces can respond to the user's touch operations and make corresponding responses.

[0034] In the embodiments of this application, the first earphone is, for example, the left earphone, and the second earphone is, for example, the right earphone.

[0035] Therefore, in scenarios where communication is established between the electronic device and the translation headset, and the sound reception method is continuous, the user can automatically switch the usage mode of the translation headset as needed without interrupting the currently executed translation business, such as collecting source language sound. This allows the dialogue translation function of the translation application to be better suited to the current usage scenario, thereby providing better service to the user and ensuring a better user experience.

[0036] According to the first aspect, the method further includes: while using the first earpiece to play the first translated language text content in the first language, continuing to collect source language sound; after using the first earpiece to play the first translated language text content in the first language, continuing to collect source language sound and displaying a sixth interface, the sixth interface including a third control, a fourth control, the first source language text content, the first translated language text content, and second source language text content that changes in real time following the collected source language sound; if the language type corresponding to the second source language text content is the first language, after the second source language text content is fully displayed, using the second earpiece to play the second translated language text content in the second language and displaying a seventh interface; if the language type corresponding to the second source language text content is the second language, after the second source language text content is fully displayed, using the first earpiece to play the second translated language text content in the first language and displaying a seventh interface; wherein, the seventh interface includes a fourth control, the first source language text content, the first translated language text content, the second source language text content, the second translated language text content, and a fifth control, the second translated language text content being the translation result of the second source language text content.

[0037] Therefore, in a scenario where the sound reception method is continuous and the translation headset is used in the second usage mode, the source language sound is continuously acquired through the speaker of the electronic device, and the translated language text content to be played is played through the first or second headset, thus enabling the two translation services of source language acquisition and translated audio playback to be performed simultaneously.

[0038] According to the first aspect, or any implementation of the first aspect above, the method further includes: during the display of the fifth control, after receiving the user's operation on the fourth control, continuing to collect source language sound, playing the first translated language text content using a first or second headset, and displaying an eighth interface, the eighth interface including the fourth control, the first source language text content, the first translated language text content, and a first window, the first window further including a second option; after receiving the user's operation on the second option, continuing to collect source language sound, playing the first translated language text content using the speaker of the electronic device or the translation headset, and displaying a ninth interface, the ninth interface including the second control, the fifth control, the first source language text content, and the first translated language text content; after playing the first translated language text content using the speaker or translation headset, continuing to collect source language sound and displaying a tenth interface, the tenth interface including the second... The interface includes a control, a third control, first source language text content, first translated language text content, and second source language text content that changes in real time following the collected source language sound. If the language type corresponding to the second source language text content is the first language, after the second source language text content is fully displayed, the translation headset is used to play the second translated language text content in the second language, and the eleventh interface is displayed. If the language type corresponding to the second source language text content is the second language, after the second source language text content is fully displayed, the speaker is used to play the second translated language text content in the first language, and the eleventh interface is displayed. The eleventh interface includes a second control, a fifth control, first source language text content, first translated language text content, first source language text content, first translated language text content, and the second translated language text content is the translation result of the second source language text content.

[0039] In this embodiment of the application, the first window, for example Figure 10 The window 2221 shown in (5) is shown in the middle.

[0040] Accordingly, the second option, for example Figure 10 Option 2221-1 is shown in (5).

[0041] Therefore, in scenarios where communication is established between the electronic device and the translation headset, and the audio reception is continuous, the user can automatically switch the headset's usage mode as needed without interrupting the current translation task, such as playing translated text content. This allows the translation application's dialogue translation function to be better suited to the current usage scenario, thereby providing better service and ensuring a better user experience.

[0042] According to the first aspect, or any implementation of the first aspect above, the method further includes: during the process of playing the first translated language text content using the first or second earphone while displaying the fifth interface, upon receiving the user's operation on the fifth control or the translation earphone, stopping the playback of the first translated language text content, continuing to collect source language sound, and displaying the twelfth interface, the twelfth interface including the third control, the fourth control, the first source language text content, the first translated language text content, and the second source language text content that changes in real time following the collected source language sound.

[0043] This includes operations on the translation earpieces, such as clicking on the first earpiece or clicking on the second earpiece.

[0044] Therefore, in scenarios where the sound reception method is continuous, by establishing a binding relationship (mapping relationship) between the fifth control and the translation headset, when the translation application is in broadcast mode, the user can click the fifth control or the translation headset at any time as needed to control the translation headset to stop playing the translated language text content, so that the user interface can quickly return to the user interface corresponding to the sound reception mode and display the latest source language text content.

[0045] Furthermore, by establishing a binding relationship between the fifth control and the translation headset, the controls displayed in the user interface can change accordingly when the translation headset is clicked. This ensures a good user experience while enabling the translation application to be better applied to various usage scenarios.

[0046] According to the first aspect, or any implementation of the first aspect above, the method further includes: during the process of collecting source language sound while displaying the twelfth interface, when the user's operation on the third control or translation headset is received, the collection of source language sound is stopped, and the thirteenth interface is displayed, the thirteenth interface including the first control, the fourth control, the first source language text content, and the first translation language text content.

[0047] Among them, the operation of the translation earpiece, for example, is a click operation on the first earpiece or a click operation on the second earpiece.

[0048] Therefore, in scenarios where the sound reception method is continuous, by establishing a binding relationship (mapping relationship) between the third control and the translation headset, when the translation application is working in the sound reception state, the user can click the third control or the translation headset at any time as needed to control the translation headset to stop collecting the source language sound and exit the sound reception state, so that the user interface can quickly return to the user interface corresponding to the non-sound reception state.

[0049] Furthermore, by establishing a binding relationship between the third control and the translation headset, the controls displayed in the user interface can change accordingly when the translation headset is clicked. This ensures a good user experience while enabling the translation application to be better applied to various usage scenarios.

[0050] Furthermore, when exiting the recording mode and returning to the non-recording mode, the second source language text content, regardless of whether the real-time changes following the captured source language sound are fully displayed, is discarded. That is, it is not displayed in the user interface corresponding to the non-recording mode, reducing the consumption of device resources.

[0051] According to the first aspect, or any implementation of the first aspect above, the method further includes: when the user's operation on the third control or the translation headset is received, and the second source language text content is fully displayed, displaying the fourteenth interface, the fourteenth interface including the first control, the fourth control, the first source language text content, the first translation language text content, the second source language text content, and the second translation language text content, the second translation language text content being the translation result of the second source language text content.

[0052] Therefore, when exiting the recording state and returning to the non-recording state, if the second source language text content, which follows the real-time changes of the collected source language sound, has been fully displayed, the second source language text content is retained, and the corresponding second translated language text content is displayed, thus avoiding the loss of useful information and ensuring the integrity of the dialogue.

[0053] According to the first aspect, or any implementation of the first aspect above, the method further includes: during the display of the fourth control, after detecting a disconnection in communication with the first or second earphone, continuing the currently executed translation service and displaying the fifteenth interface, the fifteenth interface including the second control.

[0054] Therefore, when the electronic device establishes a communication connection with both the first and second earpieces of the translation headset, and the translation headset is in the second usage mode, if it is detected that one earpiece of the translation headset, such as the first or second earpiece, has lost its communication connection with the electronic device, the usage mode of the translation headset is automatically switched back to the first usage mode. This allows the translated text content of the dialogue between the two parties to still be automatically played through different audio playback devices (before disconnection: one party through the first earpiece, and the other party through the second earpiece; after disconnection: one party through the speaker of the electronic device, and the other party through the currently connected earpiece), thereby ensuring the efficiency of the dialogue.

[0055] According to the first aspect, or any implementation of the first aspect above, the method further includes: during the display of the second control or the fourth control, after detecting that the communication connection with the translation headset has been disconnected, stopping the currently executed translation service and displaying the sixteenth interface, the sixteenth interface does not include the second control or the fourth control; wherein, when neither the second control nor the fourth control is displayed, when there is translated language text content that needs to be played, the speaker of the electronic device is used to play the translated language text content.

[0056] In this embodiment of the application, disconnecting the electronic device from the translation headset means disconnecting the communication connection from both the first and second earpieces of the translation headset.

[0057] The sixteenth interface, for example, is Figure 14 Interface 230 is shown in (3).

[0058] For specific implementation details regarding this aspect, please refer to [link / reference]. Figure 14 The description of the illustrated embodiment will not be repeated here.

[0059] According to the first aspect, or any implementation of the first aspect above, the method further includes: when neither the second control nor the fourth control is displayed, during the process of playing the translated language text content using the speaker of the electronic device, upon detecting the establishment of a communication connection with the translation headset, continuing the currently executed translation service and displaying the seventeenth interface, the seventeenth interface including the second control or the fourth control; wherein, when the second control is displayed on the seventeenth interface, if the translated language text content is in the first language, the translated language text content is played again using the speaker; if the translated language text content is in the second language, the translated language text content is played using the translation headset; when the fourth control is displayed on the seventeenth interface, if the translated language text content is in the first language, the translated language text content is played using the first headset; if the translated language text content is in the second language, the translated language text content is played using the second headset.

[0060] Therefore, once communication is established between the electronic device and the translation headset, the currently executing translation service is not interrupted. Instead, the corresponding playback device (first headset or second headset, or speaker, translation headset) is controlled to play the translated language text content directly according to the working mode of the translation headset, thus ensuring dialogue efficiency.

[0061] According to the first aspect, or any implementation of the first aspect above, the source language sound is captured using the microphone of the electronic device during the display of the third control.

[0062] That is, in some implementations, in scenarios where the sound reception method is continuous sound reception, when the translation application is working in sound reception mode, regardless of whether the translation headset is in exclusive mode or shared mode, the microphone of the electronic device can be used to collect the source language sound.

[0063] Secondly, embodiments of this application provide an electronic device. The electronic device includes: a memory and a processor, the memory and the processor being coupled; the memory stores program instructions, which, when executed by the processor, cause the electronic device to perform the methods of the first aspect or any possible implementation thereof.

[0064] Thirdly, this application provides a translation headset. The translation headset includes a memory and a processor, coupled together; the memory stores program instructions, which, when executed by the processor, cause the translation headset to perform the method in the first aspect or any possible implementation thereof.

[0065] The translation headset includes a first headset and a second headset.

[0066] The first earphone can be the master earphone, and the second earphone can be the slave earphone.

[0067] The main earpiece can be either the left or right earpiece.

[0068] Accordingly, the earphone can be either the right earphone or the left earphone.

[0069] The first and second earpieces can each be equipped with independent control chips. That is, both the first and second earpieces can respond to the user's touch operations and make corresponding responses.

[0070] The translation earphones can be either wired or wireless.

[0071] Fourthly, embodiments of this application provide a computer-readable medium for storing a computer program, the computer program including instructions for performing the method in the first aspect or any possible implementation of the first aspect.

[0072] Fifthly, embodiments of this application provide a computer program including instructions for performing the method in the first aspect or any possible implementation thereof.

[0073] Sixthly, embodiments of this application provide a chip including a processing circuit and transceiver pins. The transceiver pins and the processing circuit communicate with each other via an internal connection path. The processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive signals and to control the transmitting pin to transmit signals. Attached Figure Description

[0074] Figure 1 This is a schematic diagram of the hardware structure of an electronic device as an example.

[0075] Figure 2 A schematic diagram illustrating the software structure of an electronic device as an example;

[0076] Figure 3 This is a flowchart illustrating the user interface corresponding to a decision-making dialogue translation function as an example.

[0077] Figure 4 One example shown is based on Figure 3 The user interface diagram showing the processing logic is shown below;

[0078] Figure 5 This is a user interface diagram illustrating a usage scenario involving switching the display mode of a dialogue translation function;

[0079] Figure 6 This is an example of a user interface diagram illustrating a private mode for first-time use of headphones;

[0080] Figure 7 This is a schematic diagram of a user interface involved in a use case of switching audio channels, as an example.

[0081] Figure 8 This is an example of a user interface that displays more settings functions;

[0082] Figure 9 This is an example of a user interface diagram illustrating the use of the dialogue translation function in the exclusive mode.

[0083] Figure 10 This is an illustrative user interface diagram illustrating a usage scenario involving switching headphone usage modes.

[0084] Figure 11 This is an example illustration of a user interface for viewing detailed information about headphone usage patterns;

[0085] Figure 12 This is a user interface diagram illustrating an example of a use case where a shared mode automatically switches to a dedicated mode.

[0086] Figure 13 This is an illustrative user interface diagram illustrating a usage scenario involving a transition from not connecting headphones to connecting headphones.

[0087] Figure 14This is an illustrative user interface diagram illustrating a use case involving connecting and disconnecting headphones.

[0088] Figure 15A This is an example of a user interface diagram illustrating a use case of click-to-receive + exclusive mode in a conversational mode.

[0089] Figure 15B This is an example of a user interface diagram illustrating a use case of click-to-receive + exclusive mode in another dialogue mode.

[0090] Figure 16A This is an example of a user interface diagram illustrating a use case of click-to-receive + share mode in a dialogue mode.

[0091] Figure 16B This is an example of a user interface diagram illustrating a use case of click-to-receive + share mode in another dialogue mode.

[0092] Figure 17 This is an example of a user interface diagram illustrating a use case of continuous audio reception + private audio mode in a dialogue mode.

[0093] Figure 18 This is an illustrative user interface diagram illustrating a use case of continuous audio reception + sharing mode in a dialogue mode.

[0094] Figure 19 This is an example of a user interface diagram illustrating the switching from private mode to shared mode in a continuous audio reception scenario within a conversational mode. Detailed Implementation

[0095] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0096] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0097] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0098] In the description of the embodiments of this application, the words "exemplary," "for example," or "optionally" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "exemplary," "for example," or "optionally" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the words "exemplary," "for example," or "optionally" is intended to present the relevant concepts in a specific manner.

[0099] In the description of the embodiments of this application, the names of various controls, areas, options, and entry points are for illustrative purposes only and are not intended to limit the embodiments of this application. That is, in actual use, they may also be described with other names.

[0100] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0101] In the description of the embodiments of this application, unless otherwise stated, only one user interface can be displayed at a time. For example, if interface 201 is displayed, interface 202 will not be displayed on the current screen. Conversely, if interface 202 is displayed, interface 201 will not be displayed on the current screen. However, the elements (options, controls, entries, message bubbles, windows, etc.) included in different interfaces can be the same.

[0102] In the description of the embodiments of this application, unless otherwise stated, the dashed lines appearing in the drawings are for illustration only. That is, they are not shown in actual use.

[0103] Cross-linguistic communication refers to the exchange of information between people in two or more languages, including written, oral, and auditory communication. Its purpose is to make information in one language understandable to people speaking another language; it is a new form of communication that has emerged alongside globalization.

[0104] However, not everyone possesses the ability to learn and communicate across languages. Thus, when users speaking different languages ​​converse, language barriers can hinder communication, impacting efficiency and even leading to the failure of exchanges and collaborations.

[0105] In view of this, embodiments of this application provide a translation method aimed at solving the difficulties of cross-language communication, helping users of different languages ​​to communicate, and improving dialogue efficiency.

[0106] To better understand the technical solutions provided in the embodiments of this application, before describing the technical solutions in the embodiments of this application, the hardware and software structures of the electronic devices to which the embodiments of this application are applicable will first be described in conjunction with the accompanying drawings.

[0107] It should be noted that, in the description of the embodiments of this application, the electronic device can be any portable mobile device with a display screen.

[0108] For example, in some embodiments of this application, the electronic device is such as a smartphone, tablet computer, wearable device (such as a smartwatch), etc.

[0109] The term "smartphone" can refer to either a conventional phone or a foldable phone with an external screen. A conventional phone (also known as a candybar phone) typically includes a display screen and a back cover, which are positioned opposite each other. The size of the display screen and back cover is generally the same as the overall dimensions of a conventional phone, and their shape cannot be altered. A foldable phone, on the other hand, includes an external screen, a back cover, a hinge, and an internal screen. The internal screen is foldable, while the external screen is non-foldable; the internal screen is approximately twice the size of the external screen.

[0110] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the sole limitation on conventional equipment.

[0111] See Figure 1 The diagram illustrates, for example, a hardware structure diagram of an electronic device. Figure 1 As shown, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0112] The audio module 170 may include a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, etc.

[0113] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc., which will not be listed here, and this application does not impose any restrictions on them.

[0114] The processor 110 may include one or more processing units, which may be independent devices or integrated into one or more processors.

[0115] For example, in some implementations, processor 110 may include an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural network processing unit (NPU), etc., which will not be listed here, and this application does not limit them.

[0116] The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0117] The processor 110 may also include a memory for storing instructions and data. In some implementations, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0118] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to perform data storage.

[0119] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function, such as the translation application providing dialogue translation and simultaneous interpretation functions as described in this embodiment. The data storage area can store data created during the use of the electronic device 100, such as source language text content and translation language text content obtained using the translation application.

[0120] For example, in some implementations, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0121] The wireless communication function of the electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0122] Among them, the mobile communication module 150 can provide wireless communication solutions for electronic devices 100, including second-generation wireless telephone technology (2G), third-generation mobile communication technology (3G), fourth-generation mobile communication technology (4G), and fifth-generation mobile communication technology (5G).

[0123] The wireless communication module 160 can provide solutions for wireless communication applied to the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0124] In some implementations, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. For example, after acquiring source language text content, electronic device 100 can access a mobile network through mobile communication module 150, and then, based on the mobile network and antenna 1, access the corresponding server to translate the source language text content. Alternatively, it can access a wireless network through wireless communication module 160, and then, based on the wireless network and antenna 2, access the corresponding server to translate the source language text content.

[0125] The camera 193 is used to capture still images or videos. In some implementations, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0126] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some implementations, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0127] Specifically, in the technical solution provided in this application embodiment, the display screen 194 can be used to display various user interfaces involved in the dialogue translation function provided by the translation application, as well as various user interfaces involved in the simultaneous interpretation function, etc.

[0128] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment and is not intended to specifically limit the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, or software, or a combination of software and hardware.

[0129] In addition, it should be noted that, in Figure 1 The electronic device 100 shown also runs an operating system on its components. Examples include Apple's iOS operating system, Google's Android open-source operating system, and Microsoft's Windows operating system. These operating systems can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.

[0130] For ease of explanation, this application uses the layered architecture of the Android system as an example to illustrate the software structure of the electronic device 100.

[0131] It should be noted that although the embodiments of this application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0132] See Figure 2 The diagram illustrates, for example, the software architecture of an electronic device. Figure 2 As shown, the layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments of this application, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0133] The application layer can include a series of application packages. For example... Figure 2 As shown, the application package may include translation, settings, video, app store and other applications, which will not be listed here and this application does not impose any restrictions on them.

[0134] In the embodiments provided in this application, the translation application can provide various translation functions, such as dialogue translation and simultaneous interpretation, to help users of different languages ​​communicate and thus solve the difficulties of cross-language communication.

[0135] It should be noted that in some implementations, the launch entry point for the translation application can be set in the settings application, the control center, or a shortcut entry point can be added to the desktop.

[0136] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. In some embodiments of this application, these programming interfaces and frameworks can be described as functions. Figure 2As shown, the application framework layer may include functions such as window manager, content provider, view system, and resource manager, which will not be listed here, and this application does not impose any restrictions on them.

[0137] The Android Runtime comprises the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0138] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0139] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0140] The system library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0141] The kernel layer is the layer between hardware and software. It includes at least display drivers, camera drivers, sensor drivers, and power management drivers.

[0142] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to limit the specific operating system of the electronic device 100. In other embodiments of this application, the operating system of the electronic device 100 may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0143] Taking a mobile phone as an example of the electronic device with the above-described hardware and software structures, the translation method provided in the embodiments of this application will be specifically explained in conjunction with the accompanying drawings.

[0144] For example, in some embodiments of this application, the translation application may be a pre-installed application, that is, an application pre-installed on the desktop. In this way, the user can launch the translation application simply by clicking the icon of the translation application displayed on the desktop.

[0145] For example, in some other embodiments of this application, the translation application may also be a non-pre-installed application, that is, an application that is not pre-installed on the desktop. Typically, the launch entry for a non-pre-installed application can be integrated into the settings application or downloaded from an app store. This way, it does not occupy desktop display area, and when the user needs to use the translation application, they can manually add it to the desktop through the settings application, or directly launch the translation application from the entry provided by the settings application.

[0146] For example, in some other embodiments of this application, the translation application may also support opening via a quick service center, such as from the negative one screen of a mobile phone.

[0147] For example, in some other embodiments of this application, the translation application may also support being added to the desktop as a desktop card and then launched through the desktop card.

[0148] For example, in some other implementations of this application, the translation application can also be started and stopped through an entry point provided by the control center.

[0149] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0150] Specifically, in this embodiment of the application, the translation application is launched by clicking the icon of the translation application displayed on the desktop.

[0151] For example, in some implementations, after a user clicks the icon of the translation application displayed on the desktop, the phone responds to the user's action by launching the translation application, and the phone's user interface can display the user interface corresponding to the conversation translation function by default.

[0152] For example, in some other implementations, after a user clicks the icon of the translation application displayed on the desktop, the phone responds to the user's action by launching the translation application, and the phone's user interface can display the user interface corresponding to the simultaneous interpretation function by default.

[0153] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0154] Specifically, in this embodiment of the application, the user interface of the mobile phone defaults to displaying the user interface corresponding to the dialogue translation function after the translation application is launched.

[0155] To make it easier to understand, we will first introduce the two translation functions: dialogue translation and simultaneous interpretation.

[0156] Dialogue translation typically refers to communication between two people through a question-and-answer format. Specifically, in this embodiment, when a user activates the dialogue translation function in a translation application, translation can be performed between the two parties in a conversation, ensuring that both can understand each other's expressions. For detailed usage examples of the dialogue translation function, please refer to the following description of usage scenarios; it will not be repeated here.

[0157] Simultaneous interpreting is a technology-assisted translation method that translates content continuously to the audience without interrupting the speaker. It is widely used in large international conferences, high-level negotiations, and other events, offering high efficiency and immediacy.

[0158] Specifically, in the translation method provided in this application embodiment, after the user clicks the translation application icon displayed on the desktop, they can follow... Figure 3 The processing logic shown determines which user interface to display and then displays it.

[0159] In addition, it should be noted that, Figure 3 The processing logic shown can also be applied to other user operations that trigger the display of the user interface corresponding to the dialogue translation function, such as when a user clicks on the user interface corresponding to the translation application to select an option for the dialogue translation function (e.g., option 2021 as described in the following embodiment).

[0160] In other words, each time the user interface corresponding to the dialogue translation function is displayed, it can be done according to... Figure 3 The processing logic shown determines which user interface to display and then displays it.

[0161] For ease of explanation, this application embodiment uses the example of a user clicking the icon of a translation application displayed on the desktop to trigger the display of the user interface corresponding to the dialogue translation function, to specifically explain the processing logic of the translation method provided in this application embodiment.

[0162] like Figure 3 As shown, after the user clicks the translation application icon displayed on the desktop, the processing logic of the translation method provided in this application embodiment may include:

[0163] S101, detects whether headphones are connected.

[0164] Specifically, in this embodiment, the earphones are earphones where both the left and right earphones are equipped with independent control chips. That is, both the left and right earphones can respond to the user's touch operation and make corresponding responses.

[0165] The aforementioned earphones, both the left and right earphones, are equipped with independent control chips and can be either wired or wireless.

[0166] Taking the aforementioned headphones as an example of wireless headphones, in some embodiments of this application, the wireless headphones can be high-fidelity (HiFi) headphones.

[0167] For example, in some embodiments of this application, the wireless earphone is, for example, an in-ear HiFi earphone.

[0168] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. For ease of explanation, the earphones mentioned in the following embodiments are all wireless Loki earphones. That is, the left and right earphones can establish communication connections with the mobile phone independently.

[0169] Because different headphones have different device information, such as headphone name, device type, and identification information, the phone detects the interface used to connect the headphones, such as... Figure 1 The diagram shows the occupancy status of the headphone jack 170D, USB 130, or Bluetooth jack, etc. If it is determined that these jacks are occupied, the device information of the device occupying these jacks can be detected to determine whether headphones with independent control chips for both the left and right earpieces are currently connected.

[0170] For example, if it is determined that the headphones configured as described above are currently connected, step S102 can be performed. Otherwise, step S105 is performed.

[0171] S102, detect whether the current audio channel used for playing audio is the same channel as the headphones.

[0172] The audio channels currently used for playing audio can be the channels corresponding to the speaker in the mobile phone, the channels corresponding to the earpiece, or the channels corresponding to the headphones, etc.

[0173] For example, if it is determined that the current audio channel used for playing audio is the channel corresponding to the headphones, step S103 can be executed. Otherwise, step S104 is executed.

[0174] S103, when the audio channel being played is the same as the headphone channel, the user interface A corresponding to the dialogue translation function is displayed.

[0175] For example, in some embodiments of this application, user interface A is, for example, Figure 4 Interface 203 is shown in (3). For details about the elements included in this user interface and the purpose of each element, please refer to the description of interface 203 in the following embodiments, which will not be repeated here.

[0176] S104, when the audio channel being played is not the channel corresponding to the headphones, the user interface B corresponding to the dialogue translation function is displayed.

[0177] For example, in some embodiments of this application, the user interface B is, for example, Figure 4 Interface 204 is shown in (4). For details about the elements included in this user interface and the purpose of each element, please refer to the description of interface 204 in the following embodiments, which will not be repeated here.

[0178] S105, when no headphones are connected, displays the user interface C corresponding to the conversation translation function.

[0179] For example, in some embodiments of this application, the user interface C is, for example, Figure 4 Interface 202 is shown in (2). For details about the elements included in this user interface and the purpose of each element, please refer to the description of interface 202 in the following embodiments, which will not be repeated here.

[0180] To facilitate understanding, according to Figure 3 The following describes the scenario where the processing logic is implemented. Figure 4 Please provide a detailed explanation.

[0181] For example, in some implementations, the desktop displaying the translation application icon can be as follows: Figure 4 Interface 201 is shown in (1). The icon of the translation application is displayed in interface 201, such as icon 2011.

[0182] For example, in the case of the mobile phone's user interface display interface 201, when the user operation of clicking icon 2011 is received, the mobile phone can then proceed accordingly. Figure 3 The processing logic shown first checks whether headphones are currently connected. That is, it executes step S101 in the above embodiment.

[0183] For example, if it is detected that no headphones are currently connected (the headphones mentioned in this application embodiment refer to the headphones described in the above embodiments), after the translation application is launched, the mobile phone's user interface can display as follows: Figure 4 The interface 202 shown in (2) is where step S105 in the above embodiment is executed.

[0184] See Figure 4In (2), for example, interface 202 may include multiple elements. For example, options 2021 for selecting dialogue translation function, options 2022 for selecting simultaneous interpretation function, an icon 2023 for indicating the current display mode of dialogue translation function, an entry 2024 for viewing more settings function, a language selector 2025 for setting the interactive language, a message bubble 2026 for displaying usage instructions, a control 2027 for triggering the retrieval of source language content (voice or text) in the first language, and a control 2028 for triggering the retrieval of source language content in the second language, etc.

[0185] See also Figure 4 In (2), for example, in some implementations, the language selector 2025 may include a control 2025-1 for displaying the selected first language, a control 2025-2 for triggering a change in the first language, a control 2025-3 for displaying the selected second language, a control 2025-4 for triggering a change in the second language, and a control 2025-5 for swapping the positions of the languages ​​displayed in the controls 2025-1 and 2025-3.

[0186] See also Figure 4 In (2), for example, in some implementations, message bubble 2026 shows usage instructions described using a first source language and a second language.

[0187] For example, in some implementations, the upper half of the message bubble 2026 may display usage instructions in the language described in control 2025-3, and the lower half may display usage instructions in the language described in control 2025-1.

[0188] For example, in some other implementations, the upper half of the message bubble 2026 may be displayed as a user manual described in the first language, and the lower half as a user manual described in the second language.

[0189] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0190] See also Figure 4 In (2), for example, in some implementations, control 2027 may include control 2027-1 for triggering the acquisition of source language sound corresponding to the first language, and control 2027-2 for triggering the display of input keyboard corresponding to the first language. Control 2028 may include control 2028-1 for triggering the acquisition of source language sound corresponding to the second language, and control 2028-2 for triggering the display of input keyboard corresponding to the second language.

[0191] Taking English as the first language and Chinese as the second language as an example: When the user clicks control 2027-2, the current user interface can display keyboard 2027-3. When the user clicks control 2028-2, the current user interface can display keyboard 2028-3.

[0192] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0193] Furthermore, it should be noted that in some implementations of this application, the display modes supported by the dialogue translation function may include dialogue mode and face-to-face mode.

[0194] Understandably, in the user interface of the dialogue mode, each message bubble of dialogue content is displayed on the left and right sides of the translation content display area according to the language corresponding to the source language text content, as shown in the message bubbles 2151, message bubble 2181 and other related message bubbles in the following embodiments.

[0195] In the face-to-face mode user interface, the translation content display area is divided into two sections. One section displays the text content corresponding to the first language (including both source and translation language text content), and the other section displays the text content corresponding to the second language (including both source and translation language text content). For example... Figure 5 Interface 206 is shown in (2).

[0196] In this specific embodiment, the interface 202 displays the icon 2023 that indicates the dialog mode by default.

[0197] Furthermore, it should be noted that in some implementations, the mobile phone can respond to user actions (such as clicks or touches) on icon 2023 by displaying a window near icon 2023 for switching display modes. The user interface for displaying this window is, for example,... Figure 5 Interface 205 is shown in (1). Accordingly, a window for switching display modes is displayed on interface 205, for example, window 2051.

[0198] See Figure 5 In example (1), interface 205 may include all elements in interface 202, and window 2051 located near icon 2023. Window 2051 may include option 2051-1 for selecting a dialogue mode and option 2051-2 for selecting face-to-face mode.

[0199] For example, after the user clicks option 2051-2, the phone responds to the user's action by switching the display mode to face-to-face mode, and the user interface can change from interface 205 to... Figure 5 Interface 206 is shown in (2).

[0200] See Figure 5 In (2), for example, the interface 206 may include a translation content display area 2061 and a translation content display area 2062, a control 2063 that divides the interface 206 into the translation content display area 2061 and the translation content display area 2062, and a control 2064 for exiting the face-to-face mode.

[0201] See also Figure 5 In example (2), in some implementations, the translation content display area 2061 may include a control 2061-1 for triggering the acquisition of the source language sound corresponding to the first language, and a message bubble 2061-2 for displaying the usage instructions described in the first language. The translation content display area 2062 may include a control 2062-1 for triggering the acquisition of the source language sound corresponding to the second language, and a message bubble 2062-2 for displaying the usage instructions described in the second language.

[0202] For example, when a user clicks control 2061-1, the mobile phone responds to the user's operation and collects the source language sound corresponding to the first language. The translation content display area 2061 can display the source language text content described in the first language, and the translation content display area 2062 can display the translation language text content described in the second language (obtained by translating from the source language text content).

[0203] For example, when a user clicks control 2062-1, the mobile phone responds to the user's operation and collects the source language sound corresponding to the second language. The translation content display area 2062 can display the source language text content described in the second language, and the translation content display area 2061 can display the translation language text content described in the first language (obtained by translating from the source language text content).

[0204] Furthermore, it should be noted that in this embodiment, the control 2063 is also used to interchange the text content displayed in the translation content display area 2061 and the translation content display area 2062. That is, after clicking the control 2063, the translation content display area 2061 will display controls and text content related to the second language, and the translation content display area 2062 will display controls and text content related to the first language.

[0205] Furthermore, it should be noted that in some implementations, the control 2064 can be located anywhere on the interface 206. This embodiment only uses the example of displaying it in the upper right corner of the translation content display area 2061.

[0206] For example, after a user clicks control 2064, the phone, in response to the user's action, can exit face-to-face mode and return to conversation mode. For instance, the user interface can change from interface 206 to interface 202.

[0207] See also Figure 4 Zhong (1) and Figure 3 For example, after the user clicks icon 2011 on interface 201, when it is detected that headphones are currently connected, the following can be done: Figure 3 The processing logic shown performs channel detection. That is, it executes step S102 in the above embodiment.

[0208] For example, if it is detected that the current audio channel used for playback is the same as the headphone channel, the phone's user interface can display the following after the translation application is launched: Figure 4 The interface 203 shown in (3) is where step S103 in the above embodiment is executed.

[0209] See Figure 4 In (3), for example, interface 203 may include icon 2031, icon 2032, icon 2033, and all elements in interface 202.

[0210] Icon 2031 indicates that the phone is connected to headphones, and the current audio channel used for playback is the channel corresponding to the headphones, and the headphones are in a dedicated mode.

[0211] It should be noted that, in the embodiments of this application, the exclusive mode can be understood as one of the users conducting multilingual real-time dialogue wearing headphones to listen to the audio corresponding to the translated language text content, while the other party listens to the audio corresponding to the translated language text content through the phone speaker.

[0212] For example, in some implementations, each time the translation app restarts, or when the translation function is switched to conversation translation by selecting option 2021, if it detects that the conversation translation function is currently connected to headphones and the current audio playback channel is the channel corresponding to the headphones, the headphone usage mode can be set to dedicated mode by default. This way, when exiting the translation app or switching translation functions, there is no need to save the current headphone usage mode, reducing the translation app's footprint on the phone's storage space.

[0213] For example, in some implementations, the current headphone usage mode can be saved each time the translation application is exited or the translation function is switched. Each time the translation application is restarted, or when the translation function is switched to conversation translation by selecting option 2021, if it is detected that the conversation translation function is currently connected to the headphone and the audio playback channel is the corresponding headphone channel, the current headphone usage mode can be set to the historical usage mode. This better meets user habits and ensures a better user experience.

[0214] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0215] Furthermore, it should be noted that in this embodiment, icon 2031 can be set to a clickable state. A clickable state means that after a user clicks icon 2031, the mobile phone can respond to the user's action by displaying a window on the current user interface to switch usage modes.

[0216] For example, in some implementations, the user interface is displayed after switching usage modes, such as... Figure 10 Interface 219 shown in (2) Figure 10 Interface 221 is shown in (4). Accordingly, the window for switching usage modes can be window 2191 shown in interface 219, or window 2211 shown in interface 221. For information on using the options provided in windows 2191 and 2211 to switch usage modes, and the changes to the phone's user interface after switching usage modes, please refer to [link to relevant documentation]. Figure 10 The description of the illustrated embodiment will not be repeated here.

[0217] In addition, it should be noted that in some implementations, such as when the phone is connected to the headphones for the first time and the audio channel is the same as the headphone channel, a message bubble introducing the exclusive mode can be displayed near icon 2031 on the currently displayed interface 203.

[0218] For example, in some implementations, a user interface that displays a message bubble introducing the exclusive mode is provided, for example... Figure 6 The interface 203' shown in the figure. Accordingly, the message bubble used to introduce the exclusive mode can be message bubble 2034 displayed near icon 2031 on interface 203'.

[0219] See Figure 6 For example, message bubble 2034 can display introductory information about the exclusive mode, as well as control 2041-1.

[0220] For example, in some implementations, when a user clicks control 2034-1, the mobile phone responds to the user's action by closing message bubble 2034. That is, the user interface can change from interface 203' back to interface 203.

[0221] For example, in some other implementations, an automatic disappearance time can also be set for the message bubble 2034. That is, the message bubble 2034 disappears automatically after a set display time, such as 3 seconds (s).

[0222] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0223] See also Figure 4 In section (3), for example, icon 2032 is used to identify the use of the phone's external speaker to play the audio (translated audio) corresponding to the translated language text content of the first language (such as the language displayed in control 2025-1). That is, the audio corresponding to the translated language text content is played using the phone's corresponding audio channel, such as the audio channel corresponding to the phone's speaker.

[0224] Icon 2033 is used to identify the use of connected headphones (left and / or right headphones) to play audio (translated audio) corresponding to the translated language text content of a second language (such as the language shown in control 2025-3). That is, using the corresponding channel of the headphones connected to the mobile phone to play the audio corresponding to the translated language text content.

[0225] For example, in some implementations, icon 2032 can be located near control 2025-1 (as shown in interface 203). In this case, when the translation content display area displays the translated text content in the first language, the mobile phone can play the translated audio in the first language through the speaker.

[0226] Accordingly, icon 2033 can be located near control 2025-3 (as shown in interface 203). In this case, when the translation content display area displays the translated text content in the second language, the mobile phone can control the connected headphones to play the translated audio in the second language.

[0227] For example, in some implementations, icon 2032 can also be located near control 2025-3. In this case, when the translation content display area displays the translated text content in the second language, the mobile phone can play the translated audio in the second language through the speaker.

[0228] Accordingly, icon 2033 can be located near control 2025-1. In this case, when the translation content display area shows the translated text content in the first language, the mobile phone can control the connected headphones to play the translated audio in the first language.

[0229] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0230] See also Figure 4 Zhong (1) and Figure 3 For example, after the user clicks icon 2011 on interface 201, if it detects that headphones are currently connected, but the audio channel used for playback is not the channel corresponding to the headphones, the phone's user interface can display the following after the translation application starts: Figure 4 The interface 204 shown in (4) is where step S104 in the above embodiment is executed.

[0231] See Figure 4 In (4), for example, interface 204 may include icon 2041, as well as all elements in interface 202.

[0232] Icon 2041 indicates that the dialogue translation function is currently connected to headphones, but the audio channel used to play the audio is not the channel corresponding to the headphones.

[0233] For example, in some implementations, icons 2031 and 2041 may contain the same content, but their colors and attributes may differ. For instance, icon 2031 may display the default color (not grayed out) and be set to a clickable state, while icon 2041 may be grayed out and set to a non-clickable state. The non-clickable state means that after a user clicks icon 2041, window 2191 or window 2211 is not displayed in the current user interface.

[0234] For example, in some implementations, after the user clicks icon 2041, the phone responds to the user's action by displaying a message prompt on the current user interface, such as interface 204.

[0235] For example, in some implementations, the user interface displayed after the aforementioned message prompt box is, for example,... Figure 7 Interface 207 is shown in (1). Accordingly, the displayed message prompt box can be message prompt box 2071 displayed in interface 207.

[0236] See Figure 7In example (1), message prompt box 2071 is used to display prompt information, such as "Please switch the device sound to headphone playback before trying again", to remind the user that the current audio channel used to play the audio is not the audio channel corresponding to the headphone, and the audio of the translated language text content cannot be played using the usage mode provided by the headphone.

[0237] It should be noted that the message prompt box 2071 can be a Toast (a type of pop-up prompt box) or other controls.

[0238] Specifically, in this embodiment of the application, message prompt box 2071 is exemplified by Toast. All subsequent message prompt boxes will also be exemplified by Toast.

[0239] See also Figure 7 In example (1), in some implementations, the message prompt box 2071 may also provide an entry point 2071-1. This allows the user to directly access the audio channel settings interface by operating (e.g., clicking) the entry point 2071-1. For example, the user interface may change from interface 207 to... Figure 7 Interface 208 is shown in (2).

[0240] See Figure 7 In section (2), for example, interface 208 may include one or more elements. For example, a window 2081 for switching audio channels. Window 2081 may include options for selecting the audio channel corresponding to the mobile phone (the audio channel corresponding to the device) and options for selecting the audio channel corresponding to the headphones.

[0241] Understandably, at any given time, only one of the options for selecting the audio channel corresponding to the mobile phone and the option for selecting the audio channel corresponding to the headphones can be selected. The selected option can have the style shown in option 2081-1, and the unselected option can have the style shown in option 2081-2. Based on this principle, if the audio channel currently used for playback on the mobile phone is not the audio channel corresponding to the headphones, but rather the audio channel corresponding to the mobile phone, when the user clicks entry 2071-1 in interface 207 and the user interface changes to interface 208, the option for selecting the audio channel corresponding to the mobile phone in window 2081 can be option 2081-1, and the option for selecting the audio channel corresponding to the headphones can be option 2081-2.

[0242] See also Figure 7 In example (2), after the user clicks option 2081-2, the mobile phone responds to the user's operation by switching the device's audio channel from the local channel to the headphone's channel, and the user interface can change from interface 208 to... Figure 7 The interface 209 shown in (3) is used to switch the audio channels.

[0243] See Figure 7 In section (3), for example, interface 209 may include one or more elements. For example, window 2091 for channel switching. Window 2091 may include options for selecting the channel corresponding to the mobile phone and options for selecting the channel corresponding to the headphones. Unlike window 2081, the style of the option for selecting the channel corresponding to the mobile phone in window 2091 may be as shown in option 2091-1 (or option 2081-2), and the style of the option for selecting the channel corresponding to the headphones may be as shown in option 2091-2 (or option 2081-1).

[0244] For example, in some other implementations, the user can also change the phone's user interface to the control center's user interface through the user operation of the pull-down interface 207, and then find the window for switching audio channels in the control center's user interface. The operation of switching the audio channel played by the device through the window for switching audio channels displayed in the control center's user interface is similar to the options in the operation window 2081, and will not be described further here.

[0245] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0246] For example, after switching the device's audio channel from the local channel to the headphone's channel, the user can return to the user interface corresponding to the dialogue translation function through a specific operation.

[0247] Understandably, since the device's audio channel has been switched from the channel corresponding to the device itself to the channel corresponding to the headphones, the returned user interface can become interface 203.

[0248] For example, in some implementations, the specific operation mentioned above may be an upward swipe operation performed on interface 209 or the user interface corresponding to the control center.

[0249] For example, in some other implementations, the specific operation mentioned above may be a click operation performed in an area outside the window displayed in the user interface corresponding to the control center, such as interface 209.

[0250] For example, in some other implementations, the specific operation mentioned above may be a user operation of pressing a mechanical button on the side of the phone.

[0251] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0252] Therefore, according to Figure 3The processing logic shown can accurately determine which user interface to display in scenarios where dialogue translation is used, and then display it.

[0253] In addition, it should be noted that in the scenario where the user interface corresponding to the dialogue translation function is displayed after the translation application is launched, the option for selecting the dialogue translation function is selected by default.

[0254] Accordingly, the option for selecting simultaneous interpretation function displayed in the user interface is unselected.

[0255] For example, in some implementations, the selected option for the dialogue translation function can be displayed as shown in option 2021. That is, the words "Dialogue Translation" can be bolded and the font size can be increased.

[0256] Correspondingly, the simultaneous interpretation function option when not selected can be displayed as in option 2022. That is, the word "simultaneous interpretation" will not be bolded and the font size will not be increased.

[0257] For example, in some other implementations, the selected and unselected states can also be distinguished by different font colors, font types, etc.

[0258] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0259] For example, when launching a translation application, according to Figure 3 The processing logic shown determines which user interface to display, and after displaying it, the user can use the dialogue translation function and simultaneous interpretation function provided by the translation application according to their actual needs.

[0260] The following description, in conjunction with the accompanying diagram, illustrates the scenarios in which a user utilizes the conversation translation function provided by the translation application after it is launched.

[0261] (1) More settings related content

[0262] As described in the above embodiments, in the technical solution provided in this application, when using the dialogue translation function of a translation application, depending on whether the phone is connected to headphones and the current audio channel used for playback, the user interface corresponding to the dialogue translation function can include several types: the user interface for a scenario where headphones are not connected (e.g., interface 202), the user interface for a scenario where headphones are connected but the audio channel is not the same as the headphone channel (e.g., interface 204), and the user interface for a scenario where headphones are connected and the audio channel is the same as the headphone channel (e.g., interface 203). Therefore, different settings functions can be provided according to different usage scenarios, so that the dialogue translation function of the translation application can better meet the actual usage needs of users.

[0263] For example, in some implementations (implementation 1), the supported settings for different use cases can be determined based on the audio channel currently being played by the user. For instance, the same settings can be set for use cases 1 and 2 where the audio channel is not the same as the headphone channel.

[0264] For example, in some implementations (implementation 2), the settings functions supported by the same usage scenario can be determined based on whether headphones are connected. For instance, the same settings functions can be set for usage scenario 2 and usage scenario 3 where headphones are connected.

[0265] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0266] Specifically, in this application embodiment, taking implementation method 1 as an example, for instance, in some implementation methods, the setting functions corresponding to usage scenario 1 and usage scenario 2 can both include click to receive audio, continuous audio reception, automatic playback, click to play audio, save, clear, and settings. The setting functions corresponding to usage scenario 3 can include click to receive audio, continuous audio reception, automatic playback, save, clear, and settings. That is, in this implementation method, when headphones are connected and the current audio playback channel is the channel corresponding to the headphones, the dialogue translation function does not support the click to play audio function.

[0267] Among them, click-to-receive and continuous-receive are two mutually exclusive recording methods. That is, only one recording method can be selected at a time.

[0268] For example, in a scenario where the sound reception method is click-to-receive (the option corresponding to the click-to-receive function is selected), the user needs to manually operate the control that triggers the acquisition of the source language sound before each sentence of the source language content (speech type).

[0269] Correspondingly, in scenarios where the sound reception mode is continuous (the option corresponding to the continuous sound reception function is selected), the user only needs to manually operate the control to trigger the acquisition of the source language sound once when the conversation begins.

[0270] Automatic playback and click-to-play playback are two mutually exclusive playback methods. That is, only one playback method can be selected at a time to play the translated text content aloud.

[0271] For example, in a scenario where the broadcast method is click-to-broadcast (the option corresponding to the click-to-broadcast function is selected), the user needs to manually click the voice broadcast control corresponding to the translated language text content in order to hear the broadcast sound corresponding to that sentence of translated language text content.

[0272] Correspondingly, in scenarios where the automatic playback mode is selected (the option corresponding to the automatic playback function is selected), after each sentence of translated language text is translated, the translated language text content can be automatically played without user operation.

[0273] The option corresponding to the save function is used to trigger the phone to save the text content (source language text content and translated language text content) displayed in the current user interface.

[0274] The option corresponding to the clear function is used to trigger the phone to clear (delete) all text content displayed in the current user interface.

[0275] The settings options include options for triggering settings for the phone's desktop shortcut, voice tone, font size, and font color, allowing users to customize settings as needed.

[0276] Specifically, in some embodiments of this application, after a user clicks the entry point 2024 in the user interface corresponding to any of the above-mentioned usage scenarios, the settings function windows displayed near the entry point 2024 will all display the same settings function options. However, for the settings function window corresponding to usage scenario 3, the clickable broadcast function option is set to an unavailable state. For example, after a user clicks the entry point 2024 in the interface 202 corresponding to usage scenario 1, the interface 202 displaying the settings function window can become... Figure 8 The interface 210 shown in (1) can be changed after the user clicks the entry 2024 in the interface 204 corresponding to the usage scenario 2. The interface 204 that displays the settings function window can be changed to Figure 8 The interface 211 shown in (2) is as follows. After the user clicks on the entry 2024 in the interface 203 corresponding to usage scenario 3, the interface 203 that displays the settings function window can be changed to Figure 8 Interface 212 is shown in (3).

[0277] like Figure 8 As shown, for example, the settings function window 2101 displayed in interface 210, the settings function window 2111 displayed in interface 211, and the settings function window 2121 displayed in interface 212 all include options for settings functions such as click-to-receive, continuous reception, automatic playback, click-to-play, save, clear, and settings. However, option 2101-1 corresponding to the click-to-play function in settings function window 2101 and option 2111-1 corresponding to the click-to-play function in settings function window 2111 are both available, meaning that after the user clicks option 2101-1 or option 2111-1, the playback mode can be switched to click-to-play. However, option 2121-1 corresponding to the click-to-play function in settings function window 2121 is unavailable, meaning that after the user clicks option 2121-1, the playback mode will not switch to click-to-play, and automatic playback will remain.

[0278] For example, in some implementations, option 2121-1 can be grayed out and its properties set to unavailable (cannot be selected).

[0279] In other specific embodiments of this application, after a user clicks on entry point 2024 in the user interface corresponding to usage scenario 1 and usage scenario 2, the settings function windows displayed near entry point 2024 may all display the same settings function options, such as displaying settings function window 2101 and settings function window 2111. After a user clicks on entry point 2024 in the user interface corresponding to usage scenario 3, the settings function windows displayed near entry point 2024 may not include the option to click the broadcast function; for example, option 2121-1 may not be displayed in settings function window 2121.

[0280] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0281] In addition, it should be noted that, to ensure a good user experience, the broadcast methods corresponding to usage scenario 1 and usage scenario 2 can be recorded. This way, when switching from usage scenario 3 to usage scenario 1 (or usage scenario 2), the broadcast method can be automatically restored to the broadcast method corresponding to usage scenario 1 (or usage scenario 2).

[0282] This allows the dialogue translation function to change the available settings in the settings window in different usage scenarios, enabling the phone to automatically switch the broadcast mode according to the current usage scenario, thereby providing better service to users.

[0283] (2) Content related to exclusive mode and shared mode

[0284] It should be noted that, in the embodiments of this application, the sharing mode refers to the situation where, among users engaging in multilingual real-time dialogue, each party wears an earphone and uses their respective earphone to listen to the audio of the corresponding translated language text content.

[0285] In other words, the main difference between the exclusive mode and the shared mode lies in the playback devices used by both parties in the conversation when listening to the audio of the corresponding translated language text. To distinguish between these two usage modes of the headphones, some implementations can use different icons to identify these two usage modes and the sound source of the currently playing audio, while the other usage logic and display logic of the dialogue translation function can remain the same.

[0286] For ease of explanation, this application embodiment assumes that the display mode corresponding to the translation function is dialogue mode, the sound reception method is click-to-receive, the broadcast method is automatic broadcast, the mobile phone is currently connected to headphones, and the audio channel currently used for playing audio is the audio channel corresponding to the headphones, so as to specifically explain the usage scenarios of exclusive mode and shared mode.

[0287] 2.1 Usage Scenarios for Dedicated Mode

[0288] Taking the example that the language displayed in control 2025-1 is the first language, the icon displayed near control 2025-1 is icon 2032, the language displayed in control 2025-3 is the second language, and the icon displayed near control 2025-3 is icon 2033, for example, in some implementations (method 1), the source language sound of the first language and the source language sound of the second language can be collected by the microphone of the mobile phone, the translated audio of the first language can be played by the speaker of the mobile phone, and the translated audio of the second language can be played by the headphones.

[0289] For example, in some other implementations (method 2), the source language sound of the first language can be captured by the microphone of the mobile phone, the translated audio of the first language can be played by the speaker of the mobile phone, the source language sound of the second language can be captured by the headphones, and the translated audio of the second language can be played by the headphones.

[0290] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation on this embodiment. For ease of explanation, this application embodiment uses the above-described method 2 as an example to illustrate the use case of the exclusive mode.

[0291] For example, based on the above premise, in some implementations, when a user performs a user action that triggers the acquisition of source language sound, such as clicking control 2028-1 (or the area of ​​control 2028 excluding control 2028-2) in interface 203, the mobile phone responds to the user action, enters the sound recording state, begins to acquire source language sound through the headphones, and displays as shown below. Figure 9 Interface 213 is shown in (1).

[0292] See Figure 9 In (1), for example, interface 213 may include control 2131, as well as all elements in interface 203 except message bubble 2026 and control 2028.

[0293] Among them, control 2132 is used to indicate that the mobile phone is in radio mode.

[0294] In this embodiment, "sound receiving state" refers to the current state in which the mobile phone (or other electronic device) is picking up sound. That is, it is in the state of collecting source language sound and acquiring source language audio. For example, in the sound receiving state, the electronic device will collect the user's voice and convert the collected source language sound into source language text content, which will be displayed in real time in the translation content display area of ​​the current user interface.

[0295] For example, in some implementations, when the phone is in radio reception mode but has not yet captured the source language sound, a message bubble 2131 can also be displayed on interface 213. The message bubble 2131 is used to display a notification message to inform the user that source language sound is currently being captured.

[0296] For example, in some implementations, the prompt message displayed in message bubble 2131 may be a message described using control 2028 and the language corresponding to control 2027.

[0297] For example, when the phone is in audio reception mode and has captured the source language audio, the corresponding source language text can be displayed on the screen word by word in real time. In this case, message bubble 2131 can become Figure 9 The message bubble 2141 shown in (2) indicates that the user interface can change from interface 213 to interface 214.

[0298] See Figure 9 Zhong (1) and Figure 9 In (2), for example, interfaces 214 and 213 are largely the same. The difference is that the message bubble 2131 used to display prompt messages in interface 213 is replaced by a message bubble 2141 that displays the source language text content obtained in real time.

[0299] For example, after the user finishes speaking the source language content that needs translation, such as the complete source language text displayed in message bubble 2141, when the user clicks control 2132, the phone responds to the user's operation by exiting the audio receiving mode and returning to the non-audio receiving mode. The translated language text content corresponding to the source language text content displayed in message bubble 2131 will then be directly displayed on the screen. That is, the user interface can change from interface 214 to... Figure 9Interface 215 is shown in (3).

[0300] See Figure 9 In (3), for example, interface 215 may include message bubble 2151, and all elements in interface 203 except message bubble 2026.

[0301] See also Figure 9 In (3), for example, message bubble 2151 can display source language text content using the language description corresponding to control 2028 and translated language text content using the language description corresponding to control 2027, as well as voice broadcast control 2151-1.

[0302] Among them, the voice broadcast control 2151-1 is used to identify the translated language text content displayed in the message bubble 2151 to be broadcast in an external voice format.

[0303] Understandably, when headphones are connected and the audio channel is the same as the headphones' channel, after obtaining the translated language text content, the device automatically performs voice playback according to the playback mode of the language corresponding to the obtained translated language text content. Therefore, when the voice playback control 2151-1 is displayed in message bubble 2151, the mobile phone can control its own speaker to play the audio of the translated language text content displayed in message bubble 2151.

[0304] It should be noted that in the above scenario, the audio channel currently used by the phone to play audio (excluding audio of the translated language text content, such as music) is still the channel corresponding to the headphones. In the above scenario, only the audio of the translated language text content displayed in message bubble 2151 is being played through the channel corresponding to the phone's speaker.

[0305] Furthermore, it should be noted that, to enhance the user experience, the voice broadcast control 2151-1 can add playback animation effects during voice broadcasting, and the currently playing translated text content can be highlighted (or bolded, or changed to another color) in real time following the playback progress. For example, previously broadcast content can be highlighted. This informs the user that translated text content is currently being played.

[0306] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0307] See also Figure 9 In (3), for example, after the user clicks on control 2027-1 (or the area of ​​control 2027 excluding control 2027-2) in interface 215, the mobile phone responds to the user's operation, enters the sound recording state, starts to collect the source language sound through its own microphone, and displays as shown in the image. Figure 9 Interface 216 is shown in (4).

[0308] See Figure 9 In example (4), interface 216 may include control 2162, and all elements in interface 215 except control 2027. Among them, control 2162 is used to indicate that the mobile phone is in radio mode.

[0309] For example, in some implementations, when the phone is in radio reception mode but has not yet captured the source language sound, a message bubble 2161 can also be displayed on interface 216. The message bubble 2161 is used to display a notification message to inform the user that source language sound is currently being captured.

[0310] For example, in some implementations, the prompt message displayed in message bubble 2161 may be a message described using the language corresponding to control 2027 and control 2028.

[0311] For example, when the phone is in audio reception mode and has captured the source language audio, the corresponding source language text can be displayed on the screen word by word in real time. In this case, message bubble 2161 can become Figure 9 The message bubble 2171 shown in (5) indicates that the user interface can change from interface 216 to interface 217.

[0312] See Figure 9 Zhong (4) and Figure 9 In (5), for example, interfaces 216 and 217 are largely the same. The difference is that the message bubble 2161 used to display prompt messages in interface 216 is replaced by a message bubble 2171 that displays the source language text content obtained in real time.

[0313] For example, after the user finishes speaking the source language content that needs translation, such as the complete source language text displayed in message bubble 2171, when the user clicks control 2152, the phone responds to the user's operation by exiting the audio receiving mode and returning to the non-audio receiving mode. The translated language text content corresponding to the source language text content displayed in message bubble 2171 will then be directly displayed on the screen. That is, the user interface can change from interface 217 to... Figure 9 Interface 218 is shown in (6).

[0314] See Figure 9 In (6), for example, interface 218 may include message bubble 2181, as well as all elements in interface 215.

[0315] See also Figure 9In the middle (6), for example, message bubble 2181 can display source language text content using the language description corresponding to control 2027 and translated language text content using the language description corresponding to control 2028, as well as voice broadcast control 2181-1.

[0316] Among them, the voice broadcast control 2181-1 is used to identify the translated language text content displayed in the message bubble 2181 to be broadcast through the connected headphones.

[0317] Understandably, when headphones are connected and the audio channel is the same as the headphones' channel, after obtaining the translated language text content, the device automatically plays the audio according to the playback mode of the language corresponding to the obtained translated language text content. Therefore, when the voice playback control 2181-1 is displayed in message bubble 2181, the mobile phone can automatically control the audio channel corresponding to the headphones to play the audio of the translated language text content displayed in message bubble 2181.

[0318] It should be noted that when the earphone is used in the exclusive mode, regardless of whether the phone is currently connected to two earphones (left and right earphones) or one earphone (left or right earphone), the currently connected earphone will play the audio of the translated language text content displayed in message bubble 2181.

[0319] Furthermore, it should be noted that, to enhance the user experience, the voice broadcast control 2181-1 can also incorporate playback animation effects during voice broadcasting, and the currently playing translated text content can be highlighted (or bolded, or changed to another color) in real time, following the playback progress. For example, previously broadcast content can be highlighted. This informs the user that the currently playing translated text content is being displayed.

[0320] For example, in some implementations, when playing the audio of the translated language text content using the corresponding playback method, when the user clicks control 2151-1 or control 2181-1, the mobile phone can stop playing the audio of the translated language text content in response to the user's operation.

[0321] For example, in some other implementations, when playing the audio of the translated language text content using the corresponding playback method, if the user performs another user operation to trigger the acquisition of the source language sound, the mobile phone can respond to the user operation by stopping the playback of the translated language text content audio and entering the sound acquisition state to start acquiring the source language sound.

[0322] For example, in some other implementations, after the audio of the translated language text content has finished playing, when the user clicks control 2151-1 or control 2181-1, the mobile phone can respond to the user's operation and re-control the corresponding audio channel to play the audio of the translated language text content displayed in message bubble 2151 or message bubble 2181.

[0323] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0324] This enabled the use of the exclusive mode.

[0325] 2.2 Explanation of headphone usage mode switching scenarios

[0326] For example, when the connected headphones are in private mode and the audio channel is the channel corresponding to the headphones, for example, the display... Figure 10 In the case of interface 218 shown in (1), the user can manually click icon 2031 to trigger the phone to display a window to switch usage modes.

[0327] It's important to note that in the dedicated listening mode, the earpiece is used to play the audio of one party's translated text. In the shared listening mode, the left and right earpieces are used to play the audio of both parties' translated text, respectively. Therefore, in dedicated listening mode, the phone can establish a communication connection with both the left and right earpieces simultaneously, or with only one earpiece. In shared listening mode, the phone must establish a communication connection with both the left and right earpieces simultaneously.

[0328] Based on this, after the user clicks icon 2031, the phone responds to the user's operation by detecting whether a communication connection has been established with both the left and right earpieces at the same time, and then decides which user interface to display based on the detection result.

[0329] For example, when the phone detects that it has established communication connections with both the left and right earpieces simultaneously, the user interface can display something like this: Figure 10 Interface 219 is shown in (2).

[0330] See Figure 10 In (2), for example, interface 219 may include all elements in interface 218, and window 2191 located above interface 218. Window 2191 is an example of a window for switching usage modes.

[0331] For example, in some implementations, window 2191 may include a usage mode selectable for the currently connected headphones, a description of each usage mode, an icon, an option to select each usage mode, and an entry point to view details of each usage mode.

[0332] by Figure 10 For example, window 2191 shown in (2) can include a private mode, a description and icon of the private mode, an option 2191-1 for selecting the private mode, an entry 2191-2 for viewing the details of the private mode; a shared mode, a description and icon of the shared mode, an option 2191-3 for selecting the shared mode, an entry 2191-4 for viewing the details of the shared mode; and a control 2191-5 for closing window 2191.

[0333] Understandably, the user triggered the usage mode switching operation when the headphones were in exclusive mode, so option 2191-1 displayed in window 2191 was selected, while option 2191-3 was unselected.

[0334] See also Figure 10 In example (2), when the user clicks option 2191-3, the phone responds to the user's operation by switching the headphone usage mode from exclusive mode to shared mode and closing window 2191, displaying as shown in the image. Figure 10 Interface 220 is shown in (3).

[0335] See Figure 10 In (3), for example, interface 220 may include icon 2201, icon 2202, icon 2203, voice broadcast control 2204, voice broadcast control 2205, message prompt box 2206, and all elements in interface 218 except for icon 2031, icon 2032, icon 2033, voice broadcast control 2151-1, and voice broadcast control 2181-1.

[0336] Icon 2201 indicates that the phone is connected to headphones, and the current audio channel used for playback is the channel corresponding to the headphones, and the connected headphones are in sharing mode.

[0337] Icon 2202 is used to identify the connected earphone, such as the left earphone playing the audio corresponding to the translated language text content displayed in control 2025-1. That is, the audio corresponding to the translated language text content is played using the channel of the left earphone connected to the mobile phone.

[0338] Icon 2203 is used to identify the other earphone connected, such as the right earphone playing the audio corresponding to the translated language text content displayed in control 2025-3. That is, the right earphone connected to the phone plays the audio corresponding to the translated language text content through its corresponding channel.

[0339] The voice broadcast control 2204 is used to translate the text content in the message bubble it is located in, and broadcasts it through the connected left earphone.

[0340] The voice broadcast control 2205 is used to translate the text content in the message bubble it is located in, and broadcasts it through the connected right earphone.

[0341] The voice broadcast logic triggered when voice broadcast controls 2204 and 2205 are clicked is similar to that of voice broadcast controls 2151-1 and 2181-1. For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0342] The message prompt box 2206 is used to display a prompt message to remind the user that the headphone usage mode has been switched to sharing mode.

[0343] Understandably, message prompt 2206 can be displayed on the current user interface for a set time, such as disappearing automatically after 3 seconds.

[0344] See also Figure 10 (3) For example, similar to icon 2031, icon 2201 can also be set to a clickable state. For example, in some implementations, after the user clicks icon 2201, the mobile phone responds to the user's operation, and the user interface can change from interface 220 to... Figure 10 Interface 222 is shown in (5).

[0345] See Figure 10 In (5), for example, interface 222 may include all elements of interface 220, and window 2221 located above interface 220. Window 2221 is an example of a window for switching usage modes.

[0346] For example, window 2221 is similar to window 2191, except that option 2221-2 for selecting the shared mode is selected in window 2221, while option 2221-1 for selecting the exclusive mode is unselected.

[0347] For example, after the user clicks option 2221-1, the phone responds to the user's action by switching the headphone usage mode from shared mode to exclusive mode, closing window 2221, and displaying as shown. Figure 10 Interface 218 is shown in (1).

[0348] For example, after switching from shared mode to exclusive mode, the interface 218 can also display a message prompt box to remind the user that the headphone usage mode has been switched to exclusive mode.

[0349] In addition, it should be noted that in some implementations, when a user clicks option 2191-3 in window 2191 or option 2221-1 in window 2221, the mobile phone responds to the user's operation, can switch the usage mode, and automatically close window 2191 or window 2221.

[0350] For example, in some other implementations, the phone can also manually click on control 2191-5, or click on an area outside window 2191 (or window 2221) to trigger the phone to close window 2191 (or window 2221).

[0351] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0352] Furthermore, it should be noted that in some implementations, such as when the phone is first connected to headphones and the audio channel is the same as the headphones' channel, a message bubble introducing the sharing mode can be displayed near icon 2201 on the currently displayed interface 220. This message bubble can be similar to message bubble 2034, except that it displays introductory information about the sharing mode.

[0353] See also Figure 10 Zhong (2) and Figure 10 In the case of display window 2191 or window 2221, for example, the user can click on entry 2191-2 to view the details of the exclusive mode and click on entry 2191-4 to view the details of the shared mode.

[0354] Taking interface 219 as an example, for instance, after the user clicks on entry 2191-2, the mobile phone responds to the user's operation, and the user interface can change from interface 219 to... Figure 11 Interface 223 is shown in (1).

[0355] See Figure 11 In (1), for example, interface 223 may include window 2231 and all elements of interface 218. Window 2231 may include option 2231-1 in the selected state, option 2231-2 in the unselected state, and control 2231-3 for closing window 2231.

[0356] For example, in some implementations, option 2231-1 is used to identify that the details displayed in window 2231 are about the exclusive mode.

[0357] For example, in some implementations, when a user clicks option 2231-2, or performs a leftward swipe in window 2231, the phone responds to the user's action, and the details message displayed in window 2231 can change to details about the sharing mode. In this case, option 2231-1 can become unselected (e.g., change to...). Figure 11 Option 2241-1 shown in (2) can be selected (e.g., changed to...). Figure 11 Option 2241-2 is shown in (2). That is, window 2231 can be changed to Figure 11 The window 2241 shown in (2) and the interface 223 can be transformed into Figure 11 Interface 224 is shown in (2).

[0358] Taking interface 219 as an example, for instance, after the user clicks on entry 2191-1, the phone responds to the user's action, and the user interface can change from interface 219 to... Figure 11 Interface 224 is shown in (2).

[0359] For example, in the scenario of displaying interface 224, when the user clicks option 2241-1, or performs a user operation by swiping to the right in window 2241, the phone responds to the user operation, and the details message displayed in window 2241 can change to details about the exclusive mode. That is, window 2241 can become window 2231, and interface 224 can become interface 223.

[0360] For example, in some other implementations, the phone can be triggered to display window 2231 in the current user interface via icon 2032 or icon 2033. Alternatively, the phone can be triggered to display window 2241 in the current user interface via icon 2202 or icon 2203.

[0361] For example, in the scenario of displaying interface 218, when the user clicks icon 2032 or icon 2033, the mobile phone responds to the user's operation and can display window 2231 in the current user interface.

[0362] For example, in the scenario of displaying interface 220, when the user clicks icon 2202 or icon 2203, the mobile phone responds to the user's operation and can display window 2241 in the current user interface.

[0363] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0364] See also Figure 10 In example (1), after the user clicks icon 2031, the mobile phone responds to the user's operation. If it detects that no communication connection is currently established with both the left and right earpieces simultaneously, the user interface can display as follows: Figure 10 Interface 221 is shown in (4).

[0365] See Figure 10 In (4), for example, the window displayed in interface 221 for switching usage modes can be as shown in window 2211.

[0366] Window 2211 can be similar to window 2191, but the difference is that the options, entry points, and descriptions related to the sharing mode in window 2211 are all grayed out and set to an unavailable state. For example, if the user clicks option 2211-3, it will not trigger a switching of usage modes, and option 2211-2 will remain selected. Similarly, if the user clicks entry point 2211-4, the user interface will not display window 2231. That is, the sharing mode is unavailable in scenarios where the phone is not simultaneously connected to both the left and right earpieces.

[0367] For a description of other elements in window 2211, please refer to the description of window 2191 in the above embodiments, which will not be repeated here.

[0368] Therefore, when headphones are connected to a mobile phone, users can switch the headphone usage mode according to the actual usage scenario, thus making better use of the translation application for conversation translation.

[0369] Furthermore, it should be noted that in some implementations, such as when the headphones are in shared mode, if one earbud loses its communication connection with the phone (e.g., one earbud runs out of power or the user puts one earbud back in the charging case), the phone can automatically switch the headphone usage mode from shared mode to exclusive mode. In this case, the user interface can be changed from... Figure 12 The interface 220 shown in (1) becomes Figure 12 Interface 225 is shown in (2).

[0370] For example, in some implementations, when the phone automatically switches from shared mode to dedicated mode, a message prompt box can be displayed on interface 225, and a prompt message can be displayed in the message prompt box to inform the user that one earphone is currently disconnected, the shared mode cannot be used, and the usage mode has been switched to dedicated mode.

[0371] Furthermore, if the phone can identify which earphone has lost communication, the message box displayed on interface 225 can clearly inform the user which earphone has lost communication. For example, when the phone detects that the left earphone has lost communication, the message displayed in the message box could be as follows: Figure 12 The message prompt box 2251 shown in (2) is shown.

[0372] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0373] Therefore, when the headset connected to the phone is changed from two headsets to one headset, the headset usage mode can be automatically switched to dedicated mode. This allows the translated text content of the conversation to still be automatically played through different playback devices (before disconnection: one party through the left headset, the other through the right headset; after disconnection: one party through the phone speaker, the other through the currently connected headset), thus ensuring the efficiency of the conversation.

[0374] 2.3 Usage Scenarios of Sharing Mode

[0375] Understandably, in the shared mode usage scenario, the processing logic for controlling the phone to enter and exit the recording mode is the same as in the dedicated mode. The difference lies in the fact that, in the shared mode usage scenario, the icons of the voice broadcast control used to identify the playback method corresponding to the translated language text content in the message bubble displayed in the translation content display area are different (for example, as shown in interface 220), and the playback device that automatically performs voice broadcast after obtaining the translated language text content is different (in the shared mode usage scenario, one party plays the sound through the left earphone, and the other party plays the sound through the right earphone; in the dedicated mode usage scenario, one party plays the sound through the phone speaker, and the other party plays the sound through the currently connected earphone).

[0376] Regarding the handling logic for controlling the phone to enter and exit the sound reception state when the headphones are in the sharing mode, please refer to the description of the user interface changes after the user clicks on control 2028-1 (or the area of ​​control 2028 excluding control 2028-2), control 2132, control 2027-1 (or the area of ​​control 2027 excluding control 2027-2), and control 2162 in the above embodiment. It will not be repeated here.

[0377] This enabled the use of a sharing model.

[0378] 2.4 Usage Scenarios for Connecting Headphones

[0379] Specifically, in this embodiment, connecting the headphones will not interrupt the current translation service. That is, in scenarios where the mobile phone is in receiving or broadcasting mode (playing translated language text content), once the mobile phone and headphones establish a communication connection, the current state will not change, and the mobile phone can continue to receive source language sound in receiving mode or play translated language text content in broadcasting mode.

[0380] For example Figure 13 As shown in (1), when the mobile phone is not connected to the headset and is in the sound receiving state, the icons related to the exclusive mode or the shared mode are not displayed in the interface 226. The source language text content corresponding to the source language sound is displayed in real time in the message bubble in the translation content display area, as shown in message bubble 2261.

[0381] During the display of interface 226, when a connected headset is detected, the phone continues to work in audio reception mode to collect the source language audio. The source language text content displayed in the message bubble can change in real time following the recognized source language audio, for example, changing to... Figure 13 The message bubble 2271 shown in (2) is shown in the middle.

[0382] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0383] Furthermore, it should be noted that in this embodiment, after the mobile phone detects that headphones are connected, it can refresh the user interface according to the currently set audio channel. For example, the user interface can be changed from... Figure 13 The interface 226 shown in (1) becomes Figure 13 Interface 227 is shown in (2). That is, icons related to the exclusive mode are displayed in the current user interface.

[0384] Furthermore, it should be noted that in this embodiment, the connection of headphones also follows general headphone rules. For example, after detecting a headphone connection, audio feedback can be automatically provided to the headphones (such as playing a notification tone indicating that a communication connection with the mobile phone has been established), and a message such as "..." can be displayed on the current user interface. Figure 13 The window 2272 shown in (2) is shown in the middle.

[0385] 2.5 Usage scenarios for disconnecting headphones

[0386] For example, in some implementations (implementation 1), the current translation service can be stopped when the established communication connection with the headset is detected to be disconnected. That is, in scenarios where the mobile phone is in radio reception or broadcast mode, when the communication connection between the mobile phone and the headset is lost, the current translation service is stopped directly, and the phone returns to the non-radio reception mode.

[0387] For example, in some other implementations (implementation 1), when the headset that established the communication connection is detected to be disconnected, the current translation service can continue to be performed without interrupting the current translation service, and the source language sound can continue to be collected or the audio of the translated language text content can be played using a playback device suitable for the current scenario.

[0388] Specifically, in the embodiments of this application, we take implementation method 1 as an example. For instance, in some implementation methods, in a scenario where the mobile phone is connected to headphones and is in a private mode (such as display...), Figure 14 The interface 228 shown in (1) or the use case in shared mode (such as displaying Figure 14 When interface 229 is shown in (2), if the communication connection between the headset and the mobile phone is lost, the mobile phone will exit the radio reception state and return to the non-radio reception state. That is, control 2162 in interfaces 228 and 229 will change to control 2027. Furthermore, the voice broadcast controls displayed in the message bubble in the translation content display area, such as voice broadcast controls 2281-1 and 2291-1, can be changed to voice broadcast controls when the headset is not connected, for example... Figure 14 The voice broadcast control 2301-1 shown in (3) is shown.

[0389] Furthermore, it should be noted that after detecting a disconnection from the headset, the user interface does not display icons related to the exclusive or shared mode. For example, icons 2031, 2032, and 2033 from interface 228 are not displayed in interface 231, nor are icons 2201, 2202, and 2203 from interface 229.

[0390] For example, to remind the user that the communication connection with the headphones has been lost, in some implementations a message prompt box can be displayed in the user interface after the communication connection with the headphones is lost. For example, message prompt box 2302 displayed in interface 231 prompts the user that the mobile phone has lost the communication connection with the headphones.

[0391] Furthermore, it should be noted that in some implementations, such as scenarios where the translated language text content is displayed directly on the screen only after the source language text content has been fully displayed, the user interface displayed when the phone exits the radio reception state due to disconnection from the headphones may not display the source language text content or the corresponding translated language text content. For example, if the phone is in radio reception state and the source language text content is not fully displayed in message bubble 2282 or message bubble 2292 in the translation content display area, and the phone disconnects from the headphones, the displayed interface 230 may not display message bubble 2282 or message bubble 2292, but only retain message bubble 2281 or message bubble 2291 that has fully displayed the source language text content and the translated language text content. However, the voice broadcast control in this message bubble becomes voice broadcast control 2301-1 (e.g., Figure 14 (See message bubble 2301 shown in (2)). That is, in this implementation, when the phone exits the audio reception state due to disconnection from the headphones, the previously collected source language text content is directly discarded.

[0392] For example, in some implementations, the complete source language text content can be displayed at the level of a single sentence. That is, after displaying a complete source language text sentence, the corresponding translated language text content is directly displayed on the screen.

[0393] For example, in some other implementations, the complete source language text content can be the content corresponding to a semantically complete speech segment.

[0394] For example, in some other implementations, the complete source language text content can be the source language text content corresponding to the source language sound recognized before a set time period (e.g., 1 second) has not been recognized. For instance, during the process of recognizing source language sounds, if the current blank interval (the interval during which no source language sound has been recognized) is greater than 1 second, the currently displayed source language text content is determined to be the complete source language text content.

[0395] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0396] Furthermore, it should be noted that in some implementations, such as scenarios where the translated language text content is displayed directly after the source language text content has been fully displayed, the user interface displayed when the phone exits audio reception due to disconnection from the headphones can show the already displayed source language text content, but not the corresponding translated language text content. That is, in this implementation, when the phone exits audio reception due to disconnection from the headphones, the previously captured source language text content can be retained, but the corresponding translated language text content is not displayed in the message bubble showing that source language text content.

[0397] In some implementations of this approach, after exiting the audio reception state, the source language text content (for which the translation language text content has not yet been obtained) can be sent to the server corresponding to the translation application. Once the server returns the translation language text content, the user interface can be refreshed to display the translation language text content in the corresponding message bubble.

[0398] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0399] In addition, it should be noted that in some other implementations, such as scenarios where the translated language text content is displayed in real time following the source language text content (the translated language text content is translated word by word), the user interface displayed when the phone exits the audio reception state due to disconnection from the headphones can retain the source language text content and the translated language text content that have already been displayed.

[0400] With this implementation, when the phone normally exits the audio reception mode, such as when the user finishes speaking a complete sentence of source language text and clicks control 2162 or control 2132 to return to the non-audio reception mode, the translated language text displayed in the message bubble can be refreshed according to the complete source language text content, making the content expressed by the translated language text content more consistent with the source language text content.

[0401] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0402] (3) Relevant content about clicking on headphones in different sound reception scenarios

[0403] For example, in scenarios where headphones support touch control and audio reception, each time a user touches (clicks) the headphones, the phone responds to that user action by performing different translation services, such as entering audio reception mode to collect source language audio, or exiting audio reception / playback mode. In other words, the service performed when clicking the headphones is mapped to the service performed when clicking the corresponding control in the user interface.

[0404] For example, in some implementations, the mapping between clicking the headphones and clicking the controls displayed in the user interface can be as shown in Table 1.

[0405] Table 1. Mapping relationship between clicking the earphone and clicking the control in different audio reception scenarios.

[0406]

[0407] To better understand, the following detailed explanation is provided in conjunction with the accompanying diagram.

[0408] 3.1 Mapping instructions for clicking the headphones in the use case of "Radio + Exclusive Mode".

[0409] It should be noted that in scenarios where the sound reception method is click-to-receive and the headphone usage mode is dedicated mode, the currently connected headphones only have a mapping (correspondence) relationship with the control corresponding to the language used by the headphones. For example, the headphones are mapped to control 2028-1 (or the area of ​​control 2028 excluding control 2028-2) and control 2132.

[0410] Based on the above premises, for example, in the display Figure 15A In the case of interface 203 shown in (1), when the user clicks on any earphone that has established a communication connection with the mobile phone (or clicks on control 2028-1), the mobile phone responds to the user's operation and can switch from non-receiving mode to receiving mode to start collecting source language sound. Furthermore, the user interface can change from interface 203 to interface 213.

[0411] For example, after the source language sound is captured and the corresponding source language text content is displayed on the screen in real time, the user interface can change from interface 213 to interface 214.

[0412] See Figure 15A In example (2), after the source language text content to be translated is fully displayed, such as displaying interface 214, when the user clicks the earphone that establishes a communication connection with the mobile phone (or clicks control 2132), the mobile phone responds to the user's operation by exiting the radio reception state, stopping radio reception, returning to the non-radio reception state, and obtaining the translation language text content based on the source language text content for display. For example, the user interface can change from interface 214 to interface 215.

[0413] For example, in a scenario where a mobile phone establishes a communication connection with headphones and the audio channel is the headphones, after the source language text content and the corresponding translated language text content are displayed in the message bubble in the translation content display area of ​​interface 215, the mobile phone can automatically control the corresponding playback device to play the audio of the translated language text content according to the playback method corresponding to the translated language text content. For example Figure 15AAs shown in (3), the mobile phone will automatically control its own speaker to play the audio of the translated language text content. That is, the sound is played from the mobile phone.

[0414] See Figure 15B In example (1), when the sound reception method is click-to-receive and the headphone usage mode is exclusive mode, if the user clicks the control corresponding to the language used for playback on the device, such as control 2027-1, the phone responds to the user's operation and can switch from non-reception mode to reception mode to start collecting the source language sound. Furthermore, the source language text content corresponding to the collected source language sound can be displayed on the screen in real time, and the user interface can change from interface 203 to... Figure 15B Interface 231 is shown in (2).

[0415] See Figure 15B In example (2), after the source language text content to be translated is fully displayed, such as display interface 231, when the user clicks the earphone that establishes a communication connection with the mobile phone (or clicks control 2132), the mobile phone responds to the user's operation, stops collecting the source language sound corresponding to control 2071, obtains the translation language text content based on the source language text content for display, and begins collecting the source language sound corresponding to control 2081. In this case, control 2162 can become control 2071, and control 2081 can become control 2132. That is, the user interface can change from interface 231 to... Figure 15B Interface 232 is shown in (3).

[0416] For example, in some implementations, such as when the headphones and the phone can receive and play audio separately in dedicated mode, they can also receive and play audio simultaneously. For instance, if the translated language text needs to be played through the headphones, while the source language audio needs to be captured by the phone, the audio of the translated language text can be played through the headphones, while the source language audio is captured by the phone.

[0417] For example, in other implementations, such as when sound can only be received via a mobile phone, and the headphones and mobile phone can play sound separately, if the translated language text content to be played needs to be played via external speaker, and the source language sound to be collected also needs to be collected by the mobile phone, the playback operation can be stopped and the source language sound collection can begin.

[0418] For example, in other implementations, such as when audio can only be received via a mobile phone, and the headphones and mobile phone can play audio separately, if the translated language text content to be played needs to be played via external speaker, and the source language audio to be collected needs to be collected via headphones, the audio of the translated language text content can be played through the headphones, while the source language audio is collected via the mobile phone.

[0419] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0420] This enables control of the current translation process in a scenario where the audio reception method is click-to-receive and the headphone usage mode is a private mode, by clicking the headphone or the corresponding control in the user interface.

[0421] Furthermore, by establishing a mapping relationship between the headphones and the corresponding controls, the corresponding controls in the user interface can change accordingly when the headphones are clicked. This ensures a good user experience while enabling the translation application to be better applied to various usage scenarios.

[0422] 3.2 Mapping instructions for clicking the headphones in the use case of the radio + sharing mode.

[0423] It should be noted that in scenarios where the sound reception method is click-to-receive and the headphone usage mode is shared, the currently connected left and right headphones have mapping relationships with controls corresponding to different languages. For example, the left headphone is mapped to controls 2027 and 2162, while the right headphone is mapped to controls 2028 and 2132.

[0424] Based on the above premises, for example, in the display Figure 16A In the case of interface 233 shown in (1), when the user clicks the right earpiece that has established a communication connection with the mobile phone (or clicks control 2028-1), the mobile phone responds to the user's operation and can switch from non-receiving mode to receiving mode to start collecting source language sound. Furthermore, the source language text content corresponding to the collected source language sound can be displayed on the screen in real time, and the user interface can change from interface 233 to... Figure 16A Interface 234 is shown in (2).

[0425] See Figure 16A In example (2), after the source language text content to be translated is fully displayed, such as displaying interface 234, when the user clicks the right earpiece that has established a communication connection with the mobile phone (or clicks control 2132), the mobile phone responds to the user's operation by exiting the radio reception state, stopping radio reception, returning to the non-radio reception state, and obtaining and displaying the translated language text content based on the source language text content. For example, the user interface can change from interface 234 to... Figure 16A Interface 235 is shown in (3).

[0426] For example, in a scenario where a mobile phone establishes a communication connection with headphones and the audio channel is the headphones, after the source language text content and the corresponding translated language text content are displayed in the message bubble in the translation content display area of ​​interface 235, the mobile phone can automatically control the corresponding playback device to play the audio of the translated language text content according to the playback method corresponding to the translated language text content. For example Figure 16A As shown in (3), the phone will automatically control the left earpiece to play the audio of the translated language text content. That is, the sound is played from the left earpiece.

[0427] See also Figure 16A In example (3), after the audio of the translated language text content has finished playing, when the user clicks the left earpiece (or clicks control 2071-1), the mobile phone responds to the user's operation and can switch from non-receiving mode to receiving mode to start collecting source language sound. Furthermore, the source language text content corresponding to the collected source language sound can be displayed on the screen in real time, and the user interface can change from interface 235 to... Figure 16A Interface 236 is shown in (4).

[0428] Understandably, in other implementations, such as when playing audio of the translated language text content using the left earphone, when the user clicks the left earphone (or clicks control 2071-1), the phone responds to the user's action by stopping the playback of the translated language text content audio and starting to capture the source language sound corresponding to control 2071. Furthermore, the source language text content corresponding to the captured source language sound can be displayed on the screen in real time, and the user interface can change from interface 235 to interface 236.

[0429] See Figure 16A In section (4), for example, after fully displaying the source language text content to be translated, such as displaying interface 236, when the user clicks the right earphone (or clicks control 2162), the mobile phone responds to the user's operation by exiting the audio reception state, stopping audio reception, returning to the non-audio reception state, and obtaining and displaying the translated language text content based on the source language text content. For example, the user interface can change from interface 236 to... Figure 16A Interface 237 is shown in (5).

[0430] For example, in a scenario where a mobile phone establishes a communication connection with headphones and the audio channel is the headphones, after the source language text content and the corresponding translated language text content are displayed in the message bubble in the translation content display area of ​​interface 237, the mobile phone can automatically control the corresponding playback device to play the audio of the latest translated language text content according to the playback mode corresponding to the translated language text content. For example Figure 16A As shown in Figure (6), the phone will automatically control the right earpiece to play the audio of the translated language text content. That is, the sound is played from the right earpiece.

[0431] See Figure 16B In example (1), during the display of interface 234, if the user clicks the left earphone (or control 2081-1) directly without clicking the right earphone or control 2132, the mobile phone responds to the user's operation by stopping the acquisition of the source language sound corresponding to control 2081 and starting to acquire the source language sound corresponding to control 2071. That is, control 2132 displayed in the user interface can become control 2081, control 2071 can become control 2162, and is displayed in the current user interface as shown in the example. Figure 16B The message bubble 2382 shown in (2) is shown in the middle.

[0432] Furthermore, it should be noted that in some implementations, such as when the left earphone (or control 2081-1) is clicked, and the source language text content corresponding to control 2081 is already fully displayed, the translation content display area of ​​the current user interface can display something like... Figure 16B The message bubble 2381 shown in (2) displays the source language text corresponding to the control 2081, as well as the translated language text corresponding to the source language text. That is, the user interface in this case can be as shown in interface 238.

[0433] For example, in some other implementations, it is possible to disregard whether the source language text content corresponding to control 2081 has been fully displayed. If the user clicks the left earphone (or control 2081-1) without clicking the right earphone or control 2132, the already displayed language text content is discarded; for example, the message bubble in interface 234 is not displayed, but message bubble 2382 is displayed in the current user interface. That is, the user interface in this case can be as follows: Figure 16B Interface 238' is shown in (3).

[0434] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0435] This enables control of the current translation process in a scenario where the audio reception method is click-to-receive and the headphone usage mode is shared. This allows users to control the translation process by clicking the headphone or the corresponding control in the user interface.

[0436] Furthermore, by establishing a mapping relationship between the headphones and corresponding controls, the corresponding controls in the user interface can change accordingly when the left or right headphone is clicked. This ensures a good user experience while enabling the translation application to be better applied to various usage scenarios.

[0437] 3.3 Mapping instructions for clicking the headphones in the use case of continuous radio reception + exclusive mode.

[0438] For example, in a scenario where the headphone is used in a private mode, the user can click on entry 2024 to trigger the phone to display window 2121 in the current user interface, and then click on the option corresponding to the continuous sound reception function in window 2121 to switch the sound reception mode to continuous sound reception.

[0439] Taking an example where a user clicks entry point 2024 in interface 203, triggering the phone to display window 2121 in the current user interface, and then clicks the option corresponding to the continuous radio function in window 2121 to switch the radio mode to continuous radio. After the radio mode is switched to continuous radio, the controls corresponding to the two languages ​​can be merged into one control. For example, control 2071 and control 2081 are merged into... Figure 17 Control 2391 is shown in (1). That is, the user interface can change from interface 203 to interface 239.

[0440] Referring to 17A(1), for example, control 2391 may include control 2391-1 for triggering the acquisition of source language sound and control 2391-2 for triggering the display of input keyboard.

[0441] Among them, control 2391 is used to indicate that the mobile phone is in a non-receiving state.

[0442] In this embodiment, the non-receiving state can be understood as a state indicating that the mobile phone is not currently picking up sound or playing translated audio. That is, a state in which translation services are not being performed.

[0443] For example, after the user clicks control 2391-2, an input keyboard can be displayed in the current user interface. The displayed input keyboard can be either the input keyboard for the first language or the input keyboard for the second language. The input keyboard for the first language and the input keyboard for the second language can be switched using a language switching control on the input keyboard.

[0444] For example, in some implementations, in scenarios where the sound reception method is continuous and the headphone usage mode is exclusive mode, each currently connected headphone can have a mapping relationship with control 2391-1 (or the area of ​​control 2391 excluding control 2391-2), as well as the subsequently appearing controls 2403 and 2432.

[0445] Based on the above premises, for example, in the display Figure 17 In the case of interface 239 shown in (1), when the user clicks on any earphone that has established a communication connection with the mobile phone (or clicks on control 2391-1), the mobile phone responds to the user's operation and can switch from non-receiving mode to receiving mode to start collecting source speech sound. Furthermore, the user interface can change from interface 239 to... Figure 17Interface 240 is shown in (2).

[0446] It should be noted that each time the interface transitions from non-receiving mode to receiving mode, before the source language sound is detected, a "Listening" message bubble will be displayed on interface 240. For example, message bubbles 2401 and 2402 are shown in interface 240. Message bubble 2401 displays the language shown in control 2025-3, while message bubble 2402 displays the language shown in control 2025-1.

[0447] See also Figure 17 In (2), for example, in the radio reception state, control 2391 will change into control 2403, which indicates that the mobile phone is in the radio reception state.

[0448] For example, during the display of interface 240, if the mobile phone recognizes the source language sound, it will display the corresponding language on the screen in real time, word by word, according to the language (language) corresponding to the recognized source language sound. The message bubble on the other side will automatically disappear. Taking the language (such as Chinese) displayed in control 2025-1 as an example, the message bubble 2402 can change to the language displayed in real time, following the source language text content. Figure 17 The message bubble 2411 shown in (3) gradually disappears from the interface 240, and the final user interface becomes interface 20a.

[0449] For example, in some implementations, a disappearance transition effect can be added to the message bubble that needs to disappear. For example, it can gradually become transparent until it disappears completely.

[0450] For example, in some implementations, to improve the user experience, a transition effect can be added to each message bubble that appears. For instance, it can gradually change from transparent to fully visible.

[0451] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0452] For example, in some implementations, during the real-time display of the source language text content converted from the source language sound in message bubble 2411, if the user clicks on any of the earphones connected to the phone (or clicks on control 2403), the phone, in response to the user's operation, will stop collecting the source language sound, exit the sound receiving state, and return to the non-sound receiving state. That is, control 2403 can revert back to control 2391.

[0453] Furthermore, it should be noted that in some implementations, when exiting the audio receiving state and returning to the non-audio receiving state, if the source language text displayed in message bubble 2411 is a complete sentence in the source language, the user interface corresponding to the non-audio receiving state can display that source language text, as well as the corresponding translated language text. For example... Figure 17 The message bubble 2421 in the interface 242 shown in (4) is shown, but the audio of the translated language text content displayed in the message bubble 2421 is not played at present.

[0454] For example, in some other implementations, during the real-time display of the source language text content converted from the source language audio in message bubble 2411, if the user does not click on any of the earphones connected to the mobile phone (or click on control 2403), after displaying a complete sentence of source language text content, the translated language text content will appear in the entire sentence in message bubble 2411. That is, message bubble 2411 can become... Figure 17 The message bubble 2431 is shown in (5). Furthermore, the mobile phone will control the corresponding playback device to play the audio of the source language text content according to the playback method corresponding to the translated language text content displayed in the message bubble 2431. For example... Figure 17 As shown in (5), the phone's speaker is controlled to play the audio of the translated language text content displayed in the message bubble 2431 via external speaker. During this process, the control 2403 currently displayed in the user interface can be changed to control 2432, which indicates that the phone is in playback mode (a state of playing audio of translated language text content).

[0455] For example, during the display of interface 243, if the user clicks on any of the earphones connected to the mobile phone (or clicks on control 2432), the mobile phone, in response to the user's operation, will stop playing the audio of the translated language text content, exit the broadcast state, and return to the radio state. That is, control 2432 can revert back to control 2403.

[0456] For example, in some implementations, after interrupting the currently playing audio and returning from the playback state to the receiving state, a "Listening" message bubble can be displayed in the user interface before the source language sound is recognized. For example, Figure 17 The message bubbles 2441 and 2442 shown in the interface 244 shown in (6) are as follows. The content displayed in message bubble 2441 uses the language displayed in control 2025-3, and the content displayed in message bubble 2442 uses the language displayed in control 2025-1.

[0457] For example, in some other implementations, after interrupting the currently playing audio and returning from the playing state to the receiving state, the user interface may display only a "Listening" message bubble before the source language sound is recognized. For example, only message bubble 2441 or message bubble 2442 may be displayed.

[0458] For example, in some implementations, the decision can be made based on the message bubble containing the translated language text content that has already been displayed, whether the message bubble to be displayed before the source language sound is identified is message bubble 2441 or message bubble 2442.

[0459] For example, if message bubble 2431 is already displayed on the current user interface, and the source language in message bubble 241 is the language corresponding to control 2081, the message bubble to be displayed before the source language sound is recognized could be message bubble 2442. That is, before the source language sound is recognized, the language used in the "listening" message bubble can be the opposite of the language corresponding to the source language text content in the message bubble that has already displayed the translated language text content, or the same as the language corresponding to the translated language text content.

[0460] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the only limitation on this embodiment.

[0461] This enables users to control the current translation process by clicking on the headphones or the corresponding controls in the user interface, in scenarios where the audio reception mode is continuous and the headphone usage mode is private.

[0462] Furthermore, by establishing a mapping relationship between the headphones and the corresponding controls, the corresponding controls in the user interface can change accordingly when the headphones are clicked. This ensures a good user experience while enabling the translation application to be better applied to various usage scenarios.

[0463] 3.4 Mapping instructions for clicking the headphones in the use case of continuous radio reception + sharing mode

[0464] For example, in a scenario where the headphone is used in shared mode, the user can click on entry 2024 to trigger the phone to display window 2121 in the current user interface, and then click on the option corresponding to the continuous radio function in window 2121 to switch the radio mode to continuous radio.

[0465] Taking the example of a user clicking entry 2024 in interface 233, triggering the phone to display window 2121 in the current user interface, and then clicking the option corresponding to the continuous radio function in window 2121 to switch the radio mode to continuous radio. After the radio mode is switched to continuous radio, the controls corresponding to the two languages ​​can be merged into one control. For example, controls 2071 and 2081 are merged into control 2391. That is, the user interface can change from interface 233 to... Figure 18 Interface 245 is shown in (1).

[0466] For example, in some implementations, in scenarios where the sound reception method is continuous and the headphone usage mode is shared mode, each currently connected headphone can have a mapping relationship with control 2391-1 (or the area of ​​control 2391 excluding control 2391-2), control 2403, and control 2432.

[0467] Based on the above premises, for example, in the display Figure 18 In the case of interface 245 shown in (1), when the user clicks on any earphone that has established a communication connection with the mobile phone (or clicks on control 2391-1), the mobile phone responds to the user's operation by entering the receiving state from the non-receiving state, starting to collect source language sound, and converting the collected source language sound into source language text content in real time for display. For example, when the source language sound is collected, the user interface may display, for example, Figure 18 Interface 246 is shown in (2).

[0468] For example, in some implementations, during the real-time display of the source language text content converted from the source language sound in the message bubble 2461 of interface 246, if the user clicks on any of the earphones connected to the mobile phone (or clicks on control 2403), the mobile phone, in response to the user's operation, will stop collecting the source language sound, exit the sound receiving state, and return to the non-sound receiving state. That is, control 2403 can revert back to control 2391.

[0469] Furthermore, it should be noted that in some implementations, when exiting the audio receiving state and returning to the non-audio receiving state, if the source language text displayed in message bubble 2461 is a complete sentence in the source language, the user interface corresponding to the non-audio receiving state can display that source language text, as well as the corresponding translated language text. For example... Figure 18 The message bubble 2471 in the interface 247 shown in (3) is shown, but the audio of the translated language text content displayed in the message bubble 2471 is not being played at present.

[0470] For example, in some other implementations, during the real-time display of the source language text content converted from the source language audio in message bubble 2461, if the user does not click on any of the earphones connected to the mobile phone (or click on control 2403), after displaying a complete sentence of source language text content, the translated language text content will appear in the entire sentence in message bubble 2461. That is, message bubble 2461 can become... Figure 18 The message bubble 2481 is shown in (4). Furthermore, the mobile phone will control the corresponding playback device to play the audio of the source language text content according to the playback method corresponding to the translated language text content displayed in the message bubble 2481. For example... Figure 18 As shown in (4), the left earpiece connected to the control plays the audio of the translated language text content displayed in message bubble 2481. During this process, control 2403 displayed in the current user interface can be changed to control 2432.

[0471] For example, such as Figure 18 As shown in (4), during the display of interface 248, if the user clicks on any of the earphones that have established a communication connection with the mobile phone (or clicks on control 2432), the mobile phone will respond to the user's operation by stopping the playback of the audio of the translated language text content, exiting the broadcast state, and returning to the radio state. That is, control 2432 can revert back to control 2403.

[0472] In other words, when the sound reception mode is continuous, regardless of whether the currently connected headphones are in exclusive or shared mode, the mapping relationship between each currently connected headphone and control 2391-1 (or the area of ​​control 2391 excluding control 2391-2), control 2403, and control 2432 is the same. That is, in a scenario where the sound reception mode is continuous and the headphone usage mode is shared, the processing logic and control changes when clicking the headphones can follow the exclusive mode. For specific implementation details, please refer to the description of the embodiment regarding usage scenario 3.3 above; it will not be repeated here.

[0473] This enables control of the current translation process in a scenario where the audio reception mode is continuous and the headphone usage mode is shared, by clicking on the headphone or the corresponding control in the user interface.

[0474] Furthermore, by establishing a mapping relationship between the headphones and the corresponding controls, the corresponding controls in the user interface can change accordingly when the headphones are clicked. This ensures a good user experience while enabling the translation application to be better applied to various usage scenarios.

[0475] In addition, it should be noted that in continuous audio recording scenarios during dialogue translation, switching headphone usage modes can be done without interrupting the current translation process.

[0476] For example, in a scenario where the headphones are used in a private mode, the phone is in audio recording mode, capturing the source language audio and displaying the corresponding source language text content in real time (e.g., ...). Figure 19 During the process shown in message bubble 2491 in (1), when the user clicks icon 2031, the phone responds to the user's operation and can display the following on the current user interface: Figure 19 The interface 249 shown in (1) displays window 2191.

[0477] For example, while window 2191 is displayed on the current user interface, the mobile phone can continue to operate in audio reception mode, capturing source language audio and displaying the corresponding source language text content in real time, for example... Figure 19 The message bubble 2501 in the interface 250 shown in (2) is shown.

[0478] See also Figure 19 In example (2), after the user clicks option 2191-3 in window 2191, the mobile phone responds to the user's operation and can switch the headphone usage mode from exclusive mode to shared mode, and the user interface can change from interface 250 to... Figure 19 Interface 251 is shown in (3).

[0479] See Figure 19 In example (3), after the headphone usage mode is switched from exclusive mode to shared mode, if the user does not click control 2403 or the connected headphone, the mobile phone will still work in the sound receiving state, collect the source language sound, and display the corresponding source language text content in real time, as shown in message bubble 2511.

[0480] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended as the sole limitation of this embodiment. In actual use cases, when the headset connected to the mobile phone is in shared mode and is working in a continuous audio reception scenario for dialogue translation, the user can click icon 2201 to bring up window 2221, and then click option 2221-1 in window 2221 to switch the headset usage mode to dedicated mode without interrupting the currently executing translation service.

[0481] Therefore, when a mobile phone is connected to headphones and is in a continuous audio recording scenario for dialogue translation, the user can automatically switch the headphone usage mode as needed without interrupting the current translation business, thus making better use of the translation application for conversational communication.

[0482] Through the above-described embodiments (1) to (3), in the scenario where a user operation is received to trigger the display of the user interface corresponding to the dialogue translation function, the user interface suitable for the current usage scenario can be displayed according to the actual situation of the electronic device being connected to headphones and the selected audio channel. This makes it convenient for users to use the dialogue translation function of the translation application to conduct dialogue communication in multilingual communication scenarios, according to the actual usage scenario, by selecting different translation modes, audio channels, headphone usage modes, etc.

[0483] Furthermore, it should be understood that, in order to achieve the aforementioned functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. Based on the algorithmic steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0484] Furthermore, it should be noted that in practical application scenarios, the translation methods provided in the above embodiments implemented by electronic devices can also be executed by a chip system included in the electronic device, wherein the chip system may include a processor. The chip system can be coupled to a memory, enabling the chip system to call a computer program stored in the memory during runtime to implement the translation methods executed by the electronic device. The processor in the chip system can be an application processor or a non-application processor.

[0485] In addition, this application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned related method steps to implement the translation method in the above embodiments.

[0486] In addition, this application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform the aforementioned related steps to implement the translation method in the above embodiments.

[0487] In addition, embodiments of this application also provide a chip (which may also be a component or module), the chip may include one or more processing circuits and one or more transceiver pins; wherein, the transceiver pins and the processing circuits communicate with each other through internal connection paths, the processing circuits execute the above-mentioned related method steps to implement the translation method in the above embodiments, so as to control the receiving pin to receive signals and control the transmitting pin to transmit signals.

[0488] Furthermore, as can be seen from the above description, the electronic devices, computer-readable storage media, computer program products, or chips provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0489] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A translation method, characterized in that, Applied to electronic devices, the method includes: The first interface of the translation application is displayed. The first interface includes a first control and a second control. The first control is used to indicate that the translation application is in a non-audio reception state of the dialogue translation function. The second control is used to indicate that the translation headset is in a first usage mode. In the first usage mode, the electronic device establishes a communication connection with the first and / or second headsets of the translation headset. After receiving the user's operation on the first control or the translation headset, the system begins to collect source language sound and displays a second interface. The second interface includes the second control and a third control. The third control is used to indicate that the translation application is in the audio recording state of the dialogue translation function. After receiving the user's operation on the second control, the system continues to collect source language sound and displays a third interface. The third interface includes the second control, first source language text content that changes in real time following the collected source language sound, and a first window that includes a first option. After receiving the user's operation on the first option, the system continues to collect source language sound and displays a fourth interface. The fourth interface includes the third control, the first source language text content that changes in real time following the collected source language sound, and the fourth control. The fourth control is used to indicate that the translation headset is in a second usage mode. In the second usage mode, the electronic device establishes a communication connection with the first headset and the second headset. When the language type corresponding to the first source language text content is a second language, after the first source language text content is fully displayed, the first headphones are used to play the first translated language text content in the first language and the fifth interface is displayed. When the language type corresponding to the first source language text content is the first language, after the first source language text content is fully displayed, the second headphones are used to play the first translated language text content in the second language, and the fifth interface is displayed; The fifth interface includes the fourth control, the first source language text content, the first translation language text content, and the fifth control. The fifth control is used to indicate that the translation application is in the broadcast state of the dialogue translation function, and the first translation language text content is the translation result of the first source language text content.

2. The method according to claim 1, characterized in that, The method further includes: While using the first earphones to play the translated text content in the first language according to the first language, the source language sound continues to be collected; After playing the first translated language text content in the first language using the first earphone, the source language sound is collected again and a sixth interface is displayed. The sixth interface includes the third control, the fourth control, the first source language text content, the first translated language text content, and the second source language text content that changes in real time following the collected source language sound. When the language type corresponding to the second source language text content is the first language, after the second source language text content is fully displayed, the second headphones are used to play the second translated language text content according to the second language, and the seventh interface is displayed; When the language type corresponding to the second source language text content is the second language, after the second source language text content is fully displayed, the first headphones are used to play the second translated language text content in the first language, and the seventh interface is displayed; The seventh interface includes the fourth control, the first source language text content, the first translated language text content, the second source language text content, the second translated language text content, and the fifth control, wherein the second translated language text content is the translation result of the second source language text content.

3. The method according to claim 1, characterized in that, The method further includes: During the display of the fifth control, after receiving the user's operation on the fourth control, the source language sound continues to be collected, and the first translated language text content is played using the first or second earphone, and an eighth interface is displayed. The eighth interface includes the fourth control, the first source language text content, the first translated language text content, and the first window. The first window also includes a second option. After receiving the user's operation on the second option, the system continues to collect source language audio, plays the first translated language text content using the speaker of the electronic device or the translation headphones, and displays a ninth interface, which includes the second control, the fifth control, the first source language text content, and the first translated language text content. After playing the first translated language text content using the speaker or the translation headphones, the source language sound continues to be collected, and a tenth interface is displayed. The tenth interface includes the second control, the third control, the first source language text content, the first translated language text content, and the second source language text content that changes in real time following the collected source language sound. When the language type corresponding to the second source language text content is the first language, after the second source language text content is fully displayed, the translation headphones are used to play the second translated language text content according to the second language, and the eleventh interface is displayed; When the language type corresponding to the second source language text content is the second language, after the second source language text content is fully displayed, the speaker is used to play the second translated language text content in the first language, and the eleventh interface is displayed; The eleventh interface includes the second control, the fifth control, the first source language text content, the first translated language text content, the first source language text content, and the first translated language text content, wherein the second translated language text content is the translation result of the second source language text content.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: While displaying the fifth interface and playing the first translated language text content using the first or second earphone, upon receiving a user's operation on the fifth control or the translation earphone, the playback of the first translated language text content stops, the source language sound continues to be collected, and the twelfth interface is displayed. The twelfth interface includes the third control, the fourth control, the first source language text content, the first translated language text content, and the second source language text content that changes in real time following the collected source language sound.

5. The method according to claim 4, characterized in that, The method further includes: During the process of collecting source language audio while displaying the twelfth interface, upon receiving user input to the third control or the translation headset, the collection of source language audio stops, and the thirteenth interface is displayed. The thirteenth interface includes the first control, the fourth control, the first source language text content, and the first translation language text content.

6. The method according to claim 5, characterized in that, The method further includes: When a user operates the third control or the translation headset, and the second source language text content is fully displayed, a fourteenth interface is displayed. The fourteenth interface includes the first control, the fourth control, the first source language text content, the first translation language text content, the second source language text content, and the second translation language text content, where the second translation language text content is the translation result of the second source language text content.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: During the display of the fourth control, after detecting a disconnection between the first or second earphone, the currently executed translation service continues, and the fifteenth interface is displayed, which includes the second control.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: During the display of the second control or the fourth control, if a communication connection with the translation headset is detected to be lost, the currently executing translation service is stopped, and a sixteenth interface is displayed, which does not include the second control or the fourth control. In the absence of displaying the second control or the fourth control, when there is translated language text content that needs to be played, the electronic device's speaker is used to play the translated language text content.

9. The method according to claim 8, characterized in that, The method further includes: When the second control and the fourth control are not displayed, during the process of playing the translated language text content using the speaker of the electronic device, after a communication connection with the translation headset is detected, the currently executed translation service continues to be executed, and the seventeenth interface is displayed, which includes the second control or the fourth control; When the second control is displayed on the seventeenth interface, if the translated language text content is the first language, the translated language text content is played again using the speaker; if the translated language text content is the second language, the translated language text content is played again using the translation headphones. When the fourth control is displayed on the seventeenth interface, if the translated language text content is the first language, the translated language text content is played using the first headphones; if the translated language text content is the second language, the translated language text content is played using the second headphones.

10. The method according to any one of claims 1 to 9, characterized in that, During the display of the third control, the microphone of the electronic device is used to capture the source language sound.

11. An electronic device, characterized in that, The electronic device includes: a memory and a processor, the memory and the processor being coupled; the memory stores program instructions, which, when executed by the processor, cause the electronic device to perform the translation method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The invention includes a computer program that, when run on an electronic device, causes the electronic device to perform the translation method as described in any one of claims 1 to 10.

Images