Audio and text conversion method, electronic equipment and storage medium

Abstract

The invention provides an audio and text conversion method, electronic equipment and a storage medium, and relates to the technical field of terminals. The method comprises the steps that a first conversion panel is displayed on a first interface, the first conversion panel comprises a first mode switching control and a second mode switching control, and the first mode switching control is in a selected state; in response to a first operation of the user, the first interface is switched into a second interface, the first conversion panel is displayed on the second interface, and a first area of the second interface comprises the first multimedia file. And when the first multimedia file is played, displaying subtitles corresponding to the first multimedia file on the first conversion panel. When it is detected that the user selects the second mode switching control, a second conversion panel is displayed, first voice is collected through the microphone, and text data corresponding to the first voice is displayed on the second conversion panel. In this way, the electronic equipment can meet the requirements of a user for voice and text conversion in various application scenes.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on November 3, 2023, with application number 202311466471.7 and invention name “A method and electronic device for converting speech and text”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to an audio and text conversion method, an electronic device and a storage medium. Background Art

[0003] At present, electronic devices have greatly changed the lifestyles of users, especially portable electronic devices, which have become indispensable products for people. In order to provide better services to users, electronic devices have audio-to-text conversion functions, such as converting audio to text, or converting text to audio. For example, electronic devices can convert input voice to text through chat application software. For another example, electronic devices can convert text to voice for playback through reading application software.

[0004] However, the audio-to-text conversion function of electronic devices is not perfect and cannot meet the audio-to-text conversion needs of users in various application scenarios. Summary of the invention

[0005] The present application provides an audio and text conversion method, an electronic device and a storage medium, which are used to provide a more complete audio and text conversion function, suitable for the audio-text conversion needs of various application scenarios.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a method for converting audio and text is provided, the method comprising: displaying a first conversion panel on a first interface, the first conversion panel including a first mode switching control and a second mode switching control, and the first mode switching control is in a selected state. In response to a first operation of a user, the first interface is switched to a second interface, and the first conversion panel is displayed on the second interface, and the first area of ​​the second interface includes a first multimedia file. When the first multimedia file is played, the subtitles corresponding to the first multimedia file are displayed on the first conversion panel. When it is detected that the user has selected the second mode switching control, the second conversion panel is displayed, and the first voice is collected through a microphone, and the text data corresponding to the first voice is displayed on the second conversion panel. The second conversion panel is similar to the first conversion panel, both including the first mode switching control and the second mode switching control, but the specific functions carried are different.

[0008] In this way, the electronic device can provide subtitles of multimedia files, text data corresponding to voice data, and call text corresponding to call voice through the conversion panel, which is suitable for audio-to-text conversion modes in various application scenarios. Users can use the conversion panel to quickly switch between different application scenarios of the audio-to-text conversion mode, thereby simplifying the switching process of the audio-to-text conversion function in different application scenarios and meeting the user's needs for audio-to-text conversion in various application scenarios. Among them, the first conversion panel can provide subtitles of multimedia files, the second conversion panel can provide text data of the first voice, and the third conversion panel can provide call text corresponding to the call voice.

[0009] In a possible implementation of the first aspect, the second conversion panel corresponds to a dialogue mode or a translation mode. If the second conversion panel corresponds to the dialogue mode, the text data corresponding to the first voice is obtained by recognizing the first voice. If the second conversion panel corresponds to the translation mode, the text data corresponding to the first voice is obtained by translating the text data recognized by the first voice.

[0010] In this implementation, the second conversion panel can provide a function of converting the first voice into text data in the conversation mode. The second conversion panel can provide a function of translating the first voice into text data in the translation mode.

[0011] In another possible implementation of the first aspect, the second conversion panel further includes a setting control. In response to a user operation on the setting control, a setting page is displayed in the second conversion panel, and the setting page includes a mode selection control. In response to a user operation on the mode selection control, a conversation mode page is displayed in the second conversion panel, and the conversation mode page includes options for a conversation mode and options for a translation mode. If the option for the conversation mode is in a selected state, the second conversion panel corresponds to the conversation mode. If the option for the translation mode is in a selected state, the second conversion panel corresponds to the translation mode.

[0012] In this implementation, the user can select the translation mode or the dialogue mode through the setting control of the second conversion panel. The user can switch between the translation mode and the dialogue mode through the second conversion panel, thereby simplifying the switching process of the audio-to-text conversion function in different application scenarios and meeting the user's needs for audio-to-text conversion in various application scenarios.

[0013] And the translation function of the text data input by the user to the voice data.

[0014] In another possible implementation of the first aspect, when the second conversion panel corresponds to the translation mode, if the first input text is acquired in the second conversion panel, the translation text of the first input text is displayed in the second conversion panel, and the language types of the first input text and the translation text are different. In this implementation, the user can input the first input text through the second conversion panel, and the electronic device provides the translation text of the first input text through the second conversion panel.

[0015] In another possible implementation of the first aspect, the second conversion panel also includes a display play control. In response to a user operation on the play control, the voice data corresponding to the translated text is played. In this implementation, the second conversion panel may also provide a play control for playing the translated text, and the user may play the voice data corresponding to the translated text by clicking the play control.

[0016] In another possible implementation of the first aspect, when the second conversion panel corresponds to the dialogue mode, if the second input text is obtained in the conversion panel, the voice data corresponding to the second input text is played. In this implementation, the electronic device also provides an automatic voice playback function for the second input text. When it is inconvenient for the user to speak, the user can communicate with others by playing the voice data corresponding to the second input text, thereby providing convenience for the user.

[0017] In another possible implementation of the first aspect, the first conversion panel or the second conversion panel includes a form switching control. In response to a form switching operation of the user on the form switching control, the display form of the first conversion panel or the second conversion panel is switched from the first form to the second form, and the second form is a display form different from the first form. In this implementation, the first conversion panel or the second conversion panel is provided with a form switching control for switching between different display forms. The user can change the display form of the first conversion panel or the second conversion panel through the form switching control.

[0018] In another possible implementation of the first aspect, the first conversion panel or the second conversion panel has any one of a default form, a floating ball form, and a full-screen form. The default form is a default window, and the height of the default window is less than the height of the full-screen display area of ​​the electronic device. The floating ball form is a floating ball window, and the diameter of the floating ball window is less than a preset value. The full-screen form is a full-screen window, and the full-screen window corresponds to the full-screen display area. In this implementation, the first conversion panel or the second conversion panel has a variety of display forms, which can provide users with a variety of choices.

[0019] In another possible implementation of the first aspect, the form switching control includes a minimize control and a minimize control, and in response to a user operation on the form switching control, the display form of the first conversion panel or the second conversion panel is switched from the first form to the second form, and in response to the user's form switching operation on the minimize control, the display form of the first conversion panel is switched from the default form to the suspended ball form. Alternatively, in response to the user's form switching operation on the maximize control, the display form of the first conversion panel or the second conversion panel is switched from the default form to the full-screen form. In this implementation, the user can switch the first conversion panel or the second conversion panel from the default form to the suspended ball form through the minimize control. The user can also switch the first conversion panel or the second conversion panel from the default form to the full-screen form through the minimize control. In this way, the user can switch the display form of the first conversion panel or the second conversion panel according to demand, thereby improving the user experience.

[0020] In another possible implementation of the first aspect, the electronic device adjusts the display position of the default window or the floating ball window on the screen of the electronic device in response to the moving operation. For example, the moving operation is a drag operation acting on the shortcut key area of ​​the default window, and the electronic device moves the default window following the moving track of the moving operation in response to the moving operation. In this way, the user can adjust the display position of the default window or the floating ball window according to needs or preferences, thereby improving the user experience.

[0021] In another possible implementation of the first aspect, during the movement of the default window, if the ratio of the area of ​​the invisible area of ​​the default window to the total area is greater than a preset ratio, the first conversion panel or the second conversion panel is switched from the default form to the suspended ball form. The invisible area is the partial area where the default window exceeds the screen boundary. In this way, when the default window of the first conversion panel or the second conversion panel moves beyond the screen boundary, the electronic device switches the first conversion panel or the second conversion panel from the default form to the suspended ball form, so that the first conversion panel or the second conversion panel automatically shrinks from the default window corresponding to the default form to the suspended ball window corresponding to the suspended ball form. In this way, the electronic device can provide a solution for the first conversion panel or the second conversion panel to transform from the default form to the suspended ball form, which is convenient for user operation and improves user experience.

[0022] In another possible implementation of the first aspect, the electronic device adjusts the size of the default window in response to the size adjustment operation. In this way, the user can adjust the size of the default window, such as increasing or decreasing the size of the default window, thereby improving the user experience.

[0023] In another possible implementation of the first aspect, when the first conversion panel is closed, if the second multimedia file is played and the preset conditions are met, a first prompt message is displayed, and the first prompt message is used to prompt to turn on the conversion function between audio data and text data, and display the subtitles of the second multimedia file through the first conversion panel. The preset conditions include at least one of the playback volume of the second multimedia file is less than the preset volume, the power of the electronic device is less than the preset power, and the ambient noise collected by the electronic device is greater than the preset decibel. In this way, when the second conversion panel is not turned on, if the electronic device detects that the multimedia file is played and the preset conditions are met, the electronic device prompts to turn on the conversion function to provide subtitles of the multimedia file through the second conversion panel.

[0024] In another possible implementation of the first aspect, the text data corresponding to the first voice is in any language type among Chinese, English, Korean and Japanese. The electronic device can provide voice-to-text conversion functions in multiple language types, which is applicable to more voice-to-text conversion scenarios and meets user needs.

[0025] In another possible implementation of the first aspect, if a call voice is detected, a third conversion panel is displayed, and a call text corresponding to the call voice is displayed in the third conversion panel. In this way, the electronic device can display the call content in text on the screen. Even if the user does not hear the call voice clearly or it is inconvenient to listen to the call voice, the user can understand the call content through the call text displayed on the screen, and achieve barrier-free calls.

[0026] In another possible implementation of the first aspect, the first conversion panel or the second conversion panel includes a plurality of setting items including font size, transparency, language type of subtitles, language type of dialogue mode, source language type of translation mode, target language type of translation mode, saving panel content, clearing panel content, and broadcast timbre. The user can adjust one or more of the font size, transparency, language type of subtitles, language type of dialogue mode, source language type of translation mode, target language type of translation mode, saving panel content, clearing panel content, and broadcast timbre on the first conversion panel or the second conversion panel through the plurality of setting items provided by the first conversion panel or the second conversion panel as required.

[0027] In another possible implementation of the first aspect, in response to the timbre setting operation, the announcement timbre corresponding to the second conversion panel is set to the timbre selected by the user in the setting item of the announcement timbre. In this way, the electronic device can play the voice data using the timbre selected by the user, for example, using the timbre of the voice assistant to play the voice data corresponding to the second input text. The timbre of the voice assistant is the timbre used by the voice assistant when playing the voice.

[0028] In a second aspect, the present application provides an electronic device, comprising: a screen (or display screen), a memory, and a processor. The screen and the memory are respectively coupled to the processor. The screen is used to display an interface. The memory stores a computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method described in the first aspect and any possible implementation thereof.

[0029] In a third aspect, the present application provides a computer-readable storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation thereof.

[0030] In a fourth aspect, the present application provides a computer program product comprising program instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of an interface of a chat application software provided in an embodiment of the present application;

[0032] Figure 2 A hardware structure block diagram of a mobile phone 100, an example of an electronic device provided in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a conversion panel provided in an embodiment of the present application;

[0034] Figure 4 A flowchart of a method for converting audio and text provided in an embodiment of the present application;

[0035] Figure 5 A schematic diagram of enabling the audio-to-text conversion function provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of a first conversion panel provided in an embodiment of the present application;

[0037] Figure 7 A schematic diagram of a second conversion panel provided in an embodiment of the present application;

[0038] Figure 8 A schematic diagram of an interface for receiving text data by a second conversion panel provided in an embodiment of the present application;

[0039] Fig. 9 A schematic diagram of an interface for receiving voice data and text data by a second conversion panel provided in an embodiment of the present application;

[0040] Fig.10A schematic diagram of a third conversion panel provided in an embodiment of the present application;

[0041] Fig.11 A schematic diagram of a display form of a conversion panel provided in an embodiment of the present application;

[0042] Fig.12 A schematic diagram of a position change of a conversion panel provided in an embodiment of the present application;

[0043] Fig.13 A schematic diagram of a conversion panel provided in an embodiment of the present application switching from a default window to a floating ball window;

[0044] Fig.14 A schematic diagram of a conversion panel window size adapted to an interface display mode provided in an embodiment of the present application;

[0045] Fig.15 A schematic diagram of adjusting the window size of a conversion panel provided in an embodiment of the present application;

[0046] Fig.16 A schematic diagram of multiple setting items of a conversion panel provided in an embodiment of the present application;

[0047] Fig.17 A schematic diagram of multiple setting items of another conversion panel provided in an embodiment of the present application;

[0048] Fig.18 A schematic diagram of a first conversion panel activation prompt provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In daily life, users may often encounter the need for audio-to-text (or text) or text-to-audio due to personal reasons (such as hearing impairment, language barriers, etc.) and environmental reasons (such as noisy environment). For example, for users with hearing impairments, due to hearing impairments, users have the need for audio-to-text conversion in the following scenarios: audio and video playback scenarios, offline conversations or communication scenarios, and online voice or video call scenarios. For users without hearing impairments, there will also be audio-to-text conversion needs in multiple scenarios: in scenarios where multiple video conferences are concurrent, it is difficult for users to simultaneously obtain information from multiple video conferences, play foreign audio and video files, and have voice barriers during offline conversations or communications.

[0050] In order to meet the user's voice-to-text conversion needs, electronic devices provide audio-to-text conversion functions (hereinafter referred to as voice-to-text conversion functions) through some installed application software. For example, a chat application software is installed in the electronic device. The chat application software provides a function of converting voice to text. The interface of the chat application software is as follows: Figure 1As shown. Next to the voice message box, a "convert to text" reminder box is set. If the electronic device receives a user's click operation on the "convert to text" reminder box, the electronic device converts the voice message in the voice message box into text, and displays the text corresponding to the voice message below the voice message box. In this way, through the voice-to-text function provided by the chat application software, the user can know the content of the voice message without playing the voice message, which is convenient for the user.

[0051] However, the audio and text conversion function currently provided by electronic devices has a single application scenario and cannot meet the needs of users for multiple application scenarios. In addition, the application software that provides the audio to text conversion function and the application software that provides the text to audio conversion function in the electronic device are usually different application software. This will increase the difficulty of use for users and reduce the user experience.

[0052] In view of this, the method provided in the embodiment of the present application provides a conversion function of audio data and text data suitable for a variety of application scenarios. Specifically, the electronic device is provided with a function for providing audio and text conversion. When the electronic device turns on the audio and text conversion function, the electronic device displays a conversion panel in the first interface. The conversion panel includes a mode switching control for indicating the conversion mode, such as a first mode switching control and a second mode switching control. The user can switch between multiple conversion modes of the conversion function through the conversion panel. One conversion mode corresponds to one application scenario. In this way, the electronic device can provide audio-text conversion functions for a variety of application scenarios. In addition, the method of use is simple, which can reduce the difficulty of use for users and improve user experience.

[0053] In an embodiment of the present application, the conversion function includes a first conversion mode and a second conversion mode. The first conversion mode is used to provide a subtitle function for multimedia files. When the first mode switching control is in a selected state, the electronic device is in the first conversion mode. In the first conversion mode, if the electronic device plays a multimedia file, the subtitles of the multimedia file are displayed in the conversion panel of the electronic device. The second conversion mode is used to provide a conversion function from a first voice to text data, or to provide a conversion function from text data (which may be referred to as input text) input by a user to voice data. When the second mode switching control is in a selected state, the electronic device is in the second conversion mode. In the second conversion mode, if the electronic device collects a first voice, the electronic device displays the text data corresponding to the first voice in the conversion panel.

[0054] In some implementations, the conversion function further includes a third conversion mode. If the electronic device detects a call voice, the electronic device automatically switches to the third conversion mode and displays the call text corresponding to the call voice in the conversion panel.

[0055] It can be seen that the electronic device can provide subtitles of multimedia files, text data corresponding to voice data, and call text corresponding to call voice through the conversion panel, which is applicable to the audio-to-text conversion function in various application scenarios. Users can use the conversion panel to quickly switch between different application scenarios of the audio-to-text conversion function, thereby simplifying the switching process of the audio-to-text conversion function in different application scenarios and meeting the user's needs for audio-to-text conversion in various application scenarios.

[0056] For example, the electronic device described in the embodiments of the present application may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) / virtual reality (VR) device, a media player, a wearable device, etc. The embodiments of the present application do not impose any special restrictions on the specific form of the electronic device.

[0057] In the embodiment of the present application, the electronic device is a mobile phone 100 as an example, and the hardware structure of the electronic device is introduced through the mobile phone 100. Figure 2 As shown, the mobile phone 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0058] Among them, the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural network processor (neural-network processing unit, NPU), a driver processor, etc. Among them, different processing units can be independent devices or integrated in one or more processors. The processor 110 can be the nerve center and command center of the mobile phone 100. The processor 110 can generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0059] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0060] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, voice, music, video and other files are stored in the external memory card. In some implementations, the mobile phone 100 can also store text records of the conversion panel in the external memory card.

[0061] The internal memory 121 may be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121. For example, in an embodiment of the present application, the processor 110 may execute the instructions stored in the internal memory 121, and the internal memory 121 may include a program storage area and a data storage area.

[0062] The wireless communication function of the mobile phone 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor. In some embodiments, the antenna 1 of the mobile phone 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the mobile phone 100 can communicate with the network and other devices through wireless communication technology.

[0063] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the mobile phone 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation.

[0064] The wireless communication module 160 can provide wireless communication solutions for application in the mobile phone 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0065] In some implementations, the mobile phone 100 can communicate with other electronic devices through the mobile communication module 150 or the wireless communication module 160 to receive multimedia files such as audio files and video files transmitted by other electronic devices.

[0066] The mobile phone 100 can implement audio functions, such as audio playback and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0067] The audio module 170 is used to convert digital audio into analog audio signal output, and is also used to convert analog audio input into digital audio signals. For example, the audio module 170 can convert the voice data collected by the mobile phone 100 from analog audio signals to digital audio signals, and can also convert the audio digital signals corresponding to the text data of the mobile phone 100 into analog audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0068] The speaker 170A, also called a "speaker", is used to convert audio electrical signals into sound signals. The mobile phone 100 can listen to music or listen to hands-free calls through the speaker 170A. In some embodiments, the mobile phone 100 can play voice data corresponding to text data through the speaker 170A.

[0069] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the mobile phone 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

[0070] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The mobile phone 100 can be provided with at least one microphone 170C. In other embodiments, the mobile phone 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the mobile phone 100 can also be provided with three, four or more microphones 170C to realize the collection of sound signals (such as collecting the user's voice data), noise reduction, and can also identify the source of the sound, realize directional recording function, etc.

[0071] The sensor module 180 may include sensors such as a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a Hall sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor. The mobile phone 100 may collect various data through the sensor module 180 .

[0072] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), MiniLED, MicroLED, Micro-OLED, a quantum dot light-emitting diode (QLED), etc.

[0073] The mobile phone 100 can display the interface of the mobile phone 100 through the display screen 194, such as displaying the desktop interface, the call interface, etc. The interface displayed on the display screen 194 may include a conversion panel.

[0074] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device. In other embodiments, the electronic device may also include more or fewer modules than those provided in the above embodiments, and different interface connection methods in the above embodiments may be used between the modules, or a combination of multiple interface connection methods. The hardware structure of the electronic device provided in the embodiments of the present application can also refer to the hardware structure of the mobile phone 100. The methods in the following embodiments can all be implemented in an electronic device having the above hardware structure.

[0075] As described above, the electronic device in the embodiment of the present application has a function of converting audio data and text data, and the electronic device can provide the conversion function to the user through the conversion panel. Figure 3 As shown in (1), the conversion panel includes a shortcut key area, an audio-text transcription area and a bottom functional area. The shortcut key area is used to provide some shortcut keys of the conversion panel, such as an exit key (or exit control), a setting key (or setting control), etc. The audio-text transcription area is used to provide panel content (or transcription content), such as text data, subtitles, etc. The bottom functional area is used to provide some function keys of the conversion panel, such as keyboard keys (or keyboard controls), voice keys (or voice controls), etc. The electronic device provides the user with audio-text conversion functions corresponding to a variety of conversion modes through the conversion panel, which is suitable for audio and text conversion requirements in different application scenarios. The conversion panel provided in an embodiment of the present application and the functions provided by the conversion panel are introduced below.

[0076] The shortcut key area in the conversion panel provided by the electronic device is also provided with a first mode switching control and a second mode switching control. When the first mode switching control is in a selected state, the conversion function is in a first conversion mode. When the second mode switching control is in a selected state, the conversion function is in a second conversion mode. Exemplarily, take the first conversion mode as a subtitle mode and the second conversion mode as a dialogue mode as an example. The subtitle mode can provide subtitles corresponding to multimedia files. The dialogue mode can provide conversion of a first voice to text data, or provide conversion of text data input by a user to voice data. The conversion panel is provided with a first mode switching control corresponding to the subtitle mode and a second mode switching control corresponding to the dialogue mode. Figure 3 As shown in (2), when the first mode switching control is in the selected state, the electronic device provides a subtitle mode conversion function, and displays the subtitles of the multimedia file in the conversion panel. Figure 3 As shown in (3), when the second mode switching control is in the selected state, the electronic device provides a conversion function of the dialogue mode, displays the text data of the first voice in the conversion panel, or plays the voice data corresponding to the text data input by the user. The electronic device can switch the subtitle mode to the dialogue mode, or switch the dialogue mode to the subtitle mode under the control of the user.

[0077] In some implementations, the second conversion mode includes a conversation mode and a translation mode. The conversation mode is used to provide a conversion function from the first voice to text data and a conversion function from text data input by the user to voice data. The translation mode is used to provide a translation function from the first voice to text data and a translation function from text data input by the user to voice data.

[0078] In order to facilitate the distinction of conversion panels corresponding to different conversion modes, the method provided in an embodiment of the present application is introduced below by taking the conversion panel corresponding to the first conversion mode as the first conversion panel, the conversion panel corresponding to the second conversion mode as the first conversion panel, and the conversion panel corresponding to the third conversion mode as the third conversion panel as an example.

[0079] The following takes the example of an electronic device switching a voice-to-text conversion function from a first conversion mode to a second conversion mode to introduce the method provided by the embodiment of the present application. Figure 4 As shown, the embodiment of the present application includes the following steps:

[0080] S401, the electronic device starts the audio-text conversion function and displays a first conversion panel on the first interface.

[0081] The electronic device turns on the audio-to-text conversion function and displays a first conversion panel on the first interface. For example, the first interface is a desktop interface, and the electronic device displays the first conversion panel on the desktop interface. When the audio-to-text conversion function is in the first conversion mode, the first mode switching control in the first conversion panel is in a selected state. The first conversion mode is used to provide a subtitle conversion function for multimedia files, and may also be referred to as a subtitle mode.

[0082] It is understandable that the first conversion panel can be suspended on the interface of the electronic device in the form of a window. When the interface of the electronic device switches from the first interface to another interface (such as the second interface), the first conversion panel is not affected by the interface switching, and the electronic device continues to display the first conversion panel on the second interface.

[0083] In some implementations, the electronic device is provided with an opening entry for the voice-to-text conversion function. In one example, the electronic device adds a setting page for the voice-to-text conversion function in the setting function. If the electronic device receives an opening operation for the voice-to-text conversion function in the setting page for the voice-to-text conversion function, the electronic device turns on the voice-to-text conversion function and displays a first conversion panel on the first interface. For example, the setting function of the electronic device is provided with a smart voice option. For example, Figure 5 As shown in (1), an "intelligent transcription" option is provided in the setting page of the smart voice, and the "intelligent transcription" option is the option of the speech-to-text conversion function. The speech-to-text conversion function can also be called a transcription function, and the various conversion modes of the speech-to-text conversion function can also be called transcription modes. The "intelligent transcription" option can be used to enter the "intelligent transcription" page. In the "intelligent transcription" page, the electronic device is provided with a switch control to turn on or off the speech-to-text conversion function. If the electronic device receives a user's start-up operation to turn on the conversion function, such as the electronic device receives a user's start-up operation at the switch control, such as a click operation, the electronic device turns on the speech-to-text conversion function and displays a first conversion panel on the displayed first interface.

[0084] In some implementations, after the electronic device turns on the audio-to-text conversion function, the electronic device displays the first conversion panel of the first conversion mode by default. The distance between the first conversion panel and the lower boundary of the display area of ​​the screen is equal to the preset distance. The preset distance can be set according to the actual application scenario or demand. For example, the preset distance is 48 device independent pixels (density-independent pixel, dp). Alternatively, the position of the first conversion panel can be adjusted according to user operation. After the electronic device turns on the audio conversion function, the position of the first conversion panel is consistent with the position when the electronic device last exited the first conversion panel.

[0085] In one example, the electronic device can also add a shortcut to the audio-to-text conversion function on the desktop. For example, the "Smart Transcription" page of the electronic device also provides an option to add a "desktop shortcut". When the electronic device receives a click operation from the user at the "desktop shortcut" option, the electronic device adds a shortcut to the audio-to-text conversion function on the desktop. Figure 5 As shown in (2), a "Smart Transcription" icon is added to the desktop. The "Smart Transcription" icon is a shortcut to the audio-to-text conversion function. Through this shortcut, the user can quickly start the audio-to-text conversion function. For example, if the electronic device receives a user operation of clicking a shortcut to the audio-to-text conversion function, the electronic device starts the audio-to-text conversion function and displays a first conversion panel on the first interface.

[0086] In another example, the electronic device can also set a switch control for the audio-to-text conversion function in the drop-down menu. Figure 5 As shown in (3), a switch control of "Smart Transcription" is set in the drop-down menu of the electronic device, and the switch control of "Smart Transcription" is the switch control of the audio-to-text conversion function. When the audio-to-text conversion function is not turned on, if the electronic device receives a user's click operation at the switch control of "Smart Transcription", the electronic device turns on the audio-to-text conversion function and displays the first conversion panel on the first interface.

[0087] In another example, the electronic device can also enable the audio-to-text conversion function through a voice assistant. When the voice assistant of the electronic device is in an awake state, if the voice assistant of the electronic device receives a voice command or user operation to enable the audio-to-text conversion function, the electronic device enables the audio-to-text conversion function and displays the first conversion panel on the first interface. Figure 5 As shown in (4), when the voice assistant of the electronic device is in the awake state, if the voice assistant of the electronic device receives a voice command of "intelligent transcription" or "open face-to-face translation", or receives a user's click operation at the recommended function of "intelligent transcription" or "open face-to-face translation" prompted by the voice assistant, the electronic device turns on the audio-to-text conversion function and displays the first conversion panel on the first interface.

[0088] It is understandable that after the electronic device exits the audio-to-text conversion function, the audio-to-text conversion function can be turned on again through any of the implementation methods described in the above examples, and the first conversion panel is displayed on the first interface. For example, the electronic device exits the audio-to-text conversion function and cancels the display of the first conversion panel in response to the user operation of the user clicking the exit control in the first conversion panel. If the user wants to use the audio-to-text conversion function again, the electronic device can be controlled to turn on the audio-to-text conversion function through any of the implementation methods described in the above examples.

[0089] In some implementations, after the electronic device turns on the audio-to-text conversion function, the electronic device is in the default conversion mode of the audio conversion function. For example, the default conversion mode is the subtitle mode. After the electronic device turns on the audio-to-text conversion function, it is in the subtitle mode of the audio conversion function, and the first conversion panel corresponding to the subtitle mode is displayed on the first interface. Of course, the electronic device can also directly enter the conversion function of other conversion modes other than the default conversion mode. For example, if the electronic device receives a voice command of "turn on face-to-face translation" through a voice assistant, the electronic device enters the conversion function of the translation mode and displays the first conversion panel corresponding to the translation mode.

[0090] It is understandable that the voice-to-text conversion functions in different conversion modes are different, and the content displayed by the electronic device on the first conversion panel is also different. For example, in the first conversion mode, the electronic device provides subtitles of the multimedia file through the first conversion panel. In the second conversion mode, the electronic device provides text data corresponding to the first voice through the first conversion panel. Of course, the first conversion panels in different conversion modes may also be different in display style.

[0091] S402: In response to a first operation by a user, the electronic device switches the first interface to a second interface and displays a first conversion panel on the second interface.

[0092] The electronic device receives a first operation from the user, such as the electronic device receives a user operation of the user clicking on a video application on the desktop. In response to the first operation from the user, the electronic device switches the first interface to the second interface, such as switching the desktop interface to the video application interface. As described above, the first conversion panel is suspended on the interface of the electronic device in the form of a window, the first conversion panel is not affected by the interface switching, and the electronic device displays the first conversion panel on the second interface.

[0093] The first area of ​​the second interface includes a first multimedia file. Figure 4 As shown, the second interface displays a first multimedia file in the first area. In response to a user operation of playing the first multimedia file, the electronic device plays the first multimedia file.

[0094] S403: During the process of playing the first multimedia file, displaying subtitles corresponding to the first multimedia file in the first conversion panel.

[0095] In the first conversion mode, if the electronic device plays a first multimedia file, such as a video file, an audio file, or other first multimedia file, the electronic device obtains the audio data of the first multimedia file, identifies the audio data of the first multimedia file, and obtains text data corresponding to the audio data of the first multimedia file. The text data can be used as subtitles corresponding to the first multimedia file. Further, the electronic device displays the subtitles corresponding to the first multimedia file in the first conversion panel. For example, when the electronic device is playing a video file, a conversion panel is suspended at a preset distance from the lower boundary of the screen, and the subtitles of the video file are displayed in the first conversion panel. In this way, the electronic device can provide the user with subtitles of the first multimedia file in real time through the conversion panel.

[0096] In some implementations, the electronic device may also translate the audio data of the first multimedia file to obtain subtitles of the target language type, and display the subtitles of the target language type in the first conversion panel. For example, the source language type corresponding to the audio data of the first multimedia file is English, and the target language type is Chinese. The electronic device identifies and translates the audio data of the first multimedia file being played to obtain Chinese subtitles of the first multimedia file. Furthermore, the electronic device displays the Chinese subtitles corresponding to the multimedia file in the conversion panel.

[0097] In some examples, the first conversion panel in the subtitle mode provided by the electronic device can support subtitles in multiple languages. For example, the audio data of the first multimedia file can be in any language such as Chinese, English, Korean, Japanese, etc. The subtitles of the first multimedia file can also be in any language such as Chinese, English, Korean, Japanese, etc.

[0098] For example, Figure 6 As shown in (1), the first conversion panel in the subtitle mode has a setting control. If the electronic device receives a click operation from the user at the setting control, it enters the setting page of the subtitle mode. The setting page of the subtitle mode includes setting items for the source language type of the first multimedia file and the target language type corresponding to the subtitle, such as the setting item corresponding to the "sound source language". After entering the setting page corresponding to the "sound source language", the first conversion panel provides the source language type of the first multimedia file and the target language type corresponding to the subtitle. For example, the setting page corresponding to the "sound source language" includes multiple options such as Chinese, English, Korean, Japanese, and English to Chinese, Korean to Chinese, and Japanese to Chinese. If the "English" option is selected, it means that the source language type of the first multimedia file is English, and the target language type corresponding to the subtitle is also English. If the "English to Chinese" option is selected, it means that the source language type of the first multimedia file is English, and the target language type corresponding to the subtitle is Chinese.

[0099] In some examples, the electronic device may also display the target language type corresponding to the subtitles in the first conversion panel in the subtitle mode. Figure 6 As shown in (2), if the electronic device does not play the first multimedia file, in the first conversion panel in the subtitle mode, the electronic device displays "Language: Chinese" in the first conversion panel, indicating that the target language type corresponding to the subtitle mode is Chinese. In this way, the electronic device can prompt the user of the target voice type of the subtitles through the first conversion panel.

[0100] In some examples, in order to save the occupied area of ​​the first conversion panel, the electronic device can also automatically retract the shortcut key area of ​​the first conversion panel. For example, the first conversion panel in the subtitle mode can first display the shortcut key area. When the default window with the shortcut key area is displayed for a preset time (such as 5 seconds, 10 seconds, etc.) and no operation is received in the default window, such as Figure 6 As shown in (3), the electronic device retracts the shortcut key area of ​​the default window and displays the default window without the shortcut key area. If the electronic device receives a user operation at the default window without the shortcut key area, the electronic device calls out the shortcut key area of ​​the default window again.

[0101] In some examples, the electronic device may also display the application icon and time of the application software currently playing the first multimedia file in the first conversion panel in the subtitle mode. Figure 6 As shown in (3), during the process of the electronic device playing the first multimedia file, the transcription panel of the electronic device is provided with an application icon of the application software that plays the first multimedia file, and the time "2023 / 02 / 06 16:52", so as to prompt the user to provide subtitles for the audio data played by the application software of the application icon, and the time to prompt the user to provide subtitles.

[0102] In some examples, the electronic device can also prompt the subtitles corresponding to the currently played audio data in the first conversion panel in the subtitle mode. For example, the electronic device prompts the subtitles corresponding to the currently played audio data by the brightness of the subtitles. For example, in the subtitles displayed on the first conversion panel in the subtitle mode, part of the text is darker and part of the text is brighter. The darker part of the text is the subtitle part corresponding to the audio data of the first multimedia file that the electronic device has finished playing. The brighter part of the text is the subtitle part corresponding to the audio data that the electronic device is playing. If the subtitles are delayed, "..." can be used.

[0103] S404: In response to the second operation, the electronic device displays a second conversion panel, collects the first voice, and displays text data corresponding to the first voice in the second conversion panel.

[0104] The second operation is a user operation of selecting the second mode switching control. When the electronic device is displaying the first conversion panel of the first conversion mode, if the second operation, such as a click operation, is received at the second mode switching control of the displayed first conversion panel, the electronic device switches the first conversion panel to the second conversion panel. The second mode switching control in the second conversion panel displayed by the electronic device is in a selected state.

[0105] For example, Figure 3 As shown in (2), the shortcut key area of ​​the first conversion panel is provided with a first mode switching control and a second mode switching control. The first mode switching control corresponds to the subtitle mode, and the second mode switching control corresponds to the dialogue mode (an example of the second conversion mode). In the subtitle mode, if the electronic device receives a second operation, such as a click operation, at the "dialogue" (an example of the second mode switching control) in the first conversion panel, the electronic device switches the first conversion panel to the second conversion panel. Figure 3 As shown in (3), the electronic device displays a second conversion panel in the dialogue mode.

[0106] In the second conversion mode, the electronic device provides a function of converting offline voice to text data, and a function of converting text data input by the user to voice data. In the second conversion mode, the electronic device collects voice data through a microphone. The voice data collected by the microphone can be called the first voice. If the electronic device collects the first voice through the microphone, it converts the first voice into text data, and displays the text data converted from the first voice in the second conversion panel.

[0107] In some implementations, the second conversion panel of the second conversion mode is provided with options of a conversation mode and an option of a translation mode. If the option of the conversation mode is in a selected state, the second conversion mode is the conversation mode. If the option of the translation mode is in a selected state, the second conversion mode is the translation mode.

[0108] For example, Figure 7 As shown in (1), in the second conversion mode, the second conversion panel is provided with setting function options (or setting controls, setting items). In the setting page of the setting function options, a "dialogue language" option (or mode selection control) is provided. In the dialogue language page (or dialogue mode page) corresponding to the "dialogue language" option, an option of "monolingual dialogue" (an example of an option of dialogue mode) and an option of "bilingual dialogue" (an example of an option of translation mode) are provided. If any of the multiple options belonging to "monolingual dialogue" is in a selected state, the second conversion mode is the dialogue mode. If the option belonging to "bilingual dialogue" is in a selected state, the second conversion mode is the translation mode.

[0109] In some examples, the second conversion panel in the dialogue mode provided by the electronic device supports multiple language types. For example, the voice data (such as the first voice) collected by the electronic device and the text data converted from the voice data (such as the recognized text) can be in any language type such as Chinese, English, Korean, Japanese, etc. Correspondingly, the text data (such as the second input text) received by the electronic device in the second conversion panel and the voice data converted from the text data (such as the voice data corresponding to the second input text) can be in any language type such as Chinese, English, Korean, Japanese, etc.

[0110] For example, if Figure 7 As shown in (1), in the dialogue language page corresponding to the "Dialogue Language" option, the "Single Language Dialogue" option is provided with Chinese, English, Korean, and Japanese options. If the "Chinese" option is selected, the second conversion panel of the electronic device supports the mutual conversion function between Chinese voice data and text data.

[0111] In some examples, the second conversion panel in the translation mode provided by the electronic device also supports the setting of multiple language types. For example, the voice data (such as the first voice) collected by the electronic device can be in any language type such as Chinese, English, Korean, Japanese, etc. The text data (such as the first translated text) obtained by translating the voice data displayed by the electronic device in the second conversion panel can be in another language type different from the voice data in Chinese, English, Korean, or Japanese. The text data (such as the first input text) received by the electronic device in the second conversion panel can also be in any language type in Chinese, English, Korean, or Japanese. The voice data (such as the voice data corresponding to the first translated text, or the voice data corresponding to the second translated text) played by the electronic device can be in another language type different from the text data in Chinese, English, Korean, or Japanese. The electronic device can set the language type under user control.

[0112] In some examples, after the language type is set, the electronic device may also prompt the transcription mode and / or set the voice type in the second conversion panel.

[0113] For example, if Figure 7 As shown in (2), in the second conversion panel in the translation mode, a language selection bar for the translation mode is provided (as shown in the positions of "English" and "Chinese"). The language selection bar includes options for the source voice type and options for the target language type. The electronic device selects two different language types under the control of the user. The electronic device can provide a function of mutual conversion between voice data and text data between the two language types.

[0114] In an embodiment of the present application, the electronic device can provide conversion and translation of voice data and text data through the second conversion panel of the second conversion mode, thereby meeting the user's audio-text conversion needs in a variety of application scenarios, such as meeting the daily communication needs of hearing-impaired users and meeting the conversation needs in different language scenarios.

[0115] In the conversation mode, the electronic device collects voice data in real time. If the electronic device collects the first voice, the electronic device recognizes the first voice and obtains text data corresponding to the first voice (such as recognized text). Further, the electronic device displays the recognized text in the second conversion panel. The language type of the first voice is the same as that of the recognized text.

[0116] In some examples, the conversation mode is also used to provide a function of converting text data (such as the second input text) input by the user into voice data. If the electronic device obtains the second input text in the second conversion panel, it converts the second input text into voice data, obtains the voice data corresponding to the second input text, and automatically plays the voice data corresponding to the second input text. Figure 8 As shown, a keyboard key is provided in the second conversion panel in the dialogue mode. If the electronic device receives a click operation at the position of the keyboard key, the input keyboard is displayed to obtain the text data input by the user through the input keyboard. After the user completes the input, the "Send" control in the second conversion panel can be clicked to confirm that the second input text has been edited. If the electronic device receives a click operation of the user on the "Send" control, the electronic device displays the text data input by the user in the second conversion panel through a bubble, and the electronic device further converts the text data into voice data and automatically plays the voice data.

[0117] In this way, the electronic device can read the input text aloud to the user. When the user is not convenient to speak, the user can communicate with others through the voice data corresponding to the input text, thereby providing convenience for the user.

[0118] Optionally, the electronic device can prompt the user in the second conversion panel the text data corresponding to the voice data currently being played by highlighting it. The electronic device can also prompt the text data corresponding to the voice data being played by a play icon in the second conversion panel. For example, a play icon can be set in front of the bubble box of the text data being read aloud to prompt that the text data in the bubble box is being read aloud.

[0119] In the translation mode, the electronic device collects voice data in response to user operations. Fig. 9As shown in (1), a microphone control (or a playback control) is provided in the second conversion panel. If the electronic device receives a long press operation of the user at the microphone control, the electronic device collects voice data. When the electronic device collects the first voice, the electronic device recognizes and translates the first voice to obtain text data corresponding to the first voice (which may be referred to as a first translated text). Furthermore, the electronic device displays the first translated text in the second conversion panel. The language types of the first voice and the first translated text are different. In this way, the electronic device can provide the user with a translation function of the first voice. When the user does not understand the speech of others, the user can understand the meaning of the speech of others through the translated text displayed on the second conversion panel, thereby providing convenience for the user.

[0120] Optionally, the electronic device can collapse the language selection bar in the second conversion panel during the process of collecting voice data. Fig. 9 As shown in (2), the language selection bar in the second conversion panel is in a collapsed state. In this way, the space of the second conversion panel can be saved, so that the second conversion panel can accommodate more text boxes.

[0121] In some examples, if the electronic device receives a swipe down operation in the audio-to-text transcription area of ​​the second conversion panel, the electronic device displays a language selection bar in the second conversion panel.

[0122] Optionally, the electronic device may also provide a voice playback function for the first translated text. For example, a play icon may be provided before the bubble box of the first translated text in the second conversion panel. If the electronic device receives a click operation at the play icon, the electronic device plays the voice data corresponding to the first translated text in the bubble box.

[0123] In some examples, the translation mode is also used to provide a translation function from text data to voice data. If the electronic device obtains the first input text in the second conversion panel, it translates the first input text into text data of the target language type, obtains the translation text corresponding to the first input text (which may be referred to as the second translation text), and displays the second translation text in the second conversion panel. Fig. 9As shown in (3), a keyboard key is provided in the second conversion panel in the translation mode. If the electronic device receives a click operation at the keyboard key position, the input keyboard is displayed to obtain the text data of the source language type input by the user (such as the first input text) through the input keyboard. After the user completes editing the first input text, the user clicks the "Send" control in the second conversion panel to confirm that the first input text has been edited. When the electronic device receives the user's click operation on the "Send" control, the first input text is displayed in the second conversion panel through a bubble. The electronic device further translates the text data from the source language type to the target language type, obtains the second translated text, and displays the second translated text of the target language type below the text data of the source language type. In this way, the electronic device can provide the user with a translation function for the input text. For example, in an offline communication scenario, if the two parties in the conversation do not speak the same language, the user can show others the text of the target language type displayed on the second conversion panel to achieve communication with others and provide convenience for the user.

[0124] Optionally, the electronic device can also play the voice data corresponding to the second translated text. Fig. 9 As shown in (4), the electronic device sets a play icon before the bubble box of the text data in the second conversion panel. If the electronic device receives a click operation at the play icon, the electronic device plays the voice data corresponding to the translated text in the bubble box corresponding to the play icon.

[0125] Optionally, the source language text and the target language text in the second conversion panel can be displayed on both sides of the second conversion panel, such as the source language text is displayed on the left side of the conversion panel, and the target language text is displayed on the right side of the second conversion panel.

[0126] S405: If the electronic device detects a call voice, the electronic device displays a third conversion panel and displays a call text corresponding to the call voice in the third conversion panel.

[0127] The conversion function of the electronic device also has a third conversion mode. The third conversion mode is used to provide a call text conversion function corresponding to the call voice, and can also be called a call mode. If the electronic device receives a call voice, it automatically switches the conversion function to the third conversion mode. In the third conversion mode, the electronic device converts the call voice into call text, and displays the call text corresponding to the call voice in the third conversion panel suspended on the call interface. In this way, the electronic device can display the call content in text on the screen. Even if the user does not hear the call voice clearly or it is inconvenient to listen to the call voice, the user can understand the call content through the call text displayed on the screen to achieve barrier-free calls.

[0128] In some implementations, the third switching mode of the switching function may correspond to a separate activation mode. Fig.10 As shown in (1), if the electronic device receives a user's opening operation, such as a click operation, at the "Smart Transcription" switch control in the "Smart Transcription" page, the electronic device pops up a reminder window for whether to turn on call subtitles. If the electronic device receives a user's click operation at the "Open" control in the reminder window of call subtitles, the electronic device turns on the call text conversion function corresponding to the call mode of the third conversion panel.

[0129] In another example, when the call text conversion function corresponding to the call mode is not enabled, Fig.10 As shown in (2), a "call subtitle" control is provided in the call interface of the electronic device. If the electronic device receives a click operation from the user at the "call subtitle" control, the electronic device displays a third conversion panel of the call mode on the call interface, and displays the call text corresponding to the call voice in the third conversion panel.

[0130] Of course, the third conversion mode of the conversion function can correspond to the same activation method as the first conversion mode and the second conversion mode. For example, if the electronic device receives an activation operation from the user to activate the conversion function, the first conversion mode, the second conversion mode and the third conversion mode will be activated at the same time.

[0131] In an embodiment of the present application, the electronic device provides audio-text conversion in various application scenarios through a variety of conversion modes of the audio-text conversion function. The conversion panel can switch between different conversion modes according to different application scenarios, so that the electronic device can provide audio-text conversion solutions for all scenarios. For example, when the user is in a noisy environment and it is difficult to hear the sound of the audio and video, when the transcription function is in subtitle mode, the electronic device can automatically convert the media audio into text display, so that the user can obtain the content of the audio and video in time. When a hearing-impaired user has difficulty communicating, the electronic device can transcribe the content of the other party's voice into text display in real time. At the same time, the electronic device also supports the text input by the hearing-impaired user through the keyboard to be broadcasted through voice, so that the other party can listen to the voice, providing convenient conditions for the hearing-impaired user to communicate with others.

[0132] In order to facilitate user operation and meet the user's usage habits, in some implementations, the first conversion panel and the second conversion panel both have multiple display forms. The electronic device can switch between multiple display forms of the first conversion panel or the second conversion panel. In this way, the first conversion panel and the second conversion panel of the electronic device can not only facilitate user operation through multiple display forms, but also provide users with multiple visual changes to enhance user experience. The first conversion panel and the second conversion panel are not distinguished below, and the first conversion panel and the second conversion panel are uniformly referred to as conversion panels.

[0133] In some implementations, a mode switching control for switching between different display modes is provided in the conversion panel. If the electronic device receives a mode switching operation of the user clicking the mode switching control, the electronic device switches the switching panel from the first mode to the second mode. The first mode is a display mode different from the second mode.

[0134] Exemplarily, the conversion panel has any one of the display modes of the default mode, the floating ball mode, and the full screen mode. The first mode is any one of these display modes, and the second mode is a display mode different from the first mode among these display modes. Fig.11 As shown in (1) in . Among them, the default form of the conversion panel is a default window, and the height of the default window is less than the height of the full-screen display area of ​​the electronic device. For example, the height of the default window is the height of 2 lines of text, or 1 line of text. The floating ball form of the conversion panel is a floating ball window, and the diameter of the floating ball window is less than a preset value, such as less than 40dp, 56dp and other preset values. The full-screen form of the conversion panel is a full-screen window, and the full-screen window corresponds to the full-screen display area. The full-screen display area is the display area of ​​the electronic device when it is displayed in full screen, such as the maximum display area corresponding to the display screen of the electronic device.

[0135] Exemplarily, the form switching control includes a maximize control and a minimize control. In the default window corresponding to the default form of the conversion panel, a maximize control and a minimize control are set. If the electronic device receives a form switching operation, such as a click operation, at the maximize control in the default window of the conversion panel, the electronic device switches the conversion panel from the default window of the default form to the full-screen form in response to the form switching operation, and displays the full-screen window of the conversion panel in full screen. If the electronic device receives a form switching operation, such as a click operation, at the minimize control in the default window of the conversion panel, the electronic device switches the conversion panel from the default window of the default form to the floating ball form in response to the form switching operation, and displays the floating ball window of the conversion panel.

[0136] Correspondingly, a cancel control for canceling full-screen display is provided in the full-screen window of the conversion panel. If the electronic device receives a form switching operation, such as a click operation, at the cancel control in the full-screen window of the conversion panel, the electronic device switches the conversion panel from the full-screen window in full-screen form to the default form in response to the form switching operation, and displays the default window of the conversion panel. A minimize control may also be provided in the full-screen window of the conversion panel. If the electronic device receives a form switching operation, such as a click operation, at the minimize control in the full-screen window of the conversion panel, the electronic device switches the conversion panel from the full-screen window in full-screen form to the floating ball form in response to the form switching operation, and displays the floating ball window of the conversion panel.

[0137] In different transcription modes, the full screen form of the conversion panel can be different. For example, Fig.11 As shown in (2), in the subtitle mode, the bottom functional area of ​​the full-screen window corresponding to the full-screen form of the first conversion panel is not provided with a functional control. In the dialogue mode, the bottom functional area of ​​the full-screen window corresponding to the full-screen form of the second conversion panel is provided with a keyboard functional control for the user to input text data. In the translation mode, the bottom functional area of ​​the full-screen window corresponding to the full-screen form of the second conversion panel is provided with a keyboard functional control and a microphone functional control for the user to input voice data and text data.

[0138] Optionally, the floating ball window of the conversion panel may not be provided with a control for displaying the form switching due to size limitations. If the electronic device receives a form switching operation, such as a click operation, at the floating ball window of the conversion panel, the electronic device switches the conversion panel from the floating ball window in the form of a floating ball to a default window in the default form in response to the form switching operation.

[0139] A close control is also provided in the default window corresponding to the default form of the conversion panel and in the full-screen window corresponding to the full-screen form. If the electronic device receives a close operation of the user clicking the close control, the conversion panel is closed. Optionally, if the electronic device receives a close operation of the user clicking the close control, the user may be prompted whether to save the transcribed content in the conversion panel. For example, Fig.11 As shown in (3), when the electronic device receives a close operation in which the user clicks on the close control, the electronic device pops up a prompt box in the conversion panel asking whether to save the transcribed content. If the electronic device receives a click operation in which the user clicks on Save, the electronic device closes the conversion panel after saving the transcribed content in the conversion panel. If the electronic device receives a click operation in which the user clicks on Not to Save, the electronic device closes the conversion panel. If the electronic device receives a selection operation in which the user selects Not to be reminded again, the electronic device will no longer pop up a prompt box in which the transcribed content is saved when the conversion panel is closed next time.

[0140] It is understandable that in some implementations, when the electronic device switches the transcription mode, the transcription content in the second conversion panel will not be cleared. Fig.12 As shown in (1), in the conversation mode, the electronic device displays the text data "Nice to meet you!" in the second conversion panel. After the electronic device switches the conversation mode to the translation mode, the text data "Nice to meet you!" is still retained in the second conversion panel. The text data "Hello!" obtained by the electronic device in the translation mode and the "Hello!" obtained by translating "Hello!" are displayed below "Nice to meet you!".

[0141] In some implementations, when there are many transcribed contents in the second conversion panel, it is difficult for the electronic device to display all the transcribed contents in the audio-text transcription area of ​​the second conversion panel, and a return to top control is also provided in the second conversion panel. If the user clicks the return to top control, the electronic device displays the first transcribed content in the second conversion panel. In this way, it is convenient for the user to view the transcribed content, providing convenience for the user.

[0142] In some implementations, the conversion panel in the default form and the floating ball form supports a position movement function. If the electronic device receives a movement operation, the movement operation indicates to move the position of the default window or the floating ball window, and the electronic device adjusts the position of the default window or the floating ball window in response to the movement operation. For example, Fig.12 As shown in (2), taking the moving operation as a drag operation acting on the shortcut key area of ​​the default window as an example, the electronic device responds to the drag operation and moves the default window following the moving track of the drag operation.

[0143] It is understandable that the first conversion panel or the second conversion panel is suspended on the interface of the electronic device in the default state and the floating ball state. The first conversion panel or the second conversion panel is not affected by the interface switching, and even if the electronic device switches from one interface to another, the first conversion panel or the second conversion panel is displayed on the interface.

[0144] In some implementations, when the default window of the conversion panel moves beyond the screen boundary, the electronic device switches the conversion panel from the default state to the floating ball state, so that the conversion panel automatically shrinks from the default window corresponding to the default state to the floating ball window corresponding to the floating ball state. In this way, the electronic device can provide multiple solutions for the conversion panel to change from the default state to the floating ball state, which is convenient for user operation and improves user experience.

[0145] Exemplarily, when the conversion panel of the electronic device displays a default window in a default form, the electronic device moves the default window in response to a move operation of moving the default window. If the default window moves beyond the screen boundary, the screen of the electronic device only displays the partial area of ​​the default window that does not exceed the screen boundary, and no longer displays the partial area of ​​the default window that exceeds the screen boundary. For ease of description, the partial area of ​​the default window that exceeds the screen boundary can be referred to as an invisible area. If the ratio of the area of ​​the invisible area to the total area of ​​the default window is greater than a preset ratio, such as greater than a preset ratio of 1 / 3, 1 / 2, etc., the electronic device switches the conversion panel from the default window to the floating ball window. For example, if Fig.13 As shown in (1) of FIG. 1 , the electronic device displays a first conversion panel in a default form in the video playback interface. The first conversion panel corresponds to a subtitle mode and is used to provide subtitles for the video file. Fig.13As shown in (2), when the first conversion panel in the default form moves to the screen boundary and the ratio of the area of ​​the invisible region of the first conversion panel to the total area of ​​the default window is greater than 1 / 3, the electronic device changes the display form of the conversion panel to a floating ball form.

[0146] Optionally, the electronic device also provides a quick closing solution for the floating ball window. For example, Fig.13 As shown in (3), if the electronic device receives a long press operation at the floating ball window, the electronic device displays a trash can icon, which represents a delete function. If the electronic device receives a drag operation of dragging the floating ball window to the trash can icon, the electronic device responds to the drag operation and cancels the display of the floating ball window. Furthermore, the electronic device can also turn off the audio-to-text conversion function.

[0147] In order to provide users with a better user experience, in some implementations, the default form of the conversion panel can be different in size under different conversion modes. Exemplarily, the conversion modes of the conversion panel are subtitle mode (an example of the first conversion mode) and dialogue mode (an example of the second conversion mode) as examples. In subtitle mode, the default size of the default window of the first conversion panel can correspond to the height of 2 lines of text. The minimum height of the default window of the first conversion panel in subtitle mode can be the height of 1 line of text, and the maximum height is 2 / 3 of the full screen display area height. In dialogue mode, the default height of the default window of the second conversion panel can correspond to 3 / 5 or 1 / 2 of the full screen display area height, and the maximum height is 2 / 3 of the full screen display area height. The minimum height of the default window of the second conversion panel in dialogue mode is greater than or equal to the height of 2 single-line text boxes (or single-line text bubbles). In subtitle mode and dialogue mode, the width of the default window of the first conversion panel and the second conversion panel is less than or equal to the screen width.

[0148] Optionally, the default window size corresponding to the default form of the conversion panel can be adapted to the interface display mode of the electronic device. Fig.14 As shown, take the default window in subtitle mode as an example. If the interface of the electronic device is in portrait display mode, the width of the default window is less than or equal to the width of the display area corresponding to the portrait screen, and the height of the default window is the height of 2 lines of text. If the interface of the electronic device is in landscape display mode, the width of the default window is less than or equal to the width of the display area corresponding to the landscape screen, and the height of the default window is the height of 1 line of text. If the interface of the electronic device is in full-screen display mode of a folding screen, the width of the default window is less than or equal to the width of the display area corresponding to the full-screen of the folding screen, and the height of the default window is the height of 2 lines of text.

[0149] Of course, the size of the default conversion panel can also support manual adjustment. Exemplarily, the electronic device increases or decreases the size of the default window in response to the size adjustment operation for indicating adjustment of the default window. Fig.15 As shown, if the electronic device receives a long press operation (an example of a size adjustment operation) at the bottom of the default window, the electronic device increases the size of the default window in response to the long press operation, such as increasing the height of the default window. Optionally, the electronic device may also display an adjustment box in response to the long press operation to indicate the adjustment height of the conversion panel.

[0150] As mentioned above, the conversion panel includes setting function options. The setting page of setting function options includes multiple setting items. The user can adjust the multiple setting items of the conversion panel on the setting page. Exemplarily, the setting page of the conversion panel includes multiple setting items such as font size, transparency, language type of subtitles, language type of dialogue mode, source language type of translation mode, target language type of translation mode, saving panel content, clearing panel content, etc.

[0151] For example, in subtitle mode, the settings page of the first conversion panel is as follows: Fig.16 As shown in (1), the setting page includes multiple setting items such as sound source language (an example of the language type of subtitles), font size, transparency, save (an example of saving panel content) and clear (an example of clearing panel content). Among them, the setting items of sound source language can be referred to in the above description, which will not be repeated here.

[0152] The font size setting item is used to set the font size of the text displayed on the conversion panel. For example, the font size can be any of the following: small, standard, large, extra large, and extra large. Fig.16 As shown in (2), a font size adjustment bar (A represents the font size) is provided in the font size setting page, and the electronic device adjusts the font size of the text displayed in the conversion panel in response to a drag operation on the font size adjustment bar.

[0153] The transparency setting item is used to set the window transparency of the conversion panel. Fig.16 As shown in (3), a transparency adjustment bar is provided in the transparency setting page. The electronic device adjusts the transparency of the conversion panel in response to a drag operation on the transparency adjustment bar.

[0154] The saved settings are used to save the panel contents in the conversion panel. Fig.16 As shown in (4), the electronic device pops up a save page in response to a click operation of the save setting item. If the electronic device receives a click operation at the save control in the save page, the electronic device saves the panel content in the conversion panel.

[0155] The Clear setting is used to clear the panel contents in the conversion panel. Fig.16 As shown in (5), in response to the click operation of clearing the setting item, the electronic device pops up a clear window for clearing the conversion panel. If the electronic device receives the click operation at the clear control in the clear window, the electronic device deletes the panel content in the conversion panel.

[0156] In some implementations, the above multiple setting items can also be set in the "Smart Transcription" page. Fig.17 As shown in (1), in the "Smart Transcription" page, in addition to the switch control for turning on or off the audio-to-text conversion function and the option to add a desktop shortcut, the electronic device can also set multiple setting items such as transcription mode, font size, subtitle transparency, broadcast tone, call subtitles, etc.

[0157] The transcription mode setting item is used to set the transcription mode of the transcription function. Fig.17 As shown in (2), the transcription mode includes a subtitle mode and a dialogue mode. The user can select the subtitle mode or the dialogue mode of the conversion function. If the electronic device receives a selection operation of the user selecting the subtitle mode or the dialogue mode, the electronic device sets the conversion function to the transcription mode indicated by the selection operation. When the electronic device starts the audio-to-text conversion function for the first time under the operation of the user, the electronic device starts the conversion function of the subtitle mode by default.

[0158] The setting item of the broadcast tone is used to set the playing tone of the voice data. In response to the click operation of the setting item of the broadcast tone, the electronic device displays the broadcast tone page. Fig.17 As shown in (3), the announcement tone page includes multiple tone options, such as male and female tones pre-configured by the operating system of the electronic device. In some examples, the electronic device can also provide a custom tone, which can be a tone obtained based on the collected voice data. In response to the user's setting operation of any tone option among the multiple tone options, the electronic device sets the announcement tone corresponding to the second conversion panel to the tone selected by the user in the setting item of the announcement tone.

[0159] In this way, the electronic device can play the voice data using the timbre selected by the user, such as playing the voice data corresponding to the second input text using the timbre of the voice assistant. The timbre of the voice assistant is the timbre used by the voice assistant when playing the voice. In this way, the electronic device can provide the user with a voice broadcast function corresponding to a variety of timbres.

[0160] In order to better provide the user with the audio-text conversion function, when the audio-text conversion function is not turned on (i.e., the second conversion panel is closed), if the electronic device detects that the second multimedia file is being played and the preset conditions are met, the electronic device pops up a first prompt message in the interface, and the first prompt message is used to prompt the user to turn on the conversion function. The preset conditions can be set according to the actual application scenario or needs. For example, the preset conditions include that the playback volume of the second multimedia file is less than or equal to the preset volume, such as less than or equal to 0 (an example of a preset volume), or the power of the electronic device is less than the preset power, such as less than 40% of the total power (an example of a preset power), or the ambient noise collected by the electronic device is greater than a preset decibel, such as greater than 50 decibels (an example of a preset decibel).

[0161] For example, Fig.18 As shown, the electronic device plays a video file without turning on the first conversion panel. At this time, the playback volume of the second multimedia file of the electronic device is 0 (an example of a preset volume). The electronic device displays a capsule-style opening control in the function bar at the top of the interface. If a user click operation is received at the displayed opening control, the electronic device responds to the click operation and pops up a first prompt message at the top of the interface, and the first prompt message is used to prompt to turn on the subtitle function. If the electronic device receives a user's opening operation to turn on the subtitle function, the electronic device displays the first conversion panel of the subtitle mode in the interface, and provides subtitles for the video file through the first conversion panel.

[0162] Optionally, after the audio-to-text conversion function is turned on, if the electronic device stops playing the second multimedia file, the electronic device can automatically turn off the audio-to-text conversion function. Alternatively, after the audio-to-text conversion function is turned on, if the electronic device stops playing the second multimedia file, the function bar where the opening control is located is canceled, but the audio-to-text conversion function is still turned on, and the first conversion panel is displayed on the interface. The user can manually turn off the audio-to-text conversion function.

[0163] It should be noted that the personal information used in the technical solution of this application is limited to information for which the individual’s separate consent has been obtained, including but not limited to notifying and reminding the user to read the relevant user agreement (notification) and sign the agreement (authorization) including authorization of relevant user information before the user uses the function.

[0164] In some other embodiments of the present application, an electronic device is provided, including: a screen, a memory and one or more processors. The screen and the memory are coupled to the processors respectively. The screen is used to display an interface. The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device can perform each function or step in the above method embodiment. Of course, the electronic device can also include other hardware structures. For example, the electronic device also includes hardware structures such as sensors and communication modules. The structure of the electronic device can refer to Figure 2 The structure of the mobile phone 100 is shown.

[0165] The embodiment of the present application also provides a chip system, which is applied to an electronic device. The chip system includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected by lines. For example, the interface circuit can be used to receive signals from other devices (such as memory). For another example, the interface circuit can be used to send signals to other devices (such as processors). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instruction is executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

[0166] An embodiment of the present application also provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned electronic device, the electronic device executes each function or step in the above-mentioned method embodiment.

[0167] The present application also provides a computer program product, which, when executed on a computer, enables the computer to perform various functions or steps in the above method embodiment. For example, the computer may be the above electronic device.

[0168] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0169] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0170] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0172] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0173] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for converting audio and text, characterized in that: Applied to electronic equipment, the method comprises: Displaying a first conversion panel on a first interface, wherein the first conversion panel includes a first mode switching control and a second mode switching control, and the first mode switching control is in a selected state; In response to a first operation of the user, the first interface is switched to a second interface, and the first conversion panel is displayed on the second interface, wherein the first area of ​​the second interface includes a first multimedia file; When playing the first multimedia file, displaying subtitles corresponding to the first multimedia file on the first conversion panel; When it is detected that the user has selected the second mode switching control, the second conversion panel is displayed, and the first voice is collected through the microphone, and the text data corresponding to the first voice is displayed on the second conversion panel. The second conversion panel includes the first mode switching control and the second mode switching control.

2. The method according to claim 1, characterized in that When the second conversion panel corresponds to a conversation mode, the text data corresponding to the first voice is obtained by recognizing the first voice; When the second conversion panel corresponds to the translation mode, the text data corresponding to the first voice is obtained by translating the text data recognized by the first voice.

3. The method according to claim 2, characterized in that The second conversion panel includes a setting control; the method further includes: In response to a user operation on the setting control, displaying a setting page in the second conversion panel, the setting page including a mode selection control; In response to a user operation on the mode selection control, a conversation mode page is displayed in the second conversion panel, the conversation mode page including options for the conversation mode and options for the translation mode; if the option for the conversation mode is selected, the second conversion panel corresponds to the conversation mode; if the option for the translation mode is selected, the second conversion panel corresponds to the translation mode.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: In the case where the second conversion panel corresponds to the translation mode, if a first input text is acquired in the second conversion panel, a translation text of the first input text is displayed in the second conversion panel, and the language types of the first input text and the translation text are different.

5. The method according to claim 4, characterized in that The second conversion panel also includes a play control, and the method further includes: In response to a user operation on the play control, the voice data corresponding to the translated text is played.

6. The method according to any one of claims 2 to 5, characterized in that: In the case where the second conversion panel corresponds to the dialogue mode, if a second input text is acquired in the conversion panel, voice data corresponding to the second input text is played.

7. The method according to any one of claims 1 to 6, characterized in that The first conversion panel also includes a form switching control; the method also includes: In response to a user's mode switching operation on the mode switching control, the display mode of the first conversion panel is switched from a first mode to a second mode; the second mode is a display mode different from the first mode.

8. The method according to claim 7, characterized in that The first conversion panel has any one of the display modes of a default mode, a floating ball mode and a full screen mode; wherein, The default form of the first conversion panel is a default window, and the height of the default window is smaller than the height of the full-screen display area of ​​the electronic device; The suspended ball shape of the first conversion panel is a suspended ball window, and the diameter of the suspended ball window is smaller than a preset value; The full-screen form of the first conversion panel is a full-screen window, and the full-screen window corresponds to the full-screen display area.

9. The method according to claim 8, characterized in that The form switching control includes a minimize control and a maximize control, and the step of switching the display form of the first conversion panel from a first form to a second form in response to a user operation on the form switching control includes: In response to a user's form switching operation on the minimized control, the display form of the first conversion panel is switched from a default form to a floating ball form; or, In response to a user's mode switching operation on the maximized control, the display mode of the first conversion panel is switched from a default mode to a full-screen mode.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: In response to the moving operation, the display position of the default window or the floating ball window on the screen of the electronic device is adjusted.

11. The method according to claim 10, characterized in that The method further comprises: During the movement of the default window, if the ratio of the area of ​​the invisible area of ​​the default window to the total area is greater than a preset ratio, the first conversion panel is switched from the default form to the floating ball form; wherein the invisible area is the partial area of ​​the default window that exceeds the screen boundary.

12. The method according to any one of claims 7 to 10, characterized in that: The method further comprises: In response to the size adjustment operation, the size of the default window is adjusted.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: When the first conversion panel is closed, if the second multimedia file is played and a preset condition is met, a first prompt message is displayed, wherein the first prompt message is used to prompt to start the conversion function between audio data and text data, and the subtitles of the second multimedia file are displayed through the first conversion panel; The preset condition includes at least one of the following: the playback volume of the second multimedia file is less than a preset volume, the power of the electronic device is less than a preset power, and the environmental noise collected by the electronic device is greater than a preset decibel.

14. The method according to any one of claims 1 to 13, characterized in that The text data corresponding to the first voice is in any one of Chinese, English, Korean and Japanese.

15. The method according to any one of claims 1 to 14, characterized in that The method further comprises: If a call voice is detected, a third conversion panel is displayed, and a call text corresponding to the call voice is displayed in the third conversion panel.

16. The method according to any one of claims 1 to 15, characterized in that The first conversion panel or the second conversion panel includes multiple setting items such as font size, transparency, language type of subtitles, language type of dialogue mode, source language type of translation mode, target language type of translation mode, saving panel contents, clearing panel contents and broadcast tone.

17. The method according to claim 16, characterized in that The method further comprises: In response to the timbre setting operation, the announcement timbre corresponding to the second conversion panel is set to the timbre selected by the user in the setting item of the announcement timbre.

18. An electronic device, characterized in that: include: A screen, a memory and a processor; the screen and the memory are coupled to the processor respectively; The screen is used to display the interface; The memory stores computer program code, which includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1 to 17.

19. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 17.

Images