Audio and text conversion method, electronic device, and storage medium

Abstract

The application provides a conversion method of audio and text, an electronic device and a storage medium, and relates to the technical field of terminals. The method comprises the following steps: displaying a first conversion panel in a first interface, wherein the first conversion panel comprises a first mode switching control and a second mode switching control, and the first mode switching control is in a selected state. In response to a first operation of a user, the first interface is switched to a second interface, and the first conversion panel is displayed in the second interface, wherein a first region of the second interface comprises a first multimedia file. When the first multimedia file is played, a subtitle corresponding to the first multimedia file is displayed in the first conversion panel. When it is detected that the second mode switching control is selected by the user, a second conversion panel is displayed, a first voice is collected through a microphone, and text data corresponding to the first voice is displayed in the second conversion panel. In this way, the electronic device can meet the needs of the user for audio-text conversion in various application scenarios.

Description

[0001] The present application claims priority to the Chinese Patent Application No. 202311466471.7, filed on November 3, 2023, and entitled "A Conversion Method of Voice and Text and Electronic Device", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminal, and in particular to a conversion method of audio and text, an electronic device and a storage medium. BACKGROUND

[0003] At present, electronic devices have greatly changed the lifestyle of users, especially portable electronic devices, which have become indispensable products for people. In order to provide better services to users, electronic devices have audio text conversion functions, such as converting audio to text or converting text to audio. For example, an electronic device can convert input voice to text through a chat application software. For another example, an electronic device can convert text to voice for playing through a reading application software.

[0004] However, the audio text conversion function of the electronic device is not perfect and cannot meet the audio text conversion needs of users in various application scenarios. SUMMARY

[0005] The present application provides a conversion method of audio and text, an electronic device and a storage medium, which are used to provide more perfect audio and text conversion functions and are suitable for audio text conversion needs in various application scenarios.

[0006] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, a conversion method of audio and text is provided, the method comprising: displaying a first conversion panel in a first interface, the first conversion panel comprising a first mode switching control and a second mode switching control, the first mode switching control being in a selected state. In response to a first operation of a user, the first interface is switched to a second interface, and the first conversion panel is displayed in the second interface, the first region of the second interface comprising a first multimedia file. When playing the first multimedia file, the subtitle corresponding to the first multimedia file is displayed in the first conversion panel. When it is detected that the second mode switching control is selected by the user, a second conversion panel is displayed, and the first voice is collected through a microphone, the text data corresponding to the first voice is displayed in the second conversion panel, the second conversion panel is similar to the first conversion panel, and both of them comprise the first mode switching control and the second mode switching control, but carry different specific functions.

[0008] In this way, the electronic device can provide subtitles of a multimedia file, text data corresponding to voice data, and call text corresponding to call voice through the conversion panel, and the audio-text conversion mode is suitable for various application scenarios. The user can quickly switch between different application scenarios of the audio-text conversion mode using the conversion panel, thereby simplifying the switching process of the audio-text conversion function in different application scenarios and meeting the user's needs for audio-text conversion in various application scenarios. The first conversion panel can provide subtitles of a multimedia file, the second conversion panel can provide text data of the first voice, and the third conversion panel can provide call text corresponding to call voice.

[0009] In a possible implementation manner of the first aspect, the second conversion panel corresponds to a conversation mode or a translation mode. If the second conversion panel corresponds to the conversation mode, the text data corresponding to the first voice is obtained by recognizing the first voice. If the second conversion panel corresponds to the translation mode, the text data corresponding to the first voice is obtained by translating the text data recognized from the first voice.

[0010] In this implementation manner, the second conversion panel can provide a conversion function from the first voice to text data in the conversation mode. The second conversion panel can provide a translation function from the first voice to text data in the translation mode.

[0011] In another possible implementation manner of the first aspect, the second conversion panel further includes a setting control. In response to a user operation of the user on the setting control, a setting page is displayed in the second conversion panel, and the setting page includes a mode selection control. In response to a user operation of the user on the mode selection control, a conversation mode page is displayed in the second conversion panel, and the conversation mode page includes an option of the conversation mode and an option of the translation mode. If the option of the conversation mode is in a selected state, the second conversion panel corresponds to the conversation mode. If the option of the translation mode is in the selected state, the second conversion panel corresponds to the translation mode.

[0012] In this implementation manner, the user can select the translation mode or the conversation mode through the setting control of the second conversion panel. The user can switch between the translation mode and the conversation mode through the second conversion panel, thereby simplifying the switching process of the audio-text conversion function in different application scenarios and meeting the user's needs for audio-text conversion in various application scenarios.

[0013] and a translation function from text data input by the user to voice data.

[0014] In a possible implementation manner of the first aspect, in a case that the second conversion panel corresponds to the translation mode, if the first input text is acquired in the second conversion panel, the translated text of the first input text is displayed in the second conversion panel, and the language type of the first input text is different from that of the translated text. In this implementation manner, the user can input the first input text through the second conversion panel, and the electronic device provides the translated text of the first input text through the second conversion panel.

[0015] In a possible implementation manner of the first aspect, the second conversion panel further includes a play control. In response to a user operation on the play control, the voice data corresponding to the translated text is played. In this implementation manner, the second conversion panel further provides the play control for playing the translated text, and the user can play the voice data corresponding to the translated text by clicking the play control.

[0016] In a possible implementation manner of the first aspect, in a case that the second conversion panel corresponds to the conversation mode, if the second input text is acquired in the conversion panel, the voice data corresponding to the second input text is played. In this implementation manner, the electronic device further provides the automatic voice playing function of the second input text, and in a case that the user is inconvenient to speak, the user can communicate with others by playing the voice data corresponding to the second input text, thereby providing convenience for the user.

[0017] In a possible implementation manner of the first aspect, the first conversion panel or the second conversion panel includes a form switching control. In response to a form switching operation of the user on the form switching control, the display form of the first conversion panel or the second conversion panel is switched from the first form to the second form, and the second form is a display form different from the first form. In this implementation manner, the form switching control for switching between different display forms is arranged in the first conversion panel or the second conversion panel. The user can change the display form of the first conversion panel or the second conversion panel through the form switching control.

[0018] In a possible implementation manner of the first aspect, the first conversion panel or the second conversion panel has any one of a default form, a floating ball form, and a full-screen form. The default form is a default window, and the height of the default window is less than the height of the full-screen display area of the electronic device. The floating ball form is a floating ball window, and the diameter of the floating ball window is less than a preset value. The full-screen form is a full-screen window, and the full-screen window corresponds to the full-screen display area. In this implementation manner, the first conversion panel or the second conversion panel has multiple display forms, thereby providing diversified choices for the user.

[0019] In a possible implementation form of the first aspect, the mode switching control comprises a minimizing control and a maximizing control, and the electronic device is configured to switch the display mode of the first conversion panel or the second conversion panel from the default mode to the floating ball mode in response to a user operation of the user on the minimizing control, and switch the display mode of the first conversion panel or the second conversion panel from the default mode to the full-screen mode in response to a user operation of the user on the maximizing control. In this implementation form, the user can switch the first conversion panel or the second conversion panel from the default mode to the floating ball mode by the minimizing control. The user can also switch the first conversion panel or the second conversion panel from the default mode to the full-screen mode by the maximizing control. In this way, the user can switch the display mode of the first conversion panel or the second conversion panel according to the needs, and the user experience is improved.

[0020] In a possible implementation form of the first aspect, the electronic device is configured to adjust the display position of the default window or the floating ball window on the screen of the electronic device in response to a moving operation. For example, the moving operation is a drag operation on the shortcut area of the default window, and the electronic device is configured to move the default window along the moving track of the moving operation in response to the moving operation. In this way, the user can adjust the display position of the default window or the floating ball window according to the needs or personal preferences, and the user experience is improved.

[0021] In a possible implementation form of the first aspect, during the movement of the default window, if the ratio of the area of the invisible region of the default window to the total area is greater than a preset ratio, the electronic device is configured to switch the first conversion panel or the second conversion panel from the default mode to the floating ball mode. The invisible region is the part of the default window that is beyond the boundary of the screen. In this way, when the default window of the first conversion panel or the second conversion panel moves beyond the boundary of the screen, the electronic device switches the first conversion panel or the second conversion panel from the default mode to the floating ball mode, so that the first conversion panel or the second conversion panel automatically shrinks from the default window corresponding to the default mode to the floating ball window corresponding to the floating ball mode. In this way, the electronic device can provide a solution for the first conversion panel or the second conversion panel to change from the default mode to the floating ball mode, which is convenient for the user to operate and improves the user experience.

[0022] In a possible implementation form of the first aspect, the electronic device is configured to adjust the size of the default window in response to a size adjusting operation. In this way, the user can adjust the size of the default window, such as increasing or decreasing the size of the default window, and the user experience is improved.

[0023] In another possible implementation of the first aspect, when the first conversion panel is closed, if a second multimedia file is played and preset conditions are met, a first prompt message is displayed. This first prompt message prompts the user to enable the audio-to-text data conversion function, and the subtitles of the second multimedia file are displayed through the first conversion panel. The preset conditions include at least one of the following: the playback volume of the second multimedia file is lower than a preset volume, the battery level of the electronic device is lower than a preset battery level, and the ambient noise collected by the electronic device is greater than a preset decibel. In this way, even when the second conversion panel is not open, if the electronic device detects that a multimedia file is being played and the preset conditions are met, the electronic device prompts the user to enable the conversion function to provide the subtitles of the multimedia file through the second conversion panel.

[0024] In another possible implementation of the first aspect, the text data corresponding to the first speech is in any one of the following languages: Chinese, English, Korean, and Japanese. The electronic device can provide audio-text conversion functionality for multiple languages, making it suitable for a wider range of audio-text conversion scenarios and meeting user needs.

[0025] In another possible implementation of the first aspect, if voice is detected, a third conversion panel is displayed, showing the corresponding text of the call. In this way, the electronic device can display the call content as text on the screen. Even if the user does not hear the voice clearly or is unable to listen to it, they can understand the call content through the text displayed on the screen, achieving barrier-free communication.

[0026] In another possible implementation of the first aspect, the first or second conversion panel includes multiple settings such as font size, transparency, subtitle language type, dialogue mode language type, translation mode source language type, translation mode target language type, save panel content, clear panel content, and broadcast tone. Users can adjust one or more of these settings as needed through the multiple settings provided by the first or second conversion panel.

[0027] In another possible implementation of the first aspect, in response to a tone setting operation, the playback tone corresponding to the second conversion panel is set to the tone selected by the user in the playback tone settings. In this way, the electronic device can play voice data using the user-selected tone; for example, it can play the voice data corresponding to the second input text using the tone of a voice assistant. The voice assistant's tone is the tone used when the voice assistant plays voice.

[0028] In a second aspect, the present application provides an electronic device, comprising a screen (or display screen), a memory and a processor. The screen and the memory are coupled with the processor respectively. The screen is configured to display an interface. The memory stores computer program codes, which include computer instructions. When the computer instructions are executed by the processor, the electronic device performs the method in the first aspect and any possible implementation manner thereof.

[0029] In a third aspect, the present application provides a computer readable storage medium, which includes computer instructions. When the computer instructions are run on an electronic device, the electronic device performs the method in the first aspect and any possible implementation manner thereof.

[0030] In a fourth aspect, the present application provides a computer program product, which includes program instructions. When the program instructions are run on a computer, the computer can perform the method in the first aspect and any possible implementation manner thereof. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 An interface diagram of a chat application software provided by an embodiment of the present application;

[0032] Figure 2 A hardware structure block diagram of an example mobile phone 100 of an electronic device provided by an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a conversion panel provided by an embodiment of the present application;

[0034] Figure 4 A flowchart of an audio and text conversion method provided by an embodiment of the present application;

[0035] Figure 5 A schematic diagram of starting an audio and text conversion function provided by an embodiment of the present application;

[0036] Figure 6 A schematic diagram of a first conversion panel provided by an embodiment of the present application;

[0037] Figure 7 A schematic diagram of a second conversion panel provided by an embodiment of the present application;

[0038] Figure 8 An interface schematic diagram of a second conversion panel receiving text data provided by an embodiment of the present application;

[0039] Figure 9 An interface schematic diagram of a second conversion panel receiving voice data and text data provided by an embodiment of the present application;

[0040] Figure 10A schematic diagram of a third conversion panel provided for an embodiment of the present application;

[0041] Figure 11 A schematic diagram of a display form of a conversion panel provided for an embodiment of the present application;

[0042] Figure 12 A schematic diagram of a position change of a conversion panel provided for an embodiment of the present application;

[0043] Figure 13 A schematic diagram of a conversion panel switching from a default window to a floating ball window provided for an embodiment of the present application;

[0044] Figure 14 A schematic diagram of a window size of a conversion panel adapting to an interface display mode provided for an embodiment of the present application;

[0045] Figure 15 A schematic diagram of window size adjustment of a conversion panel provided for an embodiment of the present application;

[0046] Figure 16 A schematic diagram of multiple setting items of a conversion panel provided for an embodiment of the present application;

[0047] Figure 17 A schematic diagram of multiple setting items of another conversion panel provided for an embodiment of the present application;

[0048] Figure 18 A schematic diagram of a first conversion panel opening prompt provided for an embodiment of the present application. DETAILED DESCRIPTION

[0049] In daily life, users may often encounter the scene demand of audio to text (or character) conversion or text to audio conversion due to their own reasons (such as hearing impairment, language barrier, etc.), environmental reasons (such as noisy environment), and other factors. For example, for users with hearing impairment, users with hearing impairment have audio-text conversion needs in the following scenarios: audio-video playing scene, offline dialogue or communication scene, and online voice or video call scene. For users without hearing impairment, there are also audio-text conversion needs in multiple scenarios: in a multi-video conference concurrent scene, users have difficulty in simultaneously obtaining information of multiple video conferences, playing foreign language audio-video files, offline dialogue or communication, and voice barrier, etc.

[0050] In order to meet the audio-text conversion needs of users, electronic devices provide audio and text conversion functions (which can be simply referred to as audio-text conversion functions) through some application software installed. For example, a chat application software is installed in an electronic device. The chat application software provides a function of converting voice into text. The interface of the chat application software is as follows: Figure 1As shown in FIG. 1, the electronic device displays a voice message box in a chat interface. In the voice message box, the electronic device displays a voice message received from a user. In the voice message box, the electronic device displays a "Convert to Text" prompt box. If the electronic device receives a click operation of the user on the "Convert to Text" prompt box, the electronic device converts the voice message in the voice message box into text and displays the text corresponding to the voice message below the voice message box. In this way, the user can know the content of the voice message without playing the voice message, which facilitates the user.

[0051] However, the audio-to-text conversion function currently provided by the electronic device has a single application scenario and cannot meet the user's demand for multiple application scenarios. Moreover, the application software that provides the audio-to-text conversion function and the application software that provides the text-to-audio conversion function are usually different application software. This increases the difficulty of use for the user and reduces the user experience.

[0052] Therefore, in the method provided by the embodiments of the present application, the audio data and the text data are converted in multiple application scenarios. Specifically, the electronic device is provided with a conversion function for converting audio and text. When the electronic device is in the audio-to-text conversion mode, the electronic device displays a conversion panel in a first interface. The conversion panel includes mode switching controls for indicating conversion modes, such as a first mode switching control and a second mode switching control. The user can switch between multiple conversion modes of the conversion function through the conversion panel. One conversion mode corresponds to one application scenario. In this way, the electronic device can provide audio-to-text conversion functions in multiple application scenarios. Moreover, the use is simple, which can reduce the difficulty of use for the user and improve the user experience.

[0053] In the embodiments of the present application, the conversion function includes a first conversion mode and a second conversion mode. The first conversion mode is used to provide a subtitle function of a multimedia file. When the first mode switching control is in a selected state, the electronic device is in the first conversion mode. In the first conversion mode, if the electronic device plays a multimedia file, the electronic device displays a subtitle of the multimedia file in the conversion panel. The second conversion mode is used to provide a first speech-to-text data conversion function or a text data input by the user (which can be referred to as input text) to speech data conversion function. When the second mode switching control is in a selected state, the electronic device is in the second conversion mode. In the second conversion mode, if the electronic device collects a first speech, the electronic device displays text data corresponding to the first speech in the conversion panel.

[0054] In some implementations, the conversion function further includes a third conversion mode. If the electronic device detects a call speech, the electronic device automatically switches to the third conversion mode and displays call text corresponding to the call speech in the conversion panel.

[0055] It can be seen that the electronic device can provide text data corresponding to subtitles, voice data of a multimedia file and call text corresponding to call voice through the conversion panel, and the audio-text conversion function is suitable for various application scenarios. The user can quickly switch among different application scenarios of the audio-text conversion function using the conversion panel, thereby simplifying the switching process of the audio-text conversion function in different application scenarios and meeting the user's demand for audio-text conversion in various application scenarios.

[0056] For example, the electronic device described in the embodiments of the present application can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) \ virtual reality (VR) device, a media player, a wearable device, and the like. The embodiments of the present application do not specially limit the specific form of the electronic device.

[0057] In the embodiments of the present application, the electronic device is taken as a mobile phone 100 for example, and the hardware structure of the electronic device is introduced through the mobile phone 100. As shown in Figure 2 The mobile phone 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, and the like.

[0058] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), a driving processor, etc. Different processing units can be independent devices or integrated into one or more processors. The processor 110 can be the nerve center and command center of the mobile phone 100. The processor 110 can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0059] The memory in the processor 110 can also be configured to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or repeatedly uses. If the processor 110 needs to use the instructions or data again, it can directly call from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0060] The external memory interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external storage card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, files such as voice, music, and video are saved in the external storage card. In some implementations, the mobile phone 100 can also save the text record of the conversion panel in the external storage card.

[0061] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running the instructions stored in the internal memory 121. For example, in the embodiments of the present application, the processor 110 can execute the instructions stored in the internal memory 121, and the internal memory 121 can include a program storage area and a data storage area.

[0062] The wireless communication function of the mobile phone 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, the baseband processor, and the like. In some embodiments, the antenna 1 and the mobile communication module 150 of the mobile phone 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the mobile phone 100 can communicate with the network and other devices through wireless communication technology.

[0063] The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G and the like applied to the mobile phone 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, and the like on the received electromagnetic waves, and transmit the processed electromagnetic waves to the modem processor for demodulation.

[0064] The wireless communication module 160 can provide a solution for wireless communication including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), and the like applied to the mobile phone 100.

[0065] In some implementations, the mobile phone 100 can communicate with other electronic devices through the mobile communication module 150 or the wireless communication module 160, and receive multimedia files such as audio files and video files transmitted by other electronic devices.

[0066] The mobile phone 100 can implement audio functions such as audio playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, the application processor, and the like.

[0067] The audio module 170 is configured to convert digital audio into analog audio signals for output, and to convert analog audio input into digital audio signals. For example, the audio module 170 can convert voice data collected by the mobile phone 100 from analog audio signals into digital audio signals, and can convert audio digital signals corresponding to text data of the mobile phone 100 into analog audio signals. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0068] The speaker 170A, also referred to as a "loudspeaker", is configured to convert audio electrical signals into sound signals. The mobile phone 100 can listen to music or listen to a hands-free call through the speaker 170A. In some embodiments, the mobile phone 100 can play voice data corresponding to text data through the speaker 170A.

[0069] The receiver 170B, also referred to as a "earpiece", is configured to convert audio electrical signals into sound signals. When the mobile phone 100 answers a call or a voice message, the receiver 170B can be held close to a human ear to listen to the voice.

[0070] The microphone 170C, also referred to as a "microphone", "sound transducer", is configured to convert sound signals into electrical signals. When making a call or sending a voice message, a user can speak into the microphone 170C by holding the human mouth close to the microphone 170C to input sound signals into the microphone 170C. The mobile phone 100 can be provided with at least one microphone 170C. In other embodiments, the mobile phone 100 can be provided with two microphones 170C, in addition to collecting sound signals, to achieve noise reduction functions. In other embodiments, the mobile phone 100 can be provided with three, four or more microphones 170C to achieve sound signal collection (such as collecting voice data of the user), noise reduction, and to identify the source of the sound, to achieve directional recording functions, etc.

[0071] The sensor module 180 can include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a Hall sensor, a touch sensor, an ambient light sensor, and a bone conduction sensor, etc. The mobile phone 100 can collect various data through the sensor module 180.

[0072] The display screen 194 is configured to display images, videos, and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light emitting diode (QLED), or the like.

[0073] The mobile phone 100 can display interfaces of the mobile phone 100 through the display screen 194, such as a desktop interface, a call interface, and the like. In the interface displayed by the display screen 194, a conversion panel can be included.

[0074] It can be understood that the interface connection relationship between the modules in the embodiments is only illustrative and does not constitute a structural limitation on the electronic device. In other embodiments, the electronic device can include more or fewer modules than those provided in the above embodiments, and the modules can be connected by different interfaces or a combination of multiple interface connection manners. The hardware structure of the electronic device provided in the embodiments can also refer to the hardware structure of the mobile phone 100. The methods in the following embodiments can be implemented in the electronic device with the above hardware structure.

[0075] As described above, the electronic device in the embodiments has the conversion function of audio data and text data, and the electronic device can provide the conversion function to the user through the conversion panel. For example, Figure 3 As shown in (1) of FIG. 17, the conversion panel includes a shortcut area, an audio-text conversion area, and a bottom function area. The shortcut area is configured to provide some shortcut keys of the conversion panel, such as an exit key (or an exit control), a setting key (or a setting control), and the like. The audio-text conversion area is configured to provide panel content (or conversion content), such as text data, subtitles, and the like. The bottom function area is configured to provide some function keys of the conversion panel, such as a keyboard key (or a keyboard control), a voice key (or a voice control), and the like. The electronic device provides the audio-text conversion function corresponding to multiple conversion modes through the conversion panel, which is suitable for the conversion of audio and text in different application scenarios. The conversion panel provided in the embodiments and the functions provided by the conversion panel are introduced as follows.

[0076] The shortcut area in the conversion panel provided by the electronic device is further provided with a first mode switching control and a second mode switching control. When the first mode switching control is in a selected state, the conversion function is in a first conversion mode. When the second mode switching control is in a selected state, the conversion function is in a second conversion mode. For example, the first conversion mode is a subtitle mode, and the second conversion mode is a dialogue mode. The subtitle mode can provide subtitles corresponding to a multimedia file. The dialogue mode can provide conversion of first speech to text data or conversion of user-input text data to speech data. The conversion panel is provided with a first mode switching control corresponding to the subtitle mode and a second mode switching control corresponding to the dialogue mode. As shown in (2) of Figure 3 When the first mode switching control is in a selected state, the electronic device provides a conversion function in the subtitle mode and displays subtitles of a multimedia file in the conversion panel. As shown in (3) of Figure 3 When the second mode switching control is in a selected state, the electronic device provides a conversion function in the dialogue mode and displays text data of first speech or plays speech data corresponding to user-input text data. The electronic device can switch the subtitle mode to the dialogue mode or switch the dialogue mode to the subtitle mode under user control.

[0077] In some implementations, the second conversion mode includes a dialogue mode and a translation mode. The dialogue mode is used to provide a conversion function of first speech to text data and a conversion function of user-input text data to speech data. The translation mode is used to provide a translation function of first speech to text data and a translation function of user-input text data to speech data.

[0078] To facilitate distinguishing conversion panels corresponding to different conversion modes, the following describes a method provided by an embodiment of the present application, taking a first conversion panel corresponding to a first conversion mode, a first conversion panel corresponding to a second conversion mode, and a third conversion panel corresponding to a third conversion mode as examples.

[0079] The following describes a method provided by an embodiment of the present application, taking switching of an audio-text conversion function of an electronic device from a first conversion mode to a second conversion mode as an example. As shown in (1) of Figure 4 The embodiment of the present application includes the following steps:

[0080] S401, the electronic device starts an audio-text conversion function and displays a first conversion panel in a first interface.

[0081] The electronic device opens the audio-text conversion function and displays a first conversion panel on a first interface. For example, the first interface is a desktop interface, and the electronic device displays the first conversion panel on the desktop interface. In a case where the audio-text conversion function is a first conversion mode, a first mode switching control in the first conversion panel is in a selected state. The first conversion mode is used to provide a subtitle conversion function of a multimedia file, which can also be referred to as a subtitle mode.

[0082] It can be understood that the first conversion panel can be in a form of a window and suspended on an interface of the electronic device. When the interface of the electronic device is switched from the first interface to another interface (for example, a second interface), the first conversion panel is not affected by the interface switching, and the electronic device continues to display the first conversion panel on the second interface.

[0083] In some implementations, an opening entry of the audio-text conversion function is provided in the electronic device. In an example, the electronic device adds a setting page of the audio-text conversion function in a setting function. If the electronic device receives an opening operation of the audio-text conversion function in the setting page of the audio-text conversion function, the electronic device opens the audio-text conversion function and displays the first conversion panel on the first interface. For example, the electronic device provides a smart voice option in the setting function. For example, as shown in (1) of FIG. 1, the setting page of the smart voice provides an "intelligent transcription" option, which is an option of the audio-text conversion function. The audio-text conversion function can also be referred to as a transcription function, and various conversion modes of the audio-text conversion function can also be referred to as transcription modes. The "intelligent transcription" option can be used to enter an "intelligent transcription" page. In the "intelligent transcription" page, the electronic device is provided with a switch control for opening or closing the audio-text conversion function. If the electronic device receives an opening operation of the user to open the conversion function, for example, the electronic device receives a user opening operation, such as a click operation, at the switch control, the electronic device opens the audio-text conversion function and displays the first conversion panel on the displayed first interface. Figure 5

[0084] In some implementations, after the electronic device opens the audio-text conversion function, the electronic device displays, by default, the first conversion panel of the first conversion mode. The distance of the first conversion panel from the lower boundary of the display area of the screen is equal to a preset distance. The preset distance can be set according to actual application scenarios or requirements. For example, the preset distance is 48 device independent pixels (density-independent pixels, dp). Alternatively, the position of the first conversion panel can be adjusted according to a user operation. After the electronic device opens the audio conversion function, the position of the first conversion panel is consistent with the position of the last time the electronic device exits the first conversion panel.

[0085] ​In one example, the electronic device can also add a shortcut of the audio-text conversion function in the desktop. For example, the "Smart Transcription" page of the electronic device is also provided with an adding option of "Desktop Shortcut". When the electronic device receives a click operation of the user at the adding option of "Desktop Shortcut", the electronic device adds the shortcut of the audio-text conversion function in the desktop. For example, as shown in (2) of FIG. 10, the "Smart Transcription" icon is added in the desktop. The "Smart Transcription" icon is the shortcut of the audio-text conversion function. Through the shortcut, the user can quickly start the audio-text transcription function. For example, when the electronic device receives a user operation of the user clicking the shortcut of the audio-text conversion function, the electronic device starts the audio-text transcription function and displays the first conversion panel in the first interface. Figure 5

[0086] In another example, the electronic device can also set a switch control of the audio-text conversion function in the drop-down menu. For example, as shown in (3) of FIG. 10, the drop-down menu of the electronic device is provided with a switch control of "Smart Transcription", which is the switch control of the audio-text conversion function. When the audio-text transcription function is not started, if the electronic device receives a click operation of the user at the switch control of "Smart Transcription", the electronic device starts the audio-text conversion function and displays the first conversion panel in the first interface. Figure 5

[0087] In another example, the electronic device can also start the audio-text transcription function through the voice assistant. When the voice assistant of the electronic device is in an awakened state, if the voice assistant of the electronic device receives a voice instruction or a user operation of starting the audio-text transcription function, the electronic device starts the audio-text conversion function and displays the first conversion panel in the first interface. For example, as shown in (4) of FIG. 10, when the voice assistant of the electronic device is in an awakened state, if the voice assistant of the electronic device receives a voice instruction of "Smart Transcription" or "Turn on Face-to-Face Translation", or receives a click operation of the user at the recommended function of "Smart Transcription" or "Turn on Face-to-Face Translation" prompted by the voice assistant, the electronic device starts the audio-text conversion function and displays the first conversion panel in the first interface. Figure 5

[0088] It can be understood that after the electronic device exits the audio-text conversion function, the electronic device can start the audio-text conversion function again and display the first conversion panel in the first interface through any of the implementation manners introduced in the above examples. For example, the electronic device exits the audio-text conversion function and cancels the display of the first conversion panel in response to a user operation of the user clicking a quit control in the first conversion panel. If the user wants to use the audio-text conversion function again, the user can control the electronic device to start the audio-text conversion function through any of the implementation manners introduced in the above examples.

[0089] ​​​In some implementations, after the electronic device turns on the audio-text conversion function, the electronic device is in a default conversion mode of the audio conversion function. For example, the default conversion mode is the subtitle mode, and after the electronic device turns on the audio-text conversion function, the electronic device is in the subtitle mode of the audio conversion function, and displays a first conversion panel corresponding to the subtitle mode in the first interface. Of course, the electronic device can also directly enter the conversion function of other conversion modes other than the default conversion mode. For example, if the electronic device receives the voice instruction of "turn on face-to-face translation" through the voice assistant, the electronic device enters the conversion function of the translation mode, and displays a first conversion panel corresponding to the translation mode.

[0090] It can be understood that the audio-text conversion function is different in different conversion modes, and the content displayed by the electronic device in the first conversion panel is also different. For example, in the first conversion mode, the electronic device provides subtitles of a multimedia file through the first conversion panel. In the second conversion mode, the electronic device provides text data corresponding to the first voice through the first conversion panel. Of course, the first conversion panel in different conversion modes can also be different in display style.

[0091] S402, the electronic device switches the first interface to the second interface in response to the first operation of the user, and displays the first conversion panel in the second interface.

[0092] The electronic device receives the first operation of the user, such as the electronic device receiving the user operation of the user clicking the video application in the desktop. In response to the first operation of the user, the electronic device switches the first interface to the second interface, such as switching the desktop interface to the video application interface. As described above, the first conversion panel is in the form of a window floating on the interface of the electronic device, and the first conversion panel is not affected by the interface switching, and the electronic device displays the first conversion panel in the second interface.

[0093] The first area of the second interface includes the first multimedia file. For example, as shown in Figure 4 The second interface displays the first multimedia file in the first area. The electronic device plays the first multimedia file in response to the user operation of the user playing the first multimedia file.

[0094] S403, display subtitles corresponding to the first multimedia file in the first conversion panel during playing the first multimedia file.

[0095] In the first conversion mode, if the electronic device plays a first multimedia file, such as a video file or an audio file, the electronic device acquires the audio data of the first multimedia file, identifies the audio data, and obtains the text data corresponding to the audio data. This text data can be used as subtitles for the first multimedia file. Furthermore, the electronic device displays the subtitles corresponding to the first multimedia file in a first conversion panel. For example, while the electronic device is playing a video file, a conversion panel floats at a preset distance from the bottom edge of the screen, displaying the video file's subtitles. In this way, the electronic device can provide the user with real-time subtitles for the first multimedia file through the conversion panel.

[0096] In some implementations, the electronic device can also translate the audio data of the first multimedia file to obtain subtitles in the target language, and display the subtitles in the target language on the first conversion panel. For example, the source language of the audio data of the first multimedia file is English, and the target language is Chinese. The electronic device identifies and translates the audio data of the first multimedia file being played to obtain Chinese subtitles for the first multimedia file. Furthermore, the electronic device displays the corresponding Chinese subtitles for the multimedia file on the conversion panel.

[0097] In some examples, the first conversion panel in the subtitle mode provided by the electronic device can support subtitles in multiple languages. For example, the audio data of the first multimedia file can be in any language such as Chinese, English, Korean, or Japanese. The subtitles of the first multimedia file can also be in any language such as Chinese, English, Korean, or Japanese.

[0098] For example, such as Figure 6 As shown in (1), the first conversion panel in subtitle mode has a setting control. If the electronic device receives a user's click operation at the setting control, it enters the subtitle mode setting page. The subtitle mode setting page includes settings for the source language type of the first multimedia file and the target language type of the subtitle, such as the setting for "sound source language". After entering the setting page corresponding to "sound source language", the first conversion panel provides the source language type of the first multimedia file and the target language type of the subtitle. For example, the setting page corresponding to "sound source language" includes multiple options such as Chinese, English, Korean, Japanese, and English to Chinese, Korean to Chinese, and Japanese to Chinese. If the "English" option is selected, it means that the source language type of the first multimedia file is English, and the target language type of the subtitle is also English. If the "English to Chinese" option is selected, it means that the source language type of the first multimedia file is English, and the target language type of the subtitle is Chinese.

[0099] In some examples, the electronic device can also display the target language type corresponding to the subtitles in the first conversion panel of the subtitle mode. For example, as shown below... Figure 6 As shown in (2), if the electronic device does not play the first multimedia file, in the first conversion panel of the subtitle mode, the electronic device displays "Language: Chinese", indicating that the target language type corresponding to the subtitle mode is Chinese. In this way, the electronic device can prompt the user about the target language type of the subtitle through the first conversion panel.

[0100] In some examples, to save space on the first conversion panel, the electronic device can automatically retract the shortcut key area of ​​the first conversion panel. For example, in subtitle mode, the first conversion panel can first display the shortcut key area. If the default window with the shortcut key area has been displayed for a preset duration (e.g., 5 seconds, 10 seconds, etc.) and no operation has been received in the default window, such as... Figure 6 As shown in (3), the electronic device then collapses the shortcut area of ​​the default window and displays the default window without a shortcut area. If the electronic device receives a user operation in the default window without a shortcut area, the electronic device then brings up the shortcut area of ​​the default window again.

[0101] In some examples, the electronic device may also display the application icon and time of the application currently playing the first multimedia file in the first conversion panel of the subtitle mode. For example, as Figure 6 As shown in (3), during the process of the electronic device playing the first multimedia file, the transcription panel of the electronic device is set with the application icon of the application software that plays the first multimedia file, as well as the time "2023 / 02 / 06 16:52", so as to prompt the user to provide subtitles for the audio data played by the application software with the application icon, and to prompt the user to provide subtitles at the time.

[0102] In some examples, the electronic device can also display the subtitles corresponding to the currently playing audio data in the first conversion panel of the subtitle mode. For example, the electronic device uses the brightness of the subtitles to indicate the subtitles corresponding to the currently playing audio data. For instance, in the subtitles displayed on the first conversion panel in subtitle mode, some text is darker and some is brighter. The darker text corresponds to the subtitle portion of the audio data of the first multimedia file that the electronic device has already finished playing. The brighter text corresponds to the subtitle portion of the audio data that the electronic device is currently playing. If the subtitles are delayed, “…” can be used.

[0103] S404, the electronic device responds to the second operation by displaying a second conversion panel and acquiring the first voice, and displays the text data corresponding to the first voice in the second conversion panel.

[0104] The second operation is the user's selection of the second mode switching control. If the electronic device receives a second operation, such as a click, at the second mode switching control on the first mode switching panel while displaying the first mode switching panel, the electronic device will switch the first mode switching panel to the second mode switching panel. The second mode switching control in the second mode switching panel displayed by the electronic device will then be selected.

[0105] For example, such as Figure 3 As shown in (2), the shortcut key area of ​​the first conversion panel is equipped with a first mode switching control and a second mode switching control. The first mode switching control corresponds to the subtitle mode, and the second mode switching control corresponds to the dialog mode (an example of the second conversion mode). In subtitle mode, if the electronic device receives a second operation, such as a click operation, at the "Dialogue" (an example of the second mode switching control) in the first conversion panel, the electronic device will switch the first conversion panel to the second conversion panel. Figure 3 As shown in (3), the electronic device displays a second switching panel in dialog mode.

[0106] In the second conversion mode, the electronic device provides both offline voice-to-text data conversion and user-input text-to-voice data conversion. In this mode, the electronic device captures voice data via a microphone. The voice data captured by the microphone can be referred to as the first voice. If the electronic device captures the first voice through the microphone, it converts it into text data and displays the converted text data on the second conversion panel.

[0107] In some implementations, the second conversion panel of the second conversion mode includes options for both dialogue mode and translation mode. If the dialogue mode option is selected, the second conversion mode is dialogue mode. If the translation mode option is selected, the second conversion mode is translation mode.

[0108] For example, such as Figure 7 As shown in (1), in the second conversion mode, the second conversion panel has setting function options (or setting controls, setting items). The setting page of the setting function options has a "Dialogue Language" option (or mode selection control). In the dialogue language page (or dialogue mode page) corresponding to the "Dialogue Language" option, there are options for "Monolingual Dialogue" (an example of dialogue mode options) and "Bilingual Dialogue" (an example of translation mode options). If any of the multiple options belonging to "Monolingual Dialogue" is selected, the second conversion mode is dialogue mode. If the option belonging to "Bilingual Dialogue" is selected, the second conversion mode is translation mode.

[0109] In some examples, the second conversion panel provided by the electronic device in dialogue mode supports multiple language types. For example, the voice data collected by the electronic device (such as the first voice) and the text data converted from the voice data (such as recognized text) can both be in any language type, such as Chinese, English, Korean, or Japanese. Correspondingly, the text data received by the electronic device in the second conversion panel (such as the second input text) and the voice data converted from the text data (such as the voice data corresponding to the second input text) can both be in any language type, such as Chinese, English, Korean, or Japanese.

[0110] For example, such as Figure 7 As shown in (1), in the dialogue language page corresponding to the "dialogue language" option, there are Chinese, English, Korean, and Japanese options under "monolingual dialogue". If the "Chinese" option is selected, the second conversion panel of the electronic device supports the mutual conversion function of Chinese voice data and text data.

[0111] In some examples, the second conversion panel in the translation mode provided by the electronic device also supports multiple language settings. For example, the voice data collected by the electronic device (such as the first voice) can be in any language such as Chinese, English, Korean, or Japanese. The text data translated from the voice data and displayed on the second conversion panel by the electronic device (such as the first translated text) can be in a different language than the voice data, such as Chinese, English, Korean, or Japanese. The text data received by the electronic device on the second conversion panel (such as the first input text) can also be in any language such as Chinese, English, Korean, or Japanese. The voice data played by the electronic device (such as the voice data corresponding to the first translated text, or the voice data corresponding to the second translated text) can be in a different language than the text data, such as Chinese, English, Korean, or Japanese. The electronic device allows the user to set the language.

[0112] In some examples, after the language type is set, the electronic device can also prompt for transcription mode and / or set the speech type in the second conversion panel.

[0113] For example, such as Figure 7 As shown in (2), the second conversion panel in translation mode includes a language selection bar (as indicated by the positions of "English" and "Chinese"). This language selection bar includes options for the source speech type and the target language type. The electronic device allows the user to select two different language types. The electronic device can provide the function of converting between speech data and text data in these two language types.

[0114] In the embodiment of the present application, the electronic device can provide conversion and translation of voice data and text data through the second conversion panel in the second conversion mode, thereby meeting the voice-text conversion needs of users in various application scenarios, such as meeting the daily communication needs of hearing-impaired users, meeting the conversation needs of different language scenarios, and the like.

[0115] In the conversation mode, the electronic device collects voice data in real time. If the electronic device collects a first voice, the electronic device recognizes the first voice to obtain text data corresponding to the first voice (e.g., referred to as recognized text). Further, the electronic device displays the recognized text in the second conversion panel. The language types of the first voice and the recognized text are the same.

[0116] In some examples, the conversation mode is also used to provide a conversion function of text data input by a user (e.g., referred to as second input text) to voice data. If the electronic device obtains the second input text in the second conversion panel, the electronic device converts the second input text into voice data to obtain voice data corresponding to the second input text, and automatically plays the voice data corresponding to the second input text. For example, as shown in Figure 8 If the electronic device receives a click operation at the position of the keyboard key, the electronic device displays an input keyboard to obtain text data input by a user through the input keyboard. After the user finishes inputting, the user can click a "send" control in the second conversion panel to confirm that the second input text has been edited. If the electronic device receives a click operation of the user on the "send" control, the electronic device displays the text data input by the user in the second conversion panel through a bubble, and further converts the text data into voice data and automatically plays the voice data.

[0117] In this way, the electronic device can read the input text to the user, and in the case that the user is not convenient to speak, the user can communicate with others through voice data corresponding to the input text, thereby providing convenience for the user.

[0118] Optionally, the electronic device can prompt the user, in the second conversion panel, about text data corresponding to voice data currently being played by the electronic device in a highlighting manner. The electronic device can also prompt, in the second conversion panel, about text data corresponding to voice data being played by the electronic device through a playing icon. For example, a playing icon can be arranged in front of a bubble box of text data being read, to prompt that the text data in the bubble box is being read.

[0119] In the translation mode, the electronic device collects voice data in response to a user operation. For example, as shown in Figure 9As shown in (1) of FIG. 6, the second conversion panel is provided with a microphone control (or referred to as a play control). If the electronic device receives a long press operation of a user at the microphone control, the electronic device collects voice data. The electronic device collects a first voice, and performs recognition and translation on the first voice to obtain text data corresponding to the first voice (referred to as first translated text). Further, the electronic device displays the first translated text in the second conversion panel. The language types of the first voice and the first translated text are different. In this way, the electronic device can provide the user with a translation function of the first voice, and in the case that the user cannot understand the speech of others, the user can understand the meaning of the speech of others through the translated text displayed in the second conversion panel, thereby providing convenience for the user.

[0120] Optionally, during the process of collecting voice data, the electronic device can retract the language selection bar in the second conversion panel. As shown in Figure 9 As shown in (2) of FIG. 6, the language selection bar in the second conversion panel is in a retracted state. In this way, the space of the second conversion panel can be saved, and the second conversion panel can accommodate more text boxes.

[0121] In some examples, if the electronic device receives a downward sliding operation in the audio-text transcription area of the second conversion panel, the electronic device displays the language selection bar in the second conversion panel.

[0122] Optionally, the electronic device can also provide a voice play function of the first translated text. For example, a play icon can be provided in front of the bubble box of the first translated text in the second conversion panel. If the electronic device receives a click operation at the play icon, the electronic device plays the voice data corresponding to the first translated text in the bubble box.

[0123] In some examples, the translation mode is also used to provide a translation function from text data to voice data. If the electronic device obtains a first input text in the second conversion panel, the electronic device translates the first input text into text data of a target language type to obtain translated text corresponding to the first input text (referred to as second translated text), and displays the second translated text in the second conversion panel. For example, as shown in Figure 9As shown in (3) of FIG. 1, the second conversion panel in the translation mode is provided with a keyboard key. If the electronic device receives a click operation at the keyboard key position, the electronic device displays an input keyboard to obtain text data (e.g., the first input text) of the source language type input by the user through the input keyboard. After the user finishes editing the first input text, the user clicks the "send" control in the second conversion panel to confirm that the first input text has been edited. If the electronic device receives a click operation of the user on the "send" control, the electronic device displays the first input text in a bubble in the second conversion panel. The electronic device further translates the text data from the source language type to the target language type to obtain the second translation text, and displays the second translation text of the target language type below the text data of the source language type. In this way, the electronic device can provide the user with a translation function of input text. For example, in an offline communication scenario, if the two parties in the conversation do not understand each other, the user can show the second conversion panel to display the text of the target language type to the other party to realize communication with the other party and provide convenience for the user.

[0124] Optionally, the electronic device can also play the voice data corresponding to the second translation text. As shown in (2) of FIG. 1, the electronic device displays a play icon in the bubble frame of the text data in the second conversion panel. If the electronic device receives a click operation at the play icon, the electronic device plays the voice data corresponding to the translation text in the bubble frame corresponding to the play icon. Figure 9 As shown in (4) of FIG. 1, the electronic device displays a play icon in front of the bubble frame of the text data in the second conversion panel. If the electronic device receives a click operation at the play icon, the electronic device plays the voice data corresponding to the translation text in the bubble frame corresponding to the play icon.

[0125] Optionally, the text of the source language type and the text of the target language type in the second conversion panel can be displayed on the two sides of the second conversion panel, respectively. For example, the text of the source language type is displayed on the left side of the conversion panel, and the text of the target language type is displayed on the right side of the second conversion panel.

[0126] S405, if the electronic device detects the call voice, the electronic device displays a third conversion panel and displays the call text corresponding to the call voice in the third conversion panel.

[0127] The conversion function of the electronic device also has a third conversion mode. The third conversion mode is used to provide a call voice corresponding to a call text conversion function, which can also be referred to as a call mode. If the electronic device hears the call voice, the electronic device automatically switches the conversion function to the third conversion mode. In the third conversion mode, the electronic device converts the call voice into the call text and displays the call text corresponding to the call voice in the third conversion panel floating on the call interface. In this way, the electronic device can display the call content in the form of text on the screen. Even in the case where the user does not clearly hear the call voice or is not convenient to listen to the call voice, the user can understand the call content through the call text displayed on the screen to realize barrier-free call.

[0128] In some implementations, the third conversion mode of the conversion function can correspond to a separate activation mode. In one example, such as... Figure 10 As shown in (1), if the electronic device receives an activation operation from the user at the "Smart Transcription" switch control on the "Smart Transcription" page, such as a click operation, the electronic device will pop up a reminder window asking whether to enable call text. If the electronic device receives a click operation from the user at the "Enable" control in the call text reminder window, the electronic device will enable the call text conversion function corresponding to the call mode on the third conversion panel.

[0129] In another example, when the call text conversion function corresponding to the call mode is not enabled, such as Figure 10 As shown in (2), the electronic device has a "call text" control in its call interface. If the electronic device receives a click operation from the user at the "call text" control, the electronic device will display a third conversion panel for the call mode on the call interface and display the call text corresponding to the call voice in the third conversion panel.

[0130] Of course, the third conversion mode of the conversion function can correspond to the same activation method as the first and second conversion modes. For example, if the electronic device receives a user's activation operation to activate the conversion function, then the first, second, and third conversion modes will be activated simultaneously.

[0131] In this embodiment, the electronic device provides audio-text conversion for various application scenarios through multiple conversion modes of its audio-text conversion function. The conversion panel can switch between different conversion modes according to different application scenarios, thus providing a full-scenario audio-text conversion solution. For example, when a user is in a noisy environment and has difficulty hearing audio or video, the electronic device can automatically convert media audio into text for display when the transcription function is in subtitle mode, allowing the user to access the audio and video content in a timely manner. When a hearing-impaired user has difficulty communicating, the electronic device can transcribe the other party's speech into text for display in real time. Simultaneously, the electronic device also supports the voice playback of text entered by the hearing-impaired user via keyboard, allowing the other party to hear the voice and facilitating communication between the hearing-impaired user and others.

[0132] To facilitate user operation and meet user habits, in some implementations, both the first and second conversion panels have multiple display modes. Electronic devices can switch between these multiple display modes on either the first or second conversion panel. Thus, the first and second conversion panels of the electronic device not only facilitate user operation through multiple display modes but also provide users with various visual changes, enhancing the user experience. Hereinafter, we will not distinguish between the first and second conversion panels and will uniformly refer to them as conversion panels.

[0133] In some implementations, the conversion panel is provided with a shape switching control for switching between different display shapes. Upon receiving a shape switching operation of the shape switching control by a user, the electronic device switches the conversion panel from a first shape to a second shape. The first shape is a different display shape from the second shape.

[0134] For example, the conversion panel has any one of a default shape, a floating ball shape, and a full screen shape. The first shape is any one of the display shapes, and the second shape is a different display shape from the first shape. The default shape, the floating ball shape, and the full screen shape of the conversion panel are as shown in (1) of Figure 11 For example, the default shape of the conversion panel is a default window with a height smaller than that of the full screen display area of the electronic device. For example, the height of the default window is the height of 2 lines of text or 1 line of text. The floating ball shape of the conversion panel is a floating ball window with a diameter smaller than a preset value, such as smaller than 40dp, 56dp, or the like. The full screen shape of the conversion panel is a full screen window corresponding to the full screen display area. The full screen display area is the display area of the electronic device when displaying in full screen, such as the maximum display area corresponding to the display screen of the electronic device.

[0135] For example, the shape switching control includes a maximize control and a minimize control. The default window corresponding to the default shape of the conversion panel is provided with the maximize control and the minimize control. Upon receiving a shape switching operation, such as a click operation, at the maximize control in the default window of the conversion panel, the electronic device switches the conversion panel from the default window of the default shape to the full screen window of the full screen shape in response to the shape switching operation, and displays the full screen window of the conversion panel in full screen.

[0136] Accordingly, the full screen window of the conversion panel is provided with a cancel control for canceling the full screen display. Upon receiving a shape switching operation, such as a click operation, at the cancel control in the full screen window of the conversion panel, the electronic device switches the conversion panel from the full screen window of the full screen shape to the default window of the default shape in response to the shape switching operation, and displays the default window of the conversion panel. The full screen window of the conversion panel can also be provided with a minimize control. Upon receiving a shape switching operation, such as a click operation, at the minimize control in the full screen window of the conversion panel, the electronic device switches the conversion panel from the full screen window of the full screen shape to the floating ball window of the floating ball shape in response to the shape switching operation, and displays the floating ball window of the conversion panel.

[0137] In different transcription modes, the full-screen form of the conversion panel can be different. For example, as shown in (2) of FIG. 13, in the subtitle mode, the full-screen form of the first conversion panel does not set a function control in the bottom function area of the full-screen window. In the conversation mode, the full-screen form of the second conversion panel sets a function control of a keyboard in the bottom function area of the full-screen window for the user to input text data. In the translation mode, the full-screen form of the second conversion panel sets a function control of a keyboard and a function control of a microphone in the bottom function area of the full-screen window for the user to input voice data and text data. Figure 11

[0138] Optionally, due to the size limitation of the conversion panel, the floating ball window can not set a control for displaying mode switching. If the electronic device receives a mode switching operation, such as a click operation, at the floating ball window of the conversion panel, the electronic device switches the conversion panel from the floating ball form of the floating ball window to the default form of the default window in response to the mode switching operation.

[0139] The default window corresponding to the default form of the conversion panel and the full-screen window corresponding to the full-screen form are also provided with a close control. If the electronic device receives a close operation of the close control by the user, the electronic device closes the conversion panel. Optionally, if the electronic device receives a close operation of the close control by the user, the electronic device can also prompt the user whether to save the transcription content in the conversion panel. For example, as shown in (3) of FIG. 13, if the electronic device receives a close operation of the close control by the user, the electronic device pops up a prompt box in the conversion panel to ask whether to save the transcription content. If the electronic device receives a click operation of the user clicking on the save, the electronic device saves the transcription content in the conversion panel and then closes the conversion panel. If the electronic device receives a click operation of the user clicking on the do not save, the electronic device closes the conversion panel. If the electronic device receives a selection operation of the user selecting do not remind again, the electronic device does not pop up the prompt box to ask whether to save the transcription content when closing the conversion panel next time. Figure 11 It can be understood that in some implementations, the electronic device does not clear the transcription content in the second conversion panel when switching the transcription mode. For example, taking the electronic device switching from the conversation mode to the translation mode as an example, as shown in (1) of FIG. 13, in the conversation mode, the electronic device displays the text data “Nice to meet you!” in the second conversion panel. After the electronic device switches from the conversation mode to the translation mode, the text data “Nice to meet you!” is still retained in the second conversion panel. The text data “Hello!” obtained by the electronic device in the translation mode and the “Hello!” translated from “Hello!” are displayed below “Nice to meet you!”.

[0140] Figure 12 ​​​

[0141] In some implementations, when there is more transcribed content in the second conversion panel, the electronic device provides a return top control in the second conversion panel when the electronic device is unable to display all of the transcribed content in the audio-text transcription area of the second conversion panel. When the user clicks the return top control, the electronic device displays the first piece of transcribed content in the second conversion panel. In this way, the user can view the transcribed content, and the user experience is improved.

[0142] In some implementations, the conversion panel in the default mode and the conversion panel in the floating ball mode support a position moving function. When the electronic device receives a moving operation indicating to move the position of the default window or the floating ball window, the electronic device adjusts the position of the default window or the floating ball window in response to the moving operation. For example, as shown in (2) of FIG. 1, when the moving operation is a drag operation on the shortcut area of the default window, the electronic device moves the default window along the moving track of the drag operation. Figure 12

[0143] It can be understood that the first conversion panel or the second conversion panel is suspended on the interface of the electronic device in the default mode and the floating ball mode. The first conversion panel or the second conversion panel is not affected by the interface switching, and is displayed on the interface even if the electronic device switches from one interface to another interface.

[0144] In some implementations, when the default window of the conversion panel moves beyond the screen boundary, the electronic device switches the conversion panel from the default mode to the floating ball mode, and automatically reduces the default window corresponding to the default mode to the floating ball window corresponding to the floating ball mode. In this way, the electronic device can provide multiple schemes for the conversion panel to change from the default mode to the floating ball mode, facilitate user operation, and improve user experience.

[0145] For example, when the conversion panel of the electronic device is displayed as a default window in the default mode, the electronic device moves the default window in response to a moving operation of the default window. When the default window moves beyond the screen boundary, the screen of the electronic device only displays the part of the default window that does not exceed the screen boundary, and no longer displays the part of the default window that exceeds the screen boundary. For ease of description, the part of the default window that exceeds the screen boundary can be referred to as an invisible area. If the ratio of the area of the invisible area to the total area of the default window is greater than a preset ratio, such as 1 / 3, 1 / 2, or the like, the electronic device switches the conversion panel from the default window to the floating ball window. For example, as shown in (1) of FIG. 1, the electronic device displays the first conversion panel in the default mode in the video playing interface, and the first conversion panel corresponds to the subtitle mode and is used to provide subtitles of a video file. As shown in (2) of FIG. 1, when the default window of the first conversion panel moves beyond the screen boundary, the screen of the electronic device only displays the part of the default window that does not exceed the screen boundary, and no longer displays the part of the default window that exceeds the screen boundary. Figure 13 Figure 13 ​​As shown in (2) of FIG. 1, when the first conversion panel in the default mode moves to the screen boundary and the ratio of the area of the invisible region of the first conversion panel to the total area of the default window is greater than 1 / 3, the electronic device changes the display mode of the conversion panel to the floating ball mode.

[0146] Optionally, the electronic device further provides a quick closing scheme for the floating ball window. As shown in (3) of FIG. 1, if the electronic device receives a long press operation at the floating ball window, the electronic device displays a trash can icon, which represents a deletion function. If the electronic device receives a drag operation of dragging the floating ball window to the trash can icon, the electronic device cancels the display of the floating ball window in response to the drag operation. Further, the electronic device can also close the audio-text conversion function. Figure 13

[0147] In order to provide better user experience, in some implementations, the size of the default mode of the conversion panel can be different in different conversion modes. As an example, the conversion mode of the conversion panel is the subtitle mode (an example of the first conversion mode) and the dialogue mode (an example of the second conversion mode). In the subtitle mode, the default size of the default window of the first conversion panel can correspond to the height of 2 lines of text. The minimum height of the default window of the first conversion panel in the subtitle mode can be the height of 1 line of text, and the maximum height can be 2 / 3 of the height of the full-screen display area. In the dialogue mode, the default height of the default window of the second conversion panel can correspond to 3 / 5 or 1 / 2 of the height of the full-screen display area, and the maximum height can be 2 / 3 of the height of the full-screen display area. The minimum height of the default window of the second conversion panel in the dialogue mode is greater than or equal to the height of 2 single-line text boxes (or single-line text bubbles). In the subtitle mode and the dialogue mode, the width of the default window of the first conversion panel and the second conversion panel is less than or equal to the width of the screen.

[0148] Optionally, the size of the default window corresponding to the default mode of the conversion panel can be adapted to the interface display mode of the electronic device. As shown in (4) of FIG. 1, taking the default window in the subtitle mode as an example. If the interface of the electronic device is in a vertical screen display mode, the width of the default window is less than or equal to the width of the display area corresponding to the vertical screen, and the height of the default window is 2 lines of text. If the interface of the electronic device is in a horizontal screen display mode, the width of the default window is less than or equal to the width of the display area corresponding to the horizontal screen, and the height of the default window is 1 line of text. If the interface of the electronic device is in a full-screen display mode of a folding screen, the width of the default window is less than or equal to the width of the display area corresponding to the full screen of the folding screen, and the height of the default window is 2 lines of text. Figure 14

[0149] ​​Of course, the size of the conversion panel in the default mode can also support manual adjustment. For example, the electronic device increases or decreases the size of the default window in response to a size adjustment operation for indicating adjustment of the size of the default window. For example, as shown in Figure 15 if the electronic device receives a long press operation (an example of the size adjustment operation) at the bottom of the default window, the electronic device increases the size of the default window in response to the long press operation, such as increasing the height of the default window. Alternatively, the electronic device can also display an adjustment box in response to the long press operation for indicating the adjusted height of the conversion panel.

[0150] As mentioned above, the conversion panel includes a setting function option. The setting page of the setting function option includes a plurality of setting items. The user can adjust the plurality of setting items of the conversion panel in the setting page. For example, the setting page of the conversion panel includes a font size, a transparency, a language type of a subtitle, a language type of a dialogue mode, a source language type of a translation mode, a target language type of the translation mode, a save panel content, a clear panel content, and the like.

[0151] For example, in the subtitle mode, the setting page of the first conversion panel includes a sound source language (an example of the language type of the subtitle), a font size, a transparency, a save (an example of saving the panel content), and a clear (an example of clearing the panel content) as shown in (1) of Figure 16 . Among them, the setting item of the sound source language can be described above, and will not be repeated here.

[0152] The setting item of the font size is used to set the font size of the text displayed in the conversion panel. For example, the font size can be any one of the font sizes of small, standard, large, extra large, and extra large. As shown in (2) of Figure 16 , the setting page of the font size provides a font size adjustment bar (A represents the font size), and the electronic device adjusts the font size of the text displayed in the conversion panel in response to a drag operation on the font size adjustment bar.

[0153] The setting item of the transparency is used to set the window transparency of the conversion panel. As shown in (3) of Figure 16 , the setting page of the transparency provides a transparency adjustment bar. The electronic device adjusts the transparency of the conversion panel in response to a drag operation on the transparency adjustment bar.

[0154] The setting item of the save is used to save the panel content in the conversion panel. As shown in (4) of Figure 16 , the electronic device pops up a save page in response to a click operation of the save setting item. If the electronic device receives a click operation at the save control in the save page, the electronic device saves the panel content in the conversion panel.

[0155] The setting item of the clear is used to clear the panel content in the conversion panel. As shown inFigure 16 As shown in (5), in response to a click operation to clear settings, the electronic device pops up a clear window for clearing the conversion panel. If the electronic device receives a click operation at the clear control in the clear window, the electronic device deletes the panel content in the conversion panel.

[0156] In some implementations, the above settings can also be configured on the "Smart Transcription" page. For example, such as... Figure 17 As shown in (1), in the “Smart Transcription” page, in addition to the switch control to turn the audio-text conversion function on or off and the option to add desktop shortcuts, the electronic device can also set multiple settings such as transcription mode, font size, subtitle transparency, broadcast tone, and call subtitles.

[0157] The transcription mode setting is used to configure the conversion mode for the conversion function. For example... Figure 17 As shown in (2), the transcription modes include subtitle mode and dialogue mode. The user can select either subtitle mode or dialogue mode for the conversion function. If the electronic device receives a selection operation from the user to choose either subtitle mode or dialogue mode, the electronic device will set the conversion function to the transcription mode indicated by the selection operation. When the electronic device first starts the audio-text conversion function under user operation, the electronic device will default to starting the subtitle mode conversion function.

[0158] The playback tone settings are used to configure the playback tone of voice data. The electronic device displays the playback tone page in response to a click on the playback tone settings. For example... Figure 17 As shown in (3), the voice tone page includes multiple voice tone options, such as male and female voice tones pre-configured by the electronic device's operating system. In some examples, the electronic device can also provide a custom voice tone, which can be a voice tone obtained based on the collected speech data. In response to the user's setting operation of any of the multiple voice tone options, the electronic device sets the voice tone corresponding to the second conversion panel to the voice tone selected by the user in the voice tone settings.

[0159] In this way, electronic devices can play voice data using a user-selected voice tone, such as using the voice assistant's voice tone to play the voice data corresponding to the second input text. The voice assistant's voice tone is the tone used when playing voice. Thus, electronic devices can provide users with voice broadcast functions corresponding to multiple voice tones.

[0160] In order to better provide the user with the audio-text conversion function, the electronic device, in the case where the audio-text conversion function is not turned on (i.e., the second conversion panel is closed), if the electronic device detects that a second multimedia file is played and a preset condition is met, the electronic device pops up a first prompt information in the interface, and the first prompt information is used to prompt to turn on the conversion function. The preset condition can be set according to the actual application scene or demand. For example, the preset condition includes that the playing volume of the second multimedia file is less than or equal to a preset volume, such as less than or equal to 0 (an example of the preset volume), or the power of the electronic device is less than a preset power, such as less than 40% of the total power (an example of the preset power), or the environmental noise collected by the electronic device is greater than a preset decibel, such as greater than 50 decibels (an example of the preset decibel).

[0161] For example, as shown in FIG. 1, the electronic device plays a video file in the case where the first conversion panel is not turned on. At this time, the second multimedia file playing volume of the electronic device is 0 (an example of the preset volume). The electronic device displays an opening control in the form of a capsule in the function bar at the top of the interface. If a click operation of the user is received at the displayed opening control, the electronic device pops up a first prompt information at the top of the interface in response to the click operation, and the first prompt information is used to prompt to turn on the subtitle function. If the electronic device receives an opening operation of the user to turn on the subtitle function, the electronic device displays a first conversion panel in the form of a subtitle mode in the interface, and provides the subtitles of the video file through the first conversion panel. Figure 18

[0162] Optionally, after the audio-text conversion function is turned on, if the electronic device stops playing the second multimedia file, the electronic device can automatically turn off the audio-text conversion function. Or, after the audio-text conversion function is turned on, if the electronic device stops playing the second multimedia file, the function bar where the opening control is displayed is cancelled, but the audio-text conversion function is continued to be turned on, and the first conversion panel is displayed in the interface. The user can manually turn off the audio-text conversion function.

[0163] It should be noted that the personal information used in the technical solution of the present application is limited to information obtained with the individual consent of the user, including but not limited to informing and reminding the user to read the relevant user agreement (notification) before the user uses the function, and signing the agreement including the authorization of the relevant user information (authorization).

[0164] ​The electronic device in some embodiments of the present application also includes a screen, a memory and one or more processors. The screen and the memory are coupled to the processor. The screen is configured to display an interface. The memory stores computer program codes including computer instructions. When the computer instructions are executed by the processor, the electronic device can perform the functions or steps in the above method embodiments. Of course, the electronic device can also include other hardware structures. For example, the electronic device can also include a sensor, a communication module and other hardware structures. The structure of the electronic device can refer to the structure of the mobile phone 100 shown in FIG. 1. Figure 2 The structure of the mobile phone 100 shown in FIG. 1.

[0165] The chip system in the embodiments of the present application is applied to an electronic device. The chip system includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a circuit. For example, the interface circuit can be configured to receive signals from other devices (such as a memory). For another example, the interface circuit can be configured to send signals to other devices (such as a processor). For example, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not limited in the embodiments of the present application.

[0166] The computer readable storage medium in the embodiments of the present application includes computer instructions. When the computer instructions are run on the above electronic device, the electronic device can perform the functions or steps in the above method embodiments.

[0167] The computer program product in the embodiments of the present application can make the computer perform the functions or steps in the above method embodiments when the computer program product is run on the computer. For example, the computer can be the above electronic device.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions.

[0169] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the modules or units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another apparatus, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0170] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or a plurality of physical units, that is, can be located in one place, or can be distributed to a plurality of different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0171] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0172] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0173] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any change or replacement within the technical scope disclosed in the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of converting audio and text, characterized by, The method is applied to an electronic device, and the method comprises: displaying a first conversion panel on a first interface, the first conversion panel comprising a first mode switching control and a second mode switching control, the first mode switching control being in a selected state; in response to a first operation of a user, switching the first interface to a second interface and displaying the first conversion panel on the second interface, a first region of the second interface comprising a first multimedia file; when the first multimedia file is played, displaying a subtitle corresponding to the first multimedia file on the first conversion panel; when it is detected that the second mode switching control is selected by the user, displaying a second conversion panel, collecting first speech through a microphone, displaying text data corresponding to the first speech on the second conversion panel, the second conversion panel comprising the first mode switching control and the second mode switching control.

2. The method of claim 1, wherein, when the second conversion panel corresponds to a conversation mode, the text data corresponding to the first speech is obtained by recognizing the first speech; when the second conversion panel corresponds to a translation mode, the text data corresponding to the first speech is obtained by translating the text data recognized from the first speech.

3. The method of claim 2, wherein, the second conversion panel comprises a setting control; the method further comprises: in response to a user operation of the user on the setting control, displaying a setting page in the second conversion panel, the setting page comprising a mode selection control; in response to a user operation of the user on the mode selection control, displaying a conversation mode page in the second conversion panel, the conversation mode page comprising an option of the conversation mode and an option of the translation mode; if the option of the conversation mode is in a selected state, the second conversion panel corresponds to the conversation mode; if the option of the translation mode is in a selected state, the second conversion panel corresponds to the translation mode.

4. The method according to claim 2 or 3, characterized in that, the method further comprises: if first input text is obtained in the second conversion panel when the second conversion panel corresponds to the translation mode, displaying translated text of the first input text in the second conversion panel, the first input text and the translated text being of different language types.

5. The method of claim 4, wherein, the second conversion panel further comprises a play control, and the method further comprises: in response to a user operation of the user on the play control, playing speech data corresponding to the translated text.

6. The method of any one of claims 2, 3, or 5, wherein, if second input text is obtained in the conversion panel when the second conversion panel corresponds to the conversation mode, playing speech data corresponding to the second input text.

7. The method according to any one of claims 1-3 or 5, characterized in that, the first conversion panel further comprises a form switching control; the method further comprises: in response to a form switching operation of the user on the form switching control, switching a display form of the first conversion panel from a first form to a second form; the second form is a display form different from the first form.

8. The method of claim 7, wherein, the first conversion panel has any one of a default form, a floating ball form, and a full-screen form; wherein, The default form of the first conversion panel is a default window, and a height of the default window is less than a height of a full-screen display area of the electronic device; The floating ball form of the first conversion panel is a floating ball window, and a diameter of the floating ball window is less than a preset value; The full-screen form of the first conversion panel is a full-screen window, and the full-screen window corresponds to the full-screen display area.

9. The method of claim 8, wherein, The form switching control includes a minimizing control and a maximizing control, and the switching of the display form of the first conversion panel from a first form to a second form in response to the user operation of the user on the form switching control includes: switching the display form of the first conversion panel from the default form to the floating ball form in response to the form switching operation of the user on the minimizing control; or switching the display form of the first conversion panel from the default form to the full-screen form in response to the form switching operation of the user on the maximizing control.

10. The method according to claim 8 or 9, characterized in that, The method further includes: adjusting the display position of the default window or the floating ball window on the screen of the electronic device in response to the moving operation.

11. The method of claim 10, wherein, The method further includes: if the ratio of the area of the invisible region of the default window to the total area is greater than a preset ratio during the movement of the default window, switching the first conversion panel from the default form to the floating ball form, wherein the invisible region is a partial region of the default window beyond the screen boundary.

12. The method of claim 8 or 9, wherein, The method further includes: adjusting the size of the default window in response to the size adjusting operation.

13. The method of any one of claims 1-3, 5, 8-9, or 11, wherein, The method further includes: if a second multimedia file is played and a preset condition is met in the case of closing the first conversion panel, displaying a first prompt information, the first prompt information being used for prompting to open the conversion function of audio data and text data, and displaying the subtitle of the second multimedia file through the first conversion panel; the preset condition includes at least one of the following: the playing volume of the second multimedia file is less than a preset volume, the power of the electronic device is less than a preset power, and the environmental noise collected by the electronic device is greater than a preset decibel.

14. The method of any one of claims 1-3, 5, 8-9, or 11, wherein, The text data corresponding to the first voice adopts any one of the following language types: Chinese, English, Korean and Japanese.

15. The method of any one of claims 1-3, 5, 8-9, or 11, wherein, The method further includes: if a call voice is detected, displaying a third conversion panel and displaying the call text corresponding to the call voice in the third conversion panel.

16. The method of any one of claims 1-3, 5, 8-9, or 11, wherein, The first conversion panel or the second conversion panel includes multiple setting items of font size, transparency, language type of subtitle, language type of dialogue mode, source language type of translation mode, target language type of translation mode, saving panel content, emptying panel content and tone of voice for broadcasting.

17. The method of claim 16, wherein, The method further includes: setting the tone of voice for broadcasting corresponding to the second conversion panel to the tone of voice selected by the user in the setting item of the tone of voice for broadcasting in response to the tone setting operation.

18. An electronic device, comprising: It includes: a screen, a memory and a processor; the screen, the memory are respectively coupled with the processor; the screen is used for displaying an interface; The memory has stored therein computer program code comprising computer instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-17.

19. A computer-readable storage medium, characterized in that, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the method of any one of claims 1-17.

Images