An audio playing method

Abstract

The application provides an audio playing method. The audio playing method comprises the following steps: in response to a calling request of a target application to an audio output device, if the audio output device is in an occupied state and a target condition is met, target audio data of the target application is converted into text data; and the text data is displayed and output.

Description

Technical Field

[0001] This application relates to the field of audio playback technology, and more particularly to an audio playback method. Background Technology

[0002] Currently, the audio output device of electronic devices (such as speakers and headphones) can only be occupied by one application at a time. When an application is already occupying the audio output device, audio playback requests initiated by other applications are often rejected or the existing playback is forcibly interrupted, resulting in users being unable to access audio information content from new applications and a poor user experience. Summary of the Invention

[0003] In view of the above-mentioned technical problems existing in the current technology, this application provides an audio playback method.

[0004] This application provides an audio playback method, including: in response to a call request from a target application to an audio output device, if the audio output device is in an occupied state and the target conditions are met, converting the target audio data of the target application into text data; and displaying the text data.

[0005] In some embodiments, the target condition includes: the target application is an application whose output is displayed in a first display area.

[0006] In some embodiments, the target conditions further include: the application that puts the audio output device in an occupied state is a first application; the first application is an application displayed in a second display area; and the first display area and the second display area are display areas different from the display screen.

[0007] In some embodiments, displaying the text data includes: displaying the text data in the first display area.

[0008] In some embodiments, converting the target audio data of the target application into text data includes: establishing a forwarding path for transmitting the target audio data of the target application to the speech recognition module; and inputting the target audio data into the speech recognition module through the forwarding path to obtain text data corresponding to the target audio data.

[0009] In some embodiments, inputting the target audio data to the speech recognition module via the forwarding path includes: reading the target audio data from a target buffer based on the audio track identifier of the target audio data; the target buffer is used to store audio data of each audio track for the audio output device to read and output; forwarding the target audio data to the speech recognition module; the method further includes: muting the target audio data based on the audio track identifier.

[0010] In some embodiments, the target condition is selected from one of the following combinations: the electronic device is in target mode; an interactive operation is received indicating to output the target audio data in text.

[0011] In some embodiments, the method further includes: obtaining the audio data currently output by the audio output device; if the audio data currently output by the audio output device is audio data not belonging to the target application, determining that the audio output device is in an occupied state; and / or, the audio focus corresponding to the audio output device is held by a non-target application; the audio focus represents the application currently having the highest priority audio output permission.

[0012] In some embodiments, the display output of the text data includes: if the target application is in a first display state, the text data is first text data; if the target application is in a second display state, the text data is second text data; wherein, in the first display state, the display area occupied by the target application is smaller than the display area occupied by the target application in the second display state; and the level of detail of the content represented by the first text data is less than the level of detail of the content represented by the second text data.

[0013] In some embodiments, converting the target audio data of the target application into text data specifically includes: performing human voice detection on the target audio data; and, if the human voice detection result indicates the presence of human speech, converting at least the human speech contained in the target audio data into text data.

[0014] In some embodiments, the target audio data includes first audio data and second audio data; inputting the target audio data to the speech recognition module through the forwarding path to obtain text data corresponding to the target audio data includes: inputting the first audio data contained in the target audio data to the speech recognition module through the forwarding path to obtain text data corresponding to the target audio data; muting the target audio data based on the audio track identifier includes: muting the first audio data contained in the target audio data based on the audio track identifier; or deleting the first audio data contained in the target audio data based on the audio track identifier; wherein, the first audio data is data related to human voice; and the second audio data is data unrelated to human voice.

[0015] In some embodiments, converting the target audio data of the target application into text data includes: determining the type of the target application; and determining a conversion method for converting the audio data into text data based on the type of the target application.

[0016] This application also provides an audio playback device, including a response module and an output module. The response module is used to respond to a call request from a target application to an audio output device. If the audio output device is in an occupied state and the target conditions are met, the response module converts the target audio data of the target application into text data. The output module is used to display the text data. Attached Figure Description

[0017] In drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The drawings generally illustrate various embodiments by way of example rather than limitation and are used, together with the description and claims, to illustrate the disclosed embodiments. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0018] Figure 1 This is a first flowchart of the audio playback method according to an embodiment of this application.

[0019] Figure 2 This is a schematic diagram of the display interface of an electronic device.

[0020] Figure 3 This is a second flowchart of the audio playback method according to an embodiment of this application.

[0021] Figure 4 This is a third flowchart of the audio playback method according to an embodiment of this application.

[0022] Figure 5 This is the fourth flowchart of the audio playback method in an embodiment of this application.

[0023] Figure 6 This is the fifth flowchart of the audio playback method in an embodiment of this application.

[0024] Figure 7 This is the sixth flowchart of the audio playback method in an embodiment of this application.

[0025] Figure 8 This is a structural block diagram of an audio playback device according to an embodiment of this application. Detailed Implementation

[0026] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0027] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0028] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.

[0029] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0030] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0031] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the embodiments are merely examples of this application, which may be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0032] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0033] This application provides an audio playback method applicable to any electronic device capable of running multiple applications simultaneously and equipped with an audio output device, including but not limited to smartphones, tablets, foldable screen terminals, laptops, smart TVs, in-vehicle terminals, and smart wearable devices with displays. It is particularly suitable for terminal devices supporting split-screen mode, multi-window mode, or floating window mode. In scenarios where multiple applications run simultaneously and may concurrently call audio output resources in the aforementioned devices, this method can effectively achieve unified control and collaborative management of audio output.

[0034] like Figure 1 As shown, the audio playback method includes steps S101 and S102.

[0035] Step S101: In response to the target application's request to call the audio output device, if the audio output device is in an occupied state and the target conditions are met, the target audio data of the target application is converted into text data.

[0036] Step S102: Display the output text data.

[0037] The aforementioned audio output device can be the audio playback hardware and supporting system output channel of an electronic device, including but not limited to speakers, external headphones, Bluetooth speakers, etc., which can output audio signals to the outside.

[0038] During the operation of the aforementioned target application, it can send a request to the audio output device to play audio. Target applications include, but are not limited to, video playback applications, audio playback applications, instant messaging applications, conferencing applications, game applications, and live streaming applications.

[0039] The aforementioned audio output device being in an occupied state can be understood as the audio output device being currently occupied by an application other than the target application, that is, it is currently in an audio playback working state and cannot respond to the audio playback request of the target application.

[0040] The target audio data includes, but is not limited to, various playable audio signals such as human voice, sound effects, and background sounds. The target audio data can be stored in the target buffer of the electronic device's system. The target buffer is used to store the audio data of each audio track, which is then read and output by the audio output device.

[0041] The above target conditions can be understood as the scenario constraints for converting the target audio data of the target application into text processing, which will be explained in detail below and will not be repeated here.

[0042] Converting target audio data to text data can involve converting the entire audio data or only a portion of it; there are no specific limitations. Furthermore, the converted text data can be a word-for-word conversion or a conversion of key content.

[0043] Specifically, converting target audio data into text data can be achieved by using Automatic Speech Recognition (ASR) technology to process the speech signals in the target audio data, performing feature extraction, acoustic model matching, and language model decoding, thereby generating text data corresponding to the speech content in the target audio data.

[0044] It is understood that the conversion process from target audio data to text data can be performed based on a locally deployed speech recognition engine, or it can be achieved by sending audio data to a cloud speech recognition service and receiving the returned recognition results. This application does not make any specific limitations on this.

[0045] The above method also includes: if the audio output device is in an occupied state and the target condition is not met, adjusting the state of the audio output device based on the target strategy. Specifically, if it is determined that the target condition is not met, audio-to-text processing is not performed, but the system's default audio conflict handling strategy is followed. For example, the call request of the target application is rejected, the target application is added to the audio playback queue to wait for the currently occupied application to release the audio output device, or multiple audio streams from different applications are mixed and then output simultaneously through the audio output device.

[0046] Optionally, based on the current display area, display status, and application type of the target application, corresponding text display rules can be matched to display the processed text data in a targeted and adaptive manner within the corresponding display area of ​​the electronic device's screen, such as the first display area described below. Thus, through visual text output, the information carried by the target application's audio data is conveyed to the user without occupying the audio output channel or causing audio interference, ensuring that the user can normally access the target application's audio content.

[0047] The format of the text data may include plain text, time-stamped subtitles, or dialogue text with speaker identifiers. This application does not impose specific limitations on this, as long as the content of the target audio data can be clearly displayed.

[0048] This application, in response to a target application's request to call an audio output device, converts the target application's target audio data into text data and displays the output text data if the audio output device is occupied and the target conditions are met. This achieves the goal of not directly playing the target application's target audio data when the audio output device is occupied and the target conditions are met, but instead converting it into text data and visually displaying it. This effectively avoids the problem of audio overlap and mutual interference caused by multiple applications calling the audio output device simultaneously. Furthermore, by converting it into text data, the application can provide users with the content of the target application's target audio data, ensuring that users can normally obtain the content of the target application. This achieves unified control and collaborative management of the audio output of multiple applications.

[0049] In some embodiments, such as Figure 2 As shown, the target condition includes: the target application is the application displayed in the first display area.

[0050] In this way, by limiting the target application to the application displayed in the first display area, the audio-to-text function is precisely triggered. That is, audio-to-text processing is performed on the currently displayed application, avoiding the waste of resources caused by invalid transcription of applications that are not displayed, and ensuring that the triggering of the audio-to-text function matches the user's visual operation scenario.

[0051] Understandably, the target application is currently in a display state, and the converted text data can be directly presented in the corresponding area, allowing users to visually establish an intuitive correspondence between the text content and the application interface, thereby improving the accuracy of information acquisition and ease of operation.

[0052] Specific combination Figure 2 , Figure 2 The image shows the display interface of an electronic device's screen. The first display area in the image can be understood as being used to display the content corresponding to the target application.

[0053] In some embodiments, such as Figure 2 As shown, the target conditions also include: the application that puts the audio output device in an occupied state is a first application; the first application is an application that is displayed and output in the second display area; the first display area and the second display area are display areas different from the display screen.

[0054] Thus, by limiting the audio output device to be occupied by the first application and the first application to be located in the second display area, the audio conflict judgment conditions in the split-screen display scenario are further clarified. This enables the use of multiple windows and multiple applications to run in parallel according to the current application scenario, avoiding the problem of accidental triggering of the audio-to-text function in the single application full-screen scenario. This makes the audio-to-text adaptation scenario more accurate and specifically solves the problem of split-screen audio interference.

[0055] The first display area and the second display area are independent and non-overlapping display areas in the display screen of the electronic device. They are used to present the interfaces of different applications. The first display area and the second display area work together to form a multi-window layout in the split-screen display mode.

[0056] The aforementioned first application and target application are different applications. Specifically, the first application is the one currently occupying and controlling the audio output device, and its user interface is displayed in the second display area of ​​the screen; the target application is the one initiating the request to access the audio output device, and its user interface is displayed in the first display area of ​​the screen. When the first application is running in the second display area and continuously playing audio through the audio output device, the target application in the first display area initiates an audio access request, which identifies an audio resource conflict between two applications in different display areas within a split-screen scenario. In this scenario, the two applications are located in different display areas and are both within the user's visible range. By performing the audio-to-text processing flow of the target application, differentiated and collaborative management of the audio output of applications in the two split-screen display areas can be achieved.

[0057] In some embodiments, such as Figure 1As shown, the display output of the text data includes: displaying the text data in the first display area.

[0058] In this way, by displaying the converted text data in the first display area, the text content is kept consistent with the corresponding display area of ​​the target application, allowing users to directly view the text content corresponding to the audio within the target application's interface. Complete audio information can be obtained without switching windows, improving the convenience of information reading and aligning with users' visual habits of split-screen operation.

[0059] The display area for the aforementioned text data is the current display area of ​​the target application's interface. This display area can be located in any partition of the display screen, including but not limited to the left, right, upper, or lower half, depending on the screen layout. This application does not impose any specific limitations on this. Figure 2 The left-hand area shown is for illustrative purposes only.

[0060] In some embodiments, such as Figure 3 and Figure 4 As shown, step S101, which involves converting the target audio data of the target application into text data, includes steps S201 and S202.

[0061] Step S201: Establish a forwarding path for transmitting the target audio data of the target application to the speech recognition module.

[0062] Step S202: Input the target audio data into the speech recognition module through the forwarding path to obtain text data corresponding to the target audio data.

[0063] In this way, the directional transmission of audio data is achieved through the forwarding path, ensuring that the audio data is input to the speech recognition module completely and stably, providing reliable data support for subsequent text output, and improving the accuracy and processing efficiency of transcription.

[0064] The aforementioned voice recognition module is a functional module that is built into or external to an electronic device and has the ability to convert audio signals into text information.

[0065] The aforementioned forwarding path is a data transmission link built by the system that is independent of the audio playback channel. Through the forwarding path, the target audio data can be accurately input into the speech recognition module to obtain the corresponding text data, thus ensuring the accuracy of the output text data.

[0066] In some embodiments, such as Figure 4 and Figure 5 As shown, step S202, which involves inputting the target audio data to the speech recognition module through the forwarding path, includes steps S301 and S302.

[0067] Step S301: Based on the audio track identifier of the target audio data, read the target audio data in the target buffer; the target buffer is used to store the audio data of each audio track, and is used by the audio output device to read and output.

[0068] Step S302: Forward the target audio data to the speech recognition module.

[0069] The method further includes step S303. Step S303: Based on the audio track identifier, mute the target audio data.

[0070] In this way, the target audio data can be accurately located through the target buffer and audio track identifier, enabling separate processing of audio for a specified application. Furthermore, the target audio data is muted while the audio transcription is completed, avoiding interference caused by the output of the target audio data from the source. At the same time, it does not affect the normal playback of audio data of other applications (such as the first application), thus eliminating interference caused by the simultaneous playback of audio from multiple applications from the root.

[0071] The aforementioned target buffer is a shared memory area within the operating system of an electronic device used to store audio data generated by various applications. Specifically, when any application generates audio data to be played, this audio data is temporarily stored in the target buffer before being read and output by the audio output device. The target buffer can differentiate and store audio data from different applications according to audio track identifiers to ensure the independence of audio data between different applications.

[0072] The aforementioned audio track identifiers are identification markers used to distinguish the audio data streams corresponding to different applications, and they correspond one-to-one with the application's invocation request.

[0073] Mute the target audio data, which can be understood as invalidating the data of the corresponding audio track in the target buffer or setting its playback volume to zero, so that it is no longer read and output by the audio output device.

[0074] like Figure 4As shown, when the target application initiates a call request to the audio output device, the system first determines whether the audio output device is in use and meets the target conditions. If the result is no, the target application's target audio data is played normally. If the result is yes, the conflict interaction selection interface is entered, allowing the user to choose between "text mode" or "mixing mode". When the user selects text mode, a forwarding path is established for the target application's target audio data to be transmitted to the speech recognition module. Based on the audio track identifier of the target audio data, the corresponding target audio data is read from the target buffer. This data is then forwarded to the speech recognition module for speech-to-text processing. Simultaneously, the target audio data is muted based on the same audio track identifier. Finally, the converted text data is output on the display interface. If the user selects mixing mode, the target application's target audio data is mixed with the audio data of the first application and then output through the audio output device. This process further enriches the handling strategies in audio conflict scenarios through user interaction selection, taking into account the user's personalized needs for audio output methods in different scenarios.

[0075] In some embodiments, the target condition is selected from one of the following combinations: The electronic device is in target mode; An interactive operation was received instructing the target audio data to be output as text.

[0076] Thus, by using the electronic device being in target mode or user interaction as the trigger for audio-to-text conversion, the system adds a trigger condition for converting target audio data into text data, enabling it to handle both automatic judgment and user-controlled application scenarios. It can adapt to special silent usage scenarios such as meetings, flexibly meeting diverse and personalized audio output needs of users and expanding the applicability of the solution.

[0077] The aforementioned target mode can be understood as a pre-set mode that prioritizes outputting audio in text form, including but not limited to meeting mode, mute mode, driving mode, or office mode. The target mode represents the user's current specific preferences or constraints regarding audio output behavior.

[0078] For example, in scenarios where the target mode includes a meeting mode, the system can automatically identify the priority differences of the current applications and perform text-to-text processing on the audio data of non-critical target applications. This automatically adapts to the user's needs for muting and noise reduction in the meeting scenario, preventing irrelevant audio from interfering with the meeting process. Non-critical target applications can be understood as applications with lower priority in the current device operating scenario, such as games, short videos, and entertainment news applications. The audio content of these applications is not core information that the user is currently focused on, allowing text transcription to replace audio playback. Meanwhile, the audio of high-priority core applications such as meeting applications is played normally, thus achieving a muting and anti-interference effect while ensuring the normal output of core business audio.

[0079] Users can trigger the conversion of target audio data into text data on demand through interactive operations. These interactive operations can be manually initiated by the user to instruct the audio-to-text output, thereby improving the flexibility of human-computer interaction and user autonomy. Interactive operations can be understood as operation instructions actively input by the user through touchscreens, physical buttons, voice commands, or gestures.

[0080] In some embodiments, the method further includes: Obtain the audio data currently output by the audio output device; if the audio data currently output by the audio output device is not audio data for the target application, determine that the audio output device is in an occupied state; and / or, The audio focus corresponding to the audio output device is held by an application other than the target application; the audio focus represents the application that currently has the highest priority audio output permission.

[0081] In this way, the audio output device can be effectively used to determine whether it is in a occupied state by the audio data and / or audio focus of the audio output device, thereby improving the accuracy of the audio output device status detection, effectively avoiding the problem of misjudging the occupied state, ensuring that the audio-to-text function is accurately started when the audio output device is actually occupied, and improving the reliability of the solution.

[0082] Understandably, the two methods of determining the status based on the audio data currently output by the audio output device and the audio focus can be executed individually or in combination. When the audio focus status fails to be updated in time due to system malfunction, it can still be determined by detecting the actual output audio data, thus improving the robustness and accuracy of occupancy status detection.

[0083] The audio data currently output by the aforementioned audio output device refers to the audio signal processed by the device in real time. By detecting the source of the signal, the application to which the currently output audio belongs can be determined, thereby confirming that the audio output device is in use.

[0084] The aforementioned audio focus is a permission identifier used in the operating system of an electronic device to manage the priority of audio output. The audio focus is dynamically allocated by the system based on the application's call sequence, window focus state, or user settings. The application holding the audio focus has the highest audio output permission.

[0085] In some embodiments, such as Figure 6 As shown, the display output of the text data in step S102 includes steps S401 and S402.

[0086] Step S401: If the target application is in a first display state, the text data is first text data. Step S402: If the target application is in a second display state, the text data is second text data. Wherein, the display area occupied by the target application in the first display state is smaller than the display area occupied by the target application in the second display state; the level of detail of the content represented by the first text data is less than the level of detail of the content represented by the second text data.

[0087] In this way, by dynamically adjusting the level of detail of the text data according to the current display state of the target application, the text output content and the display state of the application window are dynamically adapted. In the small window state, the information is displayed in a concise summary text to save display resources. When the window is enlarged, the complete text content is displayed, taking into account both information integrity and interface aesthetics. This adapts to the usage scenario of dynamically changing split-screen window size, achieving the goal of adaptive matching between visualization output and display space.

[0088] Understandably, text data can be output differently depending on the display status of the target application. The first display status of a small display area corresponds to the first text data with low detail, and the second display status of a large display area corresponds to the second text data with high detail. That is, the larger the display area, the higher the application display priority and the more complete the corresponding text content.

[0089] The first and second display states mentioned above correspond to different window sizes of the target application on the display screen. In the first display state, the display area occupied by the target application is smaller than in the second display state. The first display state can be implemented as a floating window mode or a thumbnail window mode in a split-screen environment, while the second display state can be implemented as a half-screen split-screen mode or a full-screen display mode. The first and second text data are text content with different levels of detail obtained by converting the same audio data.

[0090] In some embodiments, such as Figure 7 As shown, step S101, which involves converting the target audio data of the target application into text data, specifically includes steps S501 and S502.

[0091] Step S501: Perform human voice detection on the target audio data.

[0092] Step S502: If the voice detection result indicates the presence of human speech, at least the human speech contained in the target audio data is converted into text data. Here, human speech refers to acoustic signals produced by human vocal organs that have the function of transmitting language information, and is distinct from audio with no informational value, such as pure music, short prompts, and environmental noise.

[0093] Thus, by performing human voice detection on the target audio data before audio-to-text conversion, effective human voice audio is accurately selected for transcription, while audio with no informational value, such as instrumental music, short prompts, and environmental noise, is discarded. This reduces invalid speech transcription operations, focuses on the user's core voice information needs, and improves the effectiveness of text output.

[0094] The aforementioned human voice detection can be understood as the process of identifying and determining whether there are human voice segments in the target audio data through audio signal processing algorithms. It can be implemented by deep learning acoustic models and other methods. This application does not make specific limitations on this, as long as it can accurately detect human voices.

[0095] In some embodiments, the target audio data includes first audio data and second audio data. Step S202, which involves inputting the target audio data to the speech recognition module via the forwarding path to obtain text data corresponding to the target audio data, includes: inputting the first audio data contained in the target audio data to the speech recognition module via the forwarding path to obtain text data corresponding to the target audio data.

[0096] The step S303 of silencing the target audio data based on the audio track identifier includes: silencing the first audio data contained in the target audio data based on the audio track identifier; or deleting the first audio data contained in the target audio data based on the audio track identifier; wherein the first audio data is data related to human voice; and the second audio data is data unrelated to human voice.

[0097] In this way, by separating the target audio data into first audio data and second audio data according to human voice and non-human voice, differentiated processing of audio content is achieved. By eliminating core speech interference through human voice to text conversion, this differentiated processing method not only ensures that users can obtain core semantic content through text, but also retains a more flexible playback strategy for non-critical second audio data such as background sound effects, thereby improving the refinement of audio processing and the richness of user experience.

[0098] The aforementioned second audio data, being a non-human voice portion, does not require transcription and can be flexibly processed according to actual playback needs. For example, it can be retained as background sound effects during mixing output, or it can be directly deleted without playback.

[0099] For example, the target audio data is divided into first audio data related to human voices and second audio data without human voices. Only the first audio data is converted to text and muted or deleted, while the second audio data is retained for normal output. If there is a conflict, short prompts can be allowed to be played directly, while continuous audio or human voice segments are triggered to convert to text.

[0100] In some embodiments, step S101, which involves converting the target audio data of the target application into text data, includes: determining the type of the target application; and determining a conversion method for converting the audio data into text data based on the type of the target application.

[0101] In this way, by determining the audio-to-text conversion method based on the type of the target application, the conversion strategy is accurately adapted to different application scenarios.

[0102] The above application types can be understood as being categorized according to the application's category attributes, including but not limited to conferencing and communication, short video entertainment, live interactive, online education, and media playback.

[0103] The above conversion methods can be understood as the recognition strategies and output format configurations used by the speech recognition module when converting audio to text. Different conversion methods differ in terms of recognition accuracy, response speed, and text output format. Specifically, the above conversion methods include one of the following: word-for-word transcription, keyword transcription, and scrolling subtitle transcription.

[0104] For example, for conference communication applications, high-precision word-for-word transcription ensures the integrity and rigor of meeting content, meeting the stringent accuracy requirements of business scenarios. For short video applications, summarizing or using keywords captures the essence of the video content, adapting to the characteristics of this type of content: high timeliness, dense information, and fast user browsing speed. For live streaming applications, scrolling text transcription enables real-time continuous updates, highly matching the dynamic nature of live content, thereby significantly improving the usability of text output and the naturalness of visual presentation in different scenarios.

[0105] This application also provides an audio playback device 100. For example... Figure 8As shown, the audio playback device 100 includes a response module 101 and an output module 102. The response module 101 is used to respond to a call request from a target application to the audio output device. If the audio output device is in use and the target conditions are met, the response module 101 converts the target audio data of the target application into text data. The output module 102 is used to display the text data.

[0106] This application, in response to a target application's request to call an audio output device, converts the target application's target audio data into text data and displays the output text data if the audio output device is occupied and the target conditions are met. This achieves the goal of not directly playing the target application's target audio data when the audio output device is occupied and the target conditions are met, but instead converting it into text data and visually displaying it. This effectively avoids the problem of audio overlap and mutual interference caused by multiple applications calling the audio output device simultaneously. Furthermore, by converting it into text data, the application can provide users with the content of the target application's target audio data, ensuring that users can normally obtain the content of the target application. This achieves unified control and collaborative management of the audio output of multiple applications.

[0107] In some embodiments, the target condition includes: the target application is an application whose output is displayed in a first display area.

[0108] In some embodiments, the target conditions further include: the application that puts the audio output device in an occupied state is a first application; the first application is an application displayed in a second display area; and the first display area and the second display area are display areas different from the display screen.

[0109] In some embodiments, the output module 102 is further configured to: display the text data in the first display area.

[0110] In some embodiments, the response module 101 is further configured to: establish a forwarding path for transmitting the target audio data of the target application to the speech recognition module; and input the target audio data to the speech recognition module through the forwarding path to obtain text data corresponding to the target audio data.

[0111] In some embodiments, the response module 101 is further configured to: read the target audio data from a target buffer based on the audio track identifier of the target audio data; the target buffer is used to store audio data of each audio track for the audio output device to read and output; forward the target audio data to the speech recognition module; and mute the target audio data based on the audio track identifier.

[0112] In some embodiments, the target condition is selected from one of the following combinations: the electronic device is in target mode; an interactive operation is received indicating to output the target audio data in text.

[0113] In some embodiments, the audio playback device 100 further includes an acquisition module and a determination module. The acquisition module is configured to acquire the audio data currently output by the audio output device, and the determination module is configured to determine that the audio output device is in an occupied state if the audio data currently output by the audio output device is audio data not belonging to the target application; and / or, the determination module is further configured to determine that the audio focus corresponding to the audio output device is held by an application other than the target application; the audio focus represents the application currently having the highest priority audio output permission.

[0114] In some embodiments, the output module 102 is further configured to: if the target application is in a first display state, the text data is first text data; if the target application is in a second display state, the text data is second text data; wherein, in the first display state, the display area occupied by the target application is smaller than the display area occupied by the target application in the second display state; and the level of detail of the content represented by the first text data is less than the level of detail of the content represented by the second text data.

[0115] In some embodiments, the response module 101 is further configured to: perform human voice detection on the target audio data; and, if the human voice detection result indicates the presence of human voice speech, at least convert the human voice speech contained in the target audio data into text data.

[0116] In some embodiments, the target audio data includes first audio data and second audio data. The response module 101 is further configured to: input the first audio data included in the target audio data to the speech recognition module via the forwarding path to obtain text data corresponding to the target audio data; mute the first audio data included in the target audio data based on the audio track identifier; or delete the first audio data included in the target audio data based on the audio track identifier; wherein the first audio data is data related to human voice; and the second audio data is data unrelated to human voice.

[0117] In some embodiments, the response module 101 is further configured to: determine the type of the target application; and determine the conversion method for converting the audio data into text data based on the type of the target application.

[0118] Note that the various units in the embodiments of this application can be implemented as computer-executable instructions stored in memory, which, when executed by a processor, can perform corresponding steps; they can also be implemented as hardware with corresponding logical computing capabilities; or they can be implemented as a combination of software and hardware (firmware). In some embodiments, the processor can be implemented as any of an FPGA, ASIC, DSP chip, SOC (System-on-a-Chip), MPU (e.g., but not limited to Cortex), etc. The processor can be communicatively coupled to the memory and configured to execute computer-executable instructions stored therein. The memory can include read-only memory (ROM), flash memory, random access memory (RAM), dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM, static memory (e.g., flash memory, static random access memory), etc., on which computer-executable instructions are stored in any format. The computer-executable instructions can be accessed by the processor, read from the ROM or any other suitable storage location, and loaded into the RAM for the processor to execute, to implement the wireless communication methods according to the embodiments of this application.

[0119] It should be noted that in the system of this application, the components are logically divided according to the functions they are to perform. However, this application is not limited to this and can re-divide or combine the components as needed. For example, some components can be combined into a single component, or some components can be further decomposed into more sub-components.

[0120] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the system according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such a program implementing this application can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form. Furthermore, this application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0121] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on this application that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, and such examples will be interpreted as non-exclusive.

[0122] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments can be used by those skilled in the art when reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the application. This should not be construed as an intention that a disclosed feature not claimed is necessary for any claim. Rather, the subject matter of the application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated as being able to be combined with each other in various combinations or arrangements. The scope of this application should be determined by reference to the appended claims and the full scope of their equivalents.

[0123] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. An audio playback method, comprising: In response to a call request from a target application to an audio output device, if the audio output device is in use and the target conditions are met, the target audio data of the target application is converted into text data. The text data will be displayed and output.

2. The audio playback method according to claim 1, wherein the target conditions include: The target application is the application displayed in the first display area.

3. The audio playback method according to claim 2, wherein the target condition further includes: The application that puts the audio output device in an occupied state is the first application; The first application is the application whose output is displayed in the second display area; The first display area and the second display area are different display areas of the display screen.

4. The audio playback method according to claim 2 or 3, wherein the display output of the text data includes: The text data is displayed in the first display area.

5. The audio playback method according to claim 1, wherein converting the target audio data of the target application into text data includes: Establish a forwarding path for the target audio data of the target application to the speech recognition module; The target audio data is input to the speech recognition module through the forwarding path to obtain text data corresponding to the target audio data.

6. The audio playback method according to claim 5, wherein inputting the target audio data to the speech recognition module through the forwarding path includes: Based on the audio track identifier of the target audio data, the target audio data is read from the target buffer. The target buffer is used to store the audio data of each audio track, which is then read and output by the audio output device. The target audio data is forwarded and input to the speech recognition module; The method further includes: Based on the audio track identifier, the target audio data is muted.

7. The audio playback method according to claim 1, wherein the target condition is selected from one of the following combinations: The electronic device is in target mode; An interactive operation is received instructing the target audio data to be output as text.

8. The audio playback method according to claim 1, further comprising: Obtain the audio data currently output by the audio output device; if the audio data currently output by the audio output device is not the audio data of the target application, determine that the audio output device is in an occupied state. And / or, The audio focus corresponding to the audio output device is held by an application other than the target application; the audio focus represents the application that currently has the highest priority audio output permission.

9. The audio playback method according to claim 1, wherein the display output of the text data includes: If the target application is in a first display state, the text data is the first text data; If the target application is in a second display state, the text data is second text data; In the first display state, the display area occupied by the target application is smaller than that occupied by the target application in the second display state; the level of detail of the content represented by the first text data is less than that of the content represented by the second text data.

10. The audio playback method according to claim 1, wherein converting the target audio data of the target application into text data comprises: Determine the type of the target application; The conversion method for converting the audio data into text data is determined based on the type of the target application.

Images