Information processing method and electronic equipment

Abstract

The invention discloses an information processing method and electronic equipment. The method comprises the steps of obtaining first audio data; transmitting the first audio data to a second device through a data channel so as to output the first audio data through the second device; wherein the data channel is a data channel established with the second device before the first audio data is transmitted; obtaining a playing request for the second audio data; and in response to the playing request, if the first audio data is transmitted to the second equipment through the data channel to enable the second equipment to output audio, maintaining the transmission of the first audio data through the data channel and outputting the second audio data through the audio.

Description

Technical Field

[0001] The present application relates to the field of information processing, and in particular to an information processing method and electronic equipment. Background Art

[0002] In the related art, for electronic devices such as mobile phones and computers, when playing original audio data, if there is a request to play new audio, the new audio is usually played successfully by stopping the playback of the original audio and playing the new audio, and it is impossible to play the original audio and the new audio separately. Summary of the invention

[0003] The present application provides an information processing method and an electronic device.

[0004] According to a first aspect of the present application, there is provided an information processing method, the method comprising:

[0005] obtaining first audio data;

[0006] Transmitting the first audio data to the second device through a data channel, so as to output the first audio data through the audio of the second device; wherein the data channel is a data channel established with the second device before transmitting the first audio data;

[0007] Obtaining a playback request for second audio data;

[0008] In response to the play request, if the first audio data is transmitted to the second device through the data channel to enable the second device to output audio, the first audio data is maintained to be transmitted through the data channel and the second audio data is outputted audio.

[0009] In one possible implementation, maintaining the first audio data to be transmitted through the data channel includes:

[0010] It is prohibited to notify the first application of the existence of the playback request of the second application to maintain the continuous transmission of the first audio data to the second device through the data channel, wherein the first audio data is associated with the first application; the second audio data is associated with the second application, and the first application is different from the second application.

[0011] In one embodiment, the method further comprises:

[0012] obtaining a target instruction for the first application;

[0013] In response to the target instruction, draw an interactive interface of the first application, and transmit the image of the interactive interface in a streaming media form through the data channel to the second device for display output;

[0014] in,

[0015] The obtaining of the first audio data comprises:

[0016] The first audio data is obtained in response to an interaction instruction, where the interaction instruction is determined based on input operation information received by the data channel for an image of an interaction interface of the first application displayed and output by the second device.

[0017] In one possible implementation, obtaining the first audio data includes:

[0018] In response to the target instruction, the image data and the audio data of the first file associated with the first application are parsed, and the image data and the audio data are transmitted to the display output and the audio output of the second device through the data channel in the form of streaming media.

[0019] In one embodiment, in response to the play request, if the first audio data is transmitted to the second device through the data channel so that the second device outputs audio, maintaining the transmission of the first audio data through the data channel and audio outputting the second audio data includes:

[0020] In response to the play request, query a target application currently outputting audio, the first application belonging to the target application;

[0021] Obtaining the audio output device identifier of the target application;

[0022] If the audio output device identifier of the target application does not match the target device identifier, the first audio data is maintained to be transmitted through the data channel and the second audio data is audio-outputted.

[0023] In one possible implementation, if the audio output device identifier of the target application matches the target device identifier, at least the application with the target device identifier is notified of the presence of the play request of the second application.

[0024] In one possible implementation, if the request to play the second audio data comes from the first application, it includes one of the following: maintaining the transmission of the first audio data through the data channel and transmitting the second audio data through the data channel; or, transmitting the second audio data through the data channel instead of transmitting the first audio data.

[0025] According to a second aspect of the present application, an electronic device is provided, including:

[0026] A first obtaining unit, configured to obtain first audio data;

[0027] A transmission unit, configured to transmit the first audio data to a second device through a data channel, so as to output the first audio data through the audio of the second device; wherein the data channel is a data channel established with the second device before transmitting the first audio data;

[0028] A second obtaining unit, configured to obtain a play request for second audio data;

[0029] A response unit is used to respond to the play request, if the first audio data is transmitted to the second device through the data channel so that the second device outputs audio, maintain the first audio data transmitted through the data channel and audio output the second audio data.

[0030] In one possible implementation manner, the response unit is used to:

[0031] It is prohibited to notify the first application of the existence of the playback request of the second application to maintain the continuous transmission of the first audio data to the second device through the data channel, wherein the first audio data is associated with the first application; the second audio data is associated with the second application, and the first application is different from the second application.

[0032] In one possible implementation manner, the first obtaining unit is further configured to:

[0033] obtaining a target instruction for the first application;

[0034] In response to the target instruction, draw an interactive interface of the first application, and transmit the image of the interactive interface in a streaming media form through the data channel to the second device for display output;

[0035] In response to an interaction instruction, obtaining the first audio data, wherein the interaction instruction is determined based on input operation information received by the data channel for an image of an interaction interface of the first application displayed and output by the second device;

[0036] Alternatively, the first obtaining unit is further used for:

[0037] In response to the target instruction, the image data and the audio data of the first file associated with the first application are parsed, and the image data and the audio data are transmitted to the display output and the audio output of the second device through the data channel in the form of streaming media.

[0038] According to a third aspect of the present application, an electronic device is provided, including:

[0039] at least one processor; and

[0040] a memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0042] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0043] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, which implement the method described in the present application when executed by a processor.

[0044] By utilizing the technical solution of the present application, the original audio and the new audio can be played separately.

[0045] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0047] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0048] Figure 1 The schematic diagram of the implementation process of the information processing method in the embodiment of the present application is shown Figure 1 ;

[0049] Figure 2 The schematic diagram of the implementation process of the information processing method in the embodiment of the present application is shown Figure 2 ;

[0050] Figure 3 A block diagram showing an implementation of the information processing method in an embodiment of the present application is shown;

[0051] Figure 4 A schematic diagram of the structure of an electronic device in an embodiment of the present application is shown;

[0052] Figure 5 A schematic diagram of the structure of another electronic device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0053] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0054] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0055] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] It should be understood that in the various embodiments of the present application, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0059] In actual applications, an application scenario A in which two audios (original audio and new audio) cannot be played separately is: when a mobile phone is casting a video such as a movie to a screen projection device such as a TV, laptop, or desktop for playback, if there is an incoming call request, the mobile phone stops playing the movie by playing the incoming call ringtone to remind the user of the incoming call. In this application scenario, the movie audio can be used as the original audio, and the incoming call sound can be used as the new audio. This application can be used to play the two audios separately, avoiding the bad user experience for other viewers other than the owner of the mobile phone due to only playing the incoming call ringtone and being unable to continue playing the movie.

[0060] The technical solution of the present application relates to a first device and a second device. The first device may be a device such as a mobile phone, a personal digital assistant (PAD), a tablet computer, a laptop computer, etc. The second device may be a device capable of supporting audio output, such as an audio player, a television (TV), a laptop computer, a desktop computer, etc. The information processing method of the present application is applied to the first device.

[0061] Figure 1 The schematic diagram of the implementation process of the information processing method in the embodiment of the present application is shown Figure 1 .like Figure 1 As shown, the method includes:

[0062] S (Step) 101: Obtain first audio data.

[0063] The present application involves two audio data: first audio data and second audio data. The first audio data can be used as the original audio, and the second audio data can be used as the new audio. The first audio data can be sound data such as music, sound data accompanying a video such as the sound of a video screen, or any other reasonable sound that needs to be output.

[0064] Exemplarily, taking video as an example, the first device reads a video file, parses or extracts the data of the video sound, and uses the video sound data or the data obtained by encoding the video sound as the first audio data. Taking audio as an example, the first device reads an audio file, obtains the sound data in the audio file, uses the sound data in the audio file as the first audio data, or uses the data obtained by encoding the sound data in the audio file as the first audio data. It can be understood that the first audio data in this application is not an audio file or a video file with audio, but sound data.

[0065] S102: Transmitting the first audio data to the second device via a data channel, so as to output the first audio data via the audio of the second device; wherein the data channel is a data channel established with the second device before transmitting the first audio data.

[0066] In the present application, the first device can establish a data channel with the second device to realize the transmission of audio data. If the second device is an audio player such as a speaker, a sound box, etc., the data channel can be a Bluetooth or WiFi channel between the two devices. If the second device is a TV, a desktop computer, etc., the data channel can be a wireless network or a wired network channel established between the two devices.

[0067] The first device transmits the first audio data, which is sound data, to the second device through the data channel. The second device can play the first audio data, which is sound data. Yes, the first device transmits the encoded data of the first audio data to the second device through the data channel. After decoding, the second device plays the first audio data, which is sound data. At the two device ends of the first device and the second device, the first audio data is transmitted by the first device and the sound is played at the second device end.

[0068] S103: Obtain a play request for the second audio data.

[0069] In the present application, similar to the first audio data, the second audio data is also a kind of sound data. For example, the second audio data can be an incoming call sound, music sound, video sound, or any other reasonable sound, but not an audio file or a video file with audio. When the first device detects that the second audio data needs to be played, a play request for the second audio data is generated. For example, a play request for the incoming call sound, a play request for the music sound.

[0070] S104: In response to the play request, if the first audio data is transmitted to the second device through the data channel to enable the second device to output audio, maintain the transmission of the first audio data through the data channel and output the second audio data.

[0071] In this step, the electronic device responds to the play request. When there is a play request, if the first audio data is still output by the second device, the first device continues to transmit the first audio data to the second device through the data channel so that the second device continues to output the first audio data, and the first device outputs the second audio data.

[0072] Among them, in the case of a play request, if the first audio data is still output by the second device, it means that the first audio data in the case of a play request does not need to be output by the first device, and the audio output unit of the first device such as the speaker is not occupied, then the second audio data can be output or played by the first device. In this way, the original audio and the new audio are played separately without interfering with each other. The problem in the related art that the original audio cannot be played or the playback is interrupted due to the occupation of the new audio for playback is avoided. The separate playback of the original audio and the new audio in the S101~S104 schemes provides the audience users of the original audio and the audience users of the new audio with their own good user experience.

[0073] It can be understood that some applications (programs) in the first device are sound-related applications. For example, music applications, video applications, social applications, telephone applications, etc. The two audio data (first audio data and second audio data) in this application can be sound data from these applications. Furthermore, the first audio data and the second audio data are sound data from the same application in these applications. For example, the first audio data is song 1 from a music application, and the second audio data is song 2 from a music application. The first audio data and the second audio data are sound data from different applications in these applications. For example, the first audio data is the sound from a video application, and the second audio data is the incoming call sound from a telephone application.

[0074] In the case where two audio data are derived from different applications, that is, the first application is different from the second application, the first audio data is associated with the first application, and the second audio data is associated with the second application. When there is a play request for the second audio data, if the first audio data is transmitted to the second device through the data channel to enable the second device to output audio, then the scheme of maintaining the transmission of the first audio data through the data channel is implemented by the first device using the following scheme: prohibiting notification, not notifying or intercepting notification of the existence of the play request of the second application to the first application, so as to maintain the continuous transmission of the first audio data to the second device through the data channel.

[0075] In this application, considering the competition mechanism between applications for the right to output audio, if the first application is notified of the presence of a playback request from the second application, the first device must terminate the playback of the original audio from the first application in order to play the new audio from the second application. In order to avoid terminating the playback of the original audio and to achieve the separate playback of the original audio and the new audio, in this application, when there is a playback request from the second application, the first application is not notified of the presence of other applications' playback requests for audio data, so that the original audio from the first application is continuously transmitted to the second device through the data channel, the second device continues to play the original audio, and the first device plays the new audio from the second application. In this way, the original audio and the new audio are played separately without affecting or interfering with each other.

[0076] In the case where the two audio data come from two different applications, the solution of obtaining the first audio data in S101 in the present application can be implemented by using method one or method two.

[0077] Combination Figure 2 As shown, method one:

[0078] S1010: Obtain a target instruction for the first application.

[0079] The first application in this application is an application that can support screen projection, or an application that can display its application content through screen projection. For the convenience of description, such an application is referred to as a screen projection application. The target instruction may be a screen projection instruction for the first application. When a screen projection operation on the first application is detected, a target instruction is generated. In this application, the first application may be a social application, such as WeChat, QQ, Weibo, etc.

[0080] S1011: In response to the target instruction, draw an interactive interface of the first application, and transmit the image of the interactive interface in the form of streaming media through the data channel to the second device for display output.

[0081] In this step, in response to the target instruction, the interactive interface of the first application is drawn, and the drawn interactive interface is rendered by the graphics card to obtain a frame image of the interactive interface. The frame image is transmitted to the second device through the data channel in the form of streaming media to display the image of the interactive interface on the second device side. In this way, the interactive interface of the application on the first device side is displayed on the second device side.

[0082] Streaming media is a form of media that transmits audio, video or multimedia content in real time over the network. Users can watch or listen to the content as soon as they receive the data, without having to wait for all the content to be downloaded. Common formats of streaming media include MP4, FLV, and RTMP.

[0083] For example, taking the first application as WeChat application, the above scheme can realize the display of the image of the WeChat interface of the first device on the second device side. That is, the WeChat interface is displayed on the projection side (second device side) in the form of an image. This display can be regarded as a projection display of the WeChat interface.

[0084] S1012: In response to an interaction instruction, obtaining first audio data, where the interaction instruction is determined based on input operation information of an image of an interaction interface of a first application program displayed and outputted by a second device and received through a data channel.

[0085] If the image of the interactive interface of the first application outputted by the second device contains audio data that can be listened to by the user, and the user clicks on the audio data, the second device detects the click and generates an interactive instruction. The second device transmits the interactive instruction and the operation position generated by the user in the image displayed by the second device to the first device through a data channel. The first device responds to the interactive instruction, obtains the mapping position of the aforementioned operation position in the interactive interface drawn by itself, and uses the audio that appears at the mapping position in the interactive interface drawn by itself as the audio that the user wants to listen to on the second device. The audio that the user wants to listen to on the second device can be used as the first audio data.

[0086] In S1010 to S1012, for a first application supporting screen projection, its interface image can be projected from the first device side to the second device side for display. The interface image displayed on the second device side can be operated by the user, and based on the user operation, the first audio data that can be transmitted to the second device side for playback and that the user wants to listen to can be obtained.

[0087] The present solution is described below in conjunction with application scenario B. For example, taking the first application as WeChat application and the second application as a song application as an example, application scenario B is:

[0088] The first device projects the interactive interface of the WeChat application to the second device in the form of an image. Assuming that a certain projection image is an image containing an audio data a to be played, when the first device transmits the image to the second device through a data channel in the form of streaming media, the second device will display the image containing the audio data a to be played. Because the display of the audio data a to be played on the second device indicates that it has not been played, when the user wants to play the audio data a to be played, the audio data a to be played in the image presented by the second device is selected. The second device detects the selection operation and generates an interaction instruction. The second device transmits the interaction instruction and the operation position of the user's selection operation in the image presented by the second device to the first device through the data channel. Since the image size displayed by the second device and the operation position of the user on the second device are known, the image size of the interactive interface drawn by the first device and the image size displayed by the second device are also known, so it is easy to know the mapping position of the operation position of the selection operation generated by the user on the second device in the interactive interface of the first device. The first device uses the audio data that appears at the mapping position in its interactive interface as the audio data that the user expects to play. The first device transmits the audio data that the user wishes to play to the second device through a data channel, and the second device plays the audio data, thereby achieving a good user experience of click-and-play on the second device.

[0089] In the above scheme, the first device projects the image of the application interface that can be projected on the second device side. For the operation generated in the projected image on the second device side, the first device can obtain the audio data that the user wants to hear on the second device side through the operation, and transmit the audio to the second device side through the data channel, so that the user can successfully listen to it on the second device side. It provides a technical support for the successful output of application audio on the projection end for the screen-projectable application.

[0090] When the user listens to the audio data of the screen-casting application on the second device, if there is an incoming call request in the phone application (a request to play the second audio data), the first device responds to the incoming call request without notifying the screen-casting application of the phone application's play request, so that the unplayed audio data in the screen-casting application can be continuously transmitted to the second device through the data channel, so that the unplayed audio data in the screen-casting application can be played normally to the user on the second device. At the same time, the audio output unit of the first device is used to play the incoming call sound. In layman's terms, this application scenario solution realizes the separate playback of the audio data of the screen-casting application and the incoming call ringtone, which improves the user experience.

[0091] Method 2: In response to the target instruction, the image data and audio data of the first file associated with the first application are parsed, and the image data and audio data are transmitted to the display output and audio output of the second device through the data channel in the form of streaming media.

[0092] In the second method, the first application may be a screen-casting application, such as a video application. The first file may be a video file including images and audio displayed by the first application. In the second method, the image screen part and the picture sound (audio) part of the video file can be separated by parsing the video file from the first application. For the separated image screen part, the first device can transmit it to the second device through a data channel in the form of streaming media, so that the second device can display the image screen part in the video file. For the classified picture sound part, the first device can also transmit it to the second device through a data channel in the form of streaming media, so that the second device can output the audio part of the video file. Considering that in a video file, the image screen and the picture sound are usually synchronized, based on this, in the present application, the image screen and the picture sound that need to be output synchronously can be transmitted to the second device together in the form of streaming media through a data channel, so that when the second device displays the image screen, the sound synchronized with the image screen can be synchronously output.

[0093] It can be understood that in the second method, the transmission of the image from the first device to the second device and the display on the second device is a screen projection solution. The first application can be regarded as an application that can project its associated files for display. The target instruction can be an instruction generated when there is a screen projection demand for the video file associated with the first application. For example, a screen projection operation is generated for the video file, and the first device detects the operation and generates a screen projection instruction, which can be used as a target instruction.

[0094] The aforementioned method 2 can solve the problem in the aforementioned application scenario A that only the incoming call ringtone can be played but the movie cannot be played.

[0095] Exemplarily, taking the case where the first application is a video application, the second application is a telephone application, the first file is a movie, and the first audio data is the audio data of the first file, as an example, the first device detects that there is a screen projection operation on a movie file in the video application, and generates a screen projection instruction to project the movie file to the second device. The first device responds to the screen projection instruction, parses the movie file, and separates the image part and the picture sound part. The image part and the picture sound part synchronized with the image part are transmitted to the second device through the data channel in the form of streaming media, so that when the second device displays the image, the sound synchronized with the image is played synchronously. In this way, the screen projection playback of the movie on the second device is realized.

[0096] During the movie screen casting process, if there is an incoming call request (a request for playing the second audio data) from the phone application, the first device responds to the incoming call request without notifying the video application of the phone application's play request, so that the image and sound in the movie are continuously transmitted to the second device through the data channel, and the movie is continuously played on the second device. At the same time, the audio output unit of the first device is used to play the incoming call sound.

[0097] If the first device is regarded as the local end and the second device is regarded as the opposite end. In layman's terms, the solution shown in the above application scenario can realize the normal playback of the incoming call ringtone on the local end while the movie on the local end is being cast on the opposite end. In other words, it is possible to make a call on the local end and watch a movie on the cast screen at the same time without conflict. It improves the user experience of the local end owner and other viewers.

[0098] In some embodiments, in response to the play request, if the first audio data is transmitted to the second device through the data channel so that the second device outputs audio, the scheme of maintaining the transmission of the first audio data through the data channel and audio outputting the second audio data can be implemented as follows:

[0099] In response to a play request, query the target application currently outputting audio, the first application belongs to the target application; obtain the audio output device identifier of the target application; if the audio output device identifier of the target application does not match the target device identifier, maintain the first audio data transmission through the data channel and output the second audio data. If the audio output device identifier of the target application matches the target device identifier, at least notify the application with the target device identifier that there is a play request of the second application.

[0100] It can be understood that the audio output device can be a device, apparatus or unit capable of realizing audio playback, such as a speaker, earphone, speaker, etc. The audio data is outputted by the audio output device. The audio output device that is currently outputting audio can be regarded as an occupied audio output device. The audio output device identifier can be the identifier of the device where the audio output device is located or the ID of the audio output device. For example, the audio output device identifier of the first application is the identifier of the second device where the audio output device is located. The target device is the first device or the audio output device of the first device. The target device identifier is the first device identifier or the audio output device identifier of the first device.

[0101] In the case where the first audio data is transmitted from the first device side to the second device side through the data channel for audio output on the second device side, if there is a request to play the second audio data, the (target) application that is performing audio output is queried, and the audio output device ID of such an application is read. Because the first audio data from the first application is transmitted to the second device side for playback when there is a playback request, the first application belongs to the application that is performing audio output. For example, if the application that is performing audio output is queried as the first application, and the audio output device identifier of the first application is the second device identifier, the second device identifier is a different identifier from the first device identifier, that is, the audio output device identifier of the application that is performing audio output does not match the target device identifier. This indicates that the audio output device of the first device is not currently occupied by the application that is performing audio output, and the audio data from the second application can be played through the unoccupied audio output device of the first device.

[0102] Taking the query that the applications that are currently performing audio output include the first application and the third application, the audio played by the third application is the third audio data, and the third audio data is transmitted to the third device using another channel different from the data channel of the first device to make the third device output audio (the third application and the third audio data can refer to the description of the first application and the first audio data for a similar understanding), as an example, the audio output device identifier of the third application is the third device identifier. The first application and the third application are currently performing audio output, and the audio output device identifiers of the applications currently performing audio output are the second device identifier and the third device identifier. It can be seen that the audio output device identifiers of the applications currently performing audio output - the second device identifier and the third device identifier, are different from the first device identifier. That is, the audio output device identifier of the application currently performing audio output does not match the target device identifier. This indicates that the audio output device of the first device is not currently occupied by the application that is currently performing audio output, and the audio data from the second application can be played through the audio output device of the first device that is not occupied.

[0103] Taking the query that the applications currently performing audio output include the first application and the third application, the audio played by the third application is the third audio data, and the third audio data is played by the audio output device of the first device as an example, after querying, it is found that the first application and the third application are currently performing audio output, and the audio output device identifiers of the applications currently performing audio output are the second device identifier and the first device identifier. In the audio output device identifier of the application currently performing audio output, there is an identifier that is the same as the first device identifier. That is, the audio output device identifier of the application currently performing audio output matches the target device identifier. It is explained that the audio output device of the first device is currently occupied by the application performing audio output, then the third application occupying the audio output device of the first device is notified that there is currently another application-the second application that wants to occupy the audio output device of the first device, release the occupation of the audio output device of the first device by the third application, and allocate the occupation right of the audio output device of the first device to the second application. Among them, the transfer of the occupation right from the third application to the second application can be considered as: the audio output device of the first device stops playing the third audio data and starts playing the second audio data from the second application.

[0104] It can be seen from the above scheme that if it is found that the application that is currently outputting audio is a screen-casting application, the audio of the screen-casting application can be cast and played, and does not occupy the audio output device on this side. Then, the second audio data requested to be played by the second application can be played on this side. Thereby, different audio data from different applications can be played independently of each other. If it is found that among the applications that are currently outputting audio, there are applications that use the audio output device on this side for audio playback, and there are applications that occupy the audio output device on this side, then in response to the second application's request to play the second audio data, the right to occupy the audio output device on this side will be transferred to the second application, so that the second audio data can be played on this side.

[0105] In the present application, the request for playing the second audio data may come from a second application different from the first application, or from an application the same as the first application, or from the first application itself. If the request for playing the second audio data comes from the first application, it is possible to maintain the transmission of the first audio data through the data channel and transmit the second audio data through the data channel. Alternatively, the second audio data is transmitted through the data channel instead of the first audio data.

[0106] In layman's terms, if the request to play the second audio data comes from the first application itself, when the first device continues to transmit the first audio data to the second device through the data channel, the second audio data is also transmitted to the second device through the data channel. The second device can output two types of audio data. Alternatively, the first device stops transmitting the first audio data, and transmits the second audio data to the second device through the data channel instead of the first audio output. The second device can stop playing the first audio data and play the second audio data.

[0107] Exemplarily, taking the first application as a music application that can cast the screen - Application A as an example, if Song 1 is the first audio data and is transmitted from the first device side to the second device side through the data channel for playback, there is a request for Application A to play the audio data of the accompaniment of Song 1. Then, when Song 1 is continuously transmitted to the second device side through the data channel for playback, the accompaniment of Song 1 can also be transmitted to the second device side through the data channel. The second device plays Song 1 and the accompaniment of Song 1. Alternatively, the accompaniment of Song 1 can be transmitted to the second device through the data channel instead of Song 1. The second device stops playing Song 1 and starts playing the accompaniment of Song 1.

[0108] It can be understood that the first audio data from the first application can be transmitted to the second device through the data channel for playback. For a screen-casting application such as the first application, the audio from the screen-casting application can be cast out for playback, which achieves normal playback of the audio data and does not occupy the local audio output device. If there is an application that expects to use the local audio output device for audio output in the future, the local audio output device can be occupied smoothly, and the audio expected to be output can be successfully played locally.

[0109] It can be understood that electronic devices such as mobile phones and laptops include an application layer, a software framework layer, and a hardware bottom layer. In this application, at least FW Audio (audio-related software) in the software framework layer is involved. Figure 3 The following is a description of the present application.

[0110] When application 1 has a request for audio playback, application 1 calls Audio Focus. For the application that calls Audio Focus, a system-level identifier of the application, such as the application UID, and a callback function for notifying application 1 to stop playing are generated. The UID of application 1, the package name of the audio data packet, and the callback function are stored in the AudioFocus List. If there is another application program different from application 1, such as application 2, calling Audio Focus at this time, the UID of application 2 is generated. According to the UID of application 1 and the UID of application 2, it is determined which audio output device is used to output the audio data of application 1 and the audio data of application 2. According to the UID of application 1 and the UID of application 2, the process information (Running Task Info) of application 1 and application 2 running in the background is found. The identifier (display ID) of the audio output device used by application 1 is found in the RunningTask Info of application 1. The identifier of the audio output device used by application 2 is found in the RunningTask Info of application 2. If the identifier of the audio output device used by application 1 is the identifier of the projection device, and the identifier of the audio output device used by application 2 is the identifier of the local device, then the audio output device used by application 1 is different from the audio output device to be used by application 2, then the callback function of application 1 is discarded so that the audio of application 1 continues to be played using the audio output device used by application 1. The audio of application 2 is played using the local audio output device. For the audio data of application 2 that appears during the audio playback of application 1, normal playback of application 2 is achieved, and continued playback of application 1 is achieved without interfering.

[0111] In actual applications, the terminal devices where the audio output devices used by Application 1 and Application 2 are located can be different types of devices. For example, the type of terminal device where the audio output device of Application 1 is located is a mobile phone, and the type of terminal device where the audio output device of Application 2 is located is a TV. In this way, after knowing the identifiers of the audio output devices used by Application 1 and Application 2 respectively, the output information (display info) of Application 1 and Application 2 is queried based on these two identifiers. According to the type information in the display info of Application 1 and Application 2, the types of terminal devices where the audio output devices of Application 1 and Application 2 are located are known. If the two types are different, the callback of Application 1 is discarded, and the audio data of Application 1 and Application 2 are played using the terminal devices of their respective types. If the two types are the same, the callback is used to notify Application 1 to stop playing. The audio data of Application 2 is started to be played using the same terminal device.

[0112] As can be seen from the above description, the technical solution of this application can be used to play the original audio and the new audio separately. This application can realize the solution of playing the audio of different applications from different audio output devices in the scenario where multiple audios need to be played at the same time. It ensures the normal playback of each of the multiple audios without affecting the user's experience of using each audio.

[0113] The present application provides an electronic device, which is the first device in the above-mentioned information processing method. Figure 4 As shown, the electronic device includes:

[0114] A first obtaining unit 401 is used to obtain first audio data;

[0115] A transmission unit 402, configured to transmit the first audio data to a second device through a data channel, so as to output the first audio data through the audio of the second device; wherein the data channel is a data channel established with the second device before transmitting the first audio data;

[0116] A second obtaining unit 403 is used to obtain a play request for second audio data;

[0117] The responding unit 404 is configured to respond to the play request, if the first audio data is transmitted to the second device through the data channel to enable the second device to output audio, maintain the transmission of the first audio data through the data channel and output the second audio data audio.

[0118] In some embodiments, the response unit 404 is used to:

[0119] It is prohibited to notify the first application of the existence of the playback request of the second application to maintain the continuous transmission of the first audio data to the second device through the data channel, wherein the first audio data is associated with the first application; the second audio data is associated with the second application, and the first application is different from the second application.

[0120] In some embodiments, the first obtaining unit 401 is further configured to:

[0121] obtaining a target instruction for the first application;

[0122] In response to the target instruction, draw an interactive interface of the first application, and transmit the image of the interactive interface in a streaming media form through the data channel to the second device for display output;

[0123] In response to an interaction instruction, obtaining the first audio data, wherein the interaction instruction is determined based on input operation information received by the data channel for an image of an interaction interface of the first application displayed and output by the second device;

[0124] In some embodiments, the first obtaining unit 401 is further configured to:

[0125] In response to the target instruction, the image data and the audio data of the first file associated with the first application are parsed, and the image data and the audio data are transmitted to the display output and the audio output of the second device through the data channel in the form of streaming media.

[0126] In some embodiments, the response unit 404 is further configured to:

[0127] In response to the play request, query a target application currently outputting audio, the first application belonging to the target application;

[0128] Obtaining the audio output device identifier of the target application;

[0129] If the audio output device identifier of the target application does not match the target device identifier, the first audio data is maintained to be transmitted through the data channel and the second audio data is audio-outputted.

[0130] In some embodiments, the response unit 404 is further configured to:

[0131] If the audio output device identifier of the target application matches the target device identifier, at least the application with the target device identifier is notified of the presence of the play request of the second application.

[0132] In some embodiments, the response unit 404 is further configured to:

[0133] If the request to play the second audio data comes from the first application, the method includes: maintaining the transmission of the first audio data through the data channel and transmitting the second audio data through the data channel; or, transmitting the second audio data through the data channel instead of transmitting the first audio data.

[0134] It should be noted that the first device of the embodiment of the present application solves the problem according to a principle similar to the aforementioned information processing method. Therefore, the implementation process and implementation principle of the first device can refer to the description of the implementation process and implementation principle of the aforementioned method, and the repeated parts will not be repeated.

[0135] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0136] The electronic device includes at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the information processing method described in the present application. The computer instructions are used to enable the computer to perform the information processing method described in the present application.

[0137] The present application also provides a computer program product, comprising a computer program / instruction, which implements the information processing method of the present application when executed by a processor.

[0138] Figure 5 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0139] like Figure 5 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0140] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0141] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as information processing methods. For example, in some embodiments, the information processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the information processing method in any other appropriate manner (e.g., by means of firmware).

[0142] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0144] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0147] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0148] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An information processing method, the method comprising: obtaining first audio data; Transmitting the first audio data to the second device through a data channel, so as to output the first audio data through the audio of the second device; wherein the data channel is a data channel established with the second device before transmitting the first audio data; Obtaining a playback request for second audio data; In response to the play request, if the first audio data is transmitted to the second device through the data channel to enable the second device to output audio, the first audio data is maintained to be transmitted through the data channel and the second audio data is outputted audio.

2. The method according to claim 1, wherein maintaining the first audio data for transmission through the data channel comprises: It is prohibited to notify the first application of the existence of the playback request of the second application to maintain the continuous transmission of the first audio data to the second device through the data channel, wherein the first audio data is associated with the first application; the second audio data is associated with the second application, and the first application is different from the second application.

3. The method according to claim 2, further comprising: obtaining a target instruction for the first application; In response to the target instruction, draw an interactive interface of the first application, and transmit the image of the interactive interface in a streaming media form through the data channel to the second device for display output; in, The obtaining of the first audio data comprises: The first audio data is obtained in response to an interaction instruction, where the interaction instruction is determined based on input operation information received by the data channel for an image of an interaction interface of the first application displayed and output by the second device.

4. The method according to claim 2, wherein obtaining the first audio data comprises: In response to the target instruction, the image data and the audio data of the first file associated with the first application are parsed, and the image data and the audio data are transmitted to the display output and the audio output of the second device through the data channel in the form of streaming media.

5. The method according to claim 3 or 4, wherein in response to the play request, if the first audio data is transmitted to the second device through the data channel so that the second device outputs audio, maintaining the transmission of the first audio data through the data channel and audio outputting the second audio data comprises: In response to the play request, query a target application currently outputting audio, the first application belonging to the target application; Obtaining the audio output device identifier of the target application; If the audio output device identifier of the target application does not match the target device identifier, the first audio data is maintained to be transmitted through the data channel and the second audio data is audio-outputted.

6. The method according to claim 5, if the audio output device identifier of the target application matches the target device identifier, at least notifying the application with the target device identifier of the presence of the play request of the second application.

7. The method according to claim 2, further comprising: If the request to play the second audio data comes from the first application, the method includes: maintaining the transmission of the first audio data through the data channel and transmitting the second audio data through the data channel; or, transmitting the second audio data through the data channel instead of transmitting the first audio data.

8. An electronic device comprising: A first obtaining unit, configured to obtain first audio data; A transmission unit, configured to transmit the first audio data to a second device through a data channel, so as to output the first audio data through the audio of the second device; wherein the data channel is a data channel established with the second device before transmitting the first audio data; A second obtaining unit, configured to obtain a play request for second audio data; A response unit is used to respond to the play request, if the first audio data is transmitted to the second device through the data channel so that the second device outputs audio, maintain the first audio data transmitted through the data channel and audio output the second audio data.

9. The device according to claim 8, wherein the response unit is configured to: The first application is prohibited from being notified of the presence of the play request of the second application, so as to maintain the first audio data being continuously transmitted to the second device through the data channel, wherein: The first audio data is associated with a first application; the second audio data is associated with a second application, and the first application is different from the second application.

10. The device according to claim 9, wherein the first obtaining unit is further configured to: obtaining a target instruction for the first application; In response to the target instruction, draw an interactive interface of the first application, and transmit the image of the interactive interface in a streaming media form through the data channel to the second device for display output; In response to an interaction instruction, obtaining the first audio data, wherein the interaction instruction is determined based on input operation information received by the data channel for an image of an interaction interface of the first application displayed and output by the second device; Alternatively, the first obtaining unit is further configured to: In response to the target instruction, the image data and the audio data of the first file associated with the first application are parsed, and the image data and the audio data are transmitted to the display output and the audio output of the second device through the data channel in the form of streaming media.

Images