Multimedia playing method and device and storage medium

Abstract

The invention provides a multimedia playing method and device and a storage medium, and the method comprises the steps: building an audio unit instance if a terminal application program is monitored to be in a network call mode and a multimedia playing request is received; obtaining a media stream to be played through a media player; storing an audio stream in the to-be-played media stream to the target audio buffer area, and clearing the audio stream in the to-be-played media stream to obtain a video stream in the media stream; reading an audio stream from the target audio buffer area through the audio unit instance, rendering the audio stream, and pushing the rendered audio stream to an audio output device of the terminal for playing; and playing the video stream in the to-be-played media stream by using the media player so as to display the video content corresponding to the to-be-played media stream on the video output device of the terminal, thereby realizing flexible control of playing of the multimedia content when the terminal application program is in the network call mode so as to prevent the media volume from being lowered.

Description

Technical Field

[0001] The present invention relates to the field of multimedia technology, and in particular to a multimedia playing method, device and storage medium. Background Art

[0002] When using a terminal for an audio or video call, if other audio or video content is playing at the same time, the terminal operating system will usually automatically reduce the media volume. Although this processing method ensures the clarity of the call, it also significantly affects the user's listening experience of other audio or video, especially when it comes to VoIP (voice over Internet protocol) or video conferencing, the automatic volume adjustment is difficult to meet the user's needs for multitasking. Summary of the invention

[0003] The embodiments of the present invention provide a multimedia playback method, device and storage medium, which are used to flexibly control the playback of multimedia content when a terminal application is in a network call mode to avoid the media volume being reduced.

[0004] In a first aspect, an embodiment of the present invention provides a multimedia playback method, the method comprising:

[0005] If the terminal application is detected to be in network call mode and a multimedia playback request is obtained, an audio unit instance is created;

[0006] Obtain the media stream to be played through the media player;

[0007] The audio stream in the to-be-played media stream is stored in a target audio buffer, and the audio stream in the to-be-played media stream is cleared to obtain the video stream in the to-be-played media stream;

[0008] Reading the audio stream from the target audio buffer through the audio unit instance, rendering the audio stream, and pushing the rendered audio stream to the audio output device of the terminal for playing;

[0009] The media player is used to play the video stream in the to-be-played media stream, so as to display the video content corresponding to the to-be-played media stream on the video output device of the terminal.

[0010] In a second aspect, an embodiment of the present invention provides a multimedia playback device, the device comprising:

[0011] A creation module is used to create an audio unit instance if it is monitored that the terminal application is in a network call mode and a multimedia playback request is obtained;

[0012] An acquisition module is used to acquire the media stream to be played through a media player;

[0013] A storage module, used for storing the audio stream in the to-be-played media stream into a target audio buffer, and clearing the audio stream in the to-be-played media stream to obtain the video stream in the to-be-played media stream;

[0014] A reading module, used for reading the audio stream from the target audio buffer through the audio unit instance, rendering the audio stream, and pushing the rendered audio stream to the audio output device of the terminal for playing;

[0015] The playing module is used to use the media player to play the video stream in the to-be-played media stream, so as to display the video content corresponding to the to-be-played media stream on the video output device of the terminal.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the multimedia playback method described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the multimedia playback method described in the first aspect.

[0018] In the multimedia playback scheme provided by the embodiment of the present invention, when a network call is made and multimedia content needs to be played at the same time, an audio unit instance can be used to render the audio stream to play the audio stream in the media stream to be played, and a media player can be used to play the video stream in the media stream to be played, so as to flexibly control the playback of the multimedia content to avoid the media volume being lowered. Specifically, when it is monitored that the terminal application is in a network call mode and a multimedia playback request is obtained, an audio unit is created. The media stream to be played is obtained through the media player, the audio stream in the media stream to be played is stored in the target buffer, and the audio stream in the media stream to be played is cleared to obtain the video stream in the media stream to be played. Then, the audio stream is read from the target audio buffer through the audio unit instance, the audio stream is rendered, and the rendered audio stream is pushed to the audio output device of the terminal for playback, and the video stream in the media stream to be played is played by the media player, so as to display the video content corresponding to the media stream to be played on the video output device of the terminal.

[0019] In the above scheme, the media stream to be played is obtained through the media player, the audio stream in the media stream to be played is played using the audio unit instance, and the video stream in the media stream to be played is played using the media player to flexibly control the playback of multimedia content. In addition, after the media stream to be played is obtained through the media player, the audio stream in the media stream to be played is intercepted, and the audio stream is stored in the target audio buffer. The audio unit instance reads the audio stream in the media stream to be played from the target audio buffer, and renders the audio stream to modify the playback volume and other information corresponding to the audio stream, so as to realize flexible control of the volume playback of multimedia content, so as to avoid the system automatically reducing the media playback volume when the terminal application is in the network call mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a multimedia playback method provided by an embodiment of the present invention;

[0022] Figure 2 A flowchart of real-time processing of an audio stream in a media stream to be played provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of an application of a multimedia playback method provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the structure of a multimedia playback device provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the structure of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the step timing in the following method embodiments is only an example, not a strict limitation.

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] In actual applications, when using a terminal for an audio or video call, if other audio or video content is played at the same time, in order to ensure the clarity of the call, the terminal operating system usually automatically lowers the media volume. However, when the user needs to pay attention to multimedia content at the same time, automatically lowering the media volume will affect the user's experience and cannot meet the user's needs for multitasking.

[0029] In order to solve the above technical problems, an embodiment of the present invention provides a new multimedia playback solution. In the multimedia playback solution, when the terminal application is in a network call mode and needs to play multimedia content at the same time, after the media player obtains the media stream to be played, the audio stream in the media stream to be played is intercepted by the audio processing object, and the audio stream is processed by the audio unit to play the processed audio stream, and the audio stream in the media stream to be played is cleared, so that the media player plays the video stream in the media stream to be played, and the playback of multimedia content can be flexibly controlled to avoid the media volume being reduced when the terminal is in the network call mode.

[0030] In the following, some embodiments of the present invention are described in detail in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and features in the embodiments can be combined with each other.

[0031] Figure 1 A flowchart of a multimedia playback method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the execution subject of the method may be a multimedia playback device, which may be applied to a terminal to control multimedia playback of the terminal. The method includes the following steps:

[0032] 101. If it is monitored that the terminal application is in a network call mode and a multimedia playback request is received, an audio unit instance is created.

[0033] 102. Obtain a media stream to be played through a media player.

[0034] 103. Store the audio stream in the to-be-played media stream into a target audio buffer, and clear the audio stream in the to-be-played media stream to obtain the video stream in the to-be-played media stream.

[0035] 104. Read an audio stream from a target audio buffer through an audio unit instance, render the audio stream, and push the rendered audio stream to an audio output device of the terminal for playback.

[0036] 105. Use a media player to play the video stream in the to-be-played media stream, so as to display the video content corresponding to the to-be-played media stream on a video output device of the terminal.

[0037] In actual applications, the terminal can be monitored for the usage status of the program, so that when it is monitored that the terminal application is in network call mode and needs to play multimedia content at the same time, an audio unit instance can be created immediately to play the audio stream corresponding to the multimedia content through the audio unit instance.

[0038] In specific implementation, if it is monitored that the terminal application is in the network call mode and a multimedia play request is obtained, an audio unit instance is created. The audio unit instance is mainly used to process audio data and play the corresponding audio data.

[0039] An audio unit instance can be viewed as a node in an audio processing pipeline, responsible for processing audio data and generating output. However, audio processing objects are basic elements that constitute an audio processing pipeline. Therefore, when creating an audio unit instance, it is necessary to first know the characteristics and behaviors of the audio processing objects used within the audio unit instance. Therefore, in an embodiment of the present invention, an audio processing object can be created in advance so that the audio unit obtains the required configuration information during the creation process, thereby better initializing itself to generate an audio unit instance.

[0040] Specifically, with respect to the implementation method of creating an audio unit instance, in an optional embodiment, an audio processing object is created, and the audio processing object is used to perform real-time processing of an audio stream; first configuration information corresponding to the audio processing object is determined, and the first configuration information includes a preparation callback function; when it is monitored that the terminal application is in a network call mode and a multimedia playback request is received, an audio unit instance is created based on the preparation callback function.

[0041] The audio processing object enables audio processing to occur in real time. It can intervene in the audio data stream before it passes through the hardware, thereby achieving real-time audio data processing. The callback mechanism inserts an interception point in the processing of the audio data stream, allowing the audio data to be modified, analyzed, or stored before it is transmitted to the output.

[0042] The prepare callback function is usually used to configure the input and output formats of the audio stream, etc. The prepare callback function is called before the audio stream starts processing to configure the audio unit, set the format of the audio stream, set the volume, number of channels, sampling rate and other parameters.

[0043] In practical applications, an audio unit instance is a specific instantiation form of an audio unit. Therefore, when creating an audio unit instance based on a preparation callback function, the audio unit component can be found by calling the preparation callback function; an audio unit object can be created based on the audio unit component; playback parameters corresponding to the audio unit instance and a rendering callback function corresponding to the audio unit instance can be determined; based on the playback parameters and the rendering callback function, the audio unit object can be instantiated to obtain the audio unit instance.

[0044] The playback parameters may include the input stream format of the audio unit (number of channels, sampling rate, etc.), volume value, output stream format of the audio unit, maximum number of frames rendered at one time by the audio unit, etc. The rendering callback function is used to render the audio stream to modify the audio stream. When processing the audio stream, the audio unit instance may call the rendering callback function to process the audio stream so that the audio stream in the media stream to be played is played according to the settings.

[0045] That is, in the embodiment of the present invention, when the terminal is making an Internet call and needs to play multimedia content at the same time, in order to solve the problem that the terminal operating system automatically reduces the media volume according to the priority of each audio stream, when a multimedia playback request is obtained, a new audio unit instance can be created to play the audio stream in the media stream. In addition, when the audio unit instance is used to play the audio stream in the media stream to be played, the audio unit instance can also be used to modify the audio stream so that the audio stream is played according to the modified playback parameters.

[0046] After the audio unit instance is created, when the media stream to be played is monitored and transmitted to the hardware layer, the media stream to be played is obtained through the media player, where the media player can be the system player AV Player.

[0047] Regarding the implementation method of obtaining the to-be-played media stream, in an optional embodiment, the to-be-played media stream can be obtained through a media item management object. Specifically, a media item management object is created, and second configuration information corresponding to the media item management object is determined, the second configuration information including the network address of the to-be-played media stream and the play status of the to-be-played media stream; if the play status of the to-be-played media stream is that the to-be-played media stream is transmitted to the terminal, the media player obtains the to-be-played media stream according to the network address of the to-be-played media stream.

[0048] After obtaining the media stream to be played, the media player stores the audio stream in the media stream to be played to the target audio buffer, clears the audio stream in the media stream to be played, and obtains the video stream in the media stream to be played. This not only stores the audio stream in the media stream to be played, but also enables the media player to only obtain the video stream in the media stream to be played when playing the media stream to be played, thereby achieving the purpose of muting the original media stream to be played.

[0049] Next, the audio stream of the media stream to be played is read from the target audio buffer through the audio playback unit instance, and the audio stream is rendered to modify the audio stream data so that the audio stream can be played according to the settings. After the audio playback unit instance completes the modification of the audio stream, the modified audio stream is pushed to the audio output device of the terminal for playback, so that the terminal user can clearly hear the multimedia content at a normal volume.

[0050] At the same time, in order to ensure that the terminal user can view the video content corresponding to the media stream at the same time, a media player can be used to play the video stream in the media stream to be played, so as to display the video content corresponding to the media stream to be played on the output device of the terminal.

[0051] The multimedia playback solution provided by the embodiment of the present invention obtains the media stream to be played through the media player, uses the audio unit instance to play the audio stream in the media stream to be played, and uses the media player to play the video stream in the media stream to be played, so as to flexibly control the playback of multimedia content. In addition, after the media player obtains the media stream to be played, the audio stream in the media stream to be played is intercepted, and the audio stream is stored in the target audio buffer. The audio unit instance reads the audio stream in the media stream to be played from the target audio buffer, and renders the audio stream to modify the playback volume and other information corresponding to the audio stream, so as to realize flexible control of the volume playback of multimedia content, so as to avoid the system automatically reducing the media playback volume when the terminal application is in the network call mode.

[0052] The above embodiment introduces the use of the created audio unit instance to play the audio stream in the media stream to be played, so as to flexibly control the playback volume corresponding to the multimedia content. Among them, when the audio unit instance plays the media stream to be played, it is necessary to first intercept the audio stream in the media stream to be played acquired by the media player in real time, so that the audio unit instance can read the audio stream in real time, render the audio stream, and output the rendered audio stream to the audio output device through the audio unit instance in real time for wave playback. The implementation process is described in detail below.

[0053] In the embodiment of the present invention, an audio processing object is created to process the audio stream in the media stream to be played in real time. The audio processing object can intervene in the audio stream before it passes through the hardware, insert an interception point in the processing of the audio stream through a callback mechanism, intercept the audio stream in the media stream to be played, and allow the audio data to be modified, analyzed or stored before being transmitted to the output end.

[0054] However, the audio processing object is an audio processing tool. It is not possible to directly associate the audio processing object with a specific audio track so that the audio processing object can be applied when the media player plays the media to be played. The audio stream in the media stream to be played can be intercepted by the audio processing object to perform real-time processing on the audio stream, thereby controlling the playback of the audio stream in the media stream.

[0055] Then, in actual applications, you can create an audio mixing object configured with an audio processing object and associate the audio mixing object with a media player. In this way, when the media player obtains the media stream to be played, it can use the audio processing object to perform real-time processing on the audio stream in the media stream to be played.

[0056] The audio mixing object is used to manage audio mixing effects and customize audio processing methods. You can first create an audio mixing object and an audio processing object, and determine the audio processing callback functions of different audio processing stages corresponding to the audio processing object. Add the audio processing object to the audio mixing object.

[0057] Specifically, a mutable audio mixing object can be created. After the creation is successful, the audio tracks to which the audio mixing is to be applied are specified for the audio mixing object. An audio processing object callback structure is defined to set the callback version in the callback structure and add various audio processing callback functions so as to perform customized audio processing at different stages. Then, an audio processing object is created and the audio processing callback function is configured. The audio processing object is added to the audio mixing object, and the audio mixing object is associated with the media player.

[0058] Before creating an audio processing object, a custom initialization callback function may be called first to use the initialization callback function to determine audio processing parameters corresponding to the audio processing object, allocate memory for the audio stream, and so on.

[0059] When using an audio processing object to perform real-time processing on the audio stream in the media stream to be played, the audio processing callback function corresponding to the audio processing object at different audio processing stages can be pre-set, so that the corresponding audio processing callback function can be called at different audio processing stages to process the audio stream in the media stream to be played.

[0060] The audio processing callback function may include a preparation callback function, a processing callback function, and an end callback function. The preparation callback function may be used to configure the input and output formats of the audio stream, and may also create an audio unit instance. Before starting to process the audio stream, the preparation callback function may be called to create an audio unit instance and determine the input format and output format of the audio stream corresponding to the audio unit instance.

[0061] The processing callback function can be used to process the incoming audio stream, and can modify the audio stream, analyze the audio stream, and store the audio stream in real time. When the audio stream processing starts, the processing callback function can be called to store the audio stream to the target buffer, so that the audio unit instance reads the audio stream from the target buffer, renders the audio stream, and outputs the rendered audio stream to the audio output device for playback.

[0062] The end callback function can be used to clean up and restore the state of the audio stream. When the audio stream stops processing, the end callback function is called to clear related settings, restore the state, and perform other operations.

[0063] After determining the audio processing callback functions corresponding to the audio processing object at different audio processing stages, the customized audio processing callback functions can be used at different audio processing stages to process the audio stream to control the playback of the audio stream.

[0064] In specific implementation, when it is monitored that the terminal application is in network call mode and a multimedia playback request is obtained, the audio processing object can call the preparation callback function to create an audio unit instance to use the audio unit instance to play the audio stream in the media stream to be played.

[0065] When the media stream to be played is monitored and transmitted to the terminal hardware layer, the media stream to be played is obtained through the media player. The audio processing object can call the processing callback function to intercept the audio stream in the media stream to be played, and store the intercepted audio stream in the media stream to be played to the target buffer, so that the audio unit instance can read the audio stream from the target buffer, and process the audio stream to output the processed audio stream to the audio output device for playback.

[0066] The specific implementation process of processing the media stream to be played is described in detail below in conjunction with the following embodiments.

[0067] Figure 2 A flowchart of processing a media stream to be played provided by an embodiment of the present invention, based on the above embodiment, in order to intercept the audio stream in the media stream to be played in real time, so as to use the audio unit instance to play the audio stream, so as to realize flexible control of the playback volume of the multimedia content, specifically, as follows Figure 2As shown, the method comprises the following steps:

[0068] 201. Intercept an audio stream in a to-be-played media stream through an audio processing object, and store the audio stream in a target audio buffer.

[0069] 202. Clear the audio stream in the media stream to be played, and obtain the video stream in the media stream to be played.

[0070] 203. Read the audio stream from the target audio buffer through the audio unit instance, and render the audio stream.

[0071] Among them, the audio processing object can insert an interception point in the processing of the audio stream through a callback mechanism. When the media player obtains the media stream to be played, the audio stream in the media stream to be played is intercepted by the audio processing object and the audio stream is stored in the target audio buffer.

[0072] After the media player obtains the media stream to be played, the decoding module in the media player can decode the media stream to be played to obtain the decoded media stream. Then, the audio stream and video stream contained in the media stream to be played are determined, and the audio stream is stored in the source audio buffer, and the video stream is stored in the video buffer.

[0073] Among them, the audio processing object can read the audio stream in the media stream to be played from the source audio buffer, and transfer the audio stream in the media stream to be played to the target buffer, and at the same time clear the audio stream in the media stream to be played in the source audio buffer to obtain the video stream in the media stream to be played, so that when the media player plays, it can only obtain the video stream in the media stream to be played.

[0074] Then, the audio unit instance can read the audio stream from the target audio buffer, render the audio stream, and transmit the rendered audio stream to the audio output device of the terminal for playback.

[0075] In an optional embodiment, specifically, the audio unit instance can be used to determine the audio data block to be proposed according to the number of frames currently being played and the number of frames to be played; the audio data corresponding to the audio data block can be extracted from the target audio buffer; and the audio data can be rendered using a rendering callback function to obtain rendered audio data.

[0076] Among them, when the audio unit instance is started, the rendering callback function will be called continuously to render the audio data to modify the playback parameters and other information corresponding to the audio data. Audio data is usually a large data block. Loading all of it into the system memory at one time will increase the burden on the terminal system. In an embodiment of the present invention, the audio stream can be loaded in batches. A small part of the data can be preloaded first, and then more audio data can be dynamically loaded when needed. In other words, each time the rendering callback function is used to render the audio data, the number of frames to be played corresponding to each rendering process can be set, and then, according to the number of frames currently played and the number of frames to be played, the corresponding size of audio data is read from the target audio buffer.

[0077] After the audio stream in the media stream to be played is processed, the audio processing object can call the end callback function to restore or clear the configuration corresponding to the audio unit and the configuration of the playback parameters.

[0078] In an embodiment of the present invention, an audio stream in a media stream to be played can be intercepted by an audio processing object, and the audio stream can be stored in a target audio buffer. The audio stream in the media stream to be played is cleared to obtain a video stream in the media stream to be played. The audio stream is read from the target audio buffer by an audio unit instance, and the audio stream is rendered to play the audio stream. In this way, the audio stream in the media stream to be played can be played by using the created audio unit instance to control the playback of the audio stream, thereby avoiding the media volume being reduced when the terminal application is in a network call mode.

[0079] Among them, the specific implementation process involved in the embodiments of the present invention can refer to the contents in the above embodiments, and will not be repeated here.

[0080] In order to facilitate understanding of the specific implementation process of the multimedia playback method in the above embodiment, Figure 3 The application scenario shown in the figure is an exemplary description of the multimedia playback process. In specific applications, the terminal application may be an AI interview program, through which an AI interview room can be provided for the terminal to simulate an AI video interview between a recruiter and a terminal user in the AI ​​interview room, wherein the simulated AI interview between the recruiter and the terminal user is performed in the form of playing a pre-made video file.

[0081] Among them, when playing the pre-made video file to interact with the terminal user, the corresponding media player can be configured first. Here, the system player provided by the terminal operating system can be used to play the video file. In order to avoid using the system player to play the video file, because it is currently in the network call mode, when the terminal plays the video file, the video file playback volume will be automatically reduced. In an embodiment of the present invention, an audio processing object can be created to intercept the audio stream in the video file, so as to use the audio unit instance to play the audio stream in the video file, and use the system player to play the video corresponding to the video file.

[0082] The media item to be played corresponding to the system player can be determined first, so that the system player can play the pre-made video file in real time. Specifically, a media item management object can be created, the media item management object can obtain the network address corresponding to the video file, and a listener can be added to the media item management object to monitor the playback status corresponding to the video file, so that the video file to be played can be obtained in time according to the media item management object.

[0083] Furthermore, an audio mixing object configured with an audio processing object may be created, and the created audio mixing object may be associated with a system player, so that in actual applications the system player may utilize the configured audio processing object to process audio data in a video file.

[0084] Specifically, create an audio mixing object and an audio processing object, the audio mixing object is used to manage the audio mixing effects corresponding to the media stream to be played, and the audio processing object is used to perform real-time processing on the audio stream; determine the audio processing parameters corresponding to the audio processing object; add the audio processing object to the audio mixing object, and associate the audio mixing object with the system player, so as to add the audio processing object to the system player through the audio mixing object, so as to perform real-time processing on the audio stream in the video file to be played.

[0085] When the system player is configured, you can directly use the configured system player to play video files.

[0086] In specific implementation, when the terminal user starts the AI ​​interview program, the homepage of the AI ​​interview program will display the interview link corresponding to the target interview scheduled by the terminal user. When the interview time arrives, the terminal user can enter the AI ​​interview room by clicking on the interview link. After the terminal user enters the AI ​​interview room, he will have a video conversation mode with the virtual recruiter. Among them, the virtual recruiter conducts an AI interview with the terminal user mainly by collecting the video screen corresponding to the terminal user through the server, inputting the video screen, interview position information and company-related information into the large language model to obtain the corresponding dialogue content, and inputting the dialogue content into the video generation model to generate the video content corresponding to the virtual recruiter, so as to display the video file corresponding to the virtual recruiter in the AI ​​interview interface.

[0087] Then, in the embodiment of the present invention, after the terminal user enters the AI ​​interview room, the multimedia playback solution can be used to play the video file corresponding to the virtual recruiter. Specifically, when it is monitored that the current AI interview program is in the network call mode and receives the video file corresponding to the virtual recruiter sent by the server, the audio processing object can create an audio unit instance by calling the preparation callback function.

[0088] Then, when the video file corresponding to the virtual recruitment officer sent by the server is transmitted to the terminal hardware layer, the video file to be played is obtained through the system player, the audio stream and video stream in the video file to be played are determined, the audio stream is stored in the source audio buffer, and the video stream is stored in the source video buffer.

[0089] The audio stream is read from the source audio buffer through the audio processing object, the audio stream is transferred to the target audio buffer, and the audio stream in the source audio buffer is cleared, so that the system player can only obtain the video stream in the source video buffer, so as to use the system player to play the video content, and use the created audio unit instance to play the audio stream.

[0090] Next, the audio stream is read from the target audio buffer through the audio unit instance, and the rendering callback function is called to render the audio stream, and the rendered audio stream is pushed to the audio output device of the terminal for playback. The video stream in the video content to be played is played using the system player to display the video content corresponding to the video file to be played on the video output device of the terminal.

[0091] Among them, when the AI ​​video interview is over, the audio processing object can call the end callback function to restore or clear the configuration of the audio unit and the memory occupied. At the same time, the audio processing object can also be destroyed. When the audio processing object is destroyed, the clear callback function can be called to release the memory allocated in the audio processing object, close the corresponding file, and perform other operations.

[0092] The above solution allows the terminal user to better control the playback volume of the video file corresponding to the virtual recruiter when conducting an AI video interview with the virtual recruiter, so that the terminal user can clearly hear the video file corresponding to the virtual recruiter when making an Internet call, so as to better complete the AI ​​interview with the virtual recruiter and improve the user experience.

[0093] The specific execution process and technical effects of the above steps in this embodiment are similar to the specific execution process and technical effects in the above embodiments. Please refer to the above statements for details, and no further details will be given here.

[0094] The multimedia playback device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0095] Figure 4 A schematic diagram of the structure of a multimedia playback device provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device includes: a creation module 11, an acquisition module 12, a storage module 13, a reading module 14, and a playing module 15.

[0096] The creation module 11 is used to create an audio unit instance if it is monitored that the terminal application is in a network call mode and a multimedia play request is obtained.

[0097] The acquisition module 12 is used to acquire the media stream to be played through the media player.

[0098] The storage module 13 is used to store the audio stream in the to-be-played media stream into a target audio buffer, and clear the audio stream in the to-be-played media stream to obtain the video stream in the to-be-played media stream.

[0099] The reading module 14 is used to read the audio stream from the target audio buffer through the audio unit instance, render the audio stream, and push the rendered audio stream to the audio output device of the terminal for playing.

[0100] The playing module 15 is configured to use the media player to play the video stream in the to-be-played media stream, so as to display the video content corresponding to the to-be-played media stream on the video output device of the terminal.

[0101] Optionally, the creation module 11 is specifically used to: create an audio processing object, which is used to perform real-time processing of an audio stream; determine first configuration information corresponding to the audio processing object, the first configuration information including a preparation callback function; and when it is monitored that the terminal application is in a network call mode and a multimedia playback request is received, create an audio unit instance based on the preparation callback function.

[0102] Optionally, the creation module 11 is also used to: call the preparation callback function to search for the audio unit component; create an audio unit object based on the audio unit component; determine the playback parameters corresponding to the audio unit instance and the rendering callback function corresponding to the audio unit instance; based on the playback parameters and the rendering callback function, instantiate the audio unit object to obtain an audio unit instance.

[0103] Optionally, the acquisition module 12 is specifically used to: create a media item management object; determine second configuration information corresponding to the media item management object, the second configuration information including a network address of the media stream to be played and a playback status of the media stream to be played; if the playback status of the media stream to be played is that the media stream to be played is transmitted to the terminal, then the media player acquires the media stream to be played according to the network address of the media stream to be played.

[0104] Optionally, the storage module 13 is specifically used to: intercept the audio stream in the media stream to be played through the audio processing object, store the audio stream in the target audio buffer, so that the audio unit instance reads the audio stream from the target audio buffer; clear the audio stream in the media stream to be played, and obtain the video stream in the media stream to be played.

[0105] Optionally, the device further includes a determining module, specifically configured to: determine the audio stream and the video stream included in the media stream to be played; and store the audio stream in a source audio buffer.

[0106] Optionally, the storage module 13 is further used to: transfer the audio stream from the source audio buffer to the target audio buffer through the audio processing object; and clear the audio stream in the source audio buffer.

[0107] Optionally, the reading module 14 is specifically used to: determine the audio data block to be proposed according to the number of frames currently played and the number of frames to be played through the audio unit instance; extract the audio data corresponding to the audio data block from the target audio buffer; and use the rendering callback function to render the audio data to obtain rendered audio data.

[0108] Figure 4The device shown can execute the steps in the multimedia playback method in the aforementioned embodiment. The detailed execution process and technical effects can be found in the description of the aforementioned embodiment, which will not be repeated here.

[0109] The embodiment of the present invention also provides an electronic device, such as Figure 5 As shown, the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable codes, and when the executable codes are executed by the processor 21, the processor 21 implements the multimedia playback method in the above-mentioned embodiment.

[0110] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the multimedia playback method provided in the aforementioned embodiment.

[0111] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0112] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimedia playback method, characterized in that: Applied to a terminal, the method comprises: If the terminal application is detected to be in network call mode and a multimedia playback request is obtained, an audio unit instance is created; Obtain the media stream to be played through the multimedia player; The audio stream in the to-be-played media stream is stored in a target audio buffer, and the audio stream in the to-be-played media stream is cleared to obtain the video stream in the to-be-played media stream; Reading the audio stream from the target audio buffer through the audio unit instance, rendering the audio stream, and pushing the rendered audio stream to the audio output device of the terminal for playing; The video stream in the to-be-played media stream is played by using a media player, so as to display the video content corresponding to the to-be-played media stream on the video output device of the terminal.

2. The method according to claim 1, characterized in that The step of creating an audio unit instance includes: Creating an audio processing object, wherein the audio processing object is used to process the audio stream in real time; Determine first configuration information corresponding to the audio processing object, wherein the first configuration information includes a preparation callback function; When it is monitored that the terminal application is in the network call mode and a multimedia play request is received, an audio unit instance is created based on the preparation callback function.

3. The method according to claim 2, characterized in that The step of creating an audio unit instance based on the preparation callback function includes: Calling the preparation callback function to search for the audio unit component; Based on the audio unit component, create an audio unit object; Determine the playback parameters corresponding to the audio unit instance and the rendering callback function corresponding to the audio unit instance; Based on the playback parameters and the rendering callback function, the audio unit object is instantiated to obtain an audio unit instance.

4. The method according to claim 1, characterized in that: The step of obtaining the to-be-played media stream through the media player includes: Create a media item management object; Determine second configuration information corresponding to the media item management object, the second configuration information including a network address of a to-be-played media stream and a playing status of the to-be-played media stream; If the playing state of the to-be-played media stream is that the to-be-played media stream is transmitted to the terminal, the to-be-played media stream is acquired by the media player according to the network address of the to-be-played media stream.

5. The method according to claim 2, characterized in that: The step of storing the audio stream in the to-be-played media stream into a target audio buffer includes: Intercepting the audio stream in the to-be-played media stream through the audio processing object, and storing the audio stream in a target audio buffer, so that the audio unit instance reads the audio stream from the target audio buffer; The audio stream in the media stream to be played is cleared, and the video stream in the media stream to be played is obtained.

6. The method according to claim 5, characterized in that Before storing the audio stream in the to-be-played media stream into the target audio buffer, the method further includes: Determine the audio stream and the video stream included in the media stream to be played; The audio stream is stored in a source audio buffer.

7. The method according to claim 6, characterized in that The step of intercepting the audio stream by the audio processing object and storing the audio stream in a target audio buffer includes: Transferring the audio stream from the source audio buffer to the target audio buffer through the audio processing object; Clear the audio stream in the source audio buffer.

8. The method according to claim 3, characterized in that The step of reading the audio stream from the target audio buffer through the audio unit instance and rendering the audio stream comprises: Determine the audio data block to be proposed according to the number of frames currently played and the number of frames to be played through the audio unit instance; Extracting audio data corresponding to the audio data block from the target audio buffer; The audio data is rendered using the rendering callback function to obtain rendered audio data.

9. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the multimedia playback method as described in any one of claims 1 to 8.

10. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the multimedia playback method according to any one of claims 1 to 8.

Images