Audio and video processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an audio and video processing method and device, electronic equipment and a storage medium, and the method comprises the steps: transmitting an audio and video frame to a server in an audio and video collection process, so as to enable the server to store the audio and video frame; when it is detected that the audio and video release condition is satisfied, displaying locally stored audio and video frames at an audio and video acquisition client; and when the audio and video frames issued by the server are received, replacing the audio and video frames displayed by the video acquisition client based on the audio and video frames, and issuing the audio and video frames. According to the technical scheme provided by the embodiment of the invention, the problem that certain time delay and hysteresis exist when the audio and video frames are uploaded to the server side during publishing and then pushed to other clients by the server side is solved, and the technical effect of efficiently publishing the audio and video is realized.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of audio and video processing, and in particular, to an audio and video processing method, device, electronic device, and storage medium. Background Art

[0002] With the development of computer technology, more and more application software have emerged, or corresponding application functions have been added to the application software. Optionally, the application software or the newly added application functions may be software for video shooting, video publishing, etc.

[0003] At present, the main process of video publishing is to trigger the camera function of the application software to shoot the corresponding audio and video data, and store the audio and video data locally. Based on the user's triggering operation on the publishing control, the audio and video data stored locally is uploaded to the server, so that the server can distribute the audio and video data; or the user can edit the audio and video data and send the edited audio and video data to the server, so that the server can publish the audio and video.

[0004] However, the above method has the following problems: audio and video data need to be stored both locally and on the server, which increases the encoding time and storage space usage. Furthermore, uploading audio and video data is limited by the network and equipment, resulting in a certain lag in data transmission. Correspondingly, there are also certain problems with the real-time nature of video publishing. Summary of the invention

[0005] The present invention provides an audio and video processing method, device, electronic device and storage medium to realize fast and effective processing of audio and video data, thereby achieving the technical effect of real-time audio and video publishing.

[0006] In a first aspect, an embodiment of the present disclosure provides an audio and video processing method, which is applied to a client, and the method includes:

[0007] In the process of collecting audio and video, the audio and video frames are sent to the server so that the server stores the audio and video frames;

[0008] When it is detected that the audio and video publishing conditions are met, the locally stored audio and video frames are displayed on the audio and video acquisition client;

[0009] When the audio and video frames sent by the server are received, the audio and video frames displayed by the audio and video acquisition client are replaced based on the audio and video frames and the audio and video frames are released.

[0010] In a second aspect, an embodiment of the present disclosure provides an audio and video processing method, which is applied to a server, and the method includes:

[0011] Receive and store audio and video frames sent by the audio and video acquisition client; wherein the audio and video frames are sent during the process of collecting audio and video;

[0012] When receiving the audio and video publishing instruction, the stored audio and video frames are sent to the audio and video acquisition client to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames;

[0013] Wherein, the audio and video publishing instruction is determined based on the audio and video publishing conditions.

[0014] In a third aspect, the embodiment of the present disclosure further provides an audio and video processing device, which is configured in a client, and includes:

[0015] An audio and video acquisition module, used for sending audio and video frames to a server during the process of acquiring audio and video, so that the server stores the audio and video frames;

[0016] The first audio and video playback module is used to display the locally stored audio and video frames on the audio and video acquisition client when it is detected that the audio and video publishing conditions are met;

[0017] The second audio and video playback module is used to replace the audio and video frames displayed by the audio and video acquisition client based on the audio and video frames when receiving the audio and video frames sent by the server and publish the audio and video frames.

[0018] In a fourth aspect, the embodiments of the present disclosure further provide an audio and video processing device, which is configured in a server, and includes:

[0019] An audio and video receiving module, used to receive and store audio and video frames sent by an audio and video acquisition client; wherein the audio and video frames are sent during the process of acquiring audio and video;

[0020] An audio and video sending module, configured to send the stored audio and video frames to the audio and video acquisition client upon receiving the audio and video publishing instruction, so as to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames;

[0021] Wherein, the audio and video publishing instruction is determined based on the audio and video publishing conditions.

[0022] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0023] one or more processors;

[0024] a storage device for storing one or more programs,

[0025] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio and video processing method as described in any one of the embodiments of the present disclosure.

[0026] In a sixth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the audio and video processing method as described in any one of the embodiments of the present disclosure.

[0027] The technical solution provided by the embodiments of the present disclosure can send audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video. Furthermore, when the audio and video publishing conditions are detected, in order to avoid a certain delay in the server downloading the audio and video frames, which causes the user to wait for a certain period of time and thus causes a poor user experience, the locally stored audio and video frames can be displayed in the audio and video collection client, and the audio and video frames received from the server can be replaced with the audio and video frames displayed by the audio and video collection client, and the audio and video frames sent by the server can be published, which solves the problem of certain delays and lags when the audio and video frames are uploaded when publishing and pushed to other clients by the server, and achieves the technical effect of efficient audio and video publishing. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0029] Figure 1 It is a flowchart of an audio and video processing method provided by an embodiment of the present disclosure;

[0030] Figure 2 It is a flowchart of an audio and video processing method provided by an embodiment of the present disclosure;

[0031] Figure 3 It is a flow chart of the interaction between a server and a client provided by an embodiment of the present disclosure;

[0032] Figure 4 is a structural schematic diagram of an audio and video processing device provided by an embodiment of the present disclosure;

[0033] Figure 5 is a structural schematic diagram of an audio and video processing device provided by an embodiment of the present disclosure;

[0034] Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0036] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0037] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0038] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0039] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0040] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0041] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0042] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0043] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0044] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0045] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0046] Before introducing the technical solution, an example description of the application scenario can be given. The technical solution provided by the embodiment of the present disclosure can be applied to any scenario where a client collects audio and video data and publishes the audio and video data so that other users can see the audio and video data. That is, as long as the interaction between the client and the server is involved in the audio and video processing process, the solution provided by the embodiment of the present disclosure can be used to implement it. For example, it can be applied to the scenario where the client collects audio and video and the server distributes the audio and video data. Optionally, the application scenario can be a scenario where a short video is created and published, a scenario where a special effects video is shot and published, etc.

[0047] It should be noted that the application scenarios described above are only simple explanations of the application scenarios of the solutions, and are not limitations on the application scenarios. The solutions can also be applied in other application scenarios.

[0048] It should also be noted that the audio and video processing device provided by the embodiment of the present disclosure can be integrated into an application software that supports audio and video processing functions, and the software can be installed in an electronic device, and optionally, the electronic device can be a mobile terminal or a PC, etc. The application software can be software for audio and video processing, and its specific application software will not be described one by one here, as long as audio and video processing can be realized. Optionally, audio and video processing can be audio and video acquisition or adding corresponding content to audio and video, and the content can be text, stickers, music, filters, etc.

[0049] Figure 1 It is a flow chart of an audio and video processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the situation where any client collects audio and video and the server separates the audio and video. The method can be executed by an audio and video processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0050] like Figure 1 As shown, the method includes:

[0051] S110. During the process of collecting audio and video, the audio and video frames are sent to the server, so that the server stores the audio and video frames.

[0052] The audio and video collection can be realized by the client, which can be an installed application software, and the application software can call the camera and microphone array of the terminal device to collect the corresponding audio and video. The server corresponds to the client and can be understood as a server that processes the audio and video.

[0053] Usually, the processing and synthesis of audio and video are mostly performed by the server. Therefore, when the client completes video capture, it is also necessary to send the audio and video to the server so that the server can process the audio and video. However, after the audio and video are captured locally and then sent to the server, it is limited by the network, the client's device and the video size. Based on this, the captured audio and video frame data can be encoded and sent to the server during the process of capturing audio and video, that is, it can be sent to the server while being captured, so as to avoid the need to upload it to the server when the audio and video is released. When the data volume is large, there is a problem of large lag.

[0054] Specifically, based on the objective function in the client, the corresponding camera and / or microphone array can be called to collect the corresponding audio and video frames, and the collected audio and video frames can be sent to the server so that the server can store the audio and video frames, so that they can be distributed quickly when they are released.

[0055] For example, it can be applied in short video shooting or special effect video shooting scenes, where the user can open application software A and trigger the video shooting control in the application software to start the camera and microphone array. The camera and microphone array are used to collect audio and video frames including the target object, and the audio and video frames are sent to the server during the shooting process.

[0056] The audio and video frames are sent directly to the server, or the audio and video frames are stream-encapsulated and uploaded to the server.

[0057] In this embodiment, the audio and video frames can be processed by streaming encapsulation. The advantage of this setting is that it can be uploaded while shooting. Furthermore, when the video is released or played, it can support downloading and playing at the same time because it is streaming encapsulated, so that the user can continue to download the subsequent video frames while watching the previous few video frames, avoiding the problem of having to wait for all video frames to be downloaded before playing.

[0058] S120: When it is detected that the audio and video publishing conditions are met, the locally stored audio and video frames are displayed on the audio and video acquisition client.

[0059] The audio and video publishing conditions include triggering the audio and video publishing control, or the audio and video shooting time reaches a preset time threshold, or detecting a gesture that triggers the audio and video publishing, etc. The specific publishing conditions are not limited in this embodiment, as long as the server can be informed that the audio and video needs to be published. The audio and video acquisition client is a client that collects audio and video frames.

[0060] It should be noted that, during the process of audio and video acquisition, it is usually necessary to store it locally first so that the audio and video frames can be edited. Even if it is not edited, the audio and video preview can be performed based on the locally stored audio and video frames. That is, when the audio and video frames are acquired, they need to be stored locally first.

[0061] Specifically, when it is detected that the conditions for audio and video publishing are met, the locally stored audio and video frames can be played in the audio and video acquisition client. The advantage of this is that it takes a certain amount of waiting time for the server to download the audio and video frames. During the waiting process, the locally stored audio and video can be displayed, thereby achieving the effect of no waiting for users.

[0062] In this embodiment, if the user does not add corresponding special effect elements to the collected audio and video frames, the steps S110 to S120 can be directly executed to realize the collection and release of audio and video frames. If it is necessary to add corresponding special effect elements to the audio and video, it can be: before detecting that the audio and video release conditions are met, it also includes: adding at least one special effect element to the locally stored audio and video frames to update the locally stored audio and video frames; recording the element association information of the at least one special effect element and the added attribute information in the audio and video frames, so as to merge the at least one special effect element into the audio and video frames stored on the server based on the element association information and the added attribute information; wherein the element association information includes the resource package identifier of the special effect element, and the added attribute information includes at least one of the location information and the timestamp information.

[0063] Among them, the number of at least one special effect element can be one or more, and the specific number of special effect elements matches the actual needs of the user. At least one special effect element can be a sticker, text, music, filter or beauty, that is, the special effect element can be understood as any element that is different from the original audio and video frame. In the process of adding special effect elements, the element association information corresponding to each special effect element can be recorded, and the element association information can include a resource package identifier, that is, an identifier for calling the special effect element. It can be understood that all special effect elements can be stored in a database, and the resource package corresponding to each special effect element has a unique identifier corresponding to it, and the unique identifier is the resource package identifier. At the same time, the added attributes of each special effect element in the audio and video frame can also be recorded. The added attribute information can include position information and / or timestamp information. The position information can be understood as the specific addition position of the special effect element in the audio and video frame, and the specific addition position can be represented by pixel coordinates. The added timestamp information can be the play timestamp added to the special effect element in the audio and video frame, that is, the time information of the special effect element applied in the audio and video frame. The time information is a relative time, that is, the time stamp of the application of the special effect element is based on the start frame. In other words, the added attribute information is the application information of the special effect element in the audio and video frame.

[0064] It should also be noted that the added attribute information can be one or both of timestamp information and location information. This is because different special effect elements correspond to different element attributes. Therefore, there are certain differences in the above-mentioned added attributes. For example, if music is added, there may be no location information.

[0065] Specifically, if an audio or video frame editing operation is required, and optionally, the editing operation is to add a special effect element, then the locally stored audio or video frame can be retrieved, and the special effect element can be added to the audio or video frame. In the process of adding special effect elements, the resource package identifier corresponding to each special effect element can be recorded, that is, where the resource of the special effect element is retrieved from. At the same time, the use position and / or use timestamp of the special effect element in the audio or video frame can also be recorded.

[0066] Exemplarily, if the special effect element added at the playback timestamp of the audio and video frame is 00:05-00:10 is background music A, then the element association information corresponding to background music A can be recorded as resource package identifier ID1, and the added attribute information includes timestamp information, and the timestamp information is 00:05-00:10; it can also be, for example, that the special effect element added at the playback timestamp of the audio and video frame is 00:15, the pixel coordinates are (500,500) (the pixel coordinates at this time are the center point of the text box) is text box B, and the text content XXX is entered, then the resource package identifier ID5 of text box B can be recorded, and the added attribute information includes timestamp information 00:15, and the position information is (500,500). It should also be noted that multiple special effect elements can be added to an audio and video frame, and they only need to be recorded separately when recording.

[0067] Typically, after adding special effect elements, the video can be published. If the video publishing conditions are met, the server storage identifier of the audio and video frame, as well as the element association information and corresponding added attribute information of at least one special effect element are sent to the server, so that the server retrieves the corresponding audio and video frame based on the server storage identifier, and adds special effect elements to the audio and video frame based on the element association information and the added attributes.

[0068] In the process of collecting audio and video, you can upload it to the server while collecting it, so that the collected audio and video can be stored on the server. In order to facilitate the synchronization of special effect elements to the server when the user edits the audio and video, the server can feedback the storage identifier to the client when the audio and video frame is first received and stored on the server, that is, the server storage identifier. It can be understood that the server storage identifier is the storage address identifier for storing audio and video frames, that is, the address of storing audio and video frames. Based on the storage address identifier, the stored audio and video frames can be obtained from the corresponding location.

[0069] In this embodiment, the element association information and corresponding added attribute information of all special effect elements added to the audio and video frame can be packaged into a data packet and sent to the server. That is, only the incremental information different from the collected audio and video frames needs to be packaged and sent to the client, avoiding the current method of sending all audio and video frames with added special effect elements to the server, which has a large amount of data and thus causes low transmission efficiency and long delay. The server can merge the special effect elements into the audio and video frames according to the received data packets to obtain the audio and video frames with added special effect elements.

[0070] S130: When receiving the audio and video frames sent by the server, replace the audio and video frames displayed by the audio and video acquisition client based on the audio and video frames and publish the audio and video frames.

[0071] It can be understood that when playing the audio and video frames stored locally, if the audio and video frames sent by the server are received, the audio and video frames sent by the server can replace the audio and video frames currently played by the audio and video frame acquisition client. At the same time, the audio and video frames sent by the server are published, that is, the audio and video frames sent by the server are visible to other clients.

[0072] That is to say, there is a certain duration in the process of the server's pronunciation video frame. In order to improve the user experience, the user does not need to have a certain waiting time, and the audio and video frames stored locally can be preferentially displayed in the audio and video acquisition client. In the pronunciation video frame under the server, it can be replaced to achieve the effect that the user does not perceive, thereby improving the user experience. At this time, for the user corresponding to the audio and video acquisition client, the audio and video have been published, but for users of other clients, the audio and video frames published have not yet been seen.

[0073] It should also be noted that if the server receives a data packet, it needs to obtain the corresponding special effect resource package according to the content in the data packet and merge the special effect resource package with the stored audio and video frames before sending it down, which requires a certain amount of waiting time, resulting in a poor user experience. The advantage of displaying the locally stored audio and video frames on the audio and video acquisition client is that the audio and video frames can be quickly previewed and played. For users of the audio and video acquisition client, the audio and video frames have been successfully released and are in a browsable state, which reduces the user's waiting time and improves the user's experience.

[0074] In this embodiment, the method of replacing the audio and video frames currently displayed by the audio and video acquisition client based on the audio and video frames sent by the server can be: based on the display timestamp of the audio and video acquisition client, display the audio and video frames sent by the server.

[0075] That is to say, in order to improve the smoothness of video playback and the user's imperceptibility, after the server sends down the audio and video, it can obtain the display timestamp corresponding to the audio and video frame currently played by the audio and video acquisition client, that is, the audio and video frame corresponding to the time point currently being displayed, and the sent audio and video frame can also be jumped to the corresponding display timestamp for playback to achieve the effect of smooth playback.

[0076] The technical solution provided by the embodiments of the present disclosure can send audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video. Furthermore, when the audio and video publishing conditions are detected, in order to avoid a certain delay in the server downloading the audio and video frames, which causes the user to wait for a certain period of time and thus causes a poor user experience, the locally stored audio and video frames can be displayed in the audio and video collection client, and the audio and video frames received from the server can be replaced with the audio and video frames displayed by the audio and video collection client, and the audio and video frames sent by the server can be published, which solves the problem of certain delays and lags when the audio and video frames are uploaded when publishing and pushed to other clients by the server, and achieves the technical effect of efficient audio and video publishing.

[0077] Figure 2 This is a flowchart of an audio and video processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to any situation where the client needs to collect audio and video and the server needs to separate the audio and video. The implementation of the technical solution of this embodiment can be performed by the server, and the method can be performed by an audio and video processing device, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC or a server, etc. Among them, the technical terms that are the same as or corresponding to the above embodiments are not repeated here.

[0078] like Figure 2 As shown, the method includes:

[0079] S210, receiving and storing audio and video frames sent by the audio and video acquisition client; wherein the audio and video frames are sent during the process of acquiring audio and video.

[0080] It can be understood that during the audio and video acquisition process, the acquired audio and video frames can be sent to the server. The server can receive the audio and video frames sent by the audio and video acquisition client.

[0081] When the server receives the audio and video frames, the audio and video frames can be stored. In order to facilitate the subsequent tracking of the audio and video frames, the server storage identifier can be determined based on the storage address of the audio and video frames, and the server storage identifier can be fed back to the audio and video acquisition client, so that when the audio and video acquisition client sends the audio and video frames, the server storage identifier can be carried, so that the server can store the continuously collected audio and video frames to the corresponding storage address based on the server storage identifier. At the same time, if the client has corresponding editing operations, in order to achieve the synchronization effect, the corresponding audio and video frames can be retrieved based on the server storage identifier, and the same processing as the client's editing operation can be performed. Optionally, the server storage identifier storing the audio and video frames is fed back to the audio and video acquisition client, so that the corresponding audio and video frames can be retrieved from the server based on the server storage identifier.

[0082] S220: When receiving the audio and video publishing instruction, the stored audio and video frames are sent to the audio and video acquisition client to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames.

[0083] The audio and video publishing instruction is generated when it is detected that the audio and video publishing condition is met. That is, when the audio and video publishing condition is met, the audio and video publishing instruction can be generated and sent to the server, and the server can receive the audio and video publishing instruction.

[0084] Specifically, when the server receives the audio and video publishing instruction, the server can send the stored audio and video frames to the audio and video acquisition client, and push the audio and video frames to other clients. The specific push method is not limited in this embodiment. When the audio and video acquisition client receives the audio and video frames sent by the server, the audio and video frames currently displayed by the audio and video acquisition client can be replaced with the audio and video frames sent by the server. At the same time, in order to ensure the continuity of the audio and video frame playback, the display timestamp of the audio and video frame played by the audio and video acquisition client can be determined, and the audio and video frames sent by the server can continue to be played based on the display timestamp.

[0085] It should be noted that before the server downloads the audio and video frames, the audio and video frames displayed by the audio and video acquisition client are the audio and video frames stored locally in the audio and video acquisition client.

[0086] Optionally, a server storage identifier and element association information and corresponding added attribute information of at least one special effect element sent by the audio and video acquisition client are received; based on the server storage identifier, the element association information and the corresponding added attribute information, at least one special effect element is added to the audio and video frame; wherein, at least one special effect element is generated by an editing operation on the audio and video frame on the audio and video acquisition client, the element association information includes a resource package identifier of the special effect element, and the added attribute information is position information and / or timestamp information of the special effect element in the corresponding audio and video frame.

[0087] Among them, the editing operation of the audio and video frames can be an operation of adding special effect elements to the audio and video frames, or an operation of deleting a certain frame or multiple frames, or an operation of trimming the audio and video frames, or an operation of removing an object in the audio and video frames.

[0088] Specifically, the server storage identifier, element association information of at least one special effect element, and corresponding added attribute information may be sent to the server. After receiving such information, the server may retrieve the corresponding audio and video frame and add the corresponding special effect element to it.

[0089] In this embodiment, adding corresponding special effect elements to audio and video frames in the server side can be based on the server side storage identifier, element association information and corresponding added attribute information, and at least one special effect element is added to the audio and video frame, including: retrieving the audio and video frame based on the server side storage identifier; retrieving the corresponding special effect resource package based on the resource package identifier in the element association information; determining the display time and / or display position of the corresponding special effect element in the audio and video frame based on the timestamp information and / or position information of at least one special effect element; based on the display time and / or display position, adding the special effect element in the special effect resource package to the audio and video frame to obtain an audio and video frame with at least one special effect element added.

[0090] The technical solution provided by the embodiments of the present disclosure can send audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video. Furthermore, when the audio and video publishing conditions are detected, in order to avoid a certain delay in the server downloading the audio and video frames, which causes the user to wait for a certain period of time and thus causes a poor user experience, the locally stored audio and video frames can be displayed in the audio and video collection client, and the audio and video frames received from the server can be replaced with the audio and video frames displayed by the audio and video collection client, and the audio and video frames sent by the server can be published, which solves the problem of certain delays and lags when the audio and video frames are uploaded when publishing and pushed to other clients by the server, and achieves the technical effect of efficient audio and video publishing.

[0091] Figure 3 A flow chart of the interaction between a server and a client provided in an embodiment of the present disclosure is provided. On the basis of the aforementioned embodiment, the technical solution provided in the embodiment of the present disclosure can be explained by taking the sequential processing between the server and the client as an example. The specific implementation method can refer to the detailed description of the present embodiment, wherein the technical terms that are the same as or corresponding to the aforementioned embodiment are not repeated here.

[0092] like Figure 3 As shown, the method includes:

[0093] S310: When the audio and video acquisition client acquires audio and video frames, the audio and video frames are stored locally and sent to the server.

[0094] It can be understood that in the process of shooting audio and video frames, an audio and video stream (TS stream) can be encoded and output in real time and pushed to the server. The TS stream can be understood as a streaming encapsulation of audio and video, which supports downloading only a part for playback. At the same time, the captured audio and video frames are stored locally. This method is different from the current method of only storing the captured audio and video frames locally and uploading them to the server when they are released.

[0095] S320: The server stores the received audio and video frames, and feeds back a server storage identifier to the audio and video acquisition client.

[0096] It can be understood that after the server receives the audio and video frame, it can be stored in the server and generate a server storage identifier corresponding to the audio and video frame. The server storage identifier can be the storage address of the audio and video frame, or it can be the storage connection corresponding to the audio and video frame. Further, after the server storage identifier is generated, the server storage identifier can be sent to the audio and video acquisition client.

[0097] S330: Add at least one special effect element to the locally stored audio and video frames, and determine element association information and added attribute information corresponding to the at least one special effect element.

[0098] It can be understood that at least one special effect element can be added to at least one locally stored audio and video frame on the editing page. At the same time, the element association information and added attribute information corresponding to the special effect element can be recorded. A corresponding data packet is generated based on the element association information and added attribute information of each special effect element.

[0099] S340: When it is detected that the audio and video publishing conditions are met, the locally stored audio and video frames are played on the audio and video acquisition client, and at the same time, the server storage identifier, element association information of at least one special effect element and corresponding added attribute information are sent to the server.

[0100] It can be understood that when it is detected that the audio and video publishing conditions are met, the locally stored audio and video frames can be played on the audio and video acquisition client. If a special effect element is added, the audio and video frame played can be an audio and video frame with the special effect element added. If no special effect element is added, the audio and video frame played can be an audio and video frame without the special effect element added. Further, the server storage identifier and the data packet generated based on the element association information of at least one special effect element and the corresponding added attribute information can be sent to the server. This method is different from the prior art, in which the audio and video frames obtained by shooting are stored locally, and the editing process uses this audio and video frame for editing. After the editing is completed, a new video is synthesized locally. When the new and synthesized video is uploaded to the server during publishing, there is a problem of a large amount of uploaded data, which in turn causes a delay in publishing. It is achieved that only incremental data (element association information and added attribute information of special effect elements) is uploaded, which greatly reduces the amount of data uploaded, thereby improving data processing efficiency.

[0101] S350: The server retrieves the stored audio and video frames based on the server storage identifier, and adds corresponding special effect elements to the audio and video based on element association information and added attributes of at least one special effect element.

[0102] It can be understood that the server can retrieve the stored audio and video frames based on the received server storage identifier, and retrieve the corresponding special effect resource package based on the resource package identifier in the element association information of the special effect element, and add the special effect element corresponding to the special effect resource package to the audio and video frame based on the position information and / or timestamp information in the added attribute information. Since the performance of the server is relatively good and the processing efficiency is relatively high, the special effect element can be quickly integrated into the audio and video frame to obtain the published audio and video, which greatly shortens the publishing time and saves time.

[0103] S360: Based on the server, the audio and video frames with special effect elements added are sent to the audio and video acquisition client to replace the audio and video frames played by the audio and video acquisition client.

[0104] It can be understood that after the server finishes rendering, the audio and video frames with special effect elements added can be sent to the audio and video acquisition client. After the client receives the audio and video frames sent by the server, the audio and video frames played by the audio and video acquisition client can be replaced. When replacing, the audio and video frames can be jumped to the playback position corresponding to the audio and video frames sent to the client according to the playback timestamp of the audio and video acquisition client to continue playing the audio and video frames.

[0105] S370, release audio and video.

[0106] It can be understood that when the server sends the audio and video frame to the audio and video acquisition client, it can be understood that the audio and video frame is released. At this time, other clients can browse the audio and video frame.

[0107] The technical solution provided by the embodiments of the present disclosure can send audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video. Furthermore, when the audio and video publishing conditions are detected, in order to avoid a certain delay in the server downloading the audio and video frames, which causes the user to wait for a certain period of time and thus causes a poor user experience, the locally stored audio and video frames can be displayed in the audio and video collection client, and the audio and video frames received from the server can be replaced with the audio and video frames displayed by the audio and video collection client, and the audio and video frames sent by the server can be published, which solves the problem of certain delays and lags when the audio and video frames are uploaded when publishing and pushed to other clients by the server, and achieves the technical effect of efficient audio and video publishing.

[0108] Figure 4 is a schematic diagram of the structure of an audio and video processing device provided by an embodiment of the present disclosure. The device is configured in a client, such as Figure 4 As shown, the device includes: an audio and video acquisition module 410, an audio and video first playback module 420 and an audio and video second playback module 430.

[0109] The audio and video acquisition module 410 is used to send the audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video frames; the audio and video first playback module 420 is used to display the locally stored audio and video frames on the audio and video acquisition client when it is detected that the audio and video publishing conditions are met; the audio and video second playback module 430 is used to replace the audio and video frames displayed by the audio and video acquisition client based on the audio and video frames and publish the audio and video frames when the audio and video frames sent by the server are received.

[0110] On the basis of the above technical solution, the device includes a special effect element editing module, including:

[0111] A special effect adding unit, used to add at least one special effect element to the locally stored audio and video frames to update the locally stored audio and video frames; an information recording unit, used to record element association information of the at least one special effect element and added attribute information in the audio and video frames, so as to integrate the at least one special effect element into the audio and video frames stored on the server based on the element association information and the added attribute information; wherein the element association information includes a resource package identifier of the special effect element, and the added attribute information includes at least one of location information and a timestamp.

[0112] On the basis of the above technical solutions, the device further includes:

[0113] A data sending module is used to send the server storage identifier of the audio and video frame, as well as the element association information and corresponding added attributes of the at least one special effect element to the server, so that the server can retrieve the corresponding audio and video frame based on the server storage identifier, and add special effect elements to the audio and video frame based on the element association information and the added attributes.

[0114] Based on the above technical solutions, the audio and video frames sent by the server are audio and video frames with at least one special effect element added, and the audio and video second playback module is also used to display the audio and video frames sent by the server based on the display timestamp of the audio and video acquisition client.

[0115] The technical solution provided by the embodiments of the present disclosure can send audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video. Furthermore, when the audio and video publishing conditions are detected, in order to avoid a certain delay in the server downloading the audio and video frames, which causes the user to wait for a certain period of time and thus causes a poor user experience, the locally stored audio and video frames can be displayed in the audio and video collection client, and the audio and video frames received from the server can be replaced with the audio and video frames displayed by the audio and video collection client, and the audio and video frames sent by the server can be published, which solves the problem of certain delays and lags when the audio and video frames are uploaded when publishing and pushed to other clients by the server, and achieves the technical effect of efficient audio and video publishing.

[0116] The audio and video processing device provided in the embodiments of the present disclosure can execute the audio and video processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0117] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0118] Figure 5 is a schematic diagram of the structure of an audio and video processing device provided by an embodiment of the present disclosure. The device is configured in a server, such as Figure 5 As shown, the device includes: an audio and video receiving module 510 and an audio and video sending module 520.

[0119] The audio and video receiving module 510 is used to receive and store the audio and video frames sent by the audio and video acquisition client; wherein the audio and video frames are sent during the process of acquiring audio and video; the audio and video sending module 520 is used to send the stored audio and video frames to the audio and video acquisition client when receiving the audio and video publishing instruction, so as to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames; wherein the audio and video publishing instruction is determined based on the audio and video publishing conditions.

[0120] Based on the above technical solutions, the device also includes: a data feedback module, which is used to feed back the server storage identifier storing the audio and video to the audio and video acquisition client, so as to retrieve the corresponding audio and video frames from the server based on the server storage identifier.

[0121] On the basis of the above technical solutions, the device further includes:

[0122] An attribute information receiving module is used to receive a server storage identifier and element association information and corresponding added attribute information of at least one special effect element sent by the audio and video acquisition client; a special effect synthesis module is used to add at least one special effect element to the audio and video frame based on the server storage identifier, element association information and corresponding added attribute information; wherein, the at least one special effect element is an element added to the audio and video frame on the audio and video acquisition client, the element association information includes a resource package identifier of the special effect element, and the added attribute information is the position information and / or timestamp information of the special effect element in the corresponding audio and video frame.

[0123] On the basis of the above technical solutions, the special effects synthesis module includes:

[0124] An audio and video retrieval unit is used to retrieve the audio and video frame based on the server storage identifier; a resource package retrieval unit is used to retrieve the corresponding special effect resource package based on the resource package identifier in the element association information; a display information determination unit is used to determine the display time and / or display position of the corresponding special effect element in the audio and video frame based on the timestamp information and / or position information of at least one special effect element; a special effect adding unit is used to add the special effect element in the special effect resource package to the audio and video frame based on the display time and / or display position, so as to obtain an audio and video frame with at least one special effect element added.

[0125] The technical solution provided by the embodiments of the present disclosure can send audio and video frames to the server during the process of collecting audio and video, so that the server can store the audio and video. Furthermore, when the audio and video publishing conditions are detected, in order to avoid a certain delay in the server downloading the audio and video frames, which causes the user to wait for a certain period of time and thus causes a poor user experience, the locally stored audio and video frames can be displayed in the audio and video collection client, and the audio and video frames received from the server can be replaced with the audio and video frames displayed by the audio and video collection client, and the audio and video frames sent by the server can be published, which solves the problem of certain delays and lags when the audio and video frames are uploaded when publishing and pushed to other clients by the server, and achieves the technical effect of efficient audio and video publishing.

[0126] The audio and video processing device provided in the embodiments of the present disclosure can execute the audio and video processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0127] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0128] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 6 , which shows an electronic device (eg, Figure 6 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0129] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An edit / output (I / O) interface 605 is also connected to the bus 604.

[0130] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0131] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0132] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0133] The electronic device provided in the embodiment of the present disclosure and the audio and video processing method provided in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0134] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the audio and video processing method provided by the above embodiments is implemented.

[0135] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0136] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0137] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0138] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0139] In the process of collecting audio and video, the audio and video frames are sent to the server so that the server stores the audio and video frames;

[0140] When it is detected that the audio and video publishing conditions are met, the locally stored audio and video frames are displayed on the audio and video acquisition client;

[0141] When receiving the audio and video frames sent by the server, replace the audio and video frames displayed by the audio and video acquisition client based on the audio and video frames and publish the audio and video frames. Or,

[0142] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0143] Receive and store audio and video frames sent by the audio and video acquisition client; wherein the audio and video frames are sent during the process of collecting audio and video;

[0144] When receiving the audio and video publishing instruction, the stored audio and video frames are sent to the audio and video acquisition client to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames;

[0145] Wherein, the audio and video publishing instruction is determined based on the audio and video publishing conditions.

[0146] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0149] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0150] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An audio and video processing method, characterized in that: Applied in a client, the method includes: In the process of collecting audio and video, the audio and video frames are sent to the server so that the server stores the audio and video frames; When it is detected that the audio and video publishing conditions are met, the locally stored audio and video frames are displayed on the audio and video acquisition client; When the audio and video frames sent by the server are received, the audio and video frames displayed by the audio and video acquisition client are replaced based on the audio and video frames and the audio and video frames are released.

2. The method according to claim 1, characterized in that The sending of the audio and video frames to the server includes: After the audio and video frames are stream-encapsulated, they are uploaded to the server.

3. The method according to claim 1, characterized in that Before the audio and video publishing condition is detected to be met, the method further includes: Adding at least one special effect element to the locally stored audio and video frames to update the locally stored audio and video frames; Recording element association information of the at least one special effect element and added attribute information in the audio and video frame, so as to merge the at least one special effect element into the audio and video frame stored in the server based on the element association information and the added attribute information; The element association information includes the resource package identifier of the special effect element, and the added attribute information includes at least one of location information and timestamp information.

4. The method according to claim 3, characterized in that After the audio and video publishing condition is detected to be met, the method further includes: The server storage identifier of the audio and video frame, as well as the element association information and corresponding added attribute information of the at least one special effect element are sent to the server, so that the server retrieves the corresponding audio and video frame based on the server storage identifier, and adds special effect elements to the audio and video frame based on the element association information and the added attribute information.

5. The method according to claim 4, characterized in that The audio and video frames sent by the server are audio and video frames with at least one special effect element added. When the audio and video frames displayed by the audio and video acquisition client are replaced based on the audio and video frames, the method further includes: Based on the presentation timestamp of the audio and video acquisition client, the audio and video frames sent by the server are presented.

6. An audio and video processing method, characterized in that: Applied in the server, the method includes: Receive and store audio and video frames sent by the audio and video acquisition client; wherein the audio and video frames are sent during the process of collecting audio and video; When receiving the audio and video publishing instruction, the stored audio and video frames are sent to the audio and video acquisition client to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames; The audio and video publishing instruction is generated when it is detected that the audio and video publishing conditions are met.

7. The method according to claim 6, characterized in that After storing the audio and video frames, the method further includes: Feedback the server storage identifier storing the audio and video frame to the audio and video acquisition client, so as to retrieve the corresponding audio and video frame from the server based on the server storage identifier.

8. The method according to claim 6, characterized in that Also includes: Receiving a server storage identifier and element association information and corresponding added attribute information of at least one special effect element sent by the audio and video acquisition client; Adding at least one special effect element to the audio and video frame based on the server storage identifier, element association information and corresponding added attribute information; Among them, the at least one special effect element is generated by the editing operation of the audio and video frame on the audio and video acquisition client, the element association information includes the resource package identifier of the special effect element, and the added attribute information is the position information and / or timestamp information of the special effect element in the corresponding audio and video frame.

9. The method according to claim 8, characterized in that The adding at least one special effect element to the audio and video frame based on the server storage identifier, element association information and corresponding added attribute information includes: Retrieving the audio and video frames based on the server storage identifier; Retrieving a corresponding special effect resource package based on the resource package identifier in the element association information; Based on the timestamp information and / or position information of at least one special effect element, determine the display time and / or display position of the corresponding special effect element in the audio and video frame; Based on the display time and / or display position, the special effect elements in the special effect resource package are added to the audio and video frame to obtain an audio and video frame with at least one special effect element added.

10. An audio and video processing device, characterized in that: Configured in the client, the device includes: An audio and video acquisition module, used for sending audio and video frames to a server during the process of acquiring audio and video, so that the server stores the audio and video frames; The first audio and video playback module is used to display the locally stored audio and video frames on the audio and video acquisition client when it is detected that the audio and video publishing conditions are met; The second audio and video playback module is used to replace the audio and video frames displayed by the audio and video acquisition client based on the audio and video frames when receiving the audio and video frames sent by the server and publish the audio and video frames.

11. An audio and video processing device, characterized in that: Configured in the server, the device includes: An audio and video receiving module, used to receive and store audio and video frames sent by an audio and video acquisition client; wherein the audio and video frames are sent during the process of acquiring audio and video; An audio and video sending module, configured to send the stored audio and video frames to the audio and video acquisition client upon receiving the audio and video publishing instruction, so as to replace the locally stored audio and video frames displayed by the audio and video acquisition client based on the audio and video frames; Wherein, the audio and video publishing instruction is determined based on the audio and video publishing conditions.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the audio and video processing method as described in any one of claims 1-5 or 6-9.

13. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute the audio and video processing method as described in any one of claims 1-5 or 6-9 when executed by a computer processor.

Images