Live screen processing method, device, equipment and storage medium

Abstract

The present application discloses a live video processing method, apparatus, device, and storage medium. The method includes: upon detecting a live broadcast ban instruction, obtaining a preloaded first replacement image; detecting a live video stream sent by a first client, generating a second replacement image based on the video packet type in the live video stream and the first replacement image; and sending the second replacement image to a second client, for the second client to display the image based on the second replacement image. This solution can reduce the impact on users watching the live broadcast and lower the encoding and decoding costs of the video stream.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for processing live images. Background Art

[0002] With the advancement of network and device hardware capabilities, the live streaming industry has experienced significant growth. Due to compliance requirements for live streaming, real-time review of live content is required, and appropriate measures must be taken for live streaming that violates regulations to ensure compliance.

[0003] In related technologies, when a live broadcast violation is detected, the live broadcast image is directly replaced with a fixed black screen. This replacement method can achieve the purpose of prohibiting illegal content, but because the black screen image is significantly different from the original live broadcast image, replacing it with a fixed black screen will cause changes in video stream properties, such as frame rate, bit rate, number of sent and received packets, and link bandwidth. This can cause live broadcast clarity jitter after the live broadcast resumes. Summary of the Invention

[0004] The embodiments of the present application provide a live broadcast picture processing method, apparatus, device and storage medium, which solve the problem of clarity change when adjusting the live broadcast picture in the related art, can reduce the impact on live broadcast viewing users and reduce the encoding and decoding cost of the video stream.

[0005] In a first aspect, an embodiment of the present application provides a live broadcast image processing method, which is applied to a server, and the method includes:

[0006] When a live broadcast ban instruction is detected, obtaining a preloaded first replacement image;

[0007] detecting a live video stream sent by a first client, and generating a second replacement image according to a video packet type in the live video stream and the first replacement image;

[0008] The second replacement image is sent to the second client, so that the second client performs screen display based on the second replacement image.

[0009] In a second aspect, an embodiment of the present application further provides a live image processing device, comprising:

[0010] a replacement image generation module configured to, upon detecting a live broadcast ban instruction, obtain a preloaded first replacement image, detect a live video stream sent by a first client, and generate a second replacement image based on a video packet type in the live video stream and the first replacement image;

[0011] The replacement image sending module is configured to send the second replacement image to the second client, so that the second client can display the image based on the second replacement image.

[0012] In a third aspect, an embodiment of the present application further provides a live image processing device, the device comprising:

[0013] one or more processors;

[0014] a storage device for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the live screen processing method described in the embodiment of the present application.

[0016] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to execute the live image processing method described in the embodiment of the present application.

[0017] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device executes the live screen processing method described in the embodiment of the present application.

[0018] In an embodiment of the present application, when a live broadcast ban instruction is detected, a preloaded first replacement image is obtained, the live video stream sent by the first client is detected, a second replacement image is generated based on the video packet type in the live video stream and the first replacement image, and then the second replacement image is sent to the second client for the second client to display the screen based on the second replacement image. When handling violations in live broadcasts, this solution dynamically obtains a second replacement image based on the original live video stream and the pre-generated first replacement image to replace the live screen. The second replacement image is associated with the original live video stream image, avoiding changes in video stream attributes caused by the use of fixed replacement images, which in turn leads to jitter in the live screen clarity after the live broadcast is resumed. The above solution can reduce the impact on users watching the live broadcast. At the same time, the use of the preloaded first replacement image to generate the second replacement image eliminates the need for real-time transcoding based on the live video stream to obtain the replacement image, reducing the encoding and decoding cost of the video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic diagram of an application scenario of a live broadcast image processing method provided in an embodiment of the present application;

[0020] Figure 2A flowchart of a live image processing method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a live broadcast interface provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of a method for generating a second replacement image provided in an embodiment of the present application;

[0023] Figure 5 A flowchart of another method for generating a second replacement image provided in an embodiment of the present application;

[0024] Figure 6 A flowchart of a method for reviewing a video stream during live video processing provided in an embodiment of the present application;

[0025] Figure 7 A structural block diagram of a live image processing device provided in an embodiment of the present application;

[0026] Figure 8 A structural diagram of a live image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the embodiments of the present application, and are not intended to limit the embodiments of the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the embodiments of the present application, rather than all structures.

[0028] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0029] Figure 1 A schematic diagram of an application scenario of a live image processing method provided in an embodiment of the present application is shown as follows: Figure 1As shown, the first client 01 is the device that generates the live video stream, such as the device used by the anchor when broadcasting live; the second client 02 can be the device used by the audience watching the live broadcast, or it can be the device used by other anchors. That is, after the first client 01 generates the live video stream, it can be transmitted to the second client 02 for display of the live screen. There can be multiple second clients 02, that is, the client used by each user when watching the live broadcast can be understood as a second client. When the first client 01 generates the live video stream, it forwards the live video stream through the server 03. The server 03 forwards the live video stream of the first client 01 to multiple second clients 02 for display. Optionally, the server can be a front-end server, which runs a front-end service process inside, and the front-end service process realizes the reception and forwarding of the live video stream. Optionally, when the first client 01 starts a live broadcast or performs a microphone operation in a multi-person live broadcast scenario, a corresponding communication connection is established with the server 03. When the second client 02 logs in to the live broadcast room corresponding to the corresponding second client 01, a communication connection is established with the server 03. At this time, the server 03 pushes the live video stream of the first client 01 to the second client 02. The second client 02 thereby realizes the display of the corresponding live broadcast screen, and the user can watch the live broadcast.

[0030] Figure 2 The flowchart of a live image processing method provided in an embodiment of the present application can be applied to a server and specifically includes the following steps:

[0031] Step S101: When a live broadcast ban instruction is detected, a preloaded first replacement image is obtained.

[0032] The live broadcast ban command is used to instruct a live broadcast to be banned. Optionally, it is generated when violations are discovered during the review of live broadcast content and require appropriate action. The specific review process can be completed on the current server or through a server specifically configured to run the review system. If the review is conducted through another server, the live broadcast ban command is a command received from the other server.

[0033] In one embodiment, when a live broadcast ban instruction is detected, it means that the live broadcast image corresponding to the live broadcast ban instruction has violated the regulations and needs to be processed accordingly. At this time, a preloaded first replacement image is obtained. Optionally, the first replacement image is pre-generated and pre-loaded when the corresponding service process is started.

[0034] In one embodiment, the process of generating the first replacement image is also included. Optionally, the first replacement image can be a black screen image containing SPS (Sequence Parameter Set) frames, PPS (Picture Parameter Set) frames, I frames, P frames and B frames, or it can be other special settings of screen images containing SPS frames, PPS frames, I frames, P frames and B frames. The first replacement image is an image that complies with the requirements. Among them, the SPS frame and the PPS frame contain the information parameters (such as resolution) required for initializing stream decoding, and their length is about a dozen bytes. The same video stream parameters are fixed, and the same frame can appear repeatedly. In one embodiment, the generated first replacement image includes a variety of different resolutions, such as generating first replacement images for resolutions of 480P, 720P, and 1080P, respectively, so that when the second replacement image is subsequently generated, the first replacement image with the same image resolution as the live video stream is selected.

[0035] Step S102: Detect the live video stream sent by the first client, and generate a second replacement image according to the video packet type in the live video stream and the first replacement image.

[0036] The first client may be a device that initiates a live broadcast, such as a device used by a live streamer to start a live broadcast. The first client is the device that is associated with the live broadcast ban instruction and requires live broadcast image processing. Optionally, the live broadcast ban instruction carries an identifier of the device that requires live broadcast image processing, that is, an identifier of the first client, or carries corresponding live broadcast room information, and the corresponding first client that requires live broadcast image processing is determined by querying the live broadcast room information.

[0037] When the first client executes the live broadcast function, it pushes the live video stream to the server in real time, and the server processes and forwards the live video stream. During the live image processing, the live video stream sent by the first client is inspected. If the image corresponding to the live video stream contains illegal content during the review process, the live image needs to be replaced. The image that ultimately needs to be replaced, namely the second replacement image, is generated based on the video packet type in the live video stream and the first replacement image obtained above.

[0038] Optionally, the video packet type in the live video stream may be an SPS frame type, a PPS frame type, an I frame type, a P frame type, or a B frame type, and different second replacement images are generated based on different video packet types. The difference in the images includes differences in the frame rate, bit rate, and number of associated sent and received packets.

[0039] In one embodiment, a corresponding second replacement image can be generated for image frames whose video packet types are SPS frame types and PPS frame types; a corresponding second replacement image can be generated for image frames whose video packet types are I frame types; and a corresponding second replacement image can be generated for image frames whose video packet types are P frame types and B frame types.

[0040] In one embodiment, when the video packet types in the live video stream are SPS frames and PPS frames, a second replacement image is generated by replacing the SPS frames and PPS frames in the first replacement image, that is, the SPS frames and PPS frames in the original live video stream are replaced by the SPS frames and PPS frames in the first replacement image. The two frames have similar lengths and are directly replaced.

[0041] In one embodiment, when the video packet type in the live video stream is an I-frame, the I-frame in the first replacement image is padded to generate a second replacement image. Optionally, the I-frame image of the pre-generated first replacement image is relatively small. Therefore, for the I-frame of the live video stream that is larger than the I-frame image, the smaller I-frame image can be padded to keep the size consistent or close to that of the I-frame image of the live video stream. Taking the first replacement image as a black screen image as an example, the size of its I-frame image is relatively small. In order to ensure consistency with the overall size and length of the live video stream, the I-frame in the first replacement image is padded to generate the second replacement image.

[0042] In one embodiment, when the video packet type in the live video stream is a P-frame or B-frame, a second replacement image is generated by padding the P-frames or B-frames with the same position offset as the first replacement image. The process for generating a second replacement image of the I-frame type is similar to padding the corresponding P-frames or B-frames, except that the P-frames or B-frames with the same position offset as the first replacement image are first determined. The position offset indicates the degree of offset of the P-frame or B-frame relative to the associated I-frame. For example, taking the frame sequence of I-frame, P-frame, B-frame, B-frame, and P-frame as an example, the position offset of the second P-frame is 4.

[0043] Step S103: Send the second replacement image to the second client, so that the second client can display the image based on the second replacement image.

[0044] After generating a corresponding second replacement image based on the video packet type of the different live video streams, the second replacement image is sent to a second client, which may be a client watching the live broadcast of the first client. The server replaces the video frames in the live video stream to generate corresponding second image frames, which are forwarded to the second client. In other words, the original live video stream is replaced with a video stream composed of the second replacement image, which is then displayed in real time and in compliance on the second client.

[0045] In one embodiment, upon detecting a live broadcast resume instruction, the server resumes the live broadcast image based on the I-frame in the live broadcast video stream sent by the first client. That is, during the recovery process, the server resumes the live broadcast image starting from the first detected I-frame, so that the first client can continue the normal live broadcast process. This ensures that the live broadcast image is played normally and the display effect is optimized.

[0046] In one embodiment, the second client also plays the second replacement image, which may be: the second client displays the second replacement image at the original screen position of the first client. Figure 3 As shown, Figure 3 A schematic diagram of a live broadcast interface provided in an embodiment of the present application is provided. The live broadcast interface is displayed in a second client and consists of four live broadcast sub-interfaces, including live broadcast sub-interface 1, live broadcast sub-interface 2, live broadcast sub-interface 3, and live broadcast sub-interface 4. Each live broadcast sub-interface corresponds to a live broadcast room. Assuming that the live video stream of the first client corresponding to live broadcast sub-interface 2 violates the rules, the image displayed in live broadcast sub-interface 2 is replaced with the second replacement image, and the live broadcast images of the other three live broadcast sub-interfaces, namely live broadcast sub-interface 1, live broadcast sub-interface 3, and live broadcast sub-interface 4, are not affected. This method ensures that the live broadcast process of other live broadcast rooms is not affected, optimizes the processing process of live broadcast violations, and avoids the problem of poor user experience caused by users exiting the live broadcast room due to direct disconnection of the live broadcast connection.

[0047] It can be seen from the above scheme that when a live broadcast ban instruction is detected, a preloaded first replacement image is obtained, the live video stream sent by the first client is detected, and a second replacement image is generated based on the video packet type in the live video stream and the first replacement image. The second replacement image is then sent to the second client for the second client to display the screen based on the second replacement image. When handling violations in live broadcasts, this scheme dynamically obtains the second replacement image based on the original live video stream and the pre-generated first replacement image to replace the live screen. The second replacement image is associated with the original live video stream image, avoiding the change in video stream attributes caused by the use of a fixed replacement image, which in turn causes the problem of jitter in the live screen clarity after the live broadcast is resumed. The above scheme can reduce the impact on users watching the live broadcast. At the same time, the use of the preloaded first replacement image to generate the second replacement image eliminates the need for real-time transcoding based on the live video stream to obtain the replacement image, reducing the encoding and decoding cost of the video stream.

[0048] Figure 4 A flowchart of a method for generating a second replacement image is provided in an embodiment of the present application, such as Figure 4 As shown, specifically including:

[0049] Step S201: When a live broadcast ban instruction is detected, a preloaded first replacement image is obtained.

[0050] Step S202: Detect the live video stream sent by the first client. When the video packet type in the live video stream is an I frame, determine the padding size according to the I frame size in the live video stream and the I frame size of the first replacement image.

[0051] In one embodiment, the size of the I-frame and the number of unpackings during transmission are recorded. For portions of the I-frame in the live video stream that are larger than the I-frame in the first replacement image, padding is used to keep the two consistent or similar. For example, assuming the size of the I-frame in the live video stream is 153,146 bytes, and unpacking is performed at 1,024 bytes, 150 video packets are split. Assuming the size of the I-frame in the first replacement image is 146 bytes, the calculated padding size is the difference between 153,146 and 146, which is 153,000 bytes.

[0052] Step S203: Determine the number of padding frames according to the padding size and the padding frame size, and insert the padding frames of the number of padding frames before the I frame in the first replacement image to generate a second replacement image.

[0053] In one embodiment, I-frame images are padded using padding frames. Optionally, the padding frames may be SPS frames and PPS frames. Taking the size of SPS frames and PPS frames as 23 bytes, the number of padding frames required is 153,000 divided by 23, resulting in 6,653 SPS frames and PPS frames. Optionally, during the padding process, the determined number of padding frames may be inserted before the I-frame in the first replacement image to generate a second replacement image, thereby ensuring that the size of the replaced I-frame is consistent with or similar to the size of the I-frame in the live video stream.

[0054] Step S204: Send the second replacement image to the second client, so that the second client can display the image based on the second replacement image.

[0055] From the above, it can be seen that when the video packet type in the live video stream is I frame, the padding size is determined according to the I frame size in the live video stream and the I frame size of the first replacement image, and the number of padding frames is determined according to the padding size and the padding frame size. The padding frames of the number of padding frames are inserted before the I frame in the first replacement image to generate a second replacement image, thereby ensuring the consistency of the replacement image and the original video stream image size, avoiding the change of video stream attributes caused by the use of a fixed replacement image, and then causing the problem of jitter in the live picture clarity after the live broadcast is resumed, and ensuring the consistency of the live picture clarity before and after.

[0056] Figure 5 Another flow chart of a method for generating a second replacement image is provided in an embodiment of the present application, such as Figure 5 As shown, specifically including:

[0057] Step S301: When a live broadcast ban instruction is detected, a preloaded first replacement image is obtained.

[0058] Step S302: Detect the live video stream sent by the first client. When the video packet type in the live video stream is a P frame or a B frame, count the offset of the P frame or B frame in the live video stream relative to the associated I frame through a set counter to obtain a count value.

[0059] In one embodiment, the position offset is recorded by a set counter. Optionally, when an I frame is detected, the counter is reset to zero; when a P frame or B frame is detected after the I frame, the corresponding counter value is incremented by 1, and so on. The count value obtained for each P frame or B frame is the respective position offset.

[0060] Step S303: Using the count value as a position offset relative to the I frame, searching for a corresponding P frame or B frame in the first replacement image, and filling the P frame or B frame to generate a second replacement image.

[0061] During the generation of the second replacement image, the same position offset relative to the I frame is determined based on the count value of the counter. That is, the position offset of the P frame or B frame determined in the first replacement image relative to the associated I frame is consistent with the position offset of the P frame or B frame relative to the associated I frame in the live video stream. After determining the P frame or B frame with the consistent position offset, the corresponding P frame or B frame is padded to generate the second replacement image. The specific padded process can be referred to the padded process for the I frame and will not be further described here.

[0062] Optionally, if a P frame or B frame with the same position offset cannot be found in the first replacement image, the I frame in the first replacement image is used to replace the P frame or B frame in the live video stream and fill it accordingly to generate a second replacement image.

[0063] Step S304: Send the second replacement image to the second client, so that the second client can display the image based on the second replacement image.

[0064] From the above, it can be seen that when the video packet type in the live video stream is P frame or B frame, the offset of the P frame or B frame in the live video stream relative to the associated I frame is counted by setting a counter to obtain a count value. The count value is used as the position offset relative to the I frame, and the corresponding P frame or B frame is found in the first replacement image. The P frame or B frame is filled to generate a second replacement image. The image replacement performed in this way can make the video stream composed of the replaced images consistent with the original live video stream in frame rate, bit rate, and number of transmission segmentation packets, thereby reducing the impact on live viewing users and reducing the encoding and decoding cost of the video stream.

[0065] Figure 6 A flowchart of a method for reviewing video streams during live video processing provided in an embodiment of the present application is shown as follows: Figure 6 As shown, specifically including:

[0066] Step S401: When a live broadcast ban instruction is detected, a preloaded first replacement image is obtained.

[0067] Step S402: Detect the live video stream sent by the first client, and generate a second replacement image according to the video packet type in the live video stream and the first replacement image.

[0068] Step S403: Send the second replacement image to the second client, so that the second client can display the image based on the second replacement image.

[0069] Step S404: Synchronously send the live video stream received from the first client to the review system for review, receive the live broadcast recovery instruction fed back by the review system, and when the live broadcast recovery instruction is detected, restore the live screen based on the I frame in the live video stream sent by the first client.

[0070] In one embodiment, while the server is processing the live video, it can simultaneously push the received live video stream to the review system for review. If the review process determines that it complies with regulations, a live broadcast resumption instruction will be issued accordingly. After receiving the live broadcast resumption instruction, the server restores the live video based on the I frame in the live video stream sent by the first client. This not only replaces the live video but also provides an opportunity for rectification. For live broadcasts with multiple microphones connected, this can reduce the impact on other users and viewers in the live broadcast room.

[0071] Figure 7 This is a structural block diagram of a live picture processing device provided in an embodiment of the present application. The device is used to execute the live picture processing method applied to the client provided in the above embodiment, and has the corresponding functional modules and beneficial effects of the execution method. Figure 7As shown, the device specifically includes: a replacement image generating module 101 and a replacement image sending module 102, wherein,

[0072] The replacement image generation module 101 is configured to, upon detecting a live broadcast ban instruction, obtain a preloaded first replacement image, detect a live video stream sent by a first client, and generate a second replacement image based on a video packet type in the live video stream and the first replacement image;

[0073] The replacement image sending module 102 is configured to send the second replacement image to the second client, so that the second client can display the second replacement image.

[0074] It can be seen from the above scheme that when a live broadcast ban instruction is detected, a preloaded first replacement image is obtained, the live video stream sent by the first client is detected, and a second replacement image is generated based on the video packet type in the live video stream and the first replacement image. The second replacement image is then sent to the second client for the second client to display the screen based on the second replacement image. When handling violations in live broadcasts, this scheme dynamically obtains the second replacement image based on the original live video stream and the pre-generated first replacement image to replace the live screen. The second replacement image is associated with the original live video stream image, avoiding the change in video stream attributes caused by the use of a fixed replacement image, which in turn causes the problem of jitter in the live screen clarity after the live broadcast is resumed. The above scheme can reduce the impact on users watching the live broadcast. At the same time, the use of the preloaded first replacement image to generate the second replacement image eliminates the need for real-time transcoding based on the live video stream to obtain the replacement image, reducing the encoding and decoding cost of the video stream.

[0075] In a possible embodiment, the first replacement image includes an SPS frame, a PPS frame, an I frame, a P frame, and a B frame, and the replacement image generation module 102 is configured to:

[0076] In a case where the video packet types in the live video stream are SPS frames and PPS frames, generating a second replacement image by replacing the SPS frames and PPS frames in the first replacement image;

[0077] When the video packet type in the live video stream is an I-frame, padding the I-frame in the first replacement image to generate a second replacement image;

[0078] In a case where the video packet type in the live video stream is a P frame or a B frame, the P frame or the B frame with the same position offset in the first replacement image is padded to generate a second replacement image.

[0079] In a possible embodiment, the replacement image generation module 102 is configured to:

[0080] determining a padding size according to an I-frame size in the live video stream and an I-frame size of the first replacement image, wherein the I-frame size of the first replacement image is smaller than the I-frame size in the live video stream;

[0081] Determine the number of padding frames according to the padding size and the padding frame size;

[0082] Inserting the filler frames of the number of filler frames before the I frame in the first replacement image generates a second replacement image.

[0083] In a possible embodiment, the replacement image generation module 102 is configured to:

[0084] Counting the offset of the P frame or B frame in the live video stream relative to the associated I frame by using a set counter to obtain a count value;

[0085] The count value is used as a position offset relative to the I frame, a corresponding P frame or B frame is searched in the first replacement image, and the P frame or B frame is padded to generate a second replacement image.

[0086] In a possible embodiment, the device also includes an image recovery module 103, which is configured to restore the live screen based on the I frame in the live video stream sent by the first client when a live broadcast recovery instruction is detected after the second replacement image is sent to the second client.

[0087] In a possible embodiment, the image restoration module 103 is further configured to:

[0088] Before the live broadcast resumption instruction is detected, synchronously sending the live broadcast video stream received from the first client to the review system for review;

[0089] Receive the live broadcast resumption instruction fed back by the review system.

[0090] In a possible embodiment, the replacement image generation module 101 is further configured to:

[0091] Before obtaining the preloaded first replacement image, a first replacement image that meets audit requirements is generated, where the first replacement image includes a black screen image.

[0092] Figure 8 A schematic diagram of the structure of a live image processing device provided in an embodiment of the present application is shown as follows: Figure 8 As shown, the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the number of processors 201 in the device can be one or more. Figure 8In the embodiment, a processor 201 is used as an example; the processor 201, the memory 202, the input device 203 and the output device 204 in the device can be connected by a bus or other means. Figure 8 The example of the connection via bus is taken. The memory 202, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the live screen processing method in the embodiment of the present application. The processor 201 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 202, that is, realizes the above-mentioned live screen processing method. The input device 203 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device 204 may include a display device such as a display screen.

[0093] An embodiment of the present application further provides a non-volatile storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, the method for processing a live broadcast image described in the above embodiment is used to perform the following steps:

[0094] When a live broadcast ban instruction is detected, obtaining a preloaded first replacement image;

[0095] detecting a live video stream sent by a first client, and generating a second replacement image according to a video packet type in the live video stream and the first replacement image;

[0096] The second replacement image is sent to the second client, so that the second client performs screen display based on the second replacement image.

[0097] It is worth noting that in the embodiment of the above-mentioned live image processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of this application.

[0098] In some possible implementations, various aspects of the method provided herein may also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps of the method according to the various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the live image processing method described in the embodiments of the present application. The program product may be implemented using any combination of one or more readable media.

Claims

1. A live image processing method, applied to a server, characterized in that: include: When a live broadcast ban instruction is detected, obtaining a preloaded first replacement image, where the first replacement image includes an I frame; Detecting a live video stream sent by a first client, and generating a second replacement image based on a video packet type in the live video stream and the first replacement image, wherein generating the second replacement image based on the video packet type in the live video stream and the first replacement image includes: determining a padding size based on an I-frame size in the live video stream and an I-frame size of the first replacement image, the I-frame size of the first replacement image being smaller than the I-frame size in the live video stream, determining a number of padding frames based on the padding size and the padding frame size, and inserting the number of padding frames into the I-frames in the first replacement image to generate the second replacement image; The second replacement image is sent to the second client, so that the second client performs screen display based on the second replacement image.

2. The live broadcast image processing method according to claim 1, characterized in that: The first replacement image includes an SPS frame, a PPS frame, a P frame, and a B frame, and generating a second replacement image according to a video packet type in the live video stream and the first replacement image includes: In a case where the video packet types in the live video stream are SPS frames and PPS frames, generating a second replacement image by replacing the SPS frames and PPS frames in the first replacement image; In a case where the video packet type in the live video stream is a P frame or a B frame, the P frame or the B frame with the same position offset in the first replacement image is padded to generate a second replacement image.

3. The live broadcast image processing method according to claim 2, characterized in that: Filling the P frame or B frame with the same position offset in the first replacement image to generate the second replacement image includes: Counting the offset of the P frame or B frame in the live video stream relative to the associated I frame by using a set counter to obtain a count value; The count value is used as a position offset relative to the I frame, a corresponding P frame or B frame is searched in the first replacement image, and the P frame or B frame is padded to generate a second replacement image.

4. The live broadcast image processing method according to any one of claims 1 to 3, characterized in that: After sending the second replacement image to the second client, the method further includes: When a live broadcast resumption instruction is detected, the live broadcast picture is resumed based on the I frame in the live broadcast video stream sent by the first client.

5. The live broadcast image processing method according to claim 4, characterized in that: Before the live broadcast resumption instruction is detected, the method further includes: Synchronously sending the live video stream received from the first client to the review system for review; Receive the live broadcast resumption instruction fed back by the review system.

6. The live broadcast image processing method according to any one of claims 1 to 3, characterized in that: Before obtaining the preloaded first replacement image, the method further includes: generating a first replacement image that meets audit requirements, wherein the first replacement image includes a black screen image; After sending the second replacement image to the second client, the method further includes: The second client displays the second replacement image at the screen position where the first client was originally displayed.

7. A live image processing device, characterized in that: include: a replacement image generation module configured to, upon detecting a live broadcast ban instruction, obtain a preloaded first replacement image, the first replacement image including an I-frame, detect a live video stream sent by a first client, and generate a second replacement image based on a video packet type in the live video stream and the first replacement image; the replacement image generation module being specifically configured to determine a padding size based on an I-frame size in the live video stream and an I-frame size of the first replacement image, the I-frame size of the first replacement image being smaller than the I-frame size in the live video stream, determine a number of padding frames based on the padding size and the padding frame size, and insert the number of padding frames into before the I-frame in the first replacement image to generate the second replacement image; The replacement image sending module is configured to send the second replacement image to the second client, so that the second client can display the image based on the second replacement image.

8. A live broadcast image processing device, comprising: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the one or more processors to implement the live image processing method according to any one of claims 1 to 6.

9. A non-volatile storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to execute the live image processing method according to any one of claims 1 to 6.

Images