Video processing method and device based on multiple terminals, equipment, medium and product

Abstract

The invention discloses a video processing method and device based on multiple terminals, equipment, a medium and a product, and relates to the technical field of video conferencies.The method comprises the steps that if a preset instruction that a terminal is controlled to insert additional information is received, the function of receiving an I frame request is forbidden; creating an additional processor based on the additional information insertion instruction, if the additional processor is created, starting a function of receiving an I-frame request, determining the I-frame request corresponding to the plurality of playing terminals, and forwarding the I-frame request to a preset video source server; and when an I-frame image sent by the video source server based on the I-frame request is received, carrying out an operation of inserting additional information on the I-frame image by using an additional processor to obtain a processed I-frame image, and sending the processed I-frame image to the plurality of playing terminals, so that the plurality of playing terminals play the processed I-frame image. According to the invention, the real-time performance of playing videos of the same video source by multiple terminals is improved.

Description

Technical Field

[0001] The present application relates to the field of video conferencing technology, and in particular to a multi-terminal based video processing method, device, equipment, medium and product. Background Art

[0002] In video conferencing technology, when multiple terminals need to play the same video source, the traditional processing of video stream retransmission and distribution is insufficient. In particular, when additional processing of the video stream is required, such as adding logos or subtitles, the traditional processing method will lead to efficiency problems and a decline in user experience. Specifically, when multiple terminals request key frames (I frames) at the same time, due to the processing limitations of the video source terminal, only some requests can be responded to in time, which causes other terminals to wait, thereby increasing processing delays. This delay not only affects the real-time nature of the video stream, but in some cases, it may also cause a black screen on the terminal, seriously affecting the smoothness of the video conference and the experience of the participants. Summary of the invention

[0003] The main purpose of this application is to provide a multi-terminal based video processing method, device, equipment, medium and product, aiming to improve the real-time performance of multiple terminals playing videos of the same video source.

[0004] To achieve the above purpose, the present application proposes a multi-terminal based video processing method, which is applied to a relay server, and multiple playback terminals are connected to a video source server through the relay server. The multi-terminal based video processing method includes:

[0005] If a preset control terminal insert additional information instruction is received, the function of receiving the I frame request is disabled;

[0006] An additional processor is created based on the insert additional information instruction. If the additional processor is created, a function of receiving an I frame request is enabled, and the I frame requests corresponding to multiple playback terminals are determined, and the I frame requests are forwarded to a preset video source server, wherein the additional processor is used to execute the insert additional information instruction;

[0007] When an I-frame image is received from a video source server based on an I-frame request, an additional processor is used to insert additional information into the I-frame image to obtain a processed I-frame image, and the processed I-frame image is sent to multiple playback terminals so that the multiple playback terminals can play the processed I-frame image.

[0008] In one embodiment, the additional processor includes an additional decoder and an additional encoder, and the step of creating the additional processor based on the insert additional information instruction further includes:

[0009] Based on the instruction to insert additional information, multiple additional decoders and additional encoders are created for all playback terminals, wherein one playback terminal corresponds to one additional decoder and one additional encoder.

[0010] In one embodiment, based on the instruction to insert additional information, the step of creating multiple additional decoders and additional encoders for all playback terminals includes:

[0011] If the instruction to insert additional information is an instruction to insert additional information for a station logo, then an additional decoder and an additional encoder related to the station logo are created for each playback terminal;

[0012] If the insert additional information instruction is an insert additional information instruction for subtitles, an additional decoder and an additional encoder related to the subtitles are created for each playback terminal.

[0013] In one embodiment, when an I-frame image sent by a video source server based on an I-frame request is received, the step of inserting additional information into the I-frame image using an additional processor to obtain a processed I-frame image includes:

[0014] receiving a first code stream including an I-frame image sent by a video source server based on an I-frame request, and decoding the first code stream according to an additional processor to obtain an I-frame image;

[0015] The insertion additional information sent by the control terminal is obtained, and the insertion additional information is added to the I frame image to obtain a processed I frame image.

[0016] In one embodiment, the additional processor decodes the first code stream to obtain an I frame image, including:

[0017] Decoding the first bit stream according to the additional decoder corresponding to each playback terminal to obtain an I frame image;

[0018] The step of sending the processed I frame image to multiple playback terminals includes:

[0019] For each playback terminal, the processed I frame image is encoded according to the additional encoder corresponding to the playback terminal to obtain a second code stream, and the second code stream is sent to the playback terminal.

[0020] In one embodiment, if the additional processor is created, the step of enabling the function of receiving an I frame request includes:

[0021] The timing starts from when the instruction to insert the additional information is received, and after a preset interval time, it is determined that the creation of the additional processor is completed.

[0022] In addition, to achieve the above purpose, the present application also proposes a multi-terminal based video processing device, which is arranged on a relay server, and multiple playback terminals are connected to a video source server through the relay server. The multi-terminal based video processing device includes:

[0023] The disabling module disables the function of receiving an I frame request if a preset control terminal inserts an additional information instruction;

[0024] The start module creates an additional processor based on the insert additional information instruction. If the additional processor is created, the function of receiving the I frame request is started, and the I frame request corresponding to the multiple playback terminals is determined, and the I frame request is forwarded to the preset video source server, wherein the additional processor is used to execute the insert additional information instruction;

[0025] The processing module, when receiving an I frame image sent by a video source server based on an I frame request, uses an additional processor to insert additional information into the I frame image to obtain a processed I frame image, and sends the processed I frame image to multiple playback terminals so that the multiple playback terminals can play the processed I frame image.

[0026] In addition, to achieve the above-mentioned purpose, the present application also proposes a multi-terminal based video processing device, the device comprising: a memory, a processor and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-terminal based video processing method as described above.

[0027] In addition, to achieve the above purpose, the present application also proposes a medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the multi-terminal-based video processing method as described above are implemented.

[0028] In addition, to achieve the above-mentioned purpose, the present application also provides a product, which is a computer program product. The computer program product includes a computer program. When the computer program is executed by a processor, the steps of the multi-terminal-based video processing method as described above are implemented.

[0029] One or more technical solutions proposed in this application have at least the following technical effects:

[0030] In the present application, if a preset control terminal inserts additional information instruction, the function of receiving I frame requests is disabled to avoid request conflicts; an additional processor is created based on the insert additional information instruction, and if the additional processor is created, the function of receiving I frame requests is enabled, and the I frame requests corresponding to multiple playback terminals are determined, and the I frame requests are forwarded to the preset video source server, wherein the additional processor is used to execute the insert additional information instruction; when an I frame image sent by the video source server based on the I frame request is received, the additional processor is used to insert additional information into the I frame image to obtain a processed I frame image, and the processed I frame image is sent to multiple playback terminals so that multiple playback terminals play the processed I frame image. In this way, when the video source server responds to the I frame request, it can ensure that the additional processor is used to accurately insert the required information into the I frame image, and the processed I frame image is sent to all playback terminals in a timely manner. This optimization strategy not only effectively reduces the black screen duration caused by I frame request conflicts, but also improves the overall stability and user experience of the video conferencing system. By precisely controlling the processing and distribution of I-frame images, the present invention ensures that each terminal can smoothly receive and display video content containing customized information even in special scenarios requiring additional code stream processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0032] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 This is a flowchart of a first embodiment of a multi-terminal video processing method of the present application;

[0034] Figure 2 This is a schematic diagram of the process of Embodiment 1 of the multi-terminal video processing method of the present application;

[0035] Figure 3 This is a flow chart of Embodiment 2 of the multi-terminal video processing method of the present application;

[0036] Figure 4 This is a schematic diagram of the module structure of a multi-terminal video processing device according to an embodiment of the present application;

[0037] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the multi-terminal video processing method in the embodiment of the present application.

[0038] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0040] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0041] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a terminal system, etc. The following takes the transfer server system (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0042] Based on this, this embodiment provides a multi-terminal based video processing method, which is applied to a relay server, and multiple playback terminals are connected to a video source server through the relay server. Figure 1 , Figure 1 This is a flow chart of a multi-terminal video processing method of the present application. The multi-terminal video processing method includes steps S10 to S30:

[0043] Step S10, if a preset control terminal insert additional information instruction is received, the function of receiving an I frame request is disabled;

[0044] It should be noted that in the video encoding and decoding process, I frames (key frames) are frames that can be decoded independently without relying on other frames. An I frame request refers to a request issued by a playback decoder in a playback terminal (such as a video conferencing terminal) to obtain the key frame of a video source. A playback terminal refers to a terminal device that receives and plays a video source, such as each participating terminal in a video conference. A control terminal refers to a terminal device or system used to control the video playback process (such as inserting additional information such as station logos and subtitles). An instruction to insert additional information refers to an instruction issued by a control terminal to instruct the insertion of specific information (such as station logos, subtitles, etc.) into the video content. The function of disabling / enabling the reception of I frame requests is to temporarily stop or resume the ability to receive I frame requests under certain conditions.

[0045] During a video conference, when multiple playback terminals need to play the same video source, the system will send an I-frame request to the video source server. At the same time, if the control terminal (the terminal that controls the system, which can be the system web page connected to the system) issues an instruction to insert additional information (such as requiring a logo to be inserted in all played videos), the system will receive and process this instruction.

[0046] Since the video source server usually sets up an I frame filtering mechanism (processing an I frame request every 3 seconds), it is necessary to ensure that the I frame is requested at the right time. Therefore, after receiving the instruction to insert additional information, the system will temporarily disable the function of receiving or generating I frame requests. This is to ensure that no new I frame requests are processed before the initialization of the additional information insertion process is completed, thereby avoiding the situation where the decoders of multiple terminals request I frames at a similar time, and only the first decoder can apply for the I frame, resulting in a black screen.

[0047] Step S20, creating an additional processor based on the insert additional information instruction, if the additional processor is created, then starting the function of receiving I frame requests, determining I frame requests corresponding to multiple playback terminals, and forwarding the received I frame requests to a preset video source server, wherein the additional processor is used to execute the insert additional information instruction;

[0048] It should be noted that the additional processor is a processor created to execute the insert additional information instruction, which is responsible for processing video frames (such as I frames) and inserting specified additional information. The video source server is a server that provides video source data and can respond to I frame requests and send corresponding video frames.

[0049] The system creates an additional processor based on the insert additional information instruction. This processor is responsible for executing the operation of inserting additional information, such as adding a logo to a video frame.

[0050] When the additional processor is created, the system will re-enable the function of receiving or generating I-frame requests. This means that the system is now ready to process I-frame requests and insert additional information into the video frame. The system will re-enable the function of receiving I-frame requests and forward the I-frame requests received to the video source server. It can only forward the first I-frame request received. The video source server sends the corresponding I-frames to the system based on these requests.

[0051] After an additional processor is newly added to the relay server, the additional processor obtains the bitstream that is halfway played on the video source server, which may not contain I frames. Therefore, an I frame request needs to be sent to the video source server. After the relay server decodes and re-encodes, it sends out the newly encoded bitstream. The new bitstream must have I frames at the beginning, so the playback terminal can receive I frames without sending an I frame request.

[0052] Step S30, when an I frame image is received from a video source server based on an I frame request, an additional processor is used to insert additional information into the I frame image to obtain a processed I frame image, and the processed I frame image is sent to multiple playback terminals so that the multiple playback terminals can play the processed I frame image.

[0053] It should be noted that the processed I frame image is an I frame image after the additional processor inserts additional information, and can be sent to the playback terminal for playback.

[0054] When the system receives I-frame images sent by the video source server, it uses the previously created additional processor to insert additional information into these I-frame images. After being processed by the additional processor, the I-frame images become processed I-frame images, which contain the required additional information (such as station logos). The system sends the processed I-frame images to all playback terminals that request these I-frame images. After receiving the processed I-frame images, the playback terminal can decode and play the video, and the video already contains the inserted additional information.

[0055] For example, reference Figure 2 , Figure 2 The flowchart of describes a process flow in a video processing system, especially related to the logo or subtitle adding function. When the user or the system decides to turn on the logo or subtitle during video playback, the server will start to create a decoding process. The purpose of this decoding process is to decode the received video stream so that the logo or subtitle can be inserted in the subsequent steps. It should be noted that the decoding here may refer to the decoding of the video compressed data to restore the original image frame. After the decoding process starts, the system needs to determine whether the logo or subtitle adding process has been initialized. This initialization process may include loading necessary resources (such as font files, logo images, etc.), configuring parameters (such as the position, size, color of subtitles, etc.), and preparing decoders and encoders (if re-encoding is required). Only when these initialization steps are completed can the system ensure that the subsequent video processing can proceed smoothly. If the logo and subtitle process has been initialized, the system will start to allow frame requests to be received and processed. These frame requests may be requests from the video player to obtain video frames with logos or subtitles. According to these requests, the system will insert the corresponding logo or subtitle on the decoded video frame, and may perform necessary encoding processing to generate the final video output.

[0056] In this embodiment, if a preset control terminal inserts additional information instruction, the function of receiving I frame requests is disabled to avoid request conflicts; an additional processor is created based on the insert additional information instruction, and if the additional processor is created, the function of receiving I frame requests is enabled, and I frame requests corresponding to multiple playback terminals are determined, and the I frame requests are forwarded to the preset video source server, wherein the additional processor is used to execute the insert additional information instruction; when an I frame image sent by the video source server based on the I frame request is received, the additional processor is used to insert additional information into the I frame image to obtain a processed I frame image, and the processed I frame image is sent to multiple playback terminals so that multiple playback terminals play the processed I frame image. In this way, when the video source server responds to the I frame request, it can ensure that the additional processor is used to accurately insert the required information into the I frame image, and the processed I frame image is sent to all playback terminals in a timely manner. This optimization strategy not only effectively reduces the black screen duration caused by the I frame request conflict, but also improves the overall stability and user experience of the video conferencing system. By precisely controlling the processing and distribution of I-frame images, the present invention ensures that each terminal can smoothly receive and display video content containing customized information even in special scenarios requiring additional code stream processing.

[0057] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction, and will not be repeated hereafter. On this basis, the additional processor includes an additional decoder and an additional encoder, and the step S20 also includes step A10:

[0058] Step A10: based on the instruction to insert additional information, create multiple additional decoders and additional encoders for all playback terminals, wherein one playback terminal corresponds to one additional decoder and one additional encoder.

[0059] It should be noted that the additional decoder is used to add additional information (such as station logos, subtitles, etc.). The system needs to create a separate decoder for each playback terminal. This decoder is used to decode the video stream and add additional information during the decoding process. The additional encoder is used after the additional decoder decodes and adds additional information. The system needs an encoder to re-encode these video streams with additional information so that they can be sent to the corresponding playback terminal.

[0060] When the system receives an insert additional information instruction, the system creates an additional decoder and an additional encoder in the system for each playback terminal according to the insert additional information instruction. Here, one playback terminal corresponds to one additional decoder and one additional encoder, ensuring that each playback terminal has an independent processing additional decoder and additional encoder to achieve further functions, such as executing the insert additional information instruction for each playback terminal separately, rather than executing it uniformly for all playback terminals.

[0061] A decoder is a device or software used to decode a video stream. In this embodiment, the system creates an additional decoder for each terminal to decode the video stream received from the video source. After the additional decoder decodes the video stream and adds additional information, the system needs an additional encoder to re-encode these video streams with additional information. Therefore, the system creates an additional encoder for each terminal to re-encode the decoded video stream (with additional information) into a format suitable for transmission.

[0062] This embodiment can successfully add additional information such as logos or subtitles to videos played by multiple playback terminals, while ensuring the synchronization and consistency of video streams. In this process, the creation of additional decoders and additional encoders is a key step, which ensures the correct addition and transmission of additional information.

[0063] In a feasible implementation, step A10 may include steps A101 to A102:

[0064] Step A101, if the instruction to insert additional information is an instruction to insert additional information for a station logo, then an additional decoder and an additional encoder related to the station logo are created for each playback terminal;

[0065] It should be noted that the station logo is the logo of a TV station or channel, which is usually displayed in the corner of the video screen to identify the source of the broadcast content.

[0066] Step A102, if the instruction to insert additional information is an instruction to insert additional information for subtitles, then an additional decoder and an additional encoder related to subtitles are created for each playback terminal;

[0067] It should be noted that subtitles are text descriptions in video content, usually used to provide dialogue translations, annotations or plot hints.

[0068] For different instructions for inserting additional information, the system will create different additional decoders and additional encoders. Of course, if you need to insert multiple additional information at the same time, such as subtitles and station logos at the same time, the system can also solve the needs of inserting subtitles and station logos at the same time by creating additional decoders and additional encoders.

[0069] This embodiment realizes the function of flexibly inserting additional information such as logos and subtitles into a video stream, thereby improving the flexibility of adding additional video information.

[0070] In one possible implementation, refer to Figure 3 , step S30 may include steps B10 to B20:

[0071] Step B10, receiving a first code stream including an I-frame image sent by the video source server based on an I-frame request, and decoding the first code stream according to an additional processor to obtain an I-frame image;

[0072] It should be noted that the first code stream includes a coded stream of I-frame image data, which is usually sent by a video source server and is used to transmit I-frame image information.

[0073] The video source server sends a first bitstream containing an I-frame image according to the I-frame request. This bitstream is encoded and contains compressed data of the I-frame image. Subsequently, the additional processor receives this first bitstream. The additional processor is a special decoding device or software module that can decode the encoded video data into original image data. Through the decoding process of the additional processor, the first bitstream is decoded into a displayable I-frame image. This process involves the inverse process of the video compression algorithm, such as inverse quantization, inverse transformation, deblocking filtering and other steps, and finally obtains the original I-frame image data.

[0074] Step B20, obtaining the insertion additional information sent by the control terminal, and adding the insertion additional information to the I frame image to obtain a processed I frame image.

[0075] After decoding and obtaining the I-frame image, the system needs to obtain the inserted additional information sent by the control terminal. This information may be text, icons, watermarks, etc., which are used to mark or modify the I-frame image. After receiving the inserted additional information, the system adds it to the appropriate position of the I-frame image. This adding process may involve image processing technology, such as image synthesis, transparency adjustment, etc., to ensure the natural fusion of the inserted additional information and the I-frame image. Finally, the processed I-frame image is obtained, that is, the I-frame image with the inserted additional information added. This image can be used for subsequent display, storage or transmission operations.

[0076] This embodiment realizes the function of receiving an I-frame image from a video source server and adding additional information thereto. This process involves multiple technical fields such as video decoding and image processing, ensuring accurate decoding of the I-frame image and effective addition of additional information.

[0077] In a feasible implementation, step B10 may include steps B101 to B102:

[0078] Step B101, decoding the first bit stream according to the additional decoder corresponding to each playback terminal to obtain an I frame image;

[0079] The additional processor receives the first bitstream, which is the original data stream containing all the video information. The additional processor decodes the first bitstream according to the additional decoder corresponding to each playback terminal. This step is specific because the playback terminal may require different decoders due to different hardware, software or network conditions. The result of the decoding is an I-frame image, which is a key frame in video encoding and contains complete image information.

[0080] In a feasible implementation, step S30 may include step B102:

[0081] Step B102: for each playback terminal, the processed I frame image is encoded according to the additional encoder corresponding to the playback terminal to obtain a second code stream, and the second code stream is sent to the playback terminal.

[0082] It should be noted that the second code stream is a video data stream encoded by an additional encoder, and is usually customized to adapt to the receiving capability or network conditions of a specific playback terminal.

[0083] For each playback terminal, the processed I-frame image is encoded according to its corresponding additional encoder in the system. This step is to convert the processed image into a format suitable for reception and display by the playback terminal. The result of the encoding is a second bit stream, which is a video data stream processed by a specific encoder. The second bit stream is sent to the corresponding playback terminal so that it can receive, decode and display the video content. Since all decoders are ready, it can be ensured that each terminal can correctly process and play the video content with additional information.

[0084] This embodiment implements the customized decoding, processing and encoding process of video data for different playback terminals, ensuring that the video content can be presented in the best way on various devices. This process involves multiple technical fields such as video encoding, decoding and image processing, ensuring the compatibility and high-quality transmission of video data.

[0085] Based on the first or second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first or second embodiment can be referred to the above introduction, and will not be described in detail later. Step S20 also includes step C10:

[0086] Step C10, starting the timing from when the instruction to insert the additional information is received, and after a preset interval, determining that the creation of the additional processor is completed.

[0087] It should be noted that the preset interval time is a pre-set time threshold, which is used to ensure that the system has enough time to complete the creation process of the additional processor after receiving the instruction to insert the additional information. The time interval is set according to system performance and actual needs.

[0088] When the system receives the instruction to insert additional information, it will start timing immediately. This timing process is to ensure that the system has enough time to complete the creation of the additional processor in the subsequent steps. After the timing starts, the system will wait for a preset interval time, which is set according to the system performance and actual needs to ensure that the additional processor can complete the creation process within this period of time. After waiting for the preset interval time, the system will check the creation status of the additional processor. If the additional processor has been successfully initialized and is ready to receive video source data and additional information, then the system considers that the additional processor has been created.

[0089] For example, suppose that in a video conference, 10 terminals need to play the same video stream with a station logo. When the system receives the instruction to insert the station logo, it will immediately start timing and wait for a preset interval (for example, 300 milliseconds). During these 300 milliseconds, the system will create and initialize an additional processor to process the station logo information. After 300 milliseconds, the system will superimpose the station logo information into the video stream and send it to the 10 terminals for playback.

[0090] In this embodiment, by shielding the I frame requests of all decoders and releasing the I frame requests after the initialization process of the station logo and subtitles is completed, it is ensured that all additional processors are ready when receiving the I frame. This can effectively reduce the black screen duration caused by the decoder not being ready.

[0091] In a feasible implementation manner, step B10 may include steps D10 to D20:

[0092] Step D10, detecting whether there is a preset completion mark, if there is a completion mark, determining that the creation of the additional processor is completed, wherein the mark is set when the creation of the additional processor is completed;

[0093] Step D20: If there is no completion mark, continue to execute the step of detecting whether there is a preset completion mark.

[0094] It should be noted that, in this embodiment, the preset completion flag is a specific sign or signal, which is used to indicate that the creation and initialization process of the additional processor (which may be a module for processing additional information in the video stream, such as adding watermarks, subtitles, etc.) has been completed. This flag is usually automatically set by the system after the additional processor is successfully created.

[0095] The system first checks to see if there is a preset completion flag. This flag is set automatically by the system after the add-on processor is successfully created and initialized. If the system detects this flag, it can determine that the add-on processor has been created. If the system detects the preset completion flag, it will assume that the add-on processor has been created and is ready to receive and process additional information in the video stream. If the system does not detect the preset completion flag, it will continue to perform the detection steps until the flag appears. This process is cyclical until the system confirms that the add-on processor has been created.

[0096] By introducing a preset completion flag, the system can efficiently detect the creation status of the additional processor, ensuring that it performs subsequent video information processing tasks only after it is fully initialized. The system detects the flag cyclically, and once detected, it confirms that the creation of the additional processor is complete, thereby avoiding processing errors or delays caused by the processor not being ready, and reducing the black screen duration.

[0097] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the multi-terminal video processing method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0098] The present application also provides a multi-terminal based video processing device, which is arranged on a relay server, and multiple playback terminals are connected to a video source server through the relay server. Figure 4 , the multi-terminal based video processing device comprises:

[0099] The disabling module 10 disables the function of receiving the I frame request if a preset control terminal inserts additional information instruction is received;

[0100] The start module 20 creates an additional processor based on the insert additional information instruction. If the additional processor is created, the function of receiving an I frame request is started, and the I frame request corresponding to multiple playback terminals is determined, and the I frame request is forwarded to a preset video source server, wherein the additional processor is used to execute the insert additional information instruction;

[0101] The processing module 30, when receiving an I frame image sent by the video source server based on an I frame request, uses an additional processor to insert additional information into the I frame image to obtain a processed I frame image, and sends the processed I frame image to multiple playback terminals so that the multiple playback terminals can play the processed I frame image.

[0102] The multi-terminal video processing device provided by the present application adopts the multi-terminal video processing method in the above embodiment, which can improve the real-time performance of multi-terminal video playback of the same video source. Compared with the prior art, the beneficial effects of the multi-terminal video processing device provided by the present application are the same as the beneficial effects of the multi-terminal video processing method in the above embodiment, and other technical features in the multi-terminal video processing device are the same as the features disclosed in the above embodiment method, which will not be described in detail here.

[0103] The present application provides a multi-terminal based video processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the multi-terminal based video processing method in the above-mentioned embodiment 1.

[0104] Reference below Figure 5 , which shows a schematic diagram of the structure of a multi-terminal based video processing device suitable for implementing the embodiment of the present application. The multi-terminal based video processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The multi-terminal based video processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0105] like Figure 5As shown, the multi-terminal based video processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the multi-terminal based video processing device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the multi-terminal-based video processing device to communicate with other devices wirelessly or by wire to exchange data. Although the multi-terminal-based video processing device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0106] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0107] The multi-terminal based video processing device provided by the present application adopts the multi-terminal based video processing method in the above embodiment, which can improve the real-time performance of multi-terminal video playback of the same video source. Compared with the prior art, the beneficial effects of the multi-terminal based video processing device provided by the present application are the same as the beneficial effects of the multi-terminal based video processing method provided by the above embodiment, and other technical features in the multi-terminal based video processing device are the same as the features disclosed in the method of the previous embodiment, which will not be described in detail here.

[0108] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0109] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0110] The present application provides a medium, which is a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the multi-terminal-based video processing method in the above-mentioned embodiment.

[0111] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0112] The computer-readable storage medium may be included in the multi-terminal-based video processing device; or may exist independently without being assembled into the multi-terminal-based video processing device.

[0113] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the multi-terminal-based video processing device, the multi-terminal-based video processing device:

[0114] If a preset control terminal insert additional information instruction is received, the function of receiving the I frame request is disabled;

[0115] An additional processor is created based on the insert additional information instruction. If the additional processor is created, a function of receiving an I frame request is enabled, and the I frame requests corresponding to multiple playback terminals are determined, and the I frame requests are forwarded to a preset video source server, wherein the additional processor is used to execute the insert additional information instruction;

[0116] When an I-frame image is received from a video source server based on an I-frame request, an additional processor is used to insert additional information into the I-frame image to obtain a processed I-frame image, and the processed I-frame image is sent to multiple playback terminals so that the multiple playback terminals can play the processed I-frame image.

[0117] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0118] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0119] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0120] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned multi-terminal-based video processing method, and can improve the real-time performance of multi-terminal video playback of the same video source. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the multi-terminal-based video processing method provided by the above-mentioned embodiment, and will not be described in detail here.

[0121] The present application also provides a product, which is a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the multi-terminal-based video processing method are implemented as described above.

[0122] The computer program product provided by the present application can improve the real-time performance of multiple terminals playing videos of the same video source. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the multi-terminal video processing method provided by the above embodiment, which will not be repeated here.

[0123] The above are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A video processing method based on multiple terminals, characterized in that: Applied to a relay server, multiple playback terminals are connected to a video source server through the relay server, and the multi-terminal-based video processing method includes: If a preset control terminal insert additional information instruction is received, the function of receiving the I frame request is disabled; Creating an additional processor based on the insert additional information instruction, if the additional processor is created, enabling the function of receiving the I frame request, determining the I frame requests corresponding to the plurality of playback terminals, and forwarding the I frame requests to a preset video source server, wherein the additional processor is used to execute the insert additional information instruction; When an I-frame image is received from the video source server based on the I-frame request, the additional processor is used to insert additional information into the I-frame image to obtain a processed I-frame image, and the processed I-frame image is sent to multiple playback terminals so that the multiple playback terminals can play the processed I-frame image.

2. The multi-terminal video processing method according to claim 1, characterized in that: The additional processor includes an additional decoder and an additional encoder, and the step of creating an additional processor based on the insert additional information instruction further includes: Based on the insert additional information instruction, multiple additional decoders and additional encoders are created for all the playback terminals, wherein one playback terminal corresponds to one additional decoder and one additional encoder.

3. The multi-terminal based video processing method according to claim 2, characterized in that: The step of creating a plurality of additional decoders and additional encoders for all the playback terminals based on the instruction to insert additional information comprises: If the instruction to insert additional information is an instruction to insert additional information for a station logo, then creating an additional decoder and an additional encoder related to the station logo for each of the playback terminals; If the insert additional information instruction is an insert additional information instruction for subtitles, an additional decoder and an additional encoder related to the subtitles are created for each of the playback terminals.

4. The multi-terminal based video processing method according to claim 2, characterized in that: The step of inserting additional information into the I frame image using the additional processor to obtain the processed I frame image when receiving the I frame image sent by the video source server based on the I frame request comprises: receiving a first code stream including an I-frame image sent by the video source server based on the I-frame request, and decoding the first code stream according to the additional processor to obtain the I-frame image; The inserted additional information sent by the control terminal is acquired, and the inserted additional information is added to the I frame image to obtain the processed I frame image.

5. The multi-terminal based video processing method according to claim 4, characterized in that: The additional processor decodes the first code stream to obtain an I frame image, comprising: Decoding the first bit stream according to the additional decoder corresponding to each playback terminal to obtain an I frame image; The step of sending the processed I frame image to the plurality of playback terminals comprises: For each of the playback terminals, the processed I-frame image is encoded according to the additional encoder corresponding to the playback terminal to obtain a second code stream, and the second code stream is sent to the playback terminal.

6. The multi-terminal based video processing method according to claim 1, characterized in that: If the additional processor is created, the step of enabling the function of receiving the I frame request includes: The timing starts from when the insert additional information instruction is received, and after a preset interval time, it is determined that the creation of the additional processor is completed.

7. A multi-terminal based video processing device, characterized in that: The video processing device based on multiple terminals is provided on the relay server, and multiple playback terminals are connected to the video source server through the relay server. The video processing device based on multiple terminals includes: The disabling module disables the function of receiving the I frame request if a preset control terminal inserts additional information instruction is received; An enabling module is configured to create an additional processor based on the insert additional information instruction. If the additional processor is created, a function of receiving the I frame request is enabled, and the I frame requests corresponding to the plurality of playback terminals are determined, and the I frame requests are forwarded to a preset video source server, wherein the additional processor is used to execute the insert additional information instruction; The processing module, when receiving the I frame image sent by the video source server based on the I frame request, uses the additional processor to insert additional information into the I frame image to obtain a processed I frame image, and sends the processed I frame image to multiple playback terminals so that the multiple playback terminals play the processed I frame image.

8. A multi-terminal based video processing device, characterized in that: The multi-terminal-based video processing device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the multi-terminal-based video processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the multi-terminal-based video processing method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the multi-terminal-based video processing method according to any one of claims 1 to 6 are implemented.

Images