Video transmission method and device

Abstract

The present application relates to a video transmission method and device, wherein the video transmission method comprises: slicing a knowledge image referenced by multiple to-be-encoded images in a video to be transmitted, and interleaving encoding the to-be-encoded images according to the image slicing and interleaving strategy after slicing, and at the same time, performing grammatical constraints so that multiple image frames in the video transmission process are transmitted in the order of encoding, avoiding the occurrence of bit rate spikes, so as to achieve smooth video transmission.

Description

Technical Field

[0001] The present application relates to the field of video technology, and in particular to a video transmission method and device. Background Art

[0002] AVS3 (Audio Video Coding Standard 3) is a new generation of audio and video coding standards. AVS3 is mainly aimed at efficient compression of ultra-high-definition video content, aiming to provide better coding efficiency for ultra-high-definition videos of 8K and above. In order to mine and utilize the mutual reference information between images in multiple random access intervals during encoding, when encoding (or decoding) an image, the encoder (or decoder) can select an image with similar texture content to the current encoded image (or decoded image) as a reference image. This reference image is called a knowledge image.

[0003] AVS3 stipulates that all knowledge images must use intra-frame prediction coding and be transmitted through expansion and coordination at the system layer. However, this method has a very large bit rate ratio, which will result in bit rate spikes, making it easy for mutations to occur during video transmission. Therefore, the smoothness of the video transmission process needs to be improved.

[0004] With regard to the problem in related technologies that the stability of video transmission needs to be improved, no effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a video transmission method and device are provided to solve the problem in the related art that the stability of the video transmission process needs to be improved.

[0006] In a first aspect, a video transmission method is provided in this embodiment, which is applied to an encoder side. The method includes:

[0007] Acquire multiple knowledge images referenced by multiple images to be encoded in a video to be transmitted; divide the knowledge images into multiple image slices;

[0008] According to a preset interleaving strategy, the image to be encoded is interleaved and encoded by using the multiple image slices to obtain an encoded image;

[0009] Constraining the coded image after interleaving coding according to a preset grammar to obtain a target image; and outputting the target image.

[0010] In some embodiments, the knowledge image includes a first knowledge image and a second knowledge image; the number of image slices is marked in the image slice header of the knowledge image; the first knowledge image and the second knowledge image are adjacent;

[0011] The step of interleaving and encoding the image to be encoded by using the plurality of image slices according to a preset interleaving strategy to obtain an encoded image includes:

[0012] Determining the arrangement order of the image slices according to a preset mapping relationship;

[0013] According to the arrangement order, the image slices corresponding to the first knowledge image and the second knowledge image are interleaved and encoded to obtain an encoded image.

[0014] In some embodiments, constraining the coded image after interleaving coding according to a preset grammar to obtain a target image includes:

[0015] According to a preset grammar, a reference image in the knowledge image is determined; the coding order of the reference image is located after the coding order of the knowledge images other than the reference image.

[0016] In some of the embodiments, when the first image slice of the first knowledge image is located before the image slice of the second knowledge image in the coding order, the second image slice of the first knowledge image is also located before the image slice of the second knowledge image in the coding order;

[0017] The first image slice and the second image slice are encoded in different orders.

[0018] In some of the embodiments, the image to be encoded includes a predicted image frame;

[0019] The code rate of the predicted image frame is consistent with that of the image slice; the predicted image frame is used to represent the difference between the multiple knowledge images.

[0020] In some of the embodiments, the method further comprises: adding an index mark in the image slice header of the knowledge image; the index mark comprises a display order index mark and a decoding order index mark.

[0021] In a second aspect, a video transmission method is provided in this embodiment, which is applied to a decoder side, wherein the decoder is connected to an encoder, and the method includes:

[0022] Receive a target image, and determine whether the arrangement order of image slices in the target image is consistent with a preset mapping relationship; the target image is obtained by dividing a plurality of knowledge images referenced by a plurality of images to be encoded into a plurality of image slices, and interleaving the image to be encoded through the plurality of image slices according to a preset interleaving strategy, and constraining the encoded image after the interleaving encoding according to a preset grammar;

[0023] If yes, judging whether the number of the image slices reaches a preset number threshold according to a preset encoding counter;

[0024] If so, the target image is decoded to obtain a decoded knowledge image.

[0025] In some embodiments, determining whether the number of the image slices reaches a preset number threshold includes:

[0026] Acquire a reference image in the target image, and determine the number of image slices that precede the reference image in coding order;

[0027] It is determined whether the number of image slices whose coding order precedes the reference image reaches a preset number threshold.

[0028] In some embodiments, the method further comprises:

[0029] Storing the knowledge image in a preset decoding buffer area;

[0030] At the same time, a preset number of knowledge images stored in the decoding buffer area are set to take effect.

[0031] In a third aspect, a video transmission device is provided in this embodiment, the device comprising: an encoder and a decoder; the encoder and the decoder are connected;

[0032] The encoder is used to perform the video transmission method as described in the first aspect;

[0033] The decoder is used to execute the video transmission method as described in the second aspect.

[0034] Compared with the related art, a video transmission method and device provided in this embodiment, by fragmenting the knowledge image referenced by multiple images to be encoded in the transmitted video, and interleavingly encoding the image to be encoded according to the fragmented image fragmentation and interleaving strategy, at the same time, performing grammatical constraints, so that multiple image frames in the video transmission process are transmitted in the order of encoding, so as to achieve smooth video transmission.

[0035] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0037] Figure 1 It is a hardware structure block diagram of a terminal of the video transmission method provided in an embodiment of the present application;

[0038] Figure 2 is a flowchart of a video transmission encoding method provided in an embodiment of the present application;

[0039] Figure 3 is a flow chart of another video transmission method provided by an embodiment of the present application;

[0040] Figure 4 is a flow chart of a video transmission method in the prior art;

[0041] Figure 5 It is a flow chart of the video transmission method provided in this specific embodiment. DETAILED DESCRIPTION

[0042] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0044] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1is a hardware structure block diagram of a terminal of the video transmission method provided in an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0045] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the video transmission method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0046] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0047] In the existing video compression coding technology, in order to meet the requirements of random access to video, an intra-frame prediction coding frame, i.e., an I frame, is required at intervals. For surveillance videos, since the interval between I frames is short, there is still a large correlation between I frames. In order to further eliminate the correlation between I frames, H.264 / H.265 adopts long-term reference frames, and AVS3 proposes the concept of "knowledge image (Library picture)". In order to mine and utilize the information of mutual reference between images between multiple random access intervals during encoding, when encoding (or decoding) an image, the encoder (or decoder) can select an image with a texture content similar to the current encoded image (or decoded image) as a reference image. This reference image is called a knowledge image. AVS3 stipulates that all knowledge images must also use intra-frame prediction coding-I frames, and the knowledge images are transmitted through expansion and cooperation at the system layer. On the one hand, this method requires the cooperation of the system layer to complete the corresponding encoding and decoding operations, and the interaction and coupling with the system layer are too large, and the overall system design and deployment cost is high. On the other hand, since knowledge images use intra-frame prediction coding, they have a very large (10 times to 100 times) I-frame to P-frame bit rate ratio (I frames are much larger than P frames), and bit rate "spikes" will occur, resulting in sudden changes in transmission (data loss, playback freezes).

[0048] In this embodiment, a video transmission method is proposed, which is applied on the encoder side, and the code stream of the knowledge image is interleaved in the code stream with the image encoded with the subsequent reference knowledge image in the coding slice unit to achieve smooth transmission of the knowledge image and the subsequent encoded image. Figure 2 is a flowchart of a video transmission encoding method provided in an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0049] Step S210, obtaining a plurality of knowledge images referenced by a plurality of images to be encoded in a video to be transmitted; and dividing the knowledge images into a plurality of image slices.

[0050] In this step, when the video needs to be transmitted, the image frames in the video frames generally need to be encoded and transmitted, that is, the encoder first obtains the image frames in the video, and obtains the image with texture content similar to the current image to be encoded as a reference knowledge image, and then divides the knowledge image into several image slices, preferably, the number of slices is set to 12-18 slices, and then encodes each image slice. Furthermore, the quantization parameter offset value of the knowledge image is set to retain the details of the image frame, thereby avoiding image distortion.

[0051] Step S220: According to a preset interleaving strategy, the image to be encoded is interleaved and encoded by using multiple image slices to obtain an encoded image.

[0052] In this step, the encoder interleaves the image to be encoded based on multiple image slices according to a preset interleaving strategy; wherein the preset interleaving strategy includes: interleaving according to the importance and dependency of the knowledge image corresponding to the image frame to be encoded, and the interleaving coding strategies include: frame interleaving, macroblock interleaving, row interleaving and byte interleaving, etc. The specific interleaving strategy is determined according to the source video type, video quality and target receiving device, and is not specifically limited here.

[0053] Step S230, constraining the coded image after interleaving coding according to a preset grammar to obtain a target image; and outputting the target image.

[0054] In this step, the encoder presets a constraint syntax, and constrains the coded image after interleaving coding according to the syntax to obtain a target image, so that after the decoder side obtains the target image, it determines whether the target image can be received in sequence and completely continuously according to the preset constraint syntax. The preset syntax is determined according to the sequence, continuity and integrity of the image slices in the coded image.

[0055] Through the above steps, the encoder side segments the knowledge images referenced by multiple images to be encoded in the transmitted video, and interleaves the images to be encoded according to the image segmentation and interleaving strategy after segmentation. At the same time, grammatical constraints are performed so that multiple image frames in the video transmission process are transmitted in the order of encoding, which solves the problem of bit rate spikes in the knowledge image transmission and achieves smooth video transmission.

[0056] In some of the embodiments, the knowledge image includes a first knowledge image and a second knowledge image; the number of image slices is marked in the image slice header of the knowledge image; and the first knowledge image and the second knowledge image are adjacent.

[0057] Among them, in step S220, according to a preset interleaving strategy, the image to be encoded is interleaved and encoded through multiple image slices to obtain a coded image, including: determining the arrangement order of the image slices according to a preset mapping relationship; according to the arrangement order, interleaving and encoding the image slices corresponding to the first knowledge image and the second knowledge image to obtain the coded image.

[0058] In this step, the preset mapping relationship includes the absolute mapping relationship between the image slices and the positions of the image to be encoded. According to the preset mapping relationship, the arrangement order of the image slices is determined. It can be understood that all image slices of the same knowledge image must be arranged and transmitted in the ascending order of the preset mapping relationship. Afterwards, according to the arrangement order and the preset interleaving strategy, the image slices corresponding to the multiple knowledge images are interleaved and encoded, thereby avoiding data mutation during the transmission process.

[0059] In some of the embodiments, in step S230, the coded image after interleaving coding is constrained according to a preset grammar to obtain a target image, including: determining a reference image in the knowledge image according to the preset grammar; the coding order of the reference image is located after the coding order of the knowledge image other than the reference image.

[0060] In this step, the encoder sorts the encoding order of the corresponding reference multiple knowledge images according to the transmission order of the image frames to be encoded in the video to be transmitted; further, the encoding order of the reference image is set after the encoding order of other knowledge images, so as to facilitate the judgment of whether to encode all knowledge images in combination with the image segmentation during encoding, thereby improving the integrity of the video image frame encoding.

[0061] In some of the embodiments, when the first image slice of the first knowledge image is located before the image slice of the second knowledge image in the encoding order, the second image slice of the first knowledge image is also located before the image slice of the second knowledge image in the encoding order; the encoding order of the first image slice and the second image slice is different.

[0062] In this step, when it is determined that the encoding order of at least one image slice of the first knowledge image is before the encoding order of at least one image slice of the second knowledge image, it can be understood that the encoding order of other image slices of the first knowledge image should also be before the encoding order of all image slices of the second knowledge image to achieve the sequentiality of image encoding.

[0063] In some of the embodiments, the image to be encoded includes a predicted image frame; the bit rate of the predicted image frame and the image slice is consistent; and the predicted image frame is used to represent the difference between multiple knowledge images.

[0064] In this step, the predicted image frames include P frames (Predictive-coded frames), where the P frame predicts the content of the current frame based on the previous intra-coded I frame (Intra-coded frames) or P frame. It only records the difference with the reference frame. It depends on the previous I frame or P frame to be correctly decoded, so it cannot be decoded alone. By utilizing temporal redundant information, P frames can achieve higher compression efficiency than I frames. P frames help reduce the amount of data in the video stream while maintaining good visual quality. P frames are common in video sequences, especially when there is little action or the background is relatively static.

[0065] In some of the embodiments, the data transmission method further includes: adding an index mark in the image slice header of the knowledge image; the index mark includes a display order index mark and a decoding order index mark.

[0066] In addition, a display order index mark and a decoding order index mark may be set in a knowledge image located after the knowledge image.

[0067] In this embodiment, a video transmission method is also provided. On the decoder side, knowledge images can be decoded in units of coded slices. The complete knowledge image frame can be decoded normally and placed in the decoding cache for subsequent image reference without the support and cooperation of the system layer. Figure 3 is a flow chart of another video transmission method provided in an embodiment of the present application. Figure 3 As shown, the process includes the following steps:

[0068] Step S310, receiving a target image, and determining whether the arrangement order of image slices in the target image is consistent with a preset mapping relationship; the target image is obtained by dividing a plurality of knowledge images referenced by a plurality of images to be encoded into a plurality of image slices, and interleavingly encoding the images to be encoded through a plurality of image slices according to a preset interleaving strategy, and constraining the encoded image after the interleaving encoding according to a preset grammar.

[0069] The decoder receives the interleaved encoding output by the encoder and performs grammatical constraints on the target image, and determines whether the arrangement order of the image slices in the target image is consistent with the preset mapping relationship, which can be understood as determining whether the arrangement order of the image slices is consistent. Further, the target image is a plurality of image slices obtained by dividing the knowledge image referenced by the image to be encoded, and the image slices are interleaved with a preset constrained grammar.

[0070] Step S320: If yes, determine whether the number of image slices reaches a preset number threshold according to a preset encoding counter.

[0071] In this step, when the arrangement order of image slices in the target image is consistent with the preset mapping relationship, the number of image slices in the target image is obtained according to the preset encoding counter, and it is determined whether the number of image slices reaches the preset number threshold.

[0072] Step S330: If yes, decode the target image to obtain a decoded knowledge image.

[0073] In this step, when the number of image slices reaches a preset number threshold, the target image is decoded to obtain a decoded knowledge image.

[0074] Through the above steps, when the decoder receives the target image output by the encoder after encoding, it determines the number and order of the image slices in the target image. When the order and number of the image slices are consistent with the order and number during encoding, the decoder is used to decode the target image to obtain the decoded knowledge image, so as to complete the transmission of multiple image frames in the video, while avoiding the spike of the bit rate and achieving the stability of video transmission. At the same time, there is no need to pass through the system layer during decoding.

[0075] In some of the embodiments, determining whether the number of image slices reaches a preset number threshold includes: obtaining a reference image in the target image, determining the number of image slices whose encoding order is before the reference image; and determining whether the number of image slices whose encoding order is before the reference image reaches a preset number threshold.

[0076] In some of the embodiments, the method further includes: storing the knowledge image in a preset decoding cache area; at the same time, setting the knowledge images stored in a preset number of decoding cache areas to take effect.

[0077] The present embodiment is described and illustrated by means of specific examples below.

[0078] Figure 4 is a flow chart of a video transmission method in the prior art. Figure 4 , the knowledge image in the existing AVS3 standard, since a knowledge image will be referenced by images across several random access intervals to eliminate the time correlation, for this purpose, the system layer designs a long-term cache area to store the knowledge image, and transmits the knowledge image through the cooperation and coupling of the system layer, wherein the system layer is respectively connected to the encoding layer and the decoding layer. When video transmission is required, the video is first input to the encoder in the encoding layer, and a coded image buffer is set in the encoder. At the same time, the knowledge image corresponding to the input video is cached in the knowledge image library for a long time. The coded image buffer refers to the index of the knowledge image, and the knowledge image library sends the knowledge image to the coded image buffer for encoding. Thereafter, the encoded image is transmitted to the decoding image buffer in the decoding layer through the system layer. At the same time, the knowledge image library connected to the coded image buffer sends the knowledge image to the knowledge image library connected to the decoding image buffer. At the same time, the interaction process between the decoding image buffer and the knowledge image library is consistent with the interaction process between the coded image buffer and the knowledge image library, and no further explanation is given here. After receiving the knowledge image, the decoding layer decodes and outputs the video, thereby completing the transmission of the video reference knowledge image. It can be known that the knowledge image in the prior art must be transmitted by the system layer in a dual-stream transmission including a video stream and a knowledge image stream.

[0079] This embodiment proposes a knowledge image coding slice transmission scheme, which is applied to a video transmission device, and the video transmission device includes: an encoder side and a decoder side. The encoder side interweaves the knowledge image in the code stream with the image encoded with the subsequent reference knowledge image in the code stream, so as to achieve smooth transmission of the knowledge image and the subsequent encoded image. By adding syntax and corresponding related syntax constraints in the knowledge image header, the knowledge image can be decoded in the unit of coding slice on the decoder side, and the complete knowledge image frame can be decoded normally without the support and cooperation of the system layer and put into the decoding cache for subsequent image reference. Figure 5 is a flowchart of the video transmission method provided by this specific embodiment. Figure 5 , the encoder side obtains the image frame to be encoded of the video to be transmitted, and encodes the knowledge image frame referenced by the image frame to be encoded to interweave the target image, which is transmitted in the code stream, and then the code stream of the knowledge image is interwoven with the image encoded with the subsequent reference knowledge image in the code stream in units of coding slices, so as to achieve smooth transmission of the knowledge image and the subsequent encoded image. After the decoder side obtains the target image, it decodes the target image to obtain a decoded image. The image frame to be encoded here is the image to be encoded in the aforementioned embodiment, and the knowledge image frame here is the knowledge image in the aforementioned embodiment. The knowledge image can be decoded in units of coding slices on the decoder side, and the complete knowledge image frame can be decoded normally and placed in the decoding cache for subsequent image reference without the support and cooperation of the system layer.

[0080] In one of the specific embodiments, a knowledge image coding slice transmission method is provided. First, the knowledge image is divided into several image slices. The number of slices is optionally set to 12-18 slices. At the same time, in order to ensure that the knowledge image obtains a good coding quality, its quantization parameter (Quantizer Parameter, QP) offset value is optionally set to -8. The smaller the value, the finer the quantization, the higher the image quality, and the longer the generated code stream. If the QP is small, most of the details will be retained; if the QP is increased, some details will be lost, the code rate will be reduced, but the image distortion will be enhanced and the quality will be reduced. Thereafter, each knowledge image slice is encoded, and the knowledge image slice is made to have the same code rate as the subsequent P frame during encoding, which can effectively avoid code rate spikes. Thereafter, the knowledge image coding slice and the image encoded by the subsequent reference knowledge image are interleaved and transmitted, and the relevant grammatical constraints are correspondingly performed so that the decoder can decode the complete knowledge image frame and put it into the decoding cache for reference by the subsequent image. In addition, by specifying an image encoded by a subsequent reference knowledge image, it is indicated that all knowledge image coding slices should be received before this image to determine that all knowledge image coding slices are completely received.

[0081] Further, relevant grammatical constraints include:

[0082] 1) There is only one valid knowledge image in the decoding cache at any time.

[0083] 2) Mark the number of knowledge image encoding slices in the knowledge image header.

[0084] 3) The absolute index of the current slice is marked in the slice header of each knowledge image coding slice (ie, there is an absolute mapping relationship between the current coding slice and the position of the image). The coding slice here is the image slice in the above embodiment.

[0085] 4) All coded slices of a knowledge image must be transmitted in ascending order of index.

[0086] 5) All coded slices of one knowledge image must precede all coded slices of another knowledge image.

[0087] 6) If one of all the coded slices of a knowledge image is located before the coding of a knowledge image reference image, then all the other coded slices of the knowledge image must be located before the coding of the knowledge image reference image.

[0088] 7) There can be at most one knowledge image between two random access points.

[0089] 8) The first coded slice of a knowledge image must be followed by the picture parameter set PPS of the knowledge image, and the PPS of the knowledge image must be followed by a sequence parameter set SPS.

[0090] According to the above grammatical constraints, the decoder can receive each knowledge image coding fragment in the order of index. If the index is discontinuous, it can be determined that the intermediate knowledge image coding fragment is lost. At the same time, a knowledge image coding fragment counter is set to determine whether all the coding fragments of a knowledge image are completely received, so as to determine whether the decoding operation can be started. The knowledge image coding fragment counter here is the coding counter preset in the aforementioned embodiment.

[0091] Optionally, a syntax is added to the knowledge image header to specify the interleaving method of the knowledge image coding slices and the reference knowledge image coding images, for example, marking the number of coded images interleaved between two adjacent knowledge image coding slices.

[0092] By specifying an image to be encoded with a subsequent reference knowledge image, it is indicated that all knowledge image encoding fragments should be received before this image, so as to judge whether the grammatical constraints in all knowledge image encoding fragments are completely received, that is, the preset grammar in the above-mentioned embodiment includes:

[0093] 1. Set the NALU (Network Abstraction Layer Unit) type of the knowledge image reference frame. Constrain the knowledge image referenced by the knowledge image reference frame to be the first knowledge image in the reverse order of the decoding order (i.e., the last knowledge image slice sorted). At the same time, constrain the first encoded image that references a knowledge image to be the knowledge image reference image, i.e., the reference image in the aforementioned embodiment.

[0094] 2. Parse the header information of each frame and analyze its reference image list. If an image references a knowledge image, all coded slices of the knowledge image must be received before the image. The referenced knowledge image index can be determined based on the reference frame indication inside the image header.

[0095] 3. Add syntax in the knowledge image header to mark the POI (Point of Interest, display order index) or DOI (Digital Object Identifier, decoding order index) of the image, or the delta POI or delta DOI of the POI or DOI of the first image after the knowledge image.

[0096] This embodiment slices the knowledge image, and transmits the encoded slices interleaved with the image encoded by the subsequent reference knowledge image, and performs relevant grammatical constraints accordingly, breaking through the AVS3 standard that the knowledge image must use the system layer to cooperate in dual-stream transmission, and solving the bit rate "spike" that occurs during transmission.

[0097] At the same time, by specifying an image to be encoded with a subsequent reference knowledge image, it means that all knowledge image encoding fragments should be received before this image, so as to judge whether all knowledge image encoding fragments are completely received. This solves the problem of being unable to judge whether the fragments are received completely during knowledge image transmission and improves the robustness of knowledge image encoding fragment transmission.

[0098] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a group of computer executable instructions.

[0099] In this embodiment, a video transmission device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit" and the like used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0100] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0101] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0102] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0103] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0104] S1, obtaining a plurality of knowledge images referenced by a plurality of images to be encoded in a video to be transmitted; and dividing the knowledge images into a plurality of image slices.

[0105] S2, according to a preset interleaving strategy, interleave encoding is performed on the image to be encoded through multiple image slices to obtain an encoded image.

[0106] S3, constraining the coded image after interleaving coding according to a preset syntax to obtain a target image; and outputting the target image.

[0107] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0108] In addition, in combination with the video transmission method provided in the above embodiments, a storage medium may be provided in this embodiment to implement the video transmission method. The storage medium stores a computer program, and when the computer program is executed by a processor, any one of the video transmission methods in the above embodiments is implemented.

[0109] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0110] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0111] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0112] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A video transmission method, characterized in that: Applied to the encoder side, the method includes: Acquire multiple knowledge images referenced by multiple images to be encoded in a video to be transmitted; divide the knowledge images into multiple image slices; According to a preset interleaving strategy, the image to be encoded is interleaved and encoded through the multiple image slices to obtain an encoded image; wherein the preset interleaving strategy includes: interleaving according to the importance and dependency of the knowledge image to the image to be encoded; The coded image after interleaving coding is constrained according to a preset grammar to obtain a target image; the target image is output; the preset grammar is determined according to the order, continuity and integrity of the image slices in the coded image.

2. The video transmission method according to claim 1, characterized in that: The knowledge image includes a first knowledge image and a second knowledge image; the number of image slices is marked in the image slice header of the knowledge image; the first knowledge image and the second knowledge image are adjacent; The step of interleaving and encoding the image to be encoded by using the plurality of image slices according to a preset interleaving strategy to obtain an encoded image includes: Determining the arrangement order of the image slices according to a preset mapping relationship; According to the arrangement order, the image slices corresponding to the first knowledge image and the second knowledge image are interleaved and encoded to obtain an encoded image.

3. The video transmission method according to claim 1, characterized in that: The step of constraining the coded image after interleaving coding according to a preset grammar to obtain a target image includes: According to a preset grammar, a reference image in the knowledge image is determined; the coding order of the reference image is located after the coding order of the knowledge images other than the reference image.

4. The video transmission method according to claim 2, characterized in that: When the first image slice of the first knowledge image is located before the image slice of the second knowledge image in the coding order, the second image slice of the first knowledge image is also located before the image slice of the second knowledge image in the coding order; The first image slice and the second image slice are encoded in different orders.

5. The video transmission method according to any one of claims 1 to 4, characterized in that: The image to be encoded includes a predicted image frame; The code rate of the predicted image frame is consistent with that of the image slice; the predicted image frame is used to represent the difference between the multiple knowledge images.

6. The video transmission method according to any one of claims 1 to 4, characterized in that: The method further comprises: An index mark is added to the image slice header of the knowledge image; the index mark includes a display order index mark and a decoding order index mark.

7. A video transmission method, characterized in that: Applied to the decoder side, the method comprises: Receive a target image, and determine whether the arrangement order of image slices in the target image is consistent with a preset mapping relationship; the target image is a plurality of knowledge images referenced by a plurality of images to be encoded divided into a plurality of image slices, and according to a preset interleaving strategy, the image to be encoded is interleaved and encoded through the plurality of image slices, and the coded image after the interleaving is constrained according to a preset grammar; wherein the preset interleaving strategy includes: interleaving according to the importance and dependency of the knowledge image corresponding to the image to be encoded; the preset grammar is determined according to the order, continuity and integrity of the image slices in the coded image; If yes, judging whether the number of the image slices reaches a preset number threshold according to a preset encoding counter; If so, the target image is decoded to obtain a decoded knowledge image.

8. The video transmission method according to claim 7, characterized in that: The determining whether the number of the image slices reaches a preset number threshold comprises: Acquire a reference image in the target image, and determine the number of image slices that precede the reference image in coding order; It is determined whether the number of image slices that are coded before the reference image reaches a preset number threshold.

9. The video transmission method according to claim 7, characterized in that: The method further comprises: Storing the knowledge image in a preset decoding buffer area; At the same time, a preset number of knowledge images stored in the decoding buffer area are set to take effect.

10. A video transmission device, characterized in that: The device comprises: an encoder and a decoder; the encoder and the decoder are connected; The encoder is used to perform the video transmission method according to any one of claims 1 to 6; The decoder is used to execute the video transmission method according to any one of claims 7 to 9.

Images