Image encoding method, device, electronic device and storage medium

Abstract

The present disclosure provides an image encoding method, device, electronic device and storage medium, which relate to the field of image processing and can provide a guidance scheme for encoding images at different bandwidths and / or frame rates. The specific technical scheme is: obtaining an encoding method for a first image, the first image being at least one frame image before a current frame image; acquiring the current frame image; obtaining a current transmission frame rate and a first frame rate; if the encoding method for the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image; the encoded current frame image is sent to a decoding end and a first indication message is sent to the decoding end.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image encoding method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of computer technology, image processing, especially computer-generated image processing, is also applied to more scenarios. Therefore, when encoding and decoding computer-generated images, it is necessary to cover multiple scenarios such as office and games. However, when encoding computer-generated images, different scenarios have different requirements for bandwidth and / or frame rate. However, the prior art has not yet provided a guidance solution on how to encode images at different bandwidths and / or frame rates. Summary of the invention

[0003] The embodiments of the present disclosure provide an image encoding method, device, electronic device and storage medium to provide a guidance scheme for encoding images at different bandwidths and frame rates. The technical scheme is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image encoding method, the method comprising:

[0005] Acquire a coding method of a first image, where the first image is at least one frame image before a current frame image;

[0006] Acquiring the current frame image;

[0007] Acquire a current transmission frame rate and a first frame rate, where the first frame rate is an average of frame rates when a second image is encoded and transmitted in a grating order encoding manner, and the second image is at least one frame image before the first image;

[0008] If the encoding method of the first image is a wavefront sequence encoding method, when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image;

[0009] The encoded current frame image is sent to a decoding end and a first indication message is sent to the decoding end so that the decoding end decodes the encoded current frame image. The first indication message is used to indicate to the decoding end that the encoding method of the current frame image is a raster order encoding method.

[0010] The image encoding method provided by the embodiment of the present disclosure can take the encoding mode of the first image, where the first image is at least one frame image before the current frame image; collect the current frame image; obtain the current transmission frame rate and the first frame rate, where the first frame rate is the average frame rate when the second image is encoded and transmitted using the grating sequence encoding mode, and the second image is at least one frame image before the first image; if the encoding mode of the first image is the wavefront sequence encoding mode, then when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, the current frame image is encoded using the grating sequence encoding mode to generate the encoded current frame image; the encoded current frame image is sent to a decoding end and a first indication message is sent to the decoding end so that the decoding end decodes the encoded current frame image, the first indication message is used to indicate to the decoding end that the encoding mode of the current frame image is the grating sequence encoding mode, and when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the encoding mode of the image can be switched from the wavefront sequence to the grating sequence, which can effectively improve the frame rate.

[0011] In one embodiment, after acquiring the current frame image, the method further includes:

[0012] Obtaining a collection method of the current frame image;

[0013] If the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold and the acquisition method of the current frame image is a striped acquisition method, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image.

[0014] The frame rate can be improved by encoding the current frame image in a grating sequence encoding method when the encoding method of the first image is a wavefront sequence encoding method, the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold, and the acquisition method of the current frame image is a striped acquisition method.

[0015] In one embodiment, after acquiring the current frame image, the method further includes:

[0016] Get the current transmission bandwidth and current transmission bit rate;

[0017] If the encoding method of the first image is a grating sequence encoding method, when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image;

[0018] The encoded current frame image is sent to a decoding end and a second indication message is sent to the decoding end, wherein the second indication message is used to indicate to the decoding end that the encoding method of the current frame image is a wavefront sequence encoding method.

[0019] By encoding the current frame image in a wavefront sequence when the encoding method of the first image is a raster sequence encoding method and the current bandwidth is less than the current transmission bit rate, the encoded current frame image is generated, thereby effectively reducing the bit rate.

[0020] In one embodiment, after acquiring the current frame image, the method further includes:

[0021] Obtaining a collection method of the current frame image;

[0022] If the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the acquisition method of the current frame image is a full-frame acquisition method, the wavefront sequence encoding method is used to encode the current frame image to generate an encoded current frame image.

[0023] By encoding the current frame image in a wavefront sequence encoding method when the encoding method of the first image is a raster sequence encoding method, the current bandwidth is greater than or equal to the current transmission bit rate, and the acquisition method of the current frame image is a full-frame acquisition method, the bit rate can be reduced.

[0024] In one embodiment, the method further comprises:

[0025] If the image encoding instruction input by the user is not received and the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image, the image encoding instruction includes identification information of the image encoding method, and the image encoding methods include a wavefront sequence encoding method and a grating sequence encoding method;

[0026] If an image encoding instruction input by a user is received, the current frame image is encoded using an encoding method corresponding to the identification information.

[0027] When receiving an image encoding instruction input by a user, the current frame image is encoded using an encoding method corresponding to the identification information, thereby improving the user experience.

[0028] In one embodiment, the method further comprises:

[0029] If no image encoding instruction input by the user is received and if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image.

[0030] In one embodiment, after sending the encoded current frame image to a decoding end and sending a second indication message to the decoding end, the method further includes:

[0031] If a third indication message sent by the decoding end is received, the next frame image is encoded using a grating sequence encoding method. The third indication message is used to instruct the encoding end to encode the next frame image using a grating sequence encoding method. The third indication message is sent by the decoding end to the encoding end after determining that the ratio of the second frame rate to the third frame rate is less than a second preset threshold. The second frame rate is the frame rate obtained by the decoding end when receiving the encoded image and decoding the encoded image using a wavefront sequence decoding method. The third frame rate is the average value of the frame rates obtained when the decoding end receives the fourth image and decodes the fourth image using a wavefront sequence decoding method. The fourth image is at least one frame image received by the decoding end before the encoded image.

[0032] By encoding the next frame of image using a grating sequence encoding method after receiving the third indication message sent by the decoding end, the decoding end can use a wavefront sequence decoding method to decode the encoded image, which seriously affects the frame rate, is too slow, and the picture display is not smooth enough. A third indication message is sent to the encoding end to instruct the encoding end to switch to a grating sequence encoding method to encode the next frame of image. Then, after receiving the next frame of image after encoding, the decoding end uses a grating sequence decoding method to decode the next frame of image, which makes the picture display smoother and improves the user experience.

[0033] According to a second aspect of an embodiment of the present disclosure, there is provided an image encoding apparatus, including:

[0034] An encoding mode acquisition module, used to acquire an encoding mode of a first image, where the first image is at least one frame image before a current frame image;

[0035] A current frame image acquisition module, used for acquiring the current frame image;

[0036] A current transmission frame rate acquisition module, used to acquire a current transmission frame rate and a first frame rate, wherein the first frame rate is an average frame rate when a second image is encoded and transmitted in a grating sequence encoding manner, and the second image is at least one frame image before the first image;

[0037] A current frame image encoding module, configured to, if the encoding method of the first image is a wavefront sequence encoding method, encode the current frame image in a grating sequence encoding method when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, to generate an encoded current frame image;

[0038] A current frame image sending module is used to send the encoded current frame image to a decoding end and send a first indication message to the decoding end so that the decoding end can decode the encoded current frame image. The first indication message is used to indicate to the decoding end that the encoding method of the current frame image is a raster order encoding method.

[0039] In one embodiment, the apparatus further comprises:

[0040] A current frame image acquisition mode acquisition module, used to acquire the acquisition mode of the current frame image;

[0041] The current frame encoding module is used to, if the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold and the acquisition method of the current frame image is a striped acquisition method, encode the current frame image using a grating sequence encoding method to generate an encoded current frame image.

[0042] In one embodiment, the apparatus further comprises:

[0043] The current transmission bandwidth acquisition module is used to obtain the current transmission bandwidth and the current transmission bit rate;

[0044] The current frame encoding module is used for, if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, encoding the current frame image in a wavefront sequence encoding method to generate an encoded current frame image;

[0045] The encoded current frame image is sent to a decoding end and a second indication message is sent to the decoding end, wherein the second indication message is used to indicate to the decoding end that the encoding method of the current frame image is a wavefront sequence encoding method.

[0046] In one embodiment, the apparatus further comprises:

[0047] A current frame image acquisition mode acquisition module, used to acquire the acquisition mode of the current frame image;

[0048] The current frame image encoding module is used to, if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the acquisition method of the current frame image is a full-frame acquisition method, use a wavefront sequence encoding method to encode the current frame image to generate an encoded current frame image.

[0049] In one embodiment, the current frame image encoding module is further used for:

[0050] If the image encoding instruction input by the user is not received and the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image, the image encoding instruction includes identification information of the image encoding method, and the image encoding methods include a wavefront sequence encoding method and a grating sequence encoding method;

[0051] If an image encoding instruction input by a user is received, the current frame image is encoded using an encoding method corresponding to the identification information.

[0052] In one embodiment, the current frame image encoding module is further used for:

[0053] If no image encoding instruction input by the user is received and if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image.

[0054] In one embodiment, the apparatus further comprises:

[0055] A next frame image encoding module is used to encode the next frame image by adopting a grating sequence encoding method if a third indication message sent by the decoding end is received. The third indication message is used to instruct the encoding end to encode the next frame image by adopting a grating sequence encoding method. The third indication message is sent by the decoding end to the encoding end after determining that the ratio of the second frame rate to the third frame rate is less than a second preset threshold. The second frame rate is the frame rate obtained by the decoding end when receiving the encoded image and decoding the encoded image by adopting a wavefront sequence decoding method. The third frame rate is the average value of the frame rates obtained when the decoding end receives a fourth image and decodes the fourth image by adopting a wavefront sequence decoding method. The fourth image is at least one frame of image received by the decoding end before the encoded image.

[0056] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the image encoding method described in any one of the first aspects.

[0057] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein at least one computer instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the steps performed in the image encoding method described in any one of the first aspects.

[0058] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] Figure 1 is a flowchart of an image encoding method provided by an embodiment of the present disclosure;

[0061] Figure 2 It is a schematic diagram of a grating sequence encoding method provided by an embodiment of the present disclosure;

[0062] Figure 3A This is a schematic diagram of a wavefront encoding method provided by an embodiment of the present disclosure. Figure 1 ;

[0063] Figure 3B This is a schematic diagram of a wavefront encoding method provided by an embodiment of the present disclosure. Figure 2 ;

[0064] Figure 4A is a block diagram of an encoding end provided by an embodiment of the present disclosure;

[0065] Figure 4B is a block diagram of a decoding end provided by an embodiment of the present disclosure;

[0066] Figure 5 The structure of an image encoding device provided by the embodiment of the present disclosure is shown in FIG. Figure 1 ;

[0067] Figure 6 The structure of an image encoding device provided by the embodiment of the present disclosure is shown in FIG. Figure 2 ;

[0068] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0070] Figure 1 FIG. 1 is a flowchart of an image encoding method provided by an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0071] S101: Obtain a coding method for a first image, where the first image is at least one frame image before a current frame image.

[0072] In this embodiment, the image may be encoded using a wavefront sequence encoding method or a grating sequence encoding method.

[0073] The encoding method of the raster sequence is described below. Figure 2 It is a schematic diagram of a grating order encoding method provided by an embodiment of the present disclosure. When the grating order encoding method is used to encode an image, the image needs to be divided into macroblocks. Taking a frame of an image with a resolution of 1920*1080 as an example, if a frame of an image with a resolution of 1920*1080 is divided into multiple macroblocks of 16*16 pixels, then a frame of an image with a resolution of 1920*1080 has a total of 120*68 macroblocks. The grating order encoding method uses a multi-threaded method to perform parallel encoding when encoding an image to increase the speed. Multiple threads can process multiple groups of macroblocks at the same time, and each thread encodes each of a group of macroblocks in turn. After the encoding of the current macroblock is completed, the next macroblock is encoded.

[0074] like Figure 2As shown, assuming that there are currently 4 threads participating in the encoding, the raster order encoding method is to arrange a thread to process each row of macroblocks. Thread 1 processes parallel group 1 (i.e., the first row of macroblocks). Thread 1 first encodes the macroblock numbered 1 in the figure, then encodes the macroblock numbered 2 in the figure... From left to right, encode each macroblock in the first row, until the macroblock numbered 120 in the rightmost figure. Threads 2 / 3 / 4 are similar, and each takes a row of macroblocks and encodes them from left to right. Assuming that thread 1 completes the encoding of the 120 macroblocks assigned to it first, and thread 2 / 3 / 4 has not yet completed, thread 1 will turn to the next unencoded Xth row of macroblocks (the Xth parallel group in the figure), and encode the Xth row of macroblocks from left to right. In short, each thread encodes a row of macroblocks from left to right. When the encoding of the row of macroblocks is completed, it automatically selects the nearest macroblock row that has not been encoded to continue encoding. The advantage of the raster order encoding method is that there is no need to wait between threads, that is, the encoding of each parallel group is unrelated. After thread 1 encodes the first macroblock, it automatically encodes the second macroblock without additional waiting. The disadvantage of the raster order encoding method is that it can only use the reference information between the left and right adjacent macroblocks, and does not fully utilize the reference information of the upper and lower macroblocks to reduce the bit rate. Take the macroblock numbered 123 as an example. If its content is exactly the same as that of macroblock numbered 3, the content of macroblock numbered 3 can be used as its reference, so there is no need to encode macroblock numbered 123. However, this order isolates the relationship between macroblock rows, that is, parallel groups, and this reference information is not used. Macroblock numbered 123 also has to participate in the encoding, resulting in an increase in the bit rate.

[0075] The following is an explanation of the wavefront encoding method. Similarly, when using the wavefront encoding method to encode an image, the image needs to be divided into macroblocks. Wavefront is also a multi-threaded encoding method. In essence, it has only one requirement, that is, before a macroblock is encoded, it must be ensured that the macroblocks to its left and above (if any) have been encoded. Figure 3A For example, Figure 3A This is a schematic diagram of a wavefront encoding method provided by an embodiment of the present disclosure. Figure 1 When a thread is finished executing, before taking X macroblock for encoding, it must first wait for the TOP macroblock and the LEFT macroblock to complete encoding. If one of the macroblocks is not encoded, the thread will wait until the TOP macroblock and the LEFT macroblock are encoded before encoding X macroblock.

[0076] The wavefront-order encoding method is used. When multiple threads encode an image, unlike the raster-order encoding method, each thread takes a group (such as a row) of macroblocks for processing. Instead, each thread takes macroblocks from the entire frame of the image in sequence. Figure 3B This is a schematic diagram of a wavefront encoding method provided by an embodiment of the present disclosure. Figure 2,like Figure 3B As shown, first encode macroblock No. 1. When the thread is idle, the macroblock closest to macroblock No. 1 and whose upper and left macroblocks have been encoded will be processed. If the upper and left macroblocks are not completed, the thread will execute waiting. Figure 3B The curve in shows a possible macroblock encoding order. Since multiple threads concurrently fetch macroblocks for encoding, there may be multiple orders. Figure 3B Taking the order of the middle curve as an example, before encoding macroblock No. 123, macroblocks No. 3 and No. 122 have been encoded; before encoding No. 243, macroblocks No. 242 and No. 123 have been encoded, so Figure 3B The order of the middle curves meets the requirements of the wavefront sequence encoding method.

[0077] The reason for this requirement is that the wavefront encoding method adds a reference information. Figure 3A The X macroblock in the image can refer to the information of both LEFT and TOP macroblocks. In the aforementioned raster order encoding method, the X macroblock only refers to the information of the LEFT macroblock. When the wavefront encoding method is used, if the X macroblock is the same as LEFT or TOP, it is directly marked as a left reference or top reference macroblock. After the decoding end recognizes it as this type of macroblock, it can directly copy the content of the decoded macroblock on its left or above, saving the encoding bitstream. When the wavefront encoding method is used to encode the image, the information of the LEFT macroblock and the TOP macroblock needs to be referred to when encoding the X macroblock. During encoding and decoding, it is necessary to wait until the LEFT macroblock and TOP macroblock are completed at the same time before processing X. This is the source of the wavefront requirement.

[0078] However, since the thread execution needs to wait for the completion of the left and upper macroblocks, the thread execution is delayed to a certain extent. Therefore, the wavefront encoding method may result in slow encoding speed and low frame rate.

[0079] S102: Capture the current frame image.

[0080] Exemplarily, in this embodiment, the image can be acquired by a striped acquisition method or a full-frame acquisition method. The full-frame acquisition method is to output a complete frame of image each time it is acquired; after the image is acquired by the full-frame acquisition method, the reference relationship between the upper and lower lines of macroblocks can be fully utilized when encoding the image, which is suitable for the wavefront sequence encoding method; the striped acquisition method is to acquire one or several rows of macroblocks each time, and then output the acquired one or several rows of macroblocks in sequence, that is, sliced ​​output. In this case, since the encoding end can encode according to the stripes, that is, there is no reference relationship between different strips, if the upper and lower rows of macroblocks span two different stripes, they cannot be referenced, that is, after the image is acquired by the striped acquisition method, the effect of encoding the image using the wavefront sequence encoding method may not be good, so the striped acquisition method is suitable for the encoding method that only refers to the raster sequence of the macroblock on the left.

[0081] S103, obtaining a current transmission frame rate and a first frame rate, where the first frame rate is an average frame rate when a second image is encoded and transmitted using a grating sequence encoding method, and the second image is at least one frame image before the first image.

[0082] Exemplarily, the first frame rate is the average frame rate when the second image is encoded and transmitted using the grating sequence encoding method before the current frame image, that is, the average frame rate when the grating sequence encoding method has appeared in history to encode and transmit the image (historical grating sequence average frame rate). The current transmission frame rate is the frame rate within the current preset time length, for example, the frame rate within the current 1 minute.

[0083] S104. If the encoding method of the first image is a wavefront-sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, a grating-sequence encoding method is used to encode the current frame image to generate an encoded current frame image.

[0084] In this embodiment, if the image encoding instruction input by the user is not received and the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image, and the image encoding instruction includes identification information of the image encoding method, and the image encoding method includes a wavefront sequence encoding method and a grating sequence encoding method;

[0085] If an image encoding instruction input by the user is received, the current frame image is encoded using an encoding method corresponding to the identification information.

[0086] Exemplarily, the first preset threshold value may be 60% to 80%, for example, 70%. If the image encoding instruction input by the user is not received and the encoding method of the first image is the wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold value, that is, when the image encoding instruction input by the user is not received and the current transmission frame rate is less than 70% of the historical grating sequence average frame rate, the grating sequence encoding method is used to encode the current frame image. The current transmission frame rate is less than 70% of the historical grating sequence average frame rate, which means that the frame rate of the wavefront sequence encoding method is too low and is no longer suitable for the wavefront sequence mode, so the grating sequence encoding method is switched.

[0087] It should be noted here that this embodiment only exemplarily shows the value range of the first preset threshold value, and the value of the first preset threshold value may also be other values, and this embodiment does not make any specific limitation here.

[0088] In one embodiment, after acquiring the current frame image, the method further includes:

[0089] Obtaining the acquisition mode of the current frame image;

[0090] If the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold and the acquisition method of the current frame image is a striped acquisition method, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image.

[0091] Since the current frame image is acquired in a striping acquisition method, the striping acquisition method is suitable for an encoding method that only refers to the raster order of the left macroblock. Therefore, when the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold, the current frame image is encoded in a raster order encoding method, which can improve the frame rate.

[0092] It should be noted here that in order to improve user experience, if an image encoding instruction input by the user is received, the current frame image is directly encoded using the encoding method corresponding to the identification information, without paying attention to the encoding method of the first image, the acquisition method of the current frame image, the current transmission frame rate and the first frame rate.

[0093] S105, sending the encoded current frame image to a decoding end and sending a first indication message to the decoding end so that the decoding end decodes the encoded current frame image, wherein the first indication message is used to indicate to the decoding end that the encoding method of the current frame image is a raster order encoding method.

[0094] Furthermore, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image, the encoded current frame image is sent to a decoding end and a first indication message is sent to the decoding end, so that the decoding end learns from the first message that the encoding method of the current frame image is a grating sequence encoding method, and then the decoding end decodes the encoded current frame image using a grating sequence decoding method.

[0095] The following describes how to switch the encoding mode of the current frame image to the wavefront sequence encoding mode when the encoding mode of the first image is the raster sequence encoding mode.

[0096] Exemplarily, after acquiring the current image to be encoded, the method further includes:

[0097] Get the current transmission bandwidth and current transmission bit rate;

[0098] If the encoding method of the first image is a grating sequence encoding method, when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image;

[0099] The encoded current frame image is sent to a decoding end and a second indication message is sent to the decoding end, where the second indication message is used to indicate to the decoding end that the encoding method of the current frame image is a wavefront sequence encoding method.

[0100] In this embodiment, if the image encoding instruction input by the user is not received and if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image. Further, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image, and the encoded current frame image is sent to the decoding end and a second indication message is sent to the decoding end, so that the decoding end learns from the second message that the encoding method of the current frame image is a wavefront sequence encoding method, and then the decoding end decodes the encoded current frame image using a wavefront sequence decoding method.

[0101] Exemplarily, the current transmission bit rate is the bit rate within the current preset time length, such as the bit rate within the current 1 minute, and the current transmission bandwidth is the current available bandwidth. If the image encoding instruction input by the user is not received and the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, the grating sequence encoding method is used to encode the current frame image. The current bandwidth is less than the current transmission bit rate, that is, the current transmission bit rate has exceeded the current available bandwidth, and the current available bandwidth has been occupied. At this time, the wavefront sequence encoding method should be switched to reduce the bit rate.

[0102] In another embodiment, after acquiring the current frame image, the method further includes:

[0103] Obtaining the acquisition mode of the current frame image;

[0104] If the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the acquisition method of the current frame image is a full-frame acquisition method, the wavefront sequence encoding method is used to encode the current frame image to generate an encoded current frame image.

[0105] Since the acquisition method of the current frame image is a full-frame acquisition method, which is suitable for a wavefront sequence encoding method, when the current bandwidth is greater than or equal to the current transmission bit rate and the acquisition method of the current frame image is a full-frame acquisition method, the wavefront sequence encoding method is used to encode the current frame image, which can reduce the bit rate.

[0106] It should be noted here that in order to improve user experience, if an image encoding instruction input by the user is received, the current frame image is directly encoded using the encoding method corresponding to the identification information, without paying attention to the encoding method of the first image, the acquisition method of the current frame image, the current bandwidth, the current transmission bit rate and the first bit rate.

[0107] In a third embodiment, the method further comprises:

[0108] Obtaining a first bit rate, where the first bit rate is an average bit rate when the second image is encoded and transmitted in a wavefront sequence encoding manner;

[0109] If the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the ratio of the current transmission bit stream to the first bit rate is greater than a third preset threshold, the wavefront sequence encoding method is used to encode the current frame image to generate an encoded current frame image.

[0110] Exemplarily, the first bit rate is the average bit rate when the second image is encoded and transmitted using the wavefront encoding method before the current frame image, that is, the average bit rate when the image is encoded and transmitted using the wavefront encoding method that has appeared in history (historical wavefront average bit rate). Exemplarily, the third preset threshold value can be 120% to 140%, for example, 130%.

[0111] If the encoding method of the first image is a raster sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the ratio of the current transmission bit stream to the first bit rate is greater than the third preset threshold, that is, when the current bandwidth is sufficient and the current transmission bit rate and the ratio of the current transmission bit stream to the first bit rate are greater than 130%, the raster sequence encoding method is used to encode the current frame image.

[0112] The current transmission bit rate exceeds 130% of the historical wavefront sequence average bit rate, which means that the bit rate of encoding the image using the grating sequence encoding method is too high and the encoding method should be switched to the wavefront sequence.

[0113] It should be noted that this embodiment only exemplarily shows the value range of the third preset threshold value, and the value of the third preset threshold value may also be other values, and this embodiment does not make any specific limitation here.

[0114] Furthermore, after sending the encoded current frame image to the decoding end and sending the second indication message to the decoding end, the method further includes:

[0115] If a third indication message sent by the decoding end is received, the next frame image is encoded using a grating sequence encoding method. The third indication message is used to instruct the encoding end to encode the next frame image using a grating sequence encoding method. The third indication message is sent to the encoding end by the decoding end after determining that the ratio of the second frame rate to the third frame rate is less than a second preset threshold. The second frame rate is the frame rate obtained by the decoding end when receiving the encoded image and decoding the encoded image using a wavefront sequence decoding method. The third frame rate is the average value of the frame rates obtained when the decoding end receives the fourth image and decodes the fourth image using a wavefront sequence decoding method. The fourth image is at least one frame image received by the decoding end before the encoded image.

[0116] Exemplarily, the second frame rate is the frame rate obtained when the decoding end receives the encoded image and decodes the encoded image using the wavefront decoding method, that is, the wavefront real-time frame rate. The third frame rate is the average frame rate obtained when the decoding end receives the fourth image before the encoded current frame image and decodes the fourth image using the grating decoding method, that is, the historically counted grating average frame rate. The second preset threshold value can be 60% to 80%, for example, 70%.

[0117] If the ratio of the second frame rate to the third frame rate is less than the second preset threshold, that is, the real-time frame rate of the wavefront sequence is less than 70% of the historically counted average frame rate of the grating sequence, then the wavefront sequence decoding method is used to decode the encoded image, which seriously affects the frame rate. The speed is too slow, and the picture display is not smooth enough. Switching to the grating sequence decoding method may bring a better picture experience. At this time, the decoding end sends a third indication message to the encoding end to instruct the encoding end to switch to the grating sequence encoding method to encode the next frame of the image. After receiving the next frame of the encoded image, the decoding end uses the grating sequence decoding method to decode the next frame of the image, which can give users a better picture experience.

[0118] It should be noted that this embodiment only exemplarily shows the value range of the second preset threshold value, and the value of the second preset threshold value may also be other values, and this embodiment does not make any specific limitation here.

[0119] The image encoding method provided by the embodiment of the present disclosure can take the encoding mode of the first image, where the first image is at least one frame image before the current frame image; collect the current frame image; obtain the current transmission frame rate and the first frame rate, where the first frame rate is the average frame rate when the second image is encoded and transmitted using the grating sequence encoding mode, and the second image is at least one frame image before the first image; if the encoding mode of the first image is the wavefront sequence encoding mode, then when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, the current frame image is encoded using the grating sequence encoding mode to generate the encoded current frame image; the encoded current frame image is sent to a decoding end and a first indication message is sent to the decoding end so that the decoding end decodes the encoded current frame image, the first indication message is used to indicate to the decoding end that the encoding mode of the current frame image is the grating sequence encoding mode, and when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the encoding mode of the image can be switched from the wavefront sequence to the grating sequence, which can effectively improve the frame rate.

[0120] Combine the following Figure 4A and Figure 4B , the scheme of the embodiment of the present disclosure is further described in detail. Figure 4A is a block diagram of an encoding end provided by an embodiment of the present disclosure. Figure 4A As shown in the figure, the functions of each module on the encoding end are as follows:

[0121] 401: Manual / automatic mode selection module. The system is mainly an adaptive encoding mode, but it also provides the function of manual selection by users. This module controls whether the current selection mode is manual selection or automatic selection.

[0122] 402: When the user sets the manual mode through module 401, the user interface provides a switch so that the user can specify a specific encoding mode, and wavefront order and grating order can be selected. Once selected, the mode manually selected by the user is fixed and unchanged unless the user modifies it or switches to automatic mode. When presenting the wavefront order and grating order to the user, this module can be described as "bit rate priority" and "frame rate priority" respectively.

[0123] 403: Mode judgment module. This module determines whether the encoding mode uses wavefront sequence or grating sequence. In manual mode, it reads the 402 module, which is the setting of the user through the interface, as the specific encoding mode. In automatic mode, it will read the data of the 404 / 405 / 406 / 407 / 410 modules and comprehensively determine which mode to use.

[0124] (1) The wavefront sequence and grating sequence used in this scheme are described in detail below.

[0125] The encoding and decoding of this solution is carried out by dividing the picture into macroblocks. Taking a 1920*1080 resolution image as an example, the full frame has a total of 120*68 macroblocks. This solution uses multi-threading to perform parallel encoding and decoding during encoding and decoding to increase the speed. Multiple threads can process multiple macroblocks at the same time, and each thread processes one macroblock at a time. When the encoding is completed, the next macroblock will be taken for processing. Which macroblock to take next for processing is related to the specific scanning order. Let's first look at the raster order encoding method, which can be summarized as follows: Figure 2 .

[0126] Assuming that there are 4 threads in the current system participating in encoding, the raster order scanning will arrange one thread to process each row of macroblocks. In the initial situation, thread 1 processes parallel group 1, which first encodes the macroblock numbered 1 in the figure, then the macroblock of macroblock 2, and so on from left to right to the rightmost macroblock of 120 (for example, an image with a width of 1920 pixels). Threads 2 / 3 / 4 are similar, and each takes a row of macroblocks for processing. Assuming that thread 1 first completes the encoding of the 120 macroblocks assigned to it, and threads 2 / 3 / 4 have not yet completed, thread 1 will turn to the next one (the Xth parallel group in the figure) for similar processing from left to right. In short, each thread encodes a row of macroblocks from left to right. When the work is completed, it automatically selects the nearest macroblock row that has not been processed to continue working. This order is called raster order. The advantage of this order is that there is no need to wait between threads, that is, there is no relationship between the encodings of each parallel group. After thread 1 finishes encoding the first macroblock, it automatically goes to encode the second one without additional waiting. The disadvantage of this order is that it can only use the reference information between the left and right adjacent macroblocks, and does not fully use the reference information of the upper and lower macroblocks to reduce the bit rate. For example, if the content of macroblock No. 123 is exactly the same as that of macroblock No. 3, the content of macroblock No. 3 can be used as its reference, so there is no need to encode macroblock No. 123. However, this order isolates the relationship between macroblock rows, that is, parallel groups, and this reference information is not used. Macroblock No. 123 also participates in the encoding, resulting in an increase in the bit rate.

[0127] Wavefront order is also a kind of order for multi-threaded encoding. It essentially has only one requirement, that is, before processing a macroblock, it must ensure that the macroblocks to its left and above (if any) have been processed. Figure 3A For example, when a thread is finished, it must wait for the TOP and LEFT macroblocks to be coded before it takes X for coding. If one of them is not finished, the thread will wait until it is finished, and then take X macroblock for coding.

[0128] From a macroscopic perspective, when multiple threads are processing, unlike raster order, each thread takes a group (such as a row) of macroblocks for processing, but each thread takes macroblocks from the entire frame in sequence. Figure 3B When the thread is idle, it will take the closest macroblock it has processed and the macroblock above and to the left that has been processed, and process it. If the macroblock above and to the left is not completed, it will execute waiting. Figure 3B The curve in shows a possible macroblock processing order. Note that since multiple threads concurrently fetch macroblocks, there may be multiple orders. Take the order in 3-B as an example. Focus on macroblock No. 123. Before it, No. 3 and No. 122 have been processed. Focus on No. 243. Before it, No. 242 and No. 123 have been processed. Therefore, this order meets the wavefront requirements.

[0129] The reason for this requirement is that the wavefront encoding adds a reference information. Figure 3A The X macroblock in the above raster sequence can refer to the information of both LEFT and TOP macroblocks. However, X in the above raster sequence only refers to the information of LEFT macroblock. Since the X macroblock in the wavefront sequence is the same as LEFT or TOP, it is directly marked as the left reference or top reference macroblock. After the decoding end identifies it as this type of macroblock, it can directly copy the content of the decoded macroblock on its left or top, saving the encoding bitstream. It is precisely because macroblock X needs to refer to the information of LEFT and TOP that X can be processed only after LEFT and TOP are completed at the same time during encoding and decoding. This is the source of the wavefront sequence requirement.

[0130] However, due to the need to wait for the completion of the macroblocks on the left and above, the thread execution is delayed to a certain extent. Therefore, this method may result in slow encoding speed and low frame rate. However, the severity of this situation depends on the specific operating environment, from the operating system type, CPU chip type and other execution environments. For example, if a more powerful CPU and a more mature operating system have powerful thread scheduling and management functions, this kind of waiting will rarely occur, or even be negligible, and will not cause a significant drop in frame rate. However, on some systems with weaker processing performance, especially embedded systems, the impact may be more serious. It is for this reason that this solution sets up a 410 module to monitor wavefront performance.

[0131] To summarize the two encoding and decoding modes, the raster sequence encoding and decoding speed is fast and can improve the frame rate, but the bit rate is relatively high, which is suitable for scenes such as games, focusing on high frame rate and smooth picture without stuttering; the wavefront sequence encoding has a low bit rate, but the encoding speed is relatively slow, which may lead to low frame rate. The decoding is similar. Decoding too slowly may lead to untimely decoding, resulting in frame loss and retransmission, and picture stuttering, which is suitable for office design scenes.

[0132] (2) The following describes the basis for the 403 module to adaptively determine the wavefront order / grating order.

[0133] Due to differences in operating systems, multi-threaded concurrency capabilities of CPU platforms, and different real-time conditions when the system is working (real-time conditions refer to, for example, in Windows systems, if the background is scanning for viruses, other user applications will be affected), the severity of the waiting situation during wavefront processing is different. In the mechanism of adaptive selection of working modes, this situation must be identified. If in some systems, the waiting situation of the wavefront exceeds a certain threshold, it is forced to switch to the raster order mode without waiting between threads. The monitoring method of this module is:

[0134] Collect the frame rates of the raster sequences that have appeared in history and take the average value, recorded as avg_fps_raster.

[0135] Collect the bitrates of the wavefront sequences that have appeared in history and take the average value, which is recorded as avg_bitrate_wavefront.

[0136] Get the real-time available bandwidth from the 404 module, recorded as free_bitrate.

[0137] Get the bitrate of the most recent minute from module 405, recorded as cur_bitrate.

[0138] Get the frame rate in the last minute from the 407 module, recorded as cur_fps.

[0139] If the current mode is wavefront sequence, module 103 detects the following conditions:

[0140] (Condition 1) If the current mode is manually set by the user, encode according to the mode manually set by the user; otherwise, see the following conditions;

[0141] (Condition 2) cur_fps < (avg_fps_raster * 70%)

[0142] That is, the real-time frame rate of the wavefront sequence is less than 70% of the average frame rate of the grating sequence; this means that the frame rate in the wavefront sequence mode is too low, and the wavefront sequence mode is no longer suitable for the current state, and the grating sequence will be switched.

[0143] (Condition 3) Obtain the current acquisition mode from module 406. If the current acquisition mode is striped acquisition, switch the current mode to raster order to increase the frame rate.

[0144] If the current mode is raster order, module 403 checks the following conditions:

[0145] (Condition 1) If the current mode is manually set by the user, encode according to the mode manually set by the user; otherwise, see the following conditions;

[0146] (Condition 4)cur_bitrate>free_bitrate

[0147] That is, the current bit rate has exceeded the available bandwidth, that is, the bandwidth has been fully occupied. At this time, the wavefront sequence should be switched to save the bit rate.

[0148] (Condition 3) The acquisition mode is obtained from module 406. If the current acquisition is full frame, the current mode can be switched to wavefront order to fully utilize the reference relationship between the uplink and downlink macroblocks to reduce the bit rate.

[0149] (Condition 5)cur_bitrate>(avg_bitrate_wavefront*130%)

[0150] That is, the real-time code rate of the grating sequence exceeds that of the wavefront sequence by 130%, which means that the code rate of the applied grating sequence is too high and should be switched to the wavefront sequence;

[0151] The above 403 module will detect a total of 5 conditions. If some of these conditions appear at the same time, the priority needs to be considered, and the condition with the highest priority will be the final decision criterion. The priority is as follows:

[0152] (Condition 1) > (Condition 4) > (Condition 2) > (Module 410 receives feedback from the decoding end) > (Condition 5) > (Condition 3)

[0153] That is, the overall priority is that the mode manually set by the user has the highest priority, followed by the actual bandwidth availability, then the frame rate, then the bit rate consideration of the raster sequence, and finally the acquisition method.

[0154] 404: Bandwidth detection module. In a specific environment (LAN, WAN), the network bandwidth changes dynamically. This module measures the available network bandwidth of the current environment in real time.

[0155] 405: Bitrate statistics module. Calculates the real-time bitrate of the current encoding. The unit is bps. In normal operation, the bitrate should not be greater than the available bandwidth, otherwise network transmission will have problems, resulting in frame loss and freeze.

[0156] 406: Acquisition mode detection module. Before encoding, an acquisition module is required to acquire computer images. The acquisition module itself works in two modes:

[0157] (1) Full-frame acquisition, each acquisition outputs a complete frame; in this case, the reference relationship between the upstream and downstream macroblocks can be fully utilized during encoding, and the wavefront sequence is applicable;

[0158] (2) Striped acquisition, one or several macroblock rows are acquired each time, that is, sliced ​​output. In this case, since the encoder can encode according to the strips, that is, there is no reference relationship between different strips, if the upper and lower rows of macroblocks span two different strips, there is no reference, that is, in the striped acquisition mode, the effect of using the wavefront sequence may not be good, and it is more appropriate to use the raster sequence that only refers to the macroblock on the left.

[0159] 407: Frame rate statistics module. The frame rate unit is frames per second (fps). The higher the frame rate, the smoother the picture, and the more frames the encoder can encode per second. This module counts the real-time frame rate.

[0160] 408: Wavefront order coding module, which uses wavefront order scanning of macroblocks for coding.

[0161] 409: Raster order encoding module, which uses raster order scanning macroblocks for encoding.

[0162] 410: Feedback receiving module. Receives mode exception information from the decoding end. For example, when working in wavefront mode, if the decoding end has too many multi-threaded tasks waiting, resulting in excessive decoding load (thread waiting itself consumes a lot of CPU) and abnormally slow encoding, an abnormal event will be sent to the encoding end. After receiving the event, the encoding end will notify the 403 module to switch to the raster sequence encoding mode, and notify the decoding end to switch modes through the 411 mode notification module, and maintain this mode for a period of time (such as 5 minutes).

[0163] 411: Mode notification module. After the specific mode (grating / wavefront) is determined in module 403, the decoding end needs to be notified for synchronization. Because the encoder does not use the upper macroblock reference information when using grating sequence encoding, and uses the upper macroblock reference information when using wavefront sequence encoding, the decoding end must use the same mode as the encoding end to correctly decode. When the encoding end switches modes, the mode to be switched is sent to the decoding end through this module.

[0164] The following is a description of each module on the decoding end. Figure 4B is a block diagram of a decoding end provided by an embodiment of the present disclosure. Figure 4B As shown in the figure, the functions of each module on the decoding end are as follows:

[0165] 412: Mode parsing module. Parse the mode information sent by module 411 of the encoding end and set it at the decoding end. When decoding, according to the specific mode, choose to use 415 wavefront sequence decoding module or 416 grating sequence decoding module for decoding.

[0166] 413: Wavefront monitoring and feedback module. The decoding end uses module 414 to count the real-time frame rate and the historical frame rate. Module 413 will periodically compare the real-time frame rate and the historical frame rate. If it is currently in the wavefront decoding mode, it will compare:

[0167] (1) Wavefront real-time frame rate;

[0168] (2) The average frame rate of the grating sequence obtained from historical statistics;

[0169] If the frame rate of (1) is less than 70% of (2), it means that the current parallel decoding in the wavefront order has seriously affected the frame rate and is too slow. Switching to the raster order may bring a better picture experience. At this time, it will immediately send an event to the encoder to notify the encoder to switch to the raster order. Correspondingly, the encoder receives the request in the 110 feedback receiving module, and immediately switches to the raster order after judging the decoder feedback through the mode judgment module 103, and notifies the decoding end through the 411 mode notification module. Then the codec uses the raster order for encoding and decoding at the same time.

[0170] 414: frame rate statistics module;

[0171] 415: wavefront decoding module;

[0172] 416: Raster sequence decoding module.

[0173] This solution proposes a method that can perform multi-standard decision analysis based on multiple conditions such as bit rate and frame rate, so as to adaptively select a more appropriate codec mode for the codec system. It can fully utilize the advantages of various modes without resetting the codec, monitor them and make corrections when they are obviously insufficient, dynamically switch between wavefront order and grating order, and provide users with a better application experience in real time.

[0174] For the image encoding method described in the above corresponding embodiments, the following is an embodiment of the device disclosed herein, which can be used to execute the embodiment of the method disclosed herein.

[0175] Figure 5 The structure of an image encoding device provided by the present disclosure is shown in FIG. Figure 1 .like Figure 5 As shown, the device 50 comprises:

[0176] The encoding mode acquisition module 501 is used to acquire the encoding mode of the first image, where the first image is at least one frame image before the current frame image;

[0177] The current frame image acquisition module 502 is used to acquire the current frame image;

[0178] The current transmission frame rate acquisition module 503 is used to acquire the current transmission frame rate and a first frame rate, wherein the first frame rate is an average frame rate when the second image is encoded and transmitted in a grating sequence encoding manner, and the second image is at least one frame image before the first image;

[0179] The current frame image encoding module 504 is used for, if the encoding method of the first image is the wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, encoding the current frame image in a grating sequence encoding method to generate an encoded current frame image;

[0180] The current frame image sending module 505 is used to send the encoded current frame image to the decoding end and send a first indication message to the decoding end so that the decoding end can decode the encoded current frame image. The first indication message is used to indicate to the decoding end that the encoding method of the current frame image is a raster order encoding method.

[0181] In one embodiment, Figure 6 As shown, the device 50 also includes:

[0182] The current frame image acquisition mode acquisition module 506 is used to acquire the acquisition mode of the current frame image;

[0183] The current frame encoding module 504 is used to, if the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold and the acquisition method of the current frame image is a striped acquisition method, encode the current frame image using a grating sequence encoding method to generate an encoded current frame image.

[0184] In one embodiment, the device 50 further comprises:

[0185] The current transmission bandwidth acquisition module 507 is used to acquire the current transmission bandwidth and the current transmission bit rate;

[0186] The current frame encoding module 504 is used for, if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, encoding the current frame image in a wavefront sequence encoding method to generate an encoded current frame image;

[0187] The encoded current frame image is sent to a decoding end and a second indication message is sent to the decoding end, wherein the second indication message is used to indicate to the decoding end that the encoding method of the current frame image is a wavefront sequence encoding method.

[0188] In one embodiment, the device 50 further comprises:

[0189] The current frame image acquisition mode acquisition module 506 is used to acquire the acquisition mode of the current frame image;

[0190] The current frame image encoding module 504 is used to, if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the acquisition method of the current frame image is a full-frame acquisition method, use a wavefront sequence encoding method to encode the current frame image to generate an encoded current frame image.

[0191] In one embodiment, the current frame image encoding module 504 is further used for:

[0192] If the image encoding instruction input by the user is not received and the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image, the image encoding instruction includes identification information of the image encoding method, and the image encoding methods include a wavefront sequence encoding method and a grating sequence encoding method;

[0193] If an image encoding instruction input by a user is received, the current frame image is encoded using an encoding method corresponding to the identification information.

[0194] In one embodiment, the current frame image encoding module 504 is further used for:

[0195] If no image encoding instruction input by the user is received and if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image.

[0196] In one embodiment, the device 50 further comprises:

[0197] The next frame image encoding module 508 is used to encode the next frame image by adopting a grating sequence encoding method if a third indication message sent by the decoding end is received. The third indication message is used to instruct the encoding end to encode the next frame image by adopting a grating sequence encoding method. The third indication message is sent by the decoding end to the encoding end after determining that the ratio of the second frame rate to the third frame rate is less than a second preset threshold. The second frame rate is the frame rate obtained by the decoding end when receiving the encoded image and decoding the encoded image by adopting a wavefront sequence decoding method. The third frame rate is the average value of the frame rates obtained when the decoding end receives the fourth image and decodes the fourth image by adopting a wavefront sequence decoding method. The fourth image is at least one frame of image received by the decoding end before the encoded image.

[0198] The image encoding device provided in the embodiment of the present disclosure, its implementation process and technical effects can be seen in the above Figure 1 The embodiment of FIG3 will not be described in detail here.

[0199] Figure 7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 7 As shown, the electronic device 70 includes:

[0200] A processor 701 and a memory 702 , wherein the memory 702 stores at least one computer instruction, and the instruction is loaded and executed by the processor 701 to implement the image encoding method described in the above method embodiment.

[0201] Based on the above Figure 1 In the image encoding method described in the embodiment corresponding to FIG. 3 , the embodiment of the present disclosure further provides a computer-readable storage medium. For example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The storage medium stores computer instructions for executing the above Figure 1 The image encoding method described in the embodiment corresponding to FIG. 3 will not be described in detail here.

[0202] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0203] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

Claims

1. An image coding method, characterized in that: include: Acquire a coding method of a first image, where the first image is at least one frame image before a current frame image; Acquiring the current frame image; Obtaining a current transmission frame rate and a first frame rate, wherein the first frame rate is an average of frame rates when a second image is encoded and transmitted using a raster order encoding method, and the second image is at least one frame of image before the first image; wherein the raster order encoding method refers to arranging a thread to process each group of macroblocks, and each thread encodes a row of macroblocks from left to right in sequence, and when the encoding of the row of macroblocks is completed, the nearest macroblock row that has not been encoded is automatically selected downward to continue encoding; If the encoding method of the first image is a wavefront-order encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold, the current frame image is encoded using a raster-order encoding method to generate an encoded current frame image; wherein the wavefront-order encoding method means that each thread sequentially takes macroblocks from the entire frame image for processing, and when the thread is idle, takes the macroblock closest to the most recently encoded macroblock and the macroblock above and to the left of which has completed encoding for continued encoding; The encoded current frame image is sent to a decoding end and a first indication message is sent to the decoding end so that the decoding end decodes the encoded current frame image. The first indication message is used to indicate to the decoding end that the encoding method of the current frame image is a raster order encoding method.

2. The method according to claim 1, characterized in that After acquiring the current frame image, the method further includes: Obtaining a collection method of the current frame image; If the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is greater than or equal to the first preset threshold and the acquisition method of the current frame image is a striped acquisition method, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image.

3. The method according to claim 1, characterized in that After acquiring the current image to be encoded, the method further includes: Get the current transmission bandwidth and current transmission bit rate; If the encoding method of the first image is a grating sequence encoding method, when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image; The encoded current frame image is sent to a decoding end and a second indication message is sent to the decoding end, wherein the second indication message is used to indicate to the decoding end that the encoding method of the current frame image is a wavefront sequence encoding method.

4. The method according to claim 3, characterized in that After acquiring the current frame image, the method further includes: Obtaining a collection method of the current frame image; If the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is greater than or equal to the current transmission bit rate and the acquisition method of the current frame image is a full-frame acquisition method, the wavefront sequence encoding method is used to encode the current frame image to generate an encoded current frame image.

5. The method according to claim 1, characterized in that The method further comprises: If the image encoding instruction input by the user is not received and the encoding method of the first image is a wavefront sequence encoding method, then when the ratio of the current transmission frame rate to the first frame rate is less than the first preset threshold, the current frame image is encoded using a grating sequence encoding method to generate an encoded current frame image, the image encoding instruction includes identification information of the image encoding method, and the image encoding methods include a wavefront sequence encoding method and a grating sequence encoding method; If an image encoding instruction input by a user is received, the current frame image is encoded using an encoding method corresponding to the identification information.

6. The method according to claim 5, characterized in that The method further comprises: If no image encoding instruction input by the user is received and if the encoding method of the first image is a grating sequence encoding method, then when the current bandwidth is less than the current transmission bit rate, the current frame image is encoded using a wavefront sequence encoding method to generate an encoded current frame image.

7. The method according to claim 4, characterized in that After sending the encoded current frame image to the decoding end and sending a second indication message to the decoding end, the method further includes: If a third indication message sent by the decoding end is received, the next frame image is encoded using a grating sequence encoding method. The third indication message is used to instruct the encoding end to encode the next frame image using a grating sequence encoding method. The third indication message is sent by the decoding end to the encoding end after determining that the ratio of the second frame rate to the third frame rate is less than a second preset threshold. The second frame rate is the frame rate obtained by the decoding end when receiving the encoded image and decoding the encoded image using a wavefront sequence decoding method. The third frame rate is the average value of the frame rates obtained when the decoding end receives the fourth image and decodes the fourth image using a wavefront sequence decoding method. The fourth image is at least one frame image received by the decoding end before the encoded image.

8. An image encoding device, characterized in that: include: An encoding mode acquisition module, used to acquire an encoding mode of a first image, where the first image is at least one frame image before a current frame image; A current frame image acquisition module, used for acquiring the current frame image; A current transmission frame rate acquisition module is used to acquire the current transmission frame rate and a first frame rate, wherein the first frame rate is an average frame rate when a second image is encoded and transmitted using a raster order encoding method, and the second image is at least one frame image before the first image; wherein the raster order encoding method means that each group of macroblocks is arranged to be processed by a thread, and each thread encodes a row of macroblocks from left to right in sequence, and when the encoding of the row of macroblocks is completed, the nearest macroblock row that has not been encoded is automatically selected downward to continue encoding; A current frame image encoding module, for encoding the current frame image in a raster order encoding mode to generate an encoded current frame image if the encoding mode of the first image is a wavefront order encoding mode, when the ratio of the current transmission frame rate to the first frame rate is less than a first preset threshold value; wherein the wavefront order encoding mode means that each thread sequentially takes macroblocks from the entire frame image for processing, and when the thread is idle, takes the macroblock closest to the most recently encoded macroblock and the macroblock above and to the left of which has completed encoding for continued encoding; A current frame image sending module is used to send the encoded current frame image to a decoding end and send a first indication message to the decoding end so that the decoding end can decode the encoded current frame image. The first indication message is used to indicate to the decoding end that the encoding method of the current frame image is a raster order encoding method.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores at least one computer instruction, and the instruction is loaded and executed by the processor to implement the steps performed in the image encoding method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one computer instruction, which is loaded and executed by a processor to implement the steps performed in the image encoding method according to any one of claims 1 to 7.

Images