Image coding method and device, electronic equipment and storage medium

Abstract

The invention provides an image coding method, and relates to the technical field of image processing, in particular to the technical field of video compression and image coding and decoding. According to the specific implementation scheme, for a down-sampling image sequence of an original image sequence, the correlation between each down-sampling image in the down-sampling image sequence and a reference image of the down-sampling images is determined according to an inter-frame motion estimation result of the down-sampling image sequence; determining a candidate reference frame interval of the down-sampling image according to the correlation; determining time domain motion information of a coding block in the down-sampling image according to the candidate reference frame interval; and according to the time domain motion information and the space domain motion information of the coding block in the down-sampling image, coding the original image block corresponding to the coding block. The invention further provides an image coding device, electronic equipment and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and more particularly to the fields of video compression and image encoding and decoding technologies. More specifically, the present disclosure provides an image encoding method, apparatus, electronic device, storage medium, and computer program product. Background Art

[0002] High Efficiency Video Coding (HEVC) is a new video coding standard. In HEVC, inter-frame coding utilizes the characteristic that the content between adjacent frames in the time domain is similar. By setting a search start point and a search range, the unit block that is most similar to the coding block of the current frame is found in the adjacent frames as the prediction result. The selection of the search start point will affect the search time consumption and matching accuracy of block matching. Summary of the Invention

[0003] The present disclosure provides an image encoding method, apparatus, electronic device, storage medium, and computer program product.

[0004] According to a first aspect, there is provided an image encoding method, the method comprising: for a downsampled image sequence of an original image sequence, determining the correlation between each downsampled image in the downsampled image sequence and a reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence; determining a candidate reference frame interval of the downsampled image according to the correlation; determining the temporal motion information of the coding block in the downsampled image according to the candidate reference frame interval; and encoding an original image block corresponding to the coding block according to the temporal motion information and the spatial motion information of the coding block in the downsampled image.

[0005] According to a second aspect, there is provided an image encoding apparatus, the apparatus comprising: a correlation determination module for determining the correlation between each downsampled image in the downsampled image sequence and a reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence; an interval determination module for determining a candidate reference frame interval of the downsampled image according to the correlation; a motion information determination module for determining the temporal motion information of the coding block in the downsampled image according to the candidate reference frame interval; and an encoding module for encoding an original image block corresponding to the coding block according to the temporal motion information and the spatial motion information of the coding block in the downsampled image.

[0006] According to a third aspect, there is provided an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by the present disclosure.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, and the computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0008] According to a fifth aspect, a computer program product is provided, including a computer program, where the computer program is stored on at least one of a readable storage medium and an electronic device, and the computer program, when executed by a processor, implements the method provided according to the present disclosure.

[0009] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0011] Figure 1 is a schematic diagram of a process for determining a search starting point that can apply an image coding method and apparatus according to an embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of an image coding method according to an embodiment of the present disclosure;

[0013] Figure 3A is a schematic diagram of determining a candidate reference frame interval according to an embodiment of the present disclosure;

[0014] Figure 3B is a schematic diagram of determining a candidate reference frame interval according to another embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram of determining temporal motion information of a coding block according to a candidate reference interval according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of scaling of target temporal motion information according to an embodiment of the present disclosure;

[0017] Figure 6 is a flowchart of an image coding method according to another embodiment of the present disclosure;

[0018] Figure 7 is a block diagram of an image coding apparatus according to an embodiment of the present disclosure; and

[0019] Figure 8 is a block diagram of an electronic device for an image coding method according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0021] In the technical solution of the present disclosure, the processing of collecting, storing, using, processing, transmitting, providing, and disclosing the user's personal information complies with the provisions of relevant laws and regulations and does not violate public order and good customs.

[0022] In the technical solution of the present disclosure, before obtaining or collecting the user's personal information, the user's authorization or consent is obtained.

[0023] The HEVC standard further improves the coding efficiency by adopting a variety of new compression tools (including complex coding block structures, new spatial prediction directions and intra-frame coding, complex interpolation filters, new in-loop filters, and new entropy coding schemes).

[0024] In HEVC, the process of inter-frame coding takes advantage of the similarity of the content between the current frame and the corresponding encoded reference frame in the time domain. Through the motion estimation process, the unit block in the reference frame that is most similar to the content of the coding block of the current frame is found as the prediction result. Since inter-frame coding can make the most of the similarity of video content for prediction, its prediction effect is more accurate than intra-frame coding. Moreover, inter-frame coding only needs to encode and transmit the motion vector and the prediction residual, effectively improving the compression effect of HEVC.

[0025] When performing a matching operation on the coding block of the current frame (hereinafter simply referred to as the current block), the range of motion estimation can be controlled by setting the search distance (SearchRange). When the search distance value is set larger, a more accurate motion estimation result can be obtained, improving the coding performance, but it will also cause the problem of too slow coding speed. Therefore, in the encoder, the search distance value is usually set to a value that can balance the coding speed and performance.

[0026] When performing a matching operation on the current block, another important parameter is the search start point. Based on the search start point and the search range, the unit block most similar to the current block can be found from the reference frame. If the search start point is set well, even if the searchRange is set very small, the best point can be quickly searched, and not only is the search result accurate but also the search time is very small. On the contrary, if the search start point is not selected well, not only will the search time increase, but also the search result will become inaccurate.

[0027] In a video encoding scenario, it can be divided into a lookahead process (pre-analysis process) and an encoding process. Frame type decision is an important part of the encoder's encoding. In the encoder, frame type decision can be performed through the lookahead process. First, each input image frame enters the lookahead process. When determining the relevant parameters of a frame through the lookahead process, it is stored in the encoding frame list and waits for encoding. When performing frame type decision, determining whether a scene change occurs, bitrate control, etc. in the lookahead process, the current image frame can be downsampled according to requirements, and then the intra prediction loss (intracost) and inter prediction loss (intercost) of each downsampled image block are calculated respectively. Frame type decision, determining whether a scene change occurs, bitrate control, etc. are performed based on the losses.

[0028] In the encoding process, the determination of the search start point mainly refers to the mv (motion vector) of the spatial neighboring blocks and the mv of the temporal neighboring blocks of the current block. Spatial neighboring blocks refer to the unit blocks located around the current block (such as the left side and the upper side), and temporal neighboring blocks refer to the unit blocks corresponding to the current block in the reference frame of the current frame. In actual application scenarios, there may be a situation where a unit block has a long-distance displacement between two frames. At this time, the mv of the spatial neighboring blocks and the temporal neighboring blocks may not be able to correctly reflect the long-distance displacement, and increasing the search distance will cause a large search time consumption.

[0029] In view of this, the encoding process also refers to the mv of the corresponding block of the current block in the lookahead process to determine the search start point. The lookahead process uses a downsampled picture, and the corresponding block of the current block in the lookahead process refers to the block in the downsampled image corresponding to the current block. Since the lookahead process uses a downsampled picture, relatively speaking, it can search farther.

[0030] Currently, in the lookahead process, the search start point of each block is determined only by using the spatial mv. In a complex motion scenario, this determination method is likely to cause the deviation of the search start point and increase the number of search steps.

[0031] To adapt to more long-distance searches, the present disclosure proposes an image encoding method. By adding the temporal mv to the determination of the search start point in the lookahead process, a farther mv can be searched in the lookahead process, thus playing a more reference role in the selection of the search start point in the encoding process and improving the block matching efficiency and block matching accuracy in the encoding process.

[0032] Figure 1 It is a schematic diagram of a process for determining a search start point to which an image encoding method and apparatus according to an embodiment of the present disclosure can be applied. It should be noted that Figure 1 What is shown is only an example of the process for determining a search start point to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0033] As Figure 1 shown, this embodiment includes an encoding process 110 and a pre-analysis process 120, and the pre-analysis process 120 is a lookahead process. In the encoding process 110, for an encoding block 111 of an original image to be encoded, the determination of a search start point 130 depends on the temporal motion information 112 and the spatial motion information 113 of the encoding block 111. The temporal motion information 112 may be the mv of the temporal neighboring block of the encoding block 111, and the spatial motion information 113 may be the mv of the spatial neighboring block of the encoding block 111. The determination of the search start point 130 in the encoding process 110 also depends on the pre-analysis motion information 114 provided by the pre-analysis process 120.

[0034] In the pre-analysis process 120, an encoding block 121 of a downsampled image is an encoding block corresponding to the encoding block 111 of the original image. The pre-analysis search start point of the encoding block 121 of the downsampled image depends on the temporal motion information 122 and the spatial motion information 123. The spatial motion information 123 may be the mv of the spatial neighboring block of the encoding block 121. The temporal motion information 122 may be determined according to the correlation between the downsampled image and the reference image of the downsampled image. For example, a candidate reference frame interval of the encoding block 121 may be determined according to the correlation, and the temporal motion information 122 may be determined according to the candidate reference frame interval. The temporal motion information 122 can reflect a farther distance displacement compared to the spatial motion information 123.

[0035] After the pre-analysis process 120 determines the temporal motion information 122 and the spatial motion information 123 of the encoding block 121, it may determine the pre-analysis motion information 114 according to the temporal motion information 122 and the spatial motion information 123, and then may provide the pre-analysis motion information 114 to the encoding process 110.

[0036] The encoding process 110 may determine the search start point 130 based on the temporal motion information 112, the spatial motion information 113, and the pre-analysis motion information 114 provided by the pre-analysis process 120.

[0037] Embodiments of the present disclosure add temporal motion information 122 to the pre-analysis process, jointly making decisions on the pre-analysis motion information 114 with the spatial motion information 123. Since the temporal motion information 122 can reflect long-distance displacements, introducing the temporal motion information 122 to determine the search starting point 130 can improve the block matching efficiency and block matching accuracy of the coding block 111, thereby improving the coding efficiency and coding quality.

[0038] Figure 2 is a flowchart of an image coding method according to an embodiment of the present disclosure.

[0039] As Figure 2 shown, the image coding method 200 includes operations S210 to S240.

[0040] In operation S210, for the downsampled image sequence of the original image sequence, according to the inter-frame motion estimation result of the downsampled image sequence, the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image is determined.

[0041] The original image sequence may refer to the image sequence in the coding process, and the downsampled image sequence may refer to the image sequence obtained by downsampling the original image sequence in the lookahead process. The reference image of the downsampled image may refer to the already encoded downsampled image adjacent to the downsampled image in the downsampled image sequence. Here, adjacent may mean adjacent in the frame coding order.

[0042] In the lookahead process, by performing inter-frame motion estimation on the downsampled image sequence, the intra-frame prediction loss (intracost) and inter-frame prediction loss (intercost) of the coding blocks in each downsampled image can be determined. According to the intra-frame prediction loss or the inter-frame prediction loss, it can be determined whether the coding block is an intra-coded block or an inter-coded block. For example, if the intra-frame prediction loss is greater than the inter-frame prediction loss, it can be determined that the coding block is an inter-coded block.

[0043] The proportion of inter-coded blocks in the downsampled image can be counted. If the proportion of inter-coded blocks is greater than a threshold (e.g., 50%), it indicates that most blocks of the downsampled image are encoded based on the reference image, and it can be determined that the correlation between the downsampled image and the reference image is relatively large. On the contrary, if the proportion of inter-coded blocks is not greater than the threshold, it indicates that most blocks of the downsampled image are encoded with reference to the blocks within the downsampled image frame, and it can be determined that the correlation between the downsampled image and the reference image is relatively small.

[0044] In operation S220, according to the correlation, the candidate reference frame interval of the downsampled image is determined.

[0045] A candidate reference frame interval refers to the frame interval of downsampled images that can be referred to when determining the temporal motion information of the coded blocks in the downsampled images. The correlation between the first and last frames of this frame interval in the downsampled images is relatively large. For example, for each downsampled image, if the correlation between the downsampled image and the corresponding reference image is relatively large, the frame interval corresponding to the downsampled image and the reference image can be determined as the candidate reference frame interval for each frame in this frame interval.

[0046] For example, the downsampled image is the 8th frame image in the downsampled image sequence, and the reference image of the downsampled image is the 0th frame image in the downsampled image sequence. By calculating the correlation between the 0th frame image and the 8th frame image, it can be determined whether the frame interval (0, 8) from the 0th frame to the 8th frame can be used as the candidate reference frame interval for each frame from the 0th frame to the 8th frame.

[0047] In operation S230, determine the temporal motion information of the coded blocks in the downsampled image according to the candidate reference frame interval.

[0048] For the candidate reference frame interval, the temporal motion information (temporal mv) of the candidate reference frame interval can be determined according to the motion estimation result between the first and last frames in the candidate reference frame interval, and used as the reference temporal motion information of the coded blocks in the downsampled image. Each downsampled image can have multiple candidate reference frame intervals. Correspondingly, there can be multiple reference temporal motion information of the coded blocks. The temporal motion information of the coded blocks can be determined according to multiple reference temporal motion information.

[0049] For example, the candidate reference frame intervals of the 8th frame include (0, 8), (0, 16), (8, 16), etc. The reference temporal motion information of the coded blocks can include mv(0, 8), mv(0, 16), mv(8, 16), etc. The temporal motion information of the coded blocks can be determined according to mv(0, 8), mv(0, 16), mv(8, 16).

[0050] In operation S240, encode the original image block corresponding to the coded block according to the temporal motion information and spatial motion information of the coded block in the downsampled image.

[0051] The spatial motion information of the coded block can refer to the mv of the spatial neighboring blocks of the coded block. According to the temporal motion information (temporal mv) and spatial motion information (spatial mv) of the coded block, the pre-analysis search starting point of the coded block in the reference image can be jointly determined.

[0052] For example, the mv with the lowest cost can be determined from the temporal mv and the spatial mv, and the mv with the lowest cost can be passed into the motion estimation function to obtain the pre-analysis search starting point. Then, a block matching the coding block is searched in the reference image based on the pre-analysis search starting point, and the pre-analysis motion information (pre-analysis mv) can be determined based on the motion vector between the coding block and the matching block. Then, the lookahead process can pass the pre-analysis mv to the encoding process.

[0053] The original image block in the encoding process is the block corresponding to the encoding block in the downsampled image. The encoding process can determine the search starting point of the original image in the reference image of the original image based on the pre-analysis mv provided by the pre-analysis process, combined with the spatial domain mv and temporal domain mv of the original image block, and then match the matching block of the original image block. According to the matching block of the original image block, the original image block can be inter-frame encoded.

[0054] According to the embodiments of the present disclosure, by optimizing the motion estimation process in the lookahead process, time domain motion information is introduced, and the motion information of the coding block is jointly determined together with the spatial domain motion information. This can adapt to complex scenes with intense motion, and while ensuring that the coding complexity remains basically unchanged, longer distance motion information can be obtained, and farther can be searched during block matching, thereby improving the coding speed and coding quality.

[0055] In an embodiment of the present disclosure, for a downsampled image sequence of an original image sequence, determining the correlation between each downsampled image in the downsampled image sequence and a reference image of the downsampled image based on an inter-frame motion estimation result of the downsampled image sequence includes: for each downsampled image, determining the ratio of inter-frame coding blocks between the downsampled image and the reference image based on a motion estimation result between the downsampled image and the reference image; and determining the correlation between the downsampled image and the reference image based on the inter-frame coding block ratio.

[0056] According to the motion estimation result between the downsampled image and the reference image, it can be determined whether each coding block in the downsampled image is an inter-frame coding block or an intra-frame coding block. The inter-frame coding block ratio is the ratio of the number of inter-frame coding blocks in the downsampled image to the total number of coding blocks. If the inter-frame coding block ratio is large, it means that most of the coding blocks in the downsampled image rely on the reference image for coding, and it can be considered that the correlation between the downsampled image and the corresponding reference image is large.

[0057] For example, the downsampled image is the second frame image, and the reference image is the 0th frame image. According to the motion estimation result between the second frame image and the 0th frame image, it is determined that the proportion of inter-frame coding blocks in the second frame image is 75%, and it can be determined that the second frame image and the 0th frame image have a large correlation.

[0058] In an embodiment of the present disclosure, for the downsampled image sequence of the original image sequence, determining the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence includes: determining the coding order of the downsampled images in the downsampled image sequence according to the inter-frame motion estimation result of the downsampled image sequence; and sequentially determining the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image according to the coding order.

[0059] According to the inter-frame motion estimation result of the downsampled image sequence, the frame type of each downsampled image in the downsampled image can be determined. In video coding, the common frame types are mainly the following three: I-frame (Intra-coded frame): Also known as a key frame, it is an independently coded frame that does not depend on other frames for coding. P-frame (Predictive-coded frame): It depends on the previous I-frame or P-frame for coding. B-frame (Bidirectionally predictive-coded frame): It depends on the frames before and after for coding.

[0060] The coding order can be determined according to the frame type. For example, the I-frame can be coded first, and then the P-frame and B-frame can be coded based on the I-frame. In one example, the downsampled image sequence includes frames from frame 0 to frame 16. The frame coding order can be to first code frame 0, then frame 16 is coded with reference to frame 0; next, frame 8 is coded with reference to frame 0 and frame 16; frame 4 is coded with reference to frame 0 and frame 8, and so on.

[0061] The correlation between each downsampled image and the reference image of the downsampled image can be sequentially determined according to the above coding order. Then, the candidate reference frame interval of each downsampled image can be determined according to the correlation.

[0062] The following combines Figure 3A - Figure 3B to illustrate the candidate reference frame interval of the present disclosure.

[0063] Figure 3A is a schematic diagram of determining the candidate reference frame interval according to an embodiment of the present disclosure.

[0064] In an embodiment of the present disclosure, determining the candidate reference frame interval of the downsampled image according to the correlation includes: for each downsampled image, in response to the correlation between the downsampled image and the reference image of the downsampled image meeting the correlation condition, determining the frame interval between the downsampled image and the reference image as the candidate reference frame interval of each downsampled image in the frame interval.

[0065] The correlation between the downsampled image and the reference image of the downsampled image meets the correlation condition, which may mean that the proportion of inter-coded blocks in the downsampled image is greater than a certain threshold (e.g., 50%), indicating that most of the coded blocks in the downsampled image rely on the reference image for coding. For each image located between the downsampled image and the reference image, it can be considered that the image is in a frame interval with a relatively large correlation. Therefore, the frame interval of the downsampled image and the reference image of the downsampled image can be used as the candidate reference frame interval for each downsampled image in the frame interval.

[0066] For example, as Figure 3A shown, the downsampled image sequence includes frames from frame 0 to frame 16. Frame 0 is the reference image of frame 16. By calculating the correlation between frame 0 and frame 16, when the correlation meets the correlation condition, the frame interval (16, 0) can be determined as the candidate reference frame interval for each frame from frame 0 to frame 16.

[0067] Figure 3B is a schematic diagram for determining the candidate reference frame interval according to another embodiment of the present disclosure.

[0068] In an embodiment of the present disclosure, each downsampled image includes at least one reference image, and the at least one reference image may include at least one of a forward reference image and a backward reference image. Among them, the forward reference image refers to an encoded image whose playback order is before the downsampled image, and the backward reference image refers to an encoded image whose playback order is after the downsampled image.

[0069] For example, as Figure 3B shown, for the 8th frame image, the corresponding forward reference image is frame 0, and the backward reference image is frame 16.

[0070] For each downsampled image, in response to the correlation between the downsampled image and the reference image of the downsampled image meeting the correlation condition, determining the frame interval of the downsampled image and the reference image as the candidate reference frame interval for each downsampled image in the frame interval includes at least one of the following operations: For each downsampled image, in response to the correlation between the downsampled image and the forward reference image meeting the correlation condition, determining the forward frame interval of the downsampled image and the forward reference image as the candidate reference frame interval for each downsampled image in the forward frame interval; in response to the correlation between the downsampled image and the backward reference image meeting the correlation condition, determining the backward frame interval of the downsampled image and the backward reference image as the candidate reference frame interval for each downsampled image in the backward frame interval.

[0071] For example, as Figure 3BAs shown, by calculating the correlation between the 0th frame and the 8th frame, when the correlation meets the correlation condition, the frame interval (8, 0) can be determined as the candidate reference frame interval for each frame from the 0th frame to the 8th frame.

[0072] By calculating the correlation between the 8th frame and the 16th frame, when the correlation meets the correlation condition, the frame interval (8, 16) can be determined as the candidate reference frame interval for each frame from the 8th frame to the 16th frame.

[0073] In the embodiments of the present disclosure, by determining the frame interval composed of the downsampled image with the qualified correlation and the reference image of the downsampled image as the candidate reference frame interval for each frame in this frame interval, one or more candidate reference frame intervals can be obtained for each downsampled image. Based on the candidate reference frame interval, the temporal motion information of the coding block in the downsampled image can be determined, which can improve the accuracy of the temporal motion information.

[0074] In the embodiments of the present disclosure, determining the temporal motion information of the coding block in the downsampled image according to the candidate reference frame interval includes: determining the temporal motion information of the candidate reference frame interval according to the motion estimation results of the first and last frames of the candidate reference frame interval as the reference temporal motion information; determining the temporal motion information of the coding block in the downsampled image according to the distance between the reference temporal motion information and the spatial motion information of the coding block in the downsampled image.

[0075] According to the motion estimation results between the first and last frames in the candidate reference frame interval, the temporal mv of the candidate reference frame interval can be determined. The temporal mv of the candidate reference frame interval can be used as the reference temporal motion information of the coding block. The spatial motion information can refer to the mv of the spatial neighboring blocks of the coding block.

[0076] There are multiple candidate reference frame intervals, and the reference temporal motion information includes multiple reference temporal motion information respectively corresponding to the multiple candidate reference frame intervals. The target temporal motion information can be determined from the multiple reference temporal motion information according to the distances between the spatial motion information and the multiple reference temporal motion information respectively, and the temporal motion information of the coding block in the downsampled image can be determined according to the target temporal motion information.

[0077] For example, the distance between each reference temporal mv and the spatial mv can be calculated, and the reference temporal motion information with a distance greater than the threshold can be selected from the multiple reference temporal mv to determine the temporal motion information of the coding block.

[0078] In the embodiments of the present disclosure, by selecting the reference temporal motion information with a distance greater than a threshold from the distance between the reference temporal motion information and the spatial motion information to determine the temporal motion information of the coding block, it can adapt to scenarios where long-distance MVs may actually occur, select the temporal MVs with a relatively large distance from the spatial MVs, so that long-distance MVs can be searched in complex scenarios, and the long-distance MVs are provided to the coding process, which can improve the coding efficiency and coding quality of the coding process.

[0079] The following will be combined with Figure 4 to illustrate the determination of the temporal motion information of the coding block according to the candidate reference interval of the present disclosure.

[0080] Figure 4 is a schematic diagram of determining the temporal motion information of the coding block according to the candidate reference interval according to an embodiment of the present disclosure.

[0081] According to the embodiments of the present disclosure, candidate temporal motion information meeting the distance condition is determined from multiple reference temporal motion information; and the first temporal motion information with the largest distance from the spatial motion information and the second motion information with the smallest distance from the spatial motion information are determined from the candidate temporal motion information as the target temporal motion information.

[0082] As Figure 4 shown, the downsampled image is, for example, the 4th frame, and the candidate reference frame intervals of the downsampled image include (16,0), (8,0), (4,0), (4,8), etc. The temporal MVs of each candidate reference frame interval can be calculated as the reference temporal motion information of the coding block in the downsampled image, that is, the reference temporal MVs. For example, the reference temporal MVs may include mv(16,0), mv(8,0), mv(4,0), mv(4,8), etc.

[0083] The spatial MV of the coding block in the downsampled image may refer to the MV of the spatial neighboring block of the coding block. The distances between the spatial MV and each reference temporal MV can be calculated. This distance is, for example, the Hamming distance. Then, the reference temporal MVs with a distance greater than the threshold from the spatial MV are selected as candidate temporal MVs. For example, the candidate temporal MVs may include mv(16,0), mv(8,0), and mv(4,8). Next, the candidate temporal MVs can be sorted in descending order of distance to obtain the sorting result mv(16,0), mv(4,8), mv(8,0). Finally, the first candidate temporal MV and the second candidate temporal MV corresponding to the maximum and minimum distances respectively can be selected from the sorting result as the target temporal motion information. For example, the candidate temporal MV corresponding to the maximum distance is mv(16,0), and the second candidate temporal MV corresponding to the minimum distance is mv(8,0). Both mv(16,0) and mv(8,0) are the target motion information.

[0084] In an embodiment of the present disclosure, candidate time-domain mvs corresponding to the maximum and minimum distances can cover the largest mv range. Taking the candidate time-domain mvs corresponding to the maximum and minimum distances as the target time-domain mvs and determining the time-domain motion information of the coding block according to the target time-domain mvs can search for long-distance mvs in lookahead, improving the matching efficiency and accuracy of the coding block.

[0085] According to an embodiment of the present disclosure, determining the time-domain motion information of the coding block in the downsampled image according to the target time-domain motion information includes: scaling the target time-domain motion information according to the distance and direction between the downsampled image and the reference image, and the distance and direction between the first and last frames of the candidate reference frame interval corresponding to the target time-domain motion information, to obtain the time-domain motion information of the coding block.

[0086] The target time-domain motion information is the candidate time-domain mvs corresponding to the maximum and minimum distances, and the candidate reference frame interval corresponding to the target time-domain motion information is the candidate reference frame interval corresponding to the maximum and minimum distances.

[0087] For example, the downsampled image is the 4th frame, and the reference images are the 0th frame and the 8th frame. The target time-domain motion information is the candidate time-domain mvs with the maximum and minimum spatial mv distances, such as mv(16,0) and mv(8,0) respectively. The candidate reference frame intervals corresponding to the target time-domain motion information are the frame intervals (16,0) and (8,0).

[0088] When the 4th frame refers to the 0th frame, the time-domain motion information can refer to the target time-domain motion information mv(8,0), that is, the time-domain motion information of the 8th frame relative to the 0th frame. The time-domain motion information of the 4th frame relative to the 0th frame can be obtained by scaling the target time-domain motion information mv(8,0) according to the distance and direction between the 4th frame and the 0th frame, and the distance and direction between the 8th frame and the 0th frame.

[0089] When the 4th frame refers to the 8th frame, the time-domain motion information can also refer to the target time-domain motion information mv(8,0), that is, the time-domain motion information of the 8th frame relative to the 0th frame. The time-domain motion information of the 4th frame relative to the 8th frame can be obtained by scaling the target time-domain motion information mv(8,0) according to the distance and direction between the 4th frame and the 8th frame, and the distance and direction between the 8th frame and the 0th frame.

[0090] In an embodiment of the present disclosure, the target time-domain motion information can be scaled proportionally according to the distance ratio, and then the direction of the scaled time-domain motion information can be adjusted according to the direction consistency to obtain the time-domain motion information of the coding block.

[0091] The following combines Figure 5Further description is provided for the scaling of the time-domain motion information of the present disclosure.

[0092] Figure 5 It is a schematic diagram of the scaling of the target time-domain motion information according to an embodiment of the present disclosure.

[0093] As Figure 5 shown, the downsampled image is the 4th frame image, and the corresponding reference images include the forward reference image of the 0th frame and the backward reference image of the 8th frame. Among them, the candidate reference frame interval corresponding to the target time-domain motion information is (8, 0). The distance between the first and last frames in the candidate reference frame interval (8, 0) is 8, and the direction is forward (the 8th frame references the 0th frame forward), and the target time-domain motion information is mv0. The distance between the 4th frame and the forward reference image of the 0th frame is 4, and the direction is forward. The distance between the 4th frame and the backward reference image of the 8th frame is 4, and the direction is backward.

[0094] For example, when the 4th frame references the 0th frame, according to the distance ratio, the target time-domain motion information mv0 can be scaled to obtain 0.5mv0. Since the directions of the 4th frame referencing the 0th frame and the 8th frame referencing the 0th frame are both forward, therefore, there is no need to adjust the direction of the scaled motion information, that is, the time-domain motion information of the 4th frame referencing the 0th frame is 0.5mv0.

[0095] When the 4th frame references the 8th frame, according to the distance ratio, the target time-domain motion information mv0 can be scaled to obtain 0.5mv0. Since the direction of the 4th frame referencing the 8th frame is backward and the direction of the 8th frame referencing the 0th frame is forward, and the directions are inconsistent, therefore, it is necessary to adjust the direction of the scaled motion information, that is, the time-domain motion information of the 4th frame referencing the 8th frame is -0.5mv0.

[0096] According to an embodiment of the present disclosure, the target time-domain motion information is determined based on the time-domain motion information of the candidate reference frame interval. By scaling the target time-domain motion information, the scaled time-domain motion information can be within the range of the downsampled image, which can improve the accuracy of the time-domain motion information of the coding block in the downsampled image.

[0097] According to an embodiment of the present disclosure, encoding the original image block corresponding to the coding block according to the time-domain motion information and the spatial-domain motion information of the coding block in the downsampled image includes: determining a pre-analysis search start point according to the time-domain motion information and the spatial-domain motion information of the coding block in the downsampled image; determining pre-analysis motion information according to the pre-analysis search start point; and encoding the original image block corresponding to the coding block according to the pre-analysis motion information.

[0098] After obtaining the temporal motion information of the coding block in the downsampled image, the pre-analysis search starting point can be determined in the lookahead process based on the temporal motion information and the spatial motion information. Since the temporal motion information of the coding block is determined by the target motion information, and the target motion information includes the candidate temporal mv corresponding to the maximum and minimum distances, the temporal motion information of the coding block also includes the temporal motion information corresponding to the maximum and minimum distances, which can be called the maximum temporal mv and the minimum temporal mv.

[0099] The coding costs of the maximum time domain mv, the minimum time domain mv, and the spatial domain mv can be calculated respectively to determine the mv with the minimum coding cost, and then the mv with the minimum cost is passed to the motion estimation function to obtain the pre-analysis search starting point.

[0100] After obtaining the pre-analysis search starting point, the block matching the coded block can be searched in the reference image of the downsampled image according to the pre-analysis search starting point, and then the MV between the coded block and the matching block is used as the pre-analysis MV. Subsequently, the lookahead process provides the pre-analysis MV to the encoding process.

[0101] Next, the encoding process determines the search starting point of the original image block in the reference image of the original image based on the pre-analysis MV, the time domain MV and the spatial domain MV of the original coding block, searches for the matching block corresponding to the original image block based on the search starting point, and encodes the original image block based on the matching block of the original image block.

[0102] According to the embodiments of the present disclosure, by adding time domain MV to the lookahead process, the pre-analysis MV is jointly determined by the time domain MV and the spatial domain MV, so that the lookahead process can search for long-distance motion information and pass the long-distance motion information to the encoding process. In complex scenes with intense motion, it is possible to improve the encoding efficiency and encoding quality while ensuring that the encoding complexity remains unchanged or reduced.

[0103] Figure 6 is a flowchart of an image encoding method according to another embodiment of the present disclosure.

[0104] like Figure 6 As shown, the image encoding method of this embodiment includes operations S601 to S609.

[0105] In operation S601, it is determined whether the current down-sampled image has a corresponding candidate reference frame interval. If yes, proceed to operation S602; if no, jump to operation S609.

[0106] In operation S602, it is determined whether the current coding block has spatial motion information. If yes, the process proceeds to operation S603; if no, the process jumps to operation S609.

[0107] In operation S603, traverse multiple candidate reference frame intervals of the downsampled image to determine the temporal mv of each candidate reference frame interval. Among them, according to the motion estimation between the first and last frames of each candidate reference frame interval, the temporal mv of the candidate reference frame interval can be determined, and the temporal mv of the candidate reference frame interval can be used as the reference temporal mv of the coding block in the downsampled image.

[0108] In operation S604, calculate the Hamming distance between the temporal mv of the candidate reference frame interval and the spatial mv of the coding block.

[0109] In operation S605, determine whether the Hamming distance is greater than the threshold. If so, execute operation S606; otherwise, execute operation S609.

[0110] In operation S606, add the temporal mv of the candidate reference frame interval that meets the distance condition to the candidate temporal motion information list.

[0111] The distance condition can refer to whether the distance is greater than the threshold. If the Hamming distance between the temporal mv of the candidate reference frame interval and the spatial mv of the coding block is greater than the threshold, the temporal mv of this candidate reference frame interval can be used as the candidate temporal mv of the downsampled image and added to the candidate temporal motion information list.

[0112] In operation S607, sort the candidate temporal motion information in the list based on the Hamming distance. For example, sort it in descending or ascending order based on the Hamming distance value.

[0113] In operation S608, select the maximum temporal motion information and the minimum temporal motion information after sorting and add them to the motion estimation list.

[0114] The maximum temporal motion information and the minimum temporal motion information refer to the candidate temporal mv corresponding to the maximum Hamming distance and the minimum Hamming distance. The maximum temporal motion information and the minimum temporal motion information can be scaled proportionally and then added to the motion estimation list. The motion estimation list can also include the spatial mv of the downsampled image.

[0115] Regarding the mv in the motion estimation list, it can be used to determine the determination of the pre-analysis search start point in the lookahead process.

[0116] In operation S609, do not set the temporal motion information of the coding block.

[0117] This operation can mean not introducing temporal mv in the lookahead process, and the block matching in the lookahead process determines the pre-analysis search start point only according to the spatial mv of the coding block.

[0118] Embodiments of the present disclosure can search for long-distance motion information in the lookahead process, improve the matching efficiency and matching accuracy of coding blocks by screening candidate temporal MVs whose distance from the spatial MV is greater than a threshold according to the temporal MV of the candidate reference interval, and then selecting the candidate temporal MVs corresponding to the maximum and minimum distances to determine the temporal motion information of the coding blocks in the downsampled image.

[0119] According to an embodiment of the present disclosure, the present disclosure also provides an image coding apparatus.

[0120] Figure 7 It is a block diagram of an image coding apparatus according to an embodiment of the present disclosure.

[0121] As Figure 7 shown, the image coding apparatus 700 includes a correlation determination module 710, an interval determination module 720, a motion information determination module 730, and a coding module 740.

[0122] The correlation determination module 710 is configured to determine the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image sequence according to the inter-frame motion estimation result of the downsampled image sequence for the original image sequence.

[0123] The interval determination module 720 is configured to determine the candidate reference frame interval of the downsampled image according to the correlation.

[0124] The motion information determination module 730 is configured to determine the temporal motion information of the coding blocks in the downsampled image according to the candidate reference frame interval.

[0125] The coding module 740 is configured to encode the original image block corresponding to the coding block according to the temporal motion information and the spatial motion information of the coding block in the downsampled image.

[0126] According to an embodiment of the present disclosure, the correlation determination module 710 includes a proportion determination sub-module and a first correlation determination sub-module.

[0127] The proportion determination sub-module is configured to determine the proportion of inter-frame coding blocks between the downsampled image and the reference image according to the motion estimation result between the downsampled image and the reference image.

[0128] The first correlation determination sub-module is configured to determine the correlation between the downsampled image and the reference image according to the proportion of inter-frame coding blocks.

[0129] According to an embodiment of the present disclosure, the correlation determination module 710 includes a coding order determination sub-module and a second correlation determination sub-module.

[0130] The coding order determination sub-module is used to determine the coding order of the downsampled images in the downsampled image sequence according to the inter-frame motion estimation results of the downsampled image sequence.

[0131] The second correlation determination sub-module is used to sequentially determine the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image according to the coding order.

[0132] According to an embodiment of the present disclosure, the interval determination module 720 includes an interval determination sub-module.

[0133] For each downsampled image, the interval determination sub-module is used to, in response to the correlation between the downsampled image and the reference image of the downsampled image meeting the correlation condition, determine the frame interval between the downsampled image and the reference image as the candidate reference frame interval for each downsampled image in the frame interval.

[0134] According to an embodiment of the present disclosure, each downsampled image includes at least one reference image, and the at least one reference image includes at least one of a forward reference image and a backward reference image. The interval determination sub-module includes a first interval determination unit and a second interval determination unit.

[0135] The first interval determination unit is used to, in response to the correlation between the downsampled image and the forward reference image meeting the correlation condition, determine the forward frame interval between the downsampled image and the forward reference image as the candidate reference frame interval for each downsampled image in the forward frame interval.

[0136] The second interval determination unit is used to, in response to the correlation between the downsampled image and the backward reference image meeting the correlation condition, determine the backward frame interval between the downsampled image and the backward reference image as the candidate reference frame interval for each downsampled image in the backward frame interval.

[0137] According to an embodiment of the present disclosure, the motion information determination module 730 includes a reference information determination sub-module and a motion information determination sub-module.

[0138] The reference information determination sub-module is used to determine the temporal motion information of the candidate reference frame interval according to the motion estimation results of the first and last frames of the candidate reference frame interval as the reference temporal motion information.

[0139] The motion information determination sub-module is used to determine the temporal motion information of the coded blocks in the downsampled image according to the distance between the reference temporal motion information and the spatial motion information of the coded blocks in the downsampled image.

[0140] According to an embodiment of the present disclosure, there are multiple candidate reference frame intervals, and the reference temporal motion information includes multiple reference temporal motion information respectively corresponding to the multiple candidate reference frame intervals; the motion information determination sub-module includes a target temporal motion information determination unit and a temporal motion information determination unit.

[0141] The target temporal motion information determining unit is configured to determine target temporal motion information from multiple reference temporal motion information according to the distances between the spatial motion information and the multiple reference temporal motion information respectively.

[0142] The temporal motion information determining unit is configured to determine the temporal motion information of the coding block in the downsampled image according to the target temporal motion information.

[0143] According to an embodiment of the present disclosure, the target temporal motion information determining unit includes a candidate information determining subunit and a target temporal motion information determining subunit.

[0144] The candidate information determining subunit is configured to determine candidate temporal motion information that meets the distance condition from multiple reference temporal motion information.

[0145] The target temporal motion information determining subunit is configured to determine a first temporal motion information with the largest distance from the spatial motion information and a second temporal motion information with the smallest distance from the spatial motion information from the candidate temporal motion information as the target temporal motion information.

[0146] According to an embodiment of the present disclosure, the temporal motion information determining unit includes a temporal motion information determining subunit, which is configured to scale the target temporal motion information according to the distance and direction between the downsampled image and the reference image, and the distance and direction between the first and last frames of the candidate reference frame interval corresponding to the target temporal motion information, to obtain the temporal motion information of the coding block.

[0147] According to an embodiment of the present disclosure, the encoding module 740 includes a starting point determining sub-module, a pre-analysis information determining sub-module, and an encoding sub-module.

[0148] The starting point determining sub-module is configured to determine a pre-analysis search starting point according to the temporal motion information and spatial motion information of the coding block in the downsampled image. The pre-analysis information determining sub-module is configured to determine pre-analysis motion information according to the pre-analysis search starting point. The encoding sub-module is configured to encode the original image block corresponding to the coding block according to the pre-analysis motion information.

[0149] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0150] Figure 8FIG. shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0151] As Figure 8 shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0152] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0153] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the image encoding method. For example, in some embodiments, the image encoding method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the image encoding method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the image encoding method in any other suitable manner (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-a-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0155] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0158] The systems and techniques described herein can be implemented in a computing system that includes a back-end component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of a communication network include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0159] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0160] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0161] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. An image encoding method, comprising: For a downsampled image sequence of an original image sequence, determining the correlation between each downsampled image in the downsampled image sequence and a reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence; Determining a candidate reference frame interval of the downsampled image according to the correlation; Determining the temporal motion information of an encoded block in the downsampled image according to the candidate reference frame interval; And Encoding an original image block corresponding to the encoded block according to the temporal motion information and spatial motion information of the encoded block in the downsampled image.

2. The method according to claim 1, wherein The step of, for a downsampled image sequence of an original image sequence, determining the correlation between each downsampled image in the downsampled image sequence and a reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence includes: for each downsampled image, Determining the proportion of inter-frame encoded blocks between the downsampled image and the reference image according to the motion estimation result between the downsampled image and the reference image; and Determining the correlation between the downsampled image and the reference image according to the proportion of inter-frame encoded blocks.

3. The method according to claim 1 or 2, wherein The step of, for a downsampled image sequence of an original image sequence, determining the correlation between each downsampled image in the downsampled image sequence and a reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence includes: Determining the encoding order of the downsampled images in the downsampled image sequence according to the inter-frame motion estimation result of the downsampled image sequence; and Determining the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image in sequence according to the encoding order.

4. The method according to claim 1, wherein The step of determining a candidate reference frame interval of the downsampled image according to the correlation includes: For each downsampled image, in response to the correlation between the downsampled image and the reference image of the downsampled image meeting a correlation condition, determining the frame interval between the downsampled image and the reference image as the candidate reference frame interval of each downsampled image in the frame interval.

5. The method according to claim 4, wherein Each downsampled image includes at least one reference image, and the at least one reference image includes at least one of a forward reference image and a backward reference image; the step of, for each downsampled image, in response to the correlation between the downsampled image and the reference image of the downsampled image meeting a correlation condition, determining the frame interval between the downsampled image and the reference image as the candidate reference frame interval of each downsampled image in the frame interval includes at least one of the following operations: for each downsampled image, In response to the correlation between the downsampled image and the forward reference image meeting a correlation condition, determining the forward frame interval between the downsampled image and the forward reference image as the candidate reference frame interval of each downsampled image in the forward frame interval; In response to the correlation between the downsampled image and the backward reference image meeting the correlation condition, determine the backward frame interval between the downsampled image and the backward reference image as the candidate reference frame interval for each downsampled image in the backward frame interval.

6. The method according to claim 1, wherein, The determining the temporal motion information of the coded blocks in the downsampled image according to the candidate reference frame interval includes: Determine the temporal motion information of the candidate reference frame interval as the reference temporal motion information according to the motion estimation results of the first and last frames of the candidate reference frame interval. Determine the temporal motion information of the coded blocks in the downsampled image according to the distance between the reference temporal motion information and the spatial motion information of the coded blocks in the downsampled image.

7. The method according to claim 6, wherein There are multiple candidate reference frame intervals, and the reference temporal motion information includes multiple reference temporal motion information corresponding to the multiple candidate reference frame intervals respectively; the determining the temporal motion information of the coded blocks in the downsampled image according to the distance between the reference temporal motion information and the spatial motion information of the coded blocks in the downsampled image includes: Determine the target temporal motion information from the multiple reference temporal motion information according to the distances between the spatial motion information and the multiple reference temporal motion information respectively; and Determine the temporal motion information of the coded blocks in the downsampled image according to the target temporal motion information.

8. The method according to claim 7, wherein, The determining the target temporal motion information from the multiple reference temporal motion information according to the distances between the spatial motion information and the multiple reference temporal motion information respectively includes: Determine the candidate temporal motion information meeting the distance condition from the multiple reference temporal motion information; and Determine the first temporal motion information with the maximum distance from the spatial motion information and the second temporal motion information with the minimum distance from the spatial motion information among the candidate temporal motion information as the target temporal motion information.

9. The method according to claim 7, wherein The determining the temporal motion information of the coded blocks in the downsampled image according to the target temporal motion information includes: Scale the target temporal motion information according to the distance and direction between the downsampled image and the reference image, and the distance and direction between the first and last frames of the candidate reference frame interval corresponding to the target temporal motion information to obtain the temporal motion information of the coded block.

10. The method according to claim 9, wherein, The encoding the original image block corresponding to the coded block according to the temporal motion information and spatial motion information of the coded block in the downsampled image includes: Determine the pre-analysis search start point according to the temporal motion information and spatial motion information of the coded block in the downsampled image; Determine the pre-analysis motion information according to the pre-analysis search start point; and Encode the original image block corresponding to the coded block according to the pre-analysis motion information.

11. An image encoding device, comprising: A correlation determination module, configured to determine, for the downsampled image sequence of the original image sequence, the correlation between each downsampled image in the downsampled image sequence and the reference image of the downsampled image according to the inter-frame motion estimation result of the downsampled image sequence. An interval determination module, configured to determine a candidate reference frame interval of the downsampled image according to the correlation; A motion information determination module, configured to determine temporal motion information of an encoded block in the downsampled image according to the candidate reference frame interval; And An encoding module, configured to encode an original image block corresponding to the encoded block according to the temporal motion information and spatial motion information of the encoded block in the downsampled image.

12. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program, the computer program being stored on at least one of a readable storage medium and an electronic device, and the computer program, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images