Method for determining motion estimation search range, electronic equipment and storage medium

Abstract

The invention discloses a method for determining a motion estimation search range, electronic equipment and a storage medium, and relates to the technical field of computers and video coding. The method comprises the steps that a to-be-processed video is acquired, the to-be-processed video comprises a plurality of image frames, the plurality of image frames comprise a first image frame and a second image frame, the first image frame is a current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame; distance information and index information corresponding to the to-be-processed video are determined, the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame; and determining a motion estimation search range of the second image frame based on the distance information and the index information. According to the method and the device, the technical problems of relatively low video coding speed, relatively low video compression performance and relatively high compression loss caused by performing motion estimation search in a fixed search range in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the fields of computer technology and video coding technology. Specifically, it relates to a method, an electronic device, and a storage medium for determining a motion estimation search range. Background Art

[0002] Motion estimation is an important technology in video coding and video processing, and is used to analyze the motion information between adjacent frames in a video sequence. By estimating the motion of an object or a scene in a video between different frames, more efficient video compression and coding, as well as video processing and analysis can be achieved.

[0003] Currently, the search for motion estimation starts from and centers around the motion vector prediction value (MVP), and searches within a square box with a side length of 2×search range (SR). Since SR is set to a fixed value, the video coding speed is slow, the video compression performance is low, and there is a high compression loss.

[0004] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present application provide a method, an electronic device, and a storage medium for determining a motion estimation search range, so as to at least solve the technical problem that in the related art, searching for motion estimation within a fixed search range results in a slow video coding speed, low video compression performance, and high compression loss.

[0006] According to one aspect of the embodiments of the present application, a method for determining a motion estimation search range is provided, including: obtaining a video to be processed, where the video to be processed includes: a plurality of image frames, and the plurality of image frames include: a first image frame and a second image frame, the first image frame is the current image frame in the motion estimation process of video processing, and the second image frame is at least one reference image frame associated with the first image frame; determining distance information and index information corresponding to the video to be processed, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame; and determining the motion estimation search range of the second image frame based on the distance information and the index information.

[0007] According to another aspect of the embodiments of the present application, there is also provided a method for determining a motion estimation search range, including: obtaining a network live video to be encoded, where the network live video to be encoded includes: a plurality of network live image frames to be encoded, and the plurality of network live image frames to be encoded include: a first network live image frame to be encoded and a second network live image frame to be encoded. The first network live image frame to be encoded is the current network live image frame to be encoded in the motion estimation process of network live video encoding, and the second network live image frame to be encoded is at least one reference image frame associated with the first network live image frame to be encoded; determining distance information and index information corresponding to the network live video to be encoded, where the distance information is the distance between the first network live image frame to be encoded and the second network live image frame to be encoded, and the index information is the frame index of the second network live image frame to be encoded; determining the motion estimation search range of the second network live image frame to be encoded based on the distance information and the index information.

[0008] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory storing an executable program; a processor for running the program, where when the program runs, it executes the method for determining a motion estimation search range described in any one of the above.

[0009] According to another aspect of the embodiments of the present application, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes an executable program stored therein, and when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the method for determining a motion estimation search range described in any one of the above.

[0010] In the embodiments of the present application, by obtaining a video to be processed for video encoding, then determining the minimum reference frame distance between the current image frame and the reference image frame in the motion estimation process in the video to be processed, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, an appropriate search range is adaptively determined, achieving the purpose of balancing reducing computational complexity and maintaining encoding quality, thereby realizing the technical effects of improving video encoding speed, improving encoding efficiency, and reducing video compression performance loss while maintaining video quality, and further solving the technical problem in the related art that searching for motion estimation within a fixed search range results in a slow video encoding speed, low video compression performance, and high compression loss.

[0011] It can be easily noted that the above general description and the following detailed description are for exemplifying and explaining the present application and do not constitute a limitation to the present application. Description of the Drawings

[0012] The accompanying drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0013] Figure 1 is a schematic diagram of motion estimation;

[0014] Figure 2 is a hardware structural block diagram of a computer terminal (or mobile device) for implementing a method for determining a motion estimation search range according to Embodiment 1 of the present application;

[0015] Figure 3 is a flowchart of a method for determining a motion estimation search range according to Embodiment 1 of the present application;

[0016] Figure 4 is a schematic diagram of a frame structure according to Embodiment 1 of the present application;

[0017] Figure 5 is a flowchart of a method for determining a motion estimation search range according to Embodiment 2 of the present application;

[0018] Figure 6 is a structural schematic diagram of a device for determining a motion estimation search range according to Embodiment 3 of the present application;

[0019] Figure 7 is a structural schematic diagram of another device for determining a motion estimation search range according to Embodiment 3 of the present application;

[0020] Figure 8 is a structural block diagram of a computer terminal according to an embodiment of the present application. Detailed implementation manners

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without making creative efforts shall fall within the protection scope of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] First, some of the nouns or terms that appear during the description of the embodiments of the present application are applicable to the following explanations:

[0024] Video coding: It is the process of converting digital video signals into a compressed format for storage, transmission and playback. In video coding, various algorithms and technologies are used to compress video signals to reduce the amount of data while maintaining video quality. Video coding technology is widely used in fields such as digital TV, Internet video, video conferencing, and surveillance systems.

[0025] Reference frame distance: In video coding and video compression, the reference frame distance usually refers to the time interval between the current frame and the reference frame, that is, in a video sequence, the number of frames between the current frame and the reference frame used for prediction. The selection of the reference frame distance can affect the efficiency and quality of video coding, especially in motion scenes or fast-changing scenes.

[0026] Reference frame index: In video coding and video compression, the reference frame index refers to the index number used to identify the frame used as a reference. During video compression, in order to reduce the amount of data, usually only some frames in the video sequence are encoded, while other frames are predicted inter-frame. The reference frame index is used to indicate the position of the reference frame used by the current frame in the video sequence. By using the reference frame index, the video encoder can determine the reference frame used during inter-frame prediction for motion estimation and motion compensation. It can be understood that selecting an appropriate reference frame is very important for the efficiency and quality of video compression and coding, and the correct reference frame index can help the video encoder better utilize the redundancy between frames, thus achieving a better compression effect.

[0027] Motion Estimation: It is an important part in video encoders (such as H.264, H.265, H.266 encoders), which is used to analyze the motion information between consecutive frames in order to achieve video compression in the encoder. The main purpose of motion estimation is to find the motion vectors between consecutive frames, that is, to predict the positions of pixel points in the current frame. It is usually implemented using some algorithms, such as full search algorithm, three-step search algorithm, pyramid algorithm, etc. The encoder finds the motion vectors by comparing the pixel values between the current frame and the reference frame to describe the position and motion trajectory of objects in the video, so as to reduce the redundant information between adjacent frames in the video and achieve more efficient compression.

[0028] Specifically, the process of motion estimation usually includes the following steps:

[0029] 1. Block Partitioning: The video frame is divided into small blocks, usually 8×8 or 16×16 pixel blocks.

[0030] 2. Target Matching: Between adjacent frames, by comparing the pixel values between different blocks to find the appropriate matching blocks to determine the motion vectors of the blocks.

[0031] 3. Motion Vector Estimation: According to the found appropriate matching blocks, determine the motion vectors of the current block relative to the reference block, that is, describe the displacement of the block in space.

[0032] 4. Motion Compensation: Perform motion compensation on the current frame according to the motion vectors to predict the pixel values in the current frame and reduce the redundant information between frames.

[0033] Search Range (SR) of Motion Estimation: In video coding, the search range of motion estimation refers to the search range used to find the appropriate matching blocks during inter-frame prediction. When performing motion estimation, the encoder searches for the block most similar to the block to be encoded in the current frame in the reference frame in order to find the appropriate motion vector to describe the displacement of the block. The size of the search range determines the accuracy and computational complexity of motion estimation. The larger the search range, the better the encoder can find the matching blocks, thus achieving better prediction results, but at the same time, it will increase the computational complexity. On the contrary, a smaller search range can reduce the computational complexity, but may lead to a decrease in the accuracy of motion estimation. It can be understood that choosing an appropriate search range is crucial for the efficiency and quality of video coding, and it needs to strike a balance between reducing computational complexity and maintaining coding quality in order to improve coding efficiency while maintaining video quality.

[0034] Pyramid Frame Structure: A video coding technology that utilizes different types of reference frames to achieve more efficient video compression. The pyramid frame structure usually includes the following frame types:

[0035] 1. I-frame (Key Frame): The I-frame is the key frame in the video sequence. It contains complete image information and does not rely on other frames for decoding.

[0036] 2. P-frame (Forward Prediction Frame): The P-frame is a frame predicted based on the previous I-frame or P-frame. It only contains the changed parts relative to the previous frame, so it can better utilize the redundancy between frames.

[0037] 3. B-frame (Bidirectional Prediction Frame): The B-frame is a frame predicted bidirectionally based on the adjacent I-frame or P-frame before and after. It contains the change information between the two reference frames before and after.

[0038] The pyramid frame structure can achieve a higher compression ratio while ensuring video quality by reasonably organizing the order and intervals of I-frames, P-frames, and B-frames, and is applicable to various video transmission and storage applications.

[0039] The embodiments of this application mainly solve the problem of high-complexity integer-pixel motion search. For the block to be encoded in the current frame, search is performed within the given range of the reference frame. According to a certain matching criterion, a block most similar to the current block to be encoded is found as the inter-frame prediction block of the current block to be encoded.

[0040] Currently, the motion search starts from and centers around the motion vector prediction value (MVP), and searches within a square box with a side length of 2×search range (SR). Considering the influence of the image boundary, if the box exceeds the image boundary, no search is performed. Figure 1 It is a schematic diagram of motion estimation. As Figure 1 shown, the current block to be encoded (Cur.Blk) is located in the current frame. It points to a reference block (Ref.Blk) in the reference frame through the motion vector prediction value (MVP). Then, centered on this, its search range (SR) is the gray area in the reference frame.

[0041] Generally, the larger the set search range SR is, the higher the complexity of motion estimation is, and the higher the similarity of the predicted block obtained is. Table 1 shows the performance indicators of the encoder when SR is set to 384 compared to when SR is set to 192. It can be seen from Table 1 that when SR is increased from 192 to 384, the number of frames encoded per second (Frames Per Second, FPS) in the encoding speed index increases by approximately half, the bitrate decreases by 0.56% under the same peak signal-to-noise ratio (Peak Signal-to-Noise Ratio, PSNR), the bitrate decreases by 0.59% under the same structural similarity index measure (Structural Similarity Index Measure, SSIM), the bitrate decreases by 0.67% under the same video multimethod assessment fusion (Video Multimethod Assessment Fusion, VMAF), and the bitrate decreases by 0.66% under the same video multimethod assessment fusion - negative emotion (Video Multimethod Assessment Fusion - Negative Emotion, VMAF - NEG). That is, increasing SR can improve the encoding speed, and at the same time, the compression performance of the video can also be improved while maintaining the same visual quality.

[0042] Table 1

[0043]

[0044] In the related art, there are the following defects in motion estimation based on a fixed value of SR.

[0045] Defect 1: The video encoding speed is slow, the video compression performance is low, and there is a high compression loss.

[0046] In response to the above defects, no effective solution has been proposed before this application.

[0047] Embodiment 1

[0048] According to an embodiment of the present application, a method for determining the motion estimation search range is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0049] The method embodiment provided by the first embodiment of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 2 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for determining the motion estimation search range according to Embodiment 1 of the present application. As Figure 2As shown, the computer terminal 20 (or mobile device) may include one or more processors 202 (illustrated as 202a, 202b, ……, 202n in the figure) (the processor 202 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 204 for storing data, and a transmission device 206 for communication functions. In addition, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 2 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 20 may further include more or fewer components than Figure 2 shown therein, or have a different configuration from Figure 2 that shown.

[0050] It should be noted that the above one or more processors 202 and / or other data processing circuits are generally referred to as "data processing circuits" herein. The data processing circuit may be embodied in software, hardware, firmware, or any combination thereof, in whole or in part. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of the other elements in the computer terminal 20 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is a processor control (such as the selection of a variable resistor terminal path connected to an interface).

[0051] The memory 204 may be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method for determining the motion estimation search range in the embodiments of the present application. The processor 202 executes various functional applications and data processing by running the software programs and modules stored in the memory 204, that is, implements the above-mentioned method for determining the motion estimation search range. The memory 204 may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 204 may further include a memory remotely located relative to the processor 202, and these remote memories may be connected to the computer terminal 20 through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0052] The transmission device 206 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by the communication provider of the computer terminal 20. In one example, the transmission device 206 includes a Network Interface Controller (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 206 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0053] The display can be, for example, a touch-screen Liquid Crystal Display (LCD), which enables the user to interact with the user interface of the computer terminal 20 (or mobile device).

[0054] Under the above operating environment, the present application provides a method for determining the motion estimation search range as Figure 3 shown. Figure 3 It is a flowchart of a method for determining the motion estimation search range according to Embodiment 1 of the present application. As Figure 3 shown, the method may include the following steps:

[0055] Step S31, obtain the video to be processed, where the video to be processed includes: a plurality of image frames, and the plurality of image frames include: a first image frame and a second image frame, the first image frame is the current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame;

[0056] Step S32, determine the distance information and index information corresponding to the video to be processed, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame;

[0057] Step S33, determine the motion estimation search range of the second image frame based on the distance information and the index information.

[0058] It can be understood that a video is composed of a series of consecutive image frames, and each frame is a static image. By continuously playing these image frames, a continuous dynamic image can be presented, that is, a video can be regarded as a continuous playback of a series of image frames in time.

[0059] Motion estimation is an important link in a video encoder. The video encoder compares the pixel values between the current frame and the reference frame to find the motion vector to describe the position and motion trajectory of an object in the video, so as to reduce the redundant information between adjacent frames in the video and achieve more efficient compression.

[0060] In the embodiments of the present application, the video to be processed can be understood as the video to be video-encoded. Video encoding the video can be understood as converting the original video signal into digital video data using a video encoder, so as to effectively compress the video data for more efficient storage, transmission, and processing.

[0061] The video to be processed includes a first image frame and a second image frame. The first image frame is the current image frame in the motion estimation process of video processing, which can be understood as the current frame in the video sequence in motion estimation. The second image frame is at least one reference image frame associated with the first image frame. The reference image frame can be understood as the image frame used for prediction in motion estimation, that is, the image frame used to predict the content of the current frame. Exemplarily, the reference frame is usually one or more frames before or after the current frame, which is not limited here.

[0062] The distance information can be understood as the reference frame distance, which is the distance between the first image frame and the second image frame, that is, the time interval between the current frame and the reference frame. Considering that among all the reference frames of the current frame, the reference frame closest to the current frame has the highest probability of being selected, that is, the smaller the reference frame index, the higher the probability of its being selected. Therefore, in the embodiments of the present application, the reference frame is the image frame with the smallest distance from the current frame, and the distance information is the minimum reference frame distance between the first image frame and the second image frame, that is, the minimum allowable interval distance between the current frame and the reference frame. Exemplarily, if the reference frame is before the current frame, the reference frame distance is a positive number, and if the reference frame is after the current frame, the reference frame distance is a negative number.

[0063] The index information is the frame index of the second image frame, that is, the reference frame index. In video encoding, the reference frame index is used to identify the serial number of the reference frame, that is, to determine the position of the reference frame used by the current frame in the video sequence. Exemplarily, the reference frame index can be an absolute frame number or a relative frame number relative to the current frame, which is not limited here.

[0064] The motion estimation search range can be understood as the search range used to find the matching block (i.e., the inter-frame prediction block) of the block to be encoded in the current frame when performing inter-frame prediction. It can be understood that when performing motion estimation, the encoder searches for the block most similar to the block to be encoded in the current frame in the reference frame to find a suitable motion vector to describe the displacement of the block. Therefore, the size of the search range determines the accuracy and computational complexity of motion estimation. Usually, the farther the reference frame is from the current frame, the larger the absolute value of its motion vector, and the larger the motion estimation search range required.

[0065] Considering that among all the reference frames of the current frame, the reference frame closest to the current frame has the highest probability of being selected, that is, the smaller the reference frame index, the higher the probability of its being selected. Therefore, in the embodiments of the present application, by obtaining the video to be processed for video coding, then determining the minimum reference frame distance between the current image frame and the reference image frame during the motion estimation process in the video to be processed, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, an appropriate search range can be adaptively determined, achieving a balance between reducing the computational complexity and maintaining the coding quality, and realizing the technical effects of improving the video coding speed, improving the coding efficiency, and reducing the loss of video compression performance while maintaining the video quality.

[0066] The method for determining the motion estimation search range provided by the embodiments of the present application can be, but is not limited to, applied to application scenarios involving image coding in fields such as e-commerce services, education services, legal services, medical services, conference services, social network services, financial product services, logistics services, and navigation services. For example: coding of network live videos in e-commerce services, coding of network on-demand videos in e-commerce services, coding of live class videos in education services, coding of relevant case videos in legal services, etc., which are not limited here.

[0067] By adopting the embodiments of the present application, by obtaining the video to be processed for video coding, then determining the minimum reference frame distance between the current image frame and the reference image frame during the motion estimation process in the video to be processed, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, the purpose of adaptively determining an appropriate search range and achieving a balance between reducing the computational complexity and maintaining the coding quality is achieved, thereby realizing the technical effects of improving the video coding speed, improving the coding efficiency, and reducing the loss of video compression performance while maintaining the video quality, and further solving the technical problem in the related art that the search for motion estimation within a fixed search range results in a slow video coding speed, low video compression performance, and high compression loss.

[0068] In an optional embodiment, in step S32, determining the distance information includes the following method steps:

[0069] Step S321, obtaining a first identifier and a second identifier, where the first identifier is the playback sequence identifier of the first image frame, and the second identifier is the playback sequence identifier of the second image frame;

[0070] Step S322, calculating the distance information based on the first identifier and the second identifier.

[0071] It can be understood that in video coding, since frames may be encoded and stored in different orders, in order to correctly reorganize and play the frames during decoding, each frame is assigned a Play Order Count (POC) value. The POC value is usually related to the display order of the frames and is used to enable the decoder to decode and reorganize the frames in the correct order, thereby achieving the correct playback of the video.

[0072] In the embodiments of the present application, the first identifier is the play order identifier of the first image frame, that is, the POC value of the current frame, and the second identifier is the play order identifier of the second image frame, that is, the POC value of the reference frame.

[0073] In the embodiments of the present application, when determining the distance information, the play order identifier of the first image frame, that is, the first identifier, and the play order identifier of the second image frame, that is, the second identifier, can be obtained, that is, the POC value of the current frame and the POC value of the reference frame are obtained. Then, the distance information is calculated based on the first identifier and the second identifier, that is, the distance between the current frame and the reference frame is calculated based on the POC value of the current frame and the POC value of the reference frame.

[0074] In an alternative embodiment, in step S321, obtaining the first identifier and the second identifier includes the following method steps:

[0075] Step S3211, obtaining the frame structure adopted by multiple image frames, where the frame structure includes: a frame type field and a frame play order field;

[0076] Step S3212, obtaining the first identifier and the second identifier based on the frame structure.

[0077] It can be understood that by reasonably organizing the order and interval between image frames, the frame structure can achieve a higher compression ratio while ensuring video quality. The frame structure includes a frame type field and a frame play order field. Among them, the frame type field is used to represent the type of the frame, and the frame play order field is used to represent the POC value of the frame.

[0078] In the embodiments of the present application, the frame structure adopted by multiple image frames can be a pyramid B-frame structure. Exemplarily, Figure 4 is a schematic diagram of a frame structure according to Embodiment 1 of the present application. As Figure 4 shown, the frame type field of "B82" is "B", and the frame play order field is "82", that is, it represents a B-frame with a POC value of 82. The frame type field of "P80" is "P", and the frame play order field is "80", that is, it represents a P-frame with a POC value of 80, and so on. Details are not elaborated here.

[0079] Exemplarily, the frame structure can also be a Group of Pictures (GOP) frame structure, an adaptive frame structure, or a progressive frame structure, which is not limited herein.

[0080] In the embodiments of the present application, when obtaining the first identifier and the second identifier, the frame structure adopted by multiple image frames in the video to be processed can be obtained, and based on the frame structure, the first identifier of the first image frame and the second identifier of the second image frame can be obtained, that is, the POC value of the current frame and the POC value of the reference frame are obtained based on the frame structure.

[0081] In an alternative embodiment, in step S322, calculating the distance information based on the first identifier and the second identifier includes the following method steps:

[0082] Step S3221, perform a difference calculation on the first identifier and the second identifier to obtain a calculation result;

[0083] Step S3222, select the minimum absolute value from the calculation results to obtain the distance information.

[0084] Performing a difference calculation on the first identifier and the second identifier can be understood as performing a difference calculation on the POC value of the first image frame and the POC values of at least one reference image frame included in the second image frame, so as to obtain the difference calculation result of the POC.

[0085] Considering that among all the reference frames of the current frame, the reference frame closest to the current frame has the highest probability of being selected, that is, the smaller the reference frame index, the higher the probability of its being selected. Therefore, in the embodiments of the present application, the image frame closest to the first image frame is selected as the image frame for predicting the first image frame. By selecting the minimum absolute value from the calculation results, the second image frame closest to the first image frame can be determined, and then the distance information, that is, the minimum reference frame distance, can be determined.

[0086] In the embodiments of the present application, when calculating the distance information based on the first identifier and the second identifier, a difference calculation can be performed on the first identifier and the second identifier to obtain a calculation result, and then the difference with the minimum absolute value is selected from the calculation results to obtain the distance information.

[0087] Exemplarily, as Figure 4 shown, for the B84 frame, the closest reference frames (with the minimum absolute value of the POC difference) are P80 and B88, then the minimum reference frame distance minRefDist = abs(84 - 88) = 4, and so on, which will not be elaborated here.

[0088] In an alternative embodiment, in step S33, determining the motion estimation search range of the second image frame based on the distance information and the index information includes the following method steps:

[0089] Step S331, based on the distance information and the index information, find the motion estimation search range of the second image frame from the preset search range derivation relationship, where the preset search range derivation relationship is used to record the corresponding relationship among multiple image frame distance ranges, multiple image frame index ranges, and multiple motion estimation search ranges.

[0090] The preset search range derivation relationship is used to record the corresponding relationship among multiple image frame distance ranges, multiple image frame index ranges, and multiple motion estimation search ranges, that is, it can be understood as recording the search ranges corresponding to different minimum reference frame distances and reference frame indexes.

[0091] In the embodiment of the present application, when determining the motion estimation search range of the second image frame based on the distance information and the index information, it is possible to use the preset search range derivation relationship that records the corresponding relationship among multiple image frame distance ranges, multiple image frame index ranges, and multiple motion estimation search ranges, and then determine the motion estimation search range for finding the second image frame from the preset search range derivation relationship according to the distance information and the index information.

[0092] In an alternative embodiment, the preset search range derivation relationship includes: a preset search range derivation lookup table, that is, the preset search range derivation relationship can be understood as a search range derivation lookup table preset for recording the corresponding relationship among multiple image frame distance ranges, multiple image frame index ranges, and multiple motion estimation search ranges.

[0093] The preset search range derivation lookup table includes: multiple preset search range derivation table entries, and the multiple preset search range derivation table entries include: a first field, a second field, and a third field. Among them, the first field is the image frame distance field, that is, the minRefDist field, representing the minimum reference frame distance, and various distance information is recorded in this first field. The second field is the image frame index field corresponding to the image frame distance field, that is, the refIdx field, representing the reference frame index, and various index information is recorded in this second field. The third field is the motion estimation search range field corresponding to both the image frame distance field and the image frame index field, that is, the SR field, representing the search range.

[0094] In an alternative embodiment, in step S331, finding the motion estimation search range of the second image frame from the preset search range derivation relationship based on the distance information and the index information includes the following method steps:

[0095] Step S3311, in response to the distance information being greater than or equal to the first distance and the index information being the first index, find the motion estimation search range of the second image frame from the preset search range derivation relationship as the first range;

[0096] Step S3312, in response to the distance information being less than the first distance and the index information being the first index, or the distance information being greater than or equal to the second distance and the index information being greater than or equal to the second index, find the motion estimation search range of the second image frame as the second range from the preset search range derivation relationship;

[0097] Step S3313, in response to the distance information being less than the second distance and the index information being greater than or equal to the second index, find the motion estimation search range of the second image frame as the third range from the preset search range derivation relationship.

[0098] Exemplarily, taking the first distance as 4, the second distance as 8, the first index as 0, and the second index as 1 as an example, the search range derivation lookup table can be shown in Table 2. Among them, the first range is the maximum search range, that is, maxSR. The maximum search range can be set in advance and is not limited here. The second range can be half of the maximum search range, that is, maxSR / 2. The third range can be half of the second range, that is, maxSR / 4.

[0099] When the minimum reference frame distance ≥ 4 and the reference frame index Idx = 0, the search range is maxSR; when the minimum reference frame distance < 4 and the reference frame index Idx = 0, the search range is maxSR / 2; when the minimum reference frame distance ≥ 8 and the reference frame index Idx ≥ 1, the search range is maxSR / 2; when the minimum reference frame distance < 8 and the reference frame index Idx ≥ 1, the search range is maxSR / 4.

[0100] Table 2

[0101]

[0102] It can be understood that according to Table 2 proposed in the present application, the motion estimation search range of each reference frame can be determined according to the minimum reference frame distance and the reference frame index of each reference frame, that is, a reasonable search range can be adaptively determined according to the minimum reference frame distance and the reference frame index.

[0103] Exemplarily, taking Figure 4 the pyramid B-frame structure shown in Figure 4The reference frame, minimum reference frame distance, and search range of some frames in the middle. Taking maxSR = 384 as an example, for the list0 reference frame P80 of the B84 frame, with its Idx = 0, according to Table 2, its search range is maxSR = 384. For the list0 reference frame P64 of the B84 frame, with its Idx = 1, according to Table 2, its search range is maxSR / 4 = 96. Similarly, the search ranges of other reference frames can be deduced, as shown in Table 3, which will not be elaborated here.

[0104] Table 3

[0105]

[0106] In an alternative embodiment, the method for determining the motion estimation search range further includes the following method steps:

[0107] Step S341: Select a first image block from the first image frame;

[0108] Step S342: Search for a target image block in the second image frame based on the motion estimation search range, where the target image block is an inter-frame prediction block corresponding to the first image block;

[0109] Step S342: Perform inter-frame prediction on the first image block using the target image block.

[0110] The first image block is a block to be encoded in the first image frame, that is, the current block (Cur.Blk) in the current frame.

[0111] The target image block is a reference block (Ref.Blk) in the second image frame, that is, an inter-frame prediction block in the reference frame, which can be understood as the inter-frame prediction block corresponding to the block to be encoded.

[0112] Optionally, the target image block is the image block with the highest similarity to the first image block. Exemplarily, when searching for the target image block within the motion estimation search range, the search can be performed according to a matching criterion. The matching criterion can be to search for the block most similar to the current block to be encoded. Thus, the target image block can be the image block with the highest similarity to the first image block, that is, the target image block is the block most similar to the current block to be encoded and serves as the inter-frame prediction block of the current block to be encoded.

[0113] In the embodiments of the present application, when performing video encoding processing based on the motion estimation search range, a first image block can be selected from the first image frame, that is, a block to be encoded is selected from the current frame, and then the target image block is searched and found in the second image frame according to the determined motion estimation search range, that is, the inter-frame prediction block corresponding to the block to be encoded is searched and found in the reference frame according to the search range, so as to perform inter-frame prediction on the first image block according to the found inter-frame prediction block to predict the content of the current frame, thereby reducing the redundancy of video data, enabling the video encoder to compress video data more efficiently, and improving the performance and quality of video encoding.

[0114] In an alternative embodiment, in step S342, searching for the target image block in the second image frame based on the motion estimation search range includes the following method steps:

[0115] Step S3421, obtaining the motion vector prediction value corresponding to the first image block;

[0116] Step S3422, determining a second image block in the second image frame by using the motion vector prediction value, where the second image block is a reference image block;

[0117] Step S3423, searching for the target image block within the motion estimation search range centered on the second image block.

[0118] The second image block can be understood as any reference block in the second image frame, that is, any reference block in the reference frame.

[0119] In the embodiments of the present application, when searching for the target image block in the second image frame based on the motion estimation search range, the motion vector prediction value MVP corresponding to the first image block can be obtained first, that is, the MVP corresponding to the block to be encoded is obtained, and then the second image block is determined in the second image frame by using the MVP, that is, any reference block in the reference frame is pointed to by the MVP, so as to search for the target image block within the determined motion estimation search range centered on the second image block pointed to by the MVP and find the inter-frame prediction block corresponding to the block to be encoded.

[0120] In an alternative embodiment, a graphical user interface is provided by a terminal device, and the content displayed on the graphical user interface at least partially includes an image encoding scene. The method for determining the motion estimation search range further includes the following method steps:

[0121] Step S35, in response to a first control operation performed on the graphical user interface, inputting the derivation relationship between the video to be processed and the preset search range;

[0122] Step S36: In response to a second control operation performed on the graphical user interface, determine the distance information and index information corresponding to the video to be processed, and based on the distance information and index information, search for the motion estimation search range of the second image frame from the preset search range derivation relationship;

[0123] Step S37: In response to a third control operation performed on the graphical user interface, encode the video to be processed according to the motion estimation search range to obtain an encoding result;

[0124] Step S38: Display the encoding result within the graphical user interface.

[0125] In the graphical user interface in the embodiments of the present application, at least an image encoding scenario is displayed. A user can, by performing a control operation, input the video to be processed and the preset search range derivation relationship in this image encoding scenario, determine the distance information and index information corresponding to the video to be processed, and based on the distance information and index information, search for the motion estimation search range of the second image frame from the preset search range derivation relationship, and encode the video to be processed according to the motion estimation search range to obtain an encoding result, etc. It can be understood that the above image encoding scenario can be, but is not limited to, application scenarios involving image encoding in fields such as e-commerce, education, medical care, conferences, social networks, financial products, logistics, and navigation.

[0126] The above graphical user interface further includes a first control (or a first touch area). When a first touch operation acting on the first control (or the first touch area) is detected, the video to be processed and the preset search range derivation relationship input by the user can be obtained. The above video to be processed and the preset search range derivation relationship can be input by the user from a text box in the graphical user interface through the first touch operation, or can be uploaded by the user from the graphical user interface through the first touch operation. The above first touch operation can be operations such as point selection, box selection, tick selection, conditional filtering, etc., which are not limited here.

[0127] The above graphical user interface further includes a second control (or a second touch area). When a second touch operation acting on the second control (or the second touch area) is detected, the distance information and index information corresponding to the video to be processed can be determined, and based on the distance information and index information, the motion estimation search range of the second image frame can be searched from the preset search range derivation relationship. The above second touch operation can be operations such as point selection, box selection, tick selection, conditional filtering, etc., which are not limited here.

[0128] The above-mentioned graphical user interface further includes a third control (or a third touch area). When a third touch operation on the third control (or the third touch area) is detected, the video to be processed can be encoded according to the motion estimation search range to obtain an encoding result. The above-mentioned third touch operation can be operations such as point selection, box selection, tick selection, conditional filtering, etc., which are not limited here.

[0129] After obtaining the encoding result, the encoding result can be displayed within the graphical user interface to feedback to the user.

[0130] It should be noted that the above-mentioned first touch operation, second touch operation, and third touch operation can all be operations where the user touches the display screen of the above-mentioned terminal device with a finger and touches the terminal device. This touch operation can include single-touch and multi-touch. Among them, the touch operation of each touch point can include clicking, long-pressing, hard-pressing, swiping, etc. The above-mentioned first touch operation, second touch operation, and third touch operation can also be touch operations implemented through input devices such as a mouse and a keyboard, which are not limited here.

[0131] It can be seen that in this application, by classifying using the minimum reference frame distance and the reference frame index, the motion estimation search range is determined. Specifically, by calculating the difference between all reference frames of the current frame and the identification number (POC) of the video frames of the current frame in the playback order respectively, taking the difference with the smallest absolute value as the minimum reference frame distance, and then determining the motion estimation search range of each reference frame according to the minimum reference frame distance and the reference frame index of each reference frame, the speed of the video encoder can be doubled, and there is almost no loss of compression performance.

[0132] Exemplarily, as shown in Table 4, Table 4 is the comparison data of using the method of this application and the method of the related technology under the preset maxSR = 384. As shown in Table 4, it can be seen that using the method of this application can double the speed of the video encoder (the FPS is 2.14 times the original), and there is almost no loss of compression performance (the bit rate increases by 0.1% under the same PSNR, the bit rate increases by 0.11% under the same SSIM, the bit rate increases by 0.13% under the same VMAF, and the bit rate increases by 0.14% under the same VMAF-NEG).

[0133] Table 4

[0134]

[0135] It is easy to understand that the beneficial effects of the method for determining the motion estimation search range provided by this application include the following points.

[0136] Beneficial effects (1). According to the minimum reference frame distance and the reference frame index, the present application can adaptively determine a reasonable motion estimation search range, which improves the video encoding speed, encoding efficiency, and reduces the loss of video compression performance while maintaining the video quality.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0138] In addition, it should also be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0139] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0140] Embodiment 2

[0141] Under the operating environment as in Embodiment 1, the present application provides a method for determining a motion estimation search range as shown in Figure 5 A flowchart of a method for determining a motion estimation search range according to Embodiment 2 of the present application is shown in Figure 5 As shown in Figure 5 The method includes:

[0142] Step S51: Obtain the network live video to be encoded. The network live video to be encoded includes multiple network live image frames to be encoded. The multiple network live image frames to be encoded include a first network live image frame to be encoded and a second network live image frame to be encoded. The first network live image frame to be encoded is the current network live image frame to be encoded in the motion estimation process of network live video encoding. The second network live image frame to be encoded is at least one reference image frame associated with the first network live image frame to be encoded.

[0143] Step S52: Determine the distance information and index information corresponding to the network live video to be encoded. The distance information is the distance between the first network live image frame to be encoded and the second network live image frame to be encoded. The index information is the frame index of the second network live image frame to be encoded.

[0144] Step S53: Determine the motion estimation search range of the second network live image frame to be encoded based on the distance information and the index information.

[0145] In the embodiments of the present application, the network live video to be encoded can be understood as the video to be video-encoded during the network live process. Video encoding the video can be understood as converting the original video signal into digital video data using a video encoder, so as to effectively compress the video data for more efficient storage, transmission, and processing.

[0146] The network live video to be encoded includes a first network live image frame to be encoded and a second network live image frame to be encoded. The first network live image frame to be encoded is the current network live image frame to be encoded in the motion estimation process of network live video encoding, which can be understood as the current frame in the network live video sequence during network live video encoding motion estimation. The second network live image frame to be encoded is at least one reference image frame associated with the first network live image frame to be encoded. The reference image frame can be understood as the image frame used for prediction during network live video encoding motion estimation. Exemplarily, the reference frame is usually one or more frames before or after the current frame, which is not limited here.

[0147] The distance information can be understood as the reference frame distance, which is the distance between the first network live video image frame to be encoded and the second network live video image frame to be encoded, that is, the time interval between the current frame and the reference frame. Considering that among all the reference frames of the current frame, the reference frame closest to the current frame has the highest probability of being selected, that is, the smaller the reference frame index, the higher the probability of its being selected. Therefore, in the embodiments of the present application, the reference frame is the image frame with the smallest distance from the current frame, and the distance information is the minimum reference frame distance between the first network live video image frame to be encoded and the second network live video image frame to be encoded, that is, the minimum allowable interval distance between the current frame and the reference frame. Exemplarily, if the reference frame is before the current frame, the reference frame distance is a positive number; if the reference frame is after the current frame, the reference frame distance is a negative number.

[0148] The index information is the frame index of the second network live video image frame to be encoded, that is, the reference frame index. In network live video coding, the reference frame index is used to identify the serial number of the reference frame, that is, to determine the position of the reference frame used by the current frame in the video sequence. Exemplarily, the reference frame index can be an absolute frame number or a relative frame number relative to the current frame, which is not limited here.

[0149] The motion estimation search range can be understood as the search range used to find the matching block (i.e., the inter-frame prediction block) of the block to be encoded in the current frame during inter-frame prediction. It can be understood that when performing motion estimation for network live video coding, the encoder will search for the block most similar to the block to be encoded in the current frame in the reference frame in order to find a suitable motion vector to describe the displacement of the block. Therefore, the size of the search range determines the accuracy and computational complexity of motion estimation. Generally, the farther the reference frame is from the current frame, the larger the absolute value of its motion vector, and the larger the motion estimation search range required.

[0150] Considering that among all the reference frames of the current frame, the reference frame closest to the current frame has the highest probability of being selected, that is, the smaller the reference frame index, the higher the probability of its being selected. Therefore, in the embodiments of the present application, by obtaining the network live video to be encoded for video coding, then determining the minimum reference frame distance between the current image frame and the reference image frame during the motion estimation process of network live video coding in the network live video to be encoded, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, it is possible to adaptively determine a suitable search range, achieving a balance between reducing computational complexity and maintaining coding quality, and realizing the technical effects of improving the video coding speed, improving the coding efficiency, and reducing the loss of video compression performance while maintaining the video quality.

[0151] The method for determining the motion estimation search range provided in the embodiments of the present application can also be but is not limited to being applied to application scenarios involving image coding in fields such as education services, legal services, medical services, conference services, social network services, financial product services, logistics services, and navigation services. For example, in education services, the coding of live class videos, and in legal services, the coding of relevant case videos, etc., which are not limited here.

[0152] By adopting the embodiments of the present application, by obtaining the to-be-encoded network live video to be subjected to video coding, then determining the minimum reference frame distance between the current image frame and the reference image frame in the network live video coding motion estimation process, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, thus achieving the purpose of adaptively determining a suitable search range and achieving a balance between reducing the computational complexity and maintaining the coding quality, thereby realizing the technical effects of improving the video coding speed, improving the coding efficiency, and reducing the loss of video compression performance while maintaining the video quality, and further solving the technical problem in the related art that the search for motion estimation within a fixed search range results in a slow video coding speed, a low video compression performance, and a high compression loss.

[0153] It should be noted that the preferred implementation manners of this embodiment can refer to the relevant descriptions in Embodiment 1, which will not be elaborated here.

[0154] Embodiment 3

[0155] According to the embodiments of the present application, there is also provided an apparatus embodiment for implementing the above method for determining the motion estimation search range. Figure 6 It is a schematic structural diagram of an apparatus for determining the motion estimation search range according to Embodiment 3 of the present application, as Figure 6 shown, the apparatus includes:

[0156] A first acquisition module 601, configured to acquire a to-be-processed video, where the to-be-processed video includes: a plurality of image frames, and the plurality of image frames include: a first image frame and a second image frame, the first image frame is the current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame;

[0157] A first determination module 602, configured to determine the distance information and index information corresponding to the to-be-processed video, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame;

[0158] A second determination module 603, configured to determine the motion estimation search range of the second image frame based on the distance information and the index information.

[0159] Optionally, the first determination module 602 is further configured to: obtain a first identifier and a second identifier, where the first identifier is a playback sequence identifier of a first image frame, and the second identifier is a playback sequence identifier of a second image frame; calculate distance information based on the first identifier and the second identifier.

[0160] Optionally, the first determination module 602 is further configured to: obtain a frame structure adopted by a plurality of image frames, where the frame structure includes: a frame type field and a frame playback sequence field; obtain the first identifier and the second identifier based on the frame structure.

[0161] Optionally, the first determination module 602 is further configured to: perform a difference calculation on the first identifier and the second identifier to obtain a calculation result; select the minimum absolute value from the calculation result to obtain the distance information.

[0162] Optionally, the second determination module 603 is further configured to: based on the distance information and the index information, search for a motion estimation search range of the second image frame from a preset search range derivation relationship, where the preset search range derivation relationship is used to record the corresponding relationship among a plurality of image frame distance ranges, a plurality of image frame index ranges, and a plurality of motion estimation search ranges.

[0163] Optionally, the preset search range derivation relationship includes: a preset search range derivation lookup table, where the preset search range derivation lookup table includes: a plurality of preset search range derivation table entries, and the plurality of preset search range derivation table entries include: a first field, a second field, and a third field. The first field is an image frame distance field, and the first field is used to record various distance information. The second field is an image frame index field corresponding to the image frame distance field, and the second field is used to record various index information. The third field is a motion estimation search range field corresponding to both the image frame distance field and the image frame index field.

[0164] Optionally, the second determination module 603 is further configured to: in response to the distance information being greater than or equal to a first distance and the index information being a first index, search for the motion estimation search range of the second image frame from the preset search range derivation relationship as a first range; in response to the distance information being less than the first distance and the index information being the first index, or the distance information being greater than or equal to a second distance and the index information being greater than or equal to a second index, search for the motion estimation search range of the second image frame from the preset search range derivation relationship as a second range; in response to the distance information being less than the second distance and the index information being greater than or equal to the second index, search for the motion estimation search range of the second image frame from the preset search range derivation relationship as a third range.

[0165] Optionally, it further includes: an encoding module, configured to select a first image block from a first image frame; search for a target image block in a second image frame based on a motion estimation search range, where the target image block is an inter-frame prediction block corresponding to the first image block; and perform inter-frame prediction on the first image block by using the target image block.

[0166] Optionally, the above encoding module is further configured to: obtain a predicted motion vector value corresponding to the first image block; determine a second image block in the second image frame by using the predicted motion vector value, where the second image block is a reference image block; and search for the target image block within the motion estimation search range centered on the second image block.

[0167] Optionally, a graphical user interface is provided by a terminal device, and the content displayed on the graphical user interface at least partially includes an image encoding scenario. It further includes: an interaction module, configured to, in response to a first control operation performed on the graphical user interface, input a relationship between a video to be processed and a preset search range derivation; in response to a second control operation performed on the graphical user interface, determine distance information and index information corresponding to the video to be processed, and based on the distance information and the index information, search for a motion estimation search range of a second image frame from the preset search range derivation; in response to a third control operation performed on the graphical user interface, encode the video to be processed according to the motion estimation search range to obtain an encoding result; and display the encoding result within the graphical user interface.

[0168] By adopting the embodiment of the present application, by obtaining a video to be processed for video encoding, then determining the minimum reference frame distance between a current image frame and a reference image frame in the motion estimation process in the video to be processed, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, an appropriate search range is adaptively determined, achieving the purpose of balancing between reducing computational complexity and maintaining encoding quality. Thus, the technical effects of improving the video encoding speed, improving the encoding efficiency, and reducing the loss of video compression performance while maintaining the video quality are realized, and further the technical problem in the related art that motion estimation is searched within a fixed search range, resulting in a slow video encoding speed, low video compression performance, and high compression loss is solved.

[0169] It should be noted here that the above first acquisition module 601, first determination module 602, and second determination module 603 correspond to steps S31 to S33 in Embodiment 1. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules or units may be hardware components or software components stored in a memory (for example, memory 204) and processed by one or more processors (for example, processors 202a, 202b,..., 202n). The above modules may also be part of a device and may run in the computer terminal 20 provided in Embodiment 1.

[0170] According to an embodiment of the present application, there is also provided another embodiment of a device for implementing the above determination of the motion estimation search range. Figure 7 It is a schematic structural diagram of another device for determining the motion estimation search range according to Embodiment 3 of the present application, as Figure 7 shown. The device includes:

[0171] A second acquisition module 701, configured to acquire a network live video to be encoded, where the network live video to be encoded includes: a plurality of network live image frames to be encoded, and the plurality of network live image frames to be encoded includes: a first network live image frame to be encoded and a second network live image frame to be encoded. The first network live image frame to be encoded is the current network live image frame to be encoded during the motion estimation of the network live video encoding, and the second network live image frame to be encoded is a reference image frame associated with the first network live image frame to be encoded;

[0172] A third determination module 702, configured to determine distance information and index information corresponding to the network live video to be encoded, where the distance information is the distance between the first network live image frame to be encoded and the second network live image frame to be encoded, and the index information is the frame index of the second network live image frame to be encoded;

[0173] A fourth determination module 703, configured to determine the motion estimation search range of the second network live image frame to be encoded based on the distance information and the index information.

[0174] By adopting the embodiment of the present application, by obtaining the to-be-encoded network live video to be subjected to video encoding, then determining the minimum reference frame distance between the current image frame and the reference image frame in the to-be-encoded network live video during the network live video encoding motion estimation process, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, thus achieving the purpose of adaptively determining a suitable search range and achieving a balance between reducing the computational complexity and maintaining the encoding quality, thereby realizing the technical effects of improving the video encoding speed, improving the encoding efficiency, and reducing the loss of video compression performance while maintaining the video quality, and further solving the technical problem in the related art that the search for motion estimation within a fixed search range results in a slow video encoding speed, a low video compression performance, and a high compression loss.

[0175] It should be noted here that the above-mentioned second acquisition module 701, third determination module 702, and fourth determination module 703 correspond to steps S51 to S53 in Embodiment 2. The examples and application scenarios implemented by the three modules and the corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above-mentioned module or unit can be a hardware component or a software component stored in a memory (for example, memory 204) and processed by one or more processors (for example, processors 202a, 202b,..., 202n), and the above-mentioned module can also be a part of the device and can run in the computer terminal 20 provided in Embodiment 1.

[0176] It should be noted that the preferred implementation schemes involved in the above embodiments of the present application are the same as the schemes, application scenarios, and implementation processes provided in Embodiment 1, but are not limited to the schemes provided in Embodiment 1.

[0177] Embodiment 4

[0178] The embodiment of the present application can provide a computer terminal, and this computer terminal can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above-mentioned computer terminal can also be replaced with a terminal device such as a mobile terminal.

[0179] Optionally, in this embodiment, the above-mentioned computer terminal can be located in at least one network device among multiple network devices of a computer network.

[0180] In this embodiment, the above computer terminal may execute the program code of the following steps in the method for determining the motion estimation search range: obtaining a video to be processed, where the video to be processed includes: a plurality of image frames, and the plurality of image frames include: a first image frame and a second image frame, the first image frame is the current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame; determining distance information and index information corresponding to the video to be processed, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame; determining the motion estimation search range of the second image frame based on the distance information and the index information.

[0181] Optionally, Figure 8 is a structural block diagram of a computer terminal according to an embodiment of the present application. As Figure 8 shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 802, a memory 804, a storage controller, and a peripheral interface, where the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0182] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the method and device for determining the motion estimation search range in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored therein, that is, implements the above method for determining the motion estimation search range. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely set relative to the processor, and these remote memories may be connected to the computer terminal A through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0183] The processor may call the information and application programs stored in the memory through a transmission device to execute the following steps: obtaining a video to be processed, where the video to be processed includes: a plurality of image frames, and the plurality of image frames include: a first image frame and a second image frame, the first image frame is the current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame; determining distance information and index information corresponding to the video to be processed, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame; determining the motion estimation search range of the second image frame based on the distance information and the index information.

[0184] Optionally, the above-mentioned processor may also execute program code for the following steps: obtaining a first identifier and a second identifier, where the first identifier is the playback sequence identifier of a first image frame, and the second identifier is the playback sequence identifier of a second image frame; calculating distance information based on the first identifier and the second identifier.

[0185] Optionally, the above-mentioned processor may also execute program code for the following steps: obtaining the frame structure adopted by a plurality of image frames, where the frame structure includes: a frame type field and a frame playback sequence field; obtaining the first identifier and the second identifier based on the frame structure.

[0186] Optionally, the above-mentioned processor may also execute program code for the following steps: performing a difference calculation on the first identifier and the second identifier to obtain a calculation result; selecting the minimum absolute value from the calculation result to obtain the distance information.

[0187] Optionally, the above-mentioned processor may also execute program code for the following steps: based on the distance information and the index information, searching for the motion estimation search range of the second image frame from the preset search range derivation relationship, where the preset search range derivation relationship is used to record the corresponding relationship among a plurality of image frame distance ranges, a plurality of image frame index ranges, and a plurality of motion estimation search ranges.

[0188] Optionally, the preset search range derivation relationship includes: a preset search range derivation lookup table, where the preset search range derivation lookup table includes: a plurality of preset search range derivation table entries, and the plurality of preset search range derivation table entries include: a first field, a second field, and a third field. The first field is the image frame distance field, which is used to record various distance information. The second field is the image frame index field corresponding to the image frame distance field, which is used to record various index information. The third field is the motion estimation search range field corresponding to both the image frame distance field and the image frame index field.

[0189] Optionally, the above-mentioned processor may also execute program code for the following steps: in response to the distance information being greater than or equal to a first distance and the index information being a first index, searching for the motion estimation search range of the second image frame from the preset search range derivation relationship to be a first range; in response to the distance information being less than the first distance and the index information being the first index, or the distance information being greater than or equal to a second distance and the index information being greater than or equal to a second index, searching for the motion estimation search range of the second image frame from the preset search range derivation relationship to be a second range; in response to the distance information being less than the second distance and the index information being greater than or equal to the second index, searching for the motion estimation search range of the second image frame from the preset search range derivation relationship to be a third range.

[0190] Optionally, the above-mentioned processor may also execute program code for the following steps: select a first image block from a first image frame; search for a target image block in a second image frame based on a motion estimation search range, where the target image block is an inter-frame prediction block corresponding to the first image block; perform inter-frame prediction on the first image block by using the target image block.

[0191] Optionally, the above-mentioned processor may also execute program code for the following steps: obtain a predicted value of a motion vector corresponding to the first image block; determine a second image block in the second image frame by using the predicted value of the motion vector, where the second image block is a reference image block; search for the target image block within the motion estimation search range centered on the second image block.

[0192] Optionally, by providing a graphical user interface on a terminal device, where the content displayed on the graphical user interface at least partially includes an image encoding scenario, the above-mentioned processor may also execute program code for the following steps: in response to a first control operation performed on the graphical user interface, input a relationship between a video to be processed and a preset search range derivation; in response to a second control operation performed on the graphical user interface, determine distance information and index information corresponding to the video to be processed, and based on the distance information and the index information, search for a motion estimation search range of a second image frame from the preset search range derivation; in response to a third control operation performed on the graphical user interface, encode the video to be processed according to the motion estimation search range to obtain an encoding result; display the encoding result within the graphical user interface.

[0193] By adopting the embodiment of the present application, by obtaining a video to be processed for video encoding, then determining the minimum reference frame distance between a current image frame and a reference image frame in the motion estimation process in the video to be processed, and the frame index of the reference image frame, and finally determining the motion estimation search range of the reference image frame according to the minimum reference frame distance and the reference frame index, an appropriate search range is adaptively determined, achieving the purpose of balancing between reducing computational complexity and maintaining encoding quality. Thus, the technical effects of improving the video encoding speed, improving the encoding efficiency, and reducing the loss of video compression performance while maintaining the video quality are realized, and further the technical problem in the related art that the search for motion estimation within a fixed search range results in a slow video encoding speed, low video compression performance, and high compression loss is solved.

[0194] Those of ordinary skill in the art can understand that Figure 8 the structure shown is only schematic, and the computer terminal A may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and terminal devices such as Mobile Internet Devices (MID), PAD, etc. Figure 8It does not limit the structure of the above electronic device. For example, computer terminal A may also include more or fewer components (such as network interfaces, display devices, etc.) than those shown in Figure 8 and may have a different configuration from that shown in Figure 8 .

[0195] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0196] Embodiment 5

[0197] An embodiment of the present application also provides a computer-readable storage medium. Optionally, in this embodiment, the above computer-readable storage medium may be used to store the program code executed by the method for determining the motion estimation search range provided in the first embodiment above.

[0198] Optionally, in this embodiment, the above computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0199] Optionally, in this embodiment, the computer-readable storage medium is set to store program code for performing the following steps: obtaining a video to be processed, where the video to be processed includes: a plurality of image frames, the plurality of image frames include: a first image frame and a second image frame, the first image frame is the current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame; determining distance information and index information corresponding to the video to be processed, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame; determining the motion estimation search range of the second image frame based on the distance information and the index information.

[0200] Optionally, in this embodiment, the computer-readable storage medium is set to store program code for performing the following steps: obtaining a first identifier and a second identifier, where the first identifier is the playback sequence identifier of the first image frame, and the second identifier is the playback sequence identifier of the second image frame; calculating distance information based on the first identifier and the second identifier.

[0201] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining a frame structure adopted by a plurality of image frames, where the frame structure includes: a frame type field and a frame playback order field; obtaining a first identifier and a second identifier based on the frame structure.

[0202] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: calculating a difference between the first identifier and the second identifier to obtain a calculation result; selecting the minimum absolute value from the calculation result to obtain distance information.

[0203] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: based on the distance information and the index information, searching for a motion estimation search range of a second image frame from a preset search range derivation relationship, where the preset search range derivation relationship is used to record the corresponding relationship among a plurality of image frame distance ranges, a plurality of image frame index ranges, and a plurality of motion estimation search ranges.

[0204] Optionally, the preset search range derivation relationship includes: a preset search range derivation lookup table, where the preset search range derivation lookup table includes: a plurality of preset search range derivation table entries, and the plurality of preset search range derivation table entries include: a first field, a second field, and a third field. The first field is an image frame distance field, which is used to record various distance information. The second field is an image frame index field corresponding to the image frame distance field, which is used to record various index information. The third field is a motion estimation search range field corresponding to both the image frame distance field and the image frame index field.

[0205] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: in response to the distance information being greater than or equal to a first distance and the index information being a first index, searching for a motion estimation search range of the second image frame from the preset search range derivation relationship as a first range; in response to the distance information being less than the first distance and the index information being the first index, or the distance information being greater than or equal to a second distance and the index information being greater than or equal to a second index, searching for a motion estimation search range of the second image frame from the preset search range derivation relationship as a second range; in response to the distance information being less than the second distance and the index information being greater than or equal to the second index, searching for a motion estimation search range of the second image frame from the preset search range derivation relationship as a third range.

[0206] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: selecting a first image block from a first image frame; searching for a target image block in a second image frame based on a motion estimation search range, where the target image block is an inter-frame prediction block corresponding to the first image block; and performing inter-frame prediction on the first image block by using the target image block.

[0207] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining a predicted motion vector value corresponding to a first image block; determining a second image block in a second image frame by using the predicted motion vector value, where the second image block is a reference image block; and searching for a target image block within a motion estimation search range centered on the second image block.

[0208] Optionally, a graphical user interface is provided by a terminal device, and the content displayed by the graphical user interface at least partially includes an image encoding scenario. In this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: in response to a first control operation performed on the graphical user interface, inputting a relationship between a video to be processed and a preset search range derivation; in response to a second control operation performed on the graphical user interface, determining distance information and index information corresponding to the video to be processed, and based on the distance information and the index information, searching for a motion estimation search range of a second image frame from the preset search range derivation; in response to a third control operation performed on the graphical user interface, encoding the video to be processed according to the motion estimation search range to obtain an encoding result; and displaying the encoding result within the graphical user interface.

[0209] The serial numbers of the embodiments of the present application above are for description purposes only and do not represent the advantages or disadvantages of the embodiments.

[0210] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0211] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are illustrative. For example, the division of the units is a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of units or modules can be in electrical or other forms.

[0212] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0213] In addition, each functional unit in various embodiments of the present application may be integrated into a processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0214] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs and other various media that can store program codes.

[0215] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for determining a motion estimation search range, characterized in that Including: Obtain a video to be processed, where the video to be processed includes: a plurality of image frames, and the plurality of image frames include: a first image frame and a second image frame. The first image frame is the current image frame in the video processing motion estimation process, and the second image frame is at least one reference image frame associated with the first image frame; Determine the distance information and index information corresponding to the video to be processed, where the distance information is the distance between the first image frame and the second image frame, and the index information is the frame index of the second image frame; Determine the motion estimation search range of the second image frame based on the distance information and the index information.

2. The method according to claim 1, wherein Determining the distance information includes: Obtain a first identifier and a second identifier, where the first identifier is the playback sequence identifier of the first image frame, and the second identifier is the playback sequence identifier of the second image frame; Calculate the distance information based on the first identifier and the second identifier.

3. The method according to claim 2, characterized in that, Obtaining the first identifier and the second identifier includes: Obtain the frame structure adopted by the plurality of image frames, where the frame structure includes: a frame type field and a frame playback sequence field; Obtain the first identifier and the second identifier based on the frame structure.

4. The method according to claim 2, wherein Calculating the distance information based on the first identifier and the second identifier includes: Perform a difference calculation on the first identifier and the second identifier to obtain a calculation result; Select the absolute value minimum from the calculation result to obtain the distance information.

5. The method according to claim 1, wherein Determining the motion estimation search range of the second image frame based on the distance information and the index information includes: Based on the distance information and the index information, look up the motion estimation search range of the second image frame from a preset search range derivation relationship, where the preset search range derivation relationship is used to record the corresponding relationship among a plurality of image frame distance ranges, a plurality of image frame index ranges, and a plurality of motion estimation search ranges.

6. The method according to claim 5, wherein The preset search range derivation relationship includes: a preset search range derivation lookup table, where the preset search range derivation lookup table includes: a plurality of preset search range derivation table entries, and the plurality of preset search range derivation table entries include: a first field, a second field, and a third field. The first field is an image frame distance field, and the first field is used to record multiple distance information. The second field is the image frame index field corresponding to the image frame distance field, and the second field is used to record multiple index information. The third field is the motion estimation search range field corresponding to both the image frame distance field and the image frame index field.

7. The method according to claim 6, characterized in that, Looking up the motion estimation search range of the second image frame from the preset search range derivation relationship based on the distance information and the index information includes: In response to the distance information being greater than or equal to a first distance and the index information being a first index, look up the motion estimation search range of the second image frame from the preset search range derivation relationship as a first range; In response to the distance information being less than the first distance and the index information being the first index, or the distance information being greater than or equal to the second distance and the index information being greater than or equal to the second index, find the motion estimation search range of the second image frame as the second range from the preset search range derivation relationship; In response to the distance information being less than the second distance and the index information being greater than or equal to the second index, find the motion estimation search range of the second image frame as the third range from the preset search range derivation relationship.

8. The method according to claim 1, characterized in that The method further includes: Select a first image block from the first image frame; Search for a target image block in the second image frame based on the motion estimation search range, where the target image block is an inter-frame prediction block corresponding to the first image block; Perform inter-frame prediction on the first image block using the target image block.

9. The method according to claim 8, characterized in that Searching for the target image block in the second image frame based on the motion estimation search range includes: Obtain the predicted motion vector value corresponding to the first image block; Determine a second image block in the second image frame using the predicted motion vector value, where the second image block is a reference image block; Search for the target image block within the motion estimation search range centered on the second image block.

10. The method according to claim 1, characterized in that, Provide a graphical user interface through a terminal device, and the content displayed by the graphical user interface at least partially includes an image coding scenario. The method includes: In response to a first control operation performed on the graphical user interface, input the video to be processed and the preset search range derivation relationship; In response to a second control operation performed on the graphical user interface, determine the distance information and the index information corresponding to the video to be processed, and based on the distance information and the index information, find the motion estimation search range of the second image frame from the preset search range derivation relationship; In response to a third control operation performed on the graphical user interface, encode the video to be processed according to the motion estimation search range to obtain an encoding result; Display the encoding result within the graphical user interface.

11. A method for determining a motion estimation search range, characterized in that: Obtain a network live video to be encoded, where the network live video to be encoded includes: a plurality of network live image frames to be encoded, and the plurality of network live image frames to be encoded include: a first network live image frame to be encoded and a second network live image frame to be encoded. The first network live image frame to be encoded is the current network live image frame to be encoded during the motion estimation process of network live video encoding, and the second network live image frame to be encoded is at least one reference image frame associated with the first network live image frame; Determine the distance information and the index information corresponding to the network live video to be encoded, where the distance information is the distance between the first network live image frame to be encoded and the second network live image frame to be encoded, and the index information is the frame index of the second network live image frame to be encoded; Determine the motion estimation search range of the second network live image frame to be encoded based on the distance information and the index information.

12. An electronic device, characterized in that, Comprising: A memory storing an executable program; A processor for running the program, wherein when the program runs, it executes the method for determining the motion estimation search range according to any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein when the executable program runs, it controls the device where the computer-readable storage medium is located to execute the method for determining the motion estimation search range according to any one of claims 1 to 11.

Images