Motion compensation method, main control board card and display device

Abstract

The invention belongs to the technical field of display devices, and provides a motion compensation method, a main control board card and a display device. The method comprises the following steps: receiving video data sent by a main control module; if the initial frame rate of the video data is smaller than the standard frame rate of the interface conversion module, motion compensation is carried out on the video data based on the standard frame rate, and the video data with the frame rate being the standard frame rate is obtained; and transmitting the video data obtained by motion compensation to an interface conversion module. For the video data of which the initial frame rate is smaller than the standard frame rate of the interface conversion module, motion compensation is performed before the interface conversion module, and the compensation target is that the frame rate of the video data is changed into the standard frame rate, so that the video data can be matched with the interface conversion module; the display device adopting the interface conversion module only supporting a certain fixed video frame rate can also utilize a motion compensation method to carry out image quality optimization, so that the display device has better image quality, and the low image quality problems of motion jitter, motion trailing, definition and the like are eliminated.

Description

Technical Field

[0001] The present application belongs to the technical field of display devices, and in particular, relates to a motion compensation method, a main control board and a display device. Background Art

[0002] The current display technology is developing rapidly, and the requirements for image quality are also increasing. When the video has problems such as motion jitter, motion tailing, and low clarity, it is necessary to optimize the video. For example, the use of motion estimation and motion compensation (MEMC) technology is a common video optimization method, and MEMC technology will inevitably bring about changes in the video frame rate.

[0003] The display industry has a wealth of video interfaces, and there is often a need for video interface conversion. However, some interface conversion modules can only support a certain fixed video frame rate. When the frame rate of the video does not match the frame rate of the interface conversion module, it will cause abnormal display. Therefore, for display devices that use this type of interface conversion module that only supports a fixed frame rate, in order to avoid abnormal display, image quality optimization is usually not performed. As a result, low-quality image problems such as motion jitter, motion tailing, and low clarity will occur. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a motion compensation method, a main control board and a display device, which can enable a display device using an interface conversion module that only supports a fixed video frame rate to have good image quality and eliminate low image quality problems such as motion jitter, motion smearing, and clarity.

[0005] A first aspect of an embodiment of the present application provides a motion compensation method, which is applied to a display device, wherein the display device includes a main control module, an interface conversion module, and a display module, wherein the interface conversion module is used to convert video data sent by the main control module into video data adapted to the display module, and the method includes:

[0006] Receiving video data sent by the main control module;

[0007] If the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, motion compensation is performed on the video data based on the standard frame rate to obtain video data with a frame rate of the standard frame rate;

[0008] The video data obtained by motion compensation is transmitted to the interface conversion module.

[0009] The embodiment of the present application receives the video data sent by the main control module; if the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, the video data is motion compensated based on the standard frame rate to obtain video data with a frame rate of the standard frame rate; and the video data obtained by motion compensation is transmitted to the interface conversion module. For video data with an initial frame rate less than the standard frame rate of the interface conversion module, motion compensation is performed before the interface conversion module, and the goal of compensation is to change the frame rate of the video data to the standard frame rate, so that it can be adapted to the interface conversion module, so that the display device using the interface conversion module that only supports a certain fixed video frame rate can also use the motion compensation method to optimize the image quality, so that the display device has better image quality, eliminating low image quality problems such as motion jitter, motion tailing, and clarity. And a certain optimization work is completed directly before the data enters the display device, which reduces the pressure on the terminal hardware, helps to extend its service life and maintain a good performance state.

[0010] In one embodiment, after receiving the video data sent by the main control module, the method further includes:

[0011] If the initial frame rate of the video data is equal to the standard frame rate of the interface conversion module, the video data is transmitted to the interface conversion module at the initial frame rate.

[0012] In the embodiment of the present application, if the initial frame rate of the video data is equal to the standard frame rate of the interface conversion module, the video data is transmitted to the interface conversion module at the initial frame rate. The video data whose initial frame rate is equal to the standard frame rate of the interface conversion module will not be changed in frame rate by the motion compensation method, thereby achieving selective compensation of the video data, thereby ensuring the adaptation of the frame rate of the video data to the interface conversion module and avoiding abnormal display.

[0013] In one embodiment, performing motion compensation on the video data based on the standard frame rate includes:

[0014] Based on the motion compensation condition satisfied by the video data, corresponding motion compensation is performed on the video data based on the standard frame rate.

[0015] The embodiment of the present application performs corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation conditions satisfied by the video data. A more suitable motion compensation method can be selected for different video data to perform compensation, thereby reducing the amount of data required for calculation and reducing the calculation load while ensuring the compensation effect.

[0016] In one embodiment, performing corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation condition satisfied by the video data includes:

[0017] Identify moving subjects in the video;

[0018] If there are multiple moving subjects and their distribution areas are scattered, then each adjacent frame image in the video data is compared to obtain the pixel point change state between two adjacent frame images;

[0019] Based on the pixel point change state, a motion trajectory of each moving subject between two adjacent frames of images is obtained;

[0020] generating a compensation image based on the motion trajectory of each moving subject;

[0021] Taking the standard frame rate as the target frame rate of the motion compensation, at least one compensation image is inserted between every two adjacent frames of images to compensate the entire picture.

[0022] The embodiment of the present application identifies the moving subject in the video; if there are multiple moving subjects and the distribution area is dispersed, then each adjacent frame image in the video data is compared to obtain the pixel point change state between two adjacent frames; based on the pixel point change state, the motion trajectory of each moving subject between two adjacent frames is obtained; based on the motion trajectory of each moving subject, a compensation image is generated; with the standard frame rate as the target frame rate of the motion compensation, at least one compensation image frame is inserted between every two adjacent frames to compensate the entire picture. In the case where there are multiple moving subjects and the distribution area is dispersed, the motion compensation operation is performed based on the pixel point change state of the entire image, so that all moving subjects in the entire image can achieve motion compensation, ensuring a better effect of motion compensation.

[0023] In one embodiment, performing corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation condition satisfied by the video data includes:

[0024] Identify moving subjects in the video;

[0025] If the activity range of the moving subject is within a local area in the picture, then the local areas of each adjacent frame image in the video data are compared to obtain the pixel point change state in the local area between two adjacent frames of image;

[0026] Based on the pixel point change state, a motion trajectory of a moving subject in a local area between two adjacent frames of images is obtained;

[0027] The standard frame rate is used as the target frame rate of the motion compensation, and at least one compensation image is inserted between every two frames of images to compensate for a local area in the picture.

[0028] The embodiment of the present application identifies the moving subject in the video; if the moving subject's activity range is within a local area in the picture, then the local areas of each adjacent frame image in the video data are compared to obtain the pixel point change state in the local area between two adjacent frames; based on the pixel point change state, the motion trajectory of the moving subject in the local area between two adjacent frames is obtained; with the standard frame rate as the target frame rate of the motion compensation, at least one frame of compensation image is inserted between every two frames to compensate for the local area in the picture. When the moving subject's activity range is relatively concentrated, only the moving subject in the local area is compensated, thereby ensuring a better motion compensation effect, without the need to perform motion compensation operations on the data in the rest of the picture area, thereby reducing the amount of data to be calculated and reducing the computational load.

[0029] In one embodiment, performing corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation condition satisfied by the video data includes:

[0030] Identify moving subjects in the video;

[0031] If the moving subject is one, then comparing the moving subjects of each adjacent frame image in the video data to obtain the pixel point change state of the moving subject between two adjacent frame images;

[0032] Based on the pixel point change state, a motion trajectory of the moving subject between two adjacent frames of images is obtained;

[0033] The standard frame rate is used as the target frame rate for motion compensation, and at least one compensation image is inserted between every two frames of images to compensate for the moving subject in the picture.

[0034] The embodiment of the present application identifies the moving subject in the video; if the moving subject is one, then the moving subjects of each adjacent frame image in the video data are compared to obtain the pixel point change state of the moving subject between two adjacent frames; based on the pixel point change state, the motion trajectory of the moving subject between two adjacent frames is obtained; with the standard frame rate as the target frame rate of the motion compensation, at least one frame of compensation image is inserted between every two frames to compensate the moving subject in the picture. When there is only one moving subject, only the moving subject is calculated and compensated, thereby ensuring that the best motion compensation effect is achieved, and there is no need to perform motion compensation calculations on the remaining data other than the moving subject, thereby reducing the amount of data to be calculated and reducing the calculation load.

[0035] In one embodiment, it further includes:

[0036] When n frames of compensation images are inserted between every two frames of images, the compensation image of the i-th frame is obtained based on the i-th n+1 equally divided point of the motion trajectory;

[0037] Wherein, 0<i≤n, i is an integer, and n is an integer not less than 1.

[0038] In the embodiment of the present application, when n frames of compensation images are inserted between every two frames of images, the i-th frame of the compensation image is obtained based on the i-th n+1 equally divided points of the motion trajectory; wherein 0<i≤n, i is an integer, and n is an integer not less than 1. The compensation image used in the interpolation process is located in the middle of the motion trajectory, which can achieve a more natural transition visually and further improve the effect of motion compensation on improving image quality.

[0039] A second aspect of an embodiment of the present application provides a main control board of a display device, the main control board comprising the main control module and the motion compensation module;

[0040] The main control module is configured to send video data to the motion compensation module;

[0041] The motion compensation module is configured to execute the motion compensation method provided in the first aspect of the embodiment of the present application.

[0042] A third aspect of the embodiments of the present application provides a display device, comprising a main control board and a display module as provided in the second aspect of the embodiments of the present application.

[0043] The motion compensation method provided in the first aspect of the embodiment of the present application is applied to a display device, wherein the display device includes a main control module, an interface conversion module and a display module, wherein the interface conversion module is used to convert the video data sent by the main control module into video data adapted to the display module, and receives the video data sent by the main control module; if the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, the video data is motion compensated based on the standard frame rate to obtain video data with a frame rate of the standard frame rate; and the video data obtained by the motion compensation is transmitted to the interface conversion module. For video data with an initial frame rate less than the standard frame rate of the interface conversion module, motion compensation is performed before the interface conversion module, and the goal of the compensation is to change the frame rate of the video data to the standard frame rate, so that it can be adapted to the interface conversion module, so that a display device using an interface conversion module that only supports a certain fixed video frame rate can also use the motion compensation method to optimize the image quality, so that the display device has better image quality, eliminating low image quality problems such as motion jitter, motion tailing, and clarity. Moreover, certain optimization work is completed directly before the data enters the display device, which reduces the pressure on the terminal hardware, helps to extend its service life and maintain good performance status.

[0044] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 This is a flow chart of a motion compensation method provided by an embodiment of the present application;

[0047] Figure 2 is a flow chart of a motion compensation method provided by another embodiment of the present application;

[0048] Figure 3 is a schematic diagram of a motion compensation area of ​​a motion compensation method provided by an embodiment of the present application;

[0049] Figure 4 is a flow chart of a motion compensation method provided by another embodiment of the present application;

[0050] Figure 5 is a schematic diagram of a motion compensation area of ​​a motion compensation method provided by another embodiment of the present application;

[0051] Figure 6 is a flow chart of a motion compensation method provided by another embodiment of the present application;

[0052] Figure 7 is a schematic diagram of a motion compensation area of ​​a motion compensation method provided by another embodiment of the present application;

[0053] Figure 8 It is a structural schematic diagram of the motion compensation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0058] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0060] The motion compensation method provided in the embodiment of the present application is applied to a display device, which includes a main control module, an interface conversion module and a display module. The interface conversion module is used to convert the video data sent by the main control module into video data adapted to the display module, and can be executed by the display device when running a computer program with corresponding functions, by receiving the video data sent by the main control module; if the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, the video data is motion compensated based on the standard frame rate to obtain video data with a frame rate of the standard frame rate; and the video data obtained by motion compensation is transmitted to the interface conversion module. The display device using an interface conversion module that only supports a certain fixed video frame rate can also have good image quality, eliminating low image quality problems such as motion jitter, motion tailing, and low definition.

[0061] In the application, the display device includes but is not limited to an LED display screen based on a sending card and a receiving card architecture; and display devices such as an LCD display screen, etc. The embodiments of the present application do not impose any restrictions on the specific type of the display device.

[0062] In the application, the interface conversion module is used to convert the video data in the first format sent by the main control module into the video data in the second format adapted to the display screen. And the frame rate (i.e., the standard frame rate) supported by the interface conversion module is a fixed frame rate. For example, the interface conversion module can be a VBO interface to HDMI interface module, which is used to convert the data in V-by-0neHS (VBO) format into the data in HDMI2.0 format. The standard frame rate of the interface conversion module can be 60Hz or 120Hz.

[0063] In applications, the interface conversion module can be an IT6265 chip.

[0064] like Figure 1 As shown, the motion compensation method provided in the embodiment of the present application includes the following steps S101 to S103:

[0065] Step S101: receiving video data sent by a main control module.

[0066] In the application, the original video data sent by the main control module includes a series of continuous image frames, each of which represents the picture content at a certain moment. The video data may come from various sources, such as broadcast signals, network streaming services, etc., and may have different initial frame rates, such as 30Hz, 60Hz, etc.

[0067] Step S102: If the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, motion compensation is performed on the video data based on the standard frame rate to obtain video data with a frame rate of the standard frame rate.

[0068] In the application, when the frame rate of the input video is detected to be lower than the target frame rate to be output to the interface conversion module, the motion compensation operation is performed. In this process, the motion compensation algorithm analyzes the difference between adjacent frames to estimate the movement law of the object, and then creates a new intermediate frame based on the movement law information to fill the gap in the original frame sequence. This increases the effective frame rate of the final video stream to match the target frame rate. For example, a movie with a frame rate of 30Hz is converted into video data with a frame rate of 60Hz after motion compensation.

[0069] Step S103: Transmit the video data obtained by motion compensation to the interface conversion module.

[0070] In the application, after completing the above processing, the new video stream that has been appropriately adjusted will be sent to the next link, the interface conversion module. The interface conversion module is used to convert the processed digital signal into a form suitable for output of a specific physical connection type (such as HDMI, DisplayPort, etc.), ensuring that it can be correctly received and displayed by the target display device.

[0071] The embodiment of the present application receives the video data sent by the main control module; if the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, the video data is motion compensated based on the standard frame rate to obtain video data with a frame rate of the standard frame rate; and the video data obtained by the motion compensation is transmitted to the interface conversion module. For video data with an initial frame rate less than the standard frame rate of the interface conversion module, motion compensation is performed before the interface conversion module, and the goal of the compensation is to change the frame rate of the video data to the standard frame rate, so that it can be adapted to the interface conversion module, so that the display device using the interface conversion module that only supports a certain fixed video frame rate can also use the motion compensation method to optimize the image quality, so that the display device has better image quality, eliminating low image quality problems such as motion jitter, motion tailing, and clarity. Allowing content from different sources and different specifications to be well presented under a unified standard increases the adaptability and convenience in multimedia applications. And a certain optimization work is completed directly before the data enters the display device, which reduces the pressure on the terminal hardware, helps to extend its service life and maintain a good performance state.

[0072] In one embodiment, after step S101, the following step S201 is further included:

[0073] Step S201: If the initial frame rate of the video data is equal to the standard frame rate of the interface conversion module, the video data is transmitted to the interface conversion module at the initial frame rate.

[0074] In the application, when the standard frame rate of the interface conversion module is 30Hz, if the initial frame rate is less than the standard frame rate, such as video data with a frame rate of 30Hz, motion compensation is performed according to step S102 to increase the frame rate of the video to 60Hz while optimizing the image quality. If the initial frame rate is equal to the standard frame rate, such as video data with a frame rate of 60Hz, the original video data is directly transmitted to the interface conversion module.

[0075] In the embodiment of the present application, if the initial frame rate of the video data is equal to the standard frame rate of the interface conversion module, the video data is transmitted to the interface conversion module at the initial frame rate. The video data whose initial frame rate is equal to the standard frame rate of the interface conversion module will not be changed in frame rate by the motion compensation method, thereby achieving selective compensation of the video data, thereby ensuring the adaptation of the frame rate of the video data to the interface conversion module and avoiding abnormal display.

[0076] In one embodiment, performing motion compensation on video data based on a standard frame rate includes the following step S301:

[0077] Step S301 : Based on the motion compensation conditions satisfied by the video data, corresponding motion compensation is performed on the video data based on a standard frame rate.

[0078] In applications, the distribution of moving subjects can be distinguished, and different motion compensation methods can be used according to the distribution of moving subjects. This can not only improve the quality of video processing, but also effectively manage computing resources, thereby providing users with a better visual experience.

[0079] The embodiment of the present application performs corresponding motion compensation on the video data based on the motion compensation conditions satisfied by the video data and based on the standard frame rate. A more suitable motion compensation method can be selected for different video data to perform compensation, thereby reducing the amount of data required for calculation and reducing the calculation load while ensuring the compensation effect.

[0080] In one embodiment, Figure 2 As shown, step S301 includes the following steps S311 to S315:

[0081] Step S311: Identify the moving subject in the video.

[0082] In applications, image processing technology can be used to identify moving subjects in video frames. For example, background subtraction, optical flow, feature point detection, deep learning and other methods can be used to identify moving subjects in videos, providing a basis for subsequent processing.

[0083] Step S312: If there are multiple moving subjects and their distribution areas are dispersed, then each adjacent frame image in the video data is compared to obtain the pixel point change state between two adjacent frame images.

[0084] In the application, if there are many subjects in the video and the distribution area of ​​the subjects is not concentrated, for example, a group of animals run quickly across the entire screen, then the data of the entire screen is included in the calculation, that is, global motion compensation is performed, such as Figure 3 As shown. A pixel-by-pixel method can be used to compare two adjacent frames to identify which pixels have changed. For example, the difference method, that is, calculating the difference in the pixel value of the corresponding position between the current frame and the previous frame, can obtain the pixel change state between two adjacent frames. By comparing each adjacent frame, the pixel change state between each two adjacent frames can be obtained.

[0085] Step S313: based on the pixel change state, obtain the motion trajectory of each moving subject between two adjacent frames of images.

[0086] In the application, based on the pixel change state obtained in the previous step, the specific motion path of each moving subject is further analyzed. For example, the motion vector clustering method can be used to cluster similar optical flow vectors to form the motion trajectory of each moving subject. The multi-target tracking method can also be used to track the position of each moving subject in different frames in combination with time continuity. Through the above method, the motion path of each moving subject between two adjacent frames (i.e., one frame interval) can be clearly depicted.

[0087] Step S314: Generate a compensated image based on the motion trajectory of each moving subject.

[0088] In the application, an intermediate compensation frame, i.e., a compensation image, is generated according to the motion trajectory of each moving subject. Specifically, a linear interpolation method can be used to generate a new frame based on the position and speed of adjacent frames. A nonlinear interpolation method can also be used to use more complex interpolation methods, such as B-spline curves or cubic spline interpolation, to obtain a smoother effect. A new frame synthesis method can also be used, that is, a new frame is synthesized based on the motion trajectory and image content, rather than simply interpolating. When synthesizing a new frame, the midpoint or the remaining equally divided points in the motion trajectory can be selected as the pixel position to generate a new compensation image, i.e., a synthesized new frame.

[0089] Step S315 , using the standard frame rate as the target frame rate for motion compensation, inserting at least one compensation image frame between every two adjacent image frames to compensate the entire picture.

[0090] In the application, the generated compensation frame (compensation image) is inserted into the original video sequence by inserting at least one compensation image frame between every two adjacent frames, so that the final output video reaches the required standard frame rate. For example, if the frame rate of the original video is 30Hz and the target frame rate is 60Hz, then a compensation frame is inserted between every two original frames. When inserting the compensation frame, the compensation frame is inserted at a predetermined interval and the timestamp of the video is updated to ensure that all frames are evenly distributed on the timeline.

[0091] The embodiment of the present application identifies the moving subject in the video; if there are multiple moving subjects and the distribution area is dispersed, then each adjacent frame image in the video data is compared to obtain the pixel point change state between the two adjacent frames; based on the pixel point change state, the motion trajectory of each moving subject between the two adjacent frames is obtained; based on the motion trajectory of each moving subject, a compensation image is generated; with the standard frame rate as the target frame rate of motion compensation, at least one frame of compensation image is inserted between each two adjacent frames to compensate the entire picture. For the case where there are multiple moving subjects and the distribution area is dispersed, the motion compensation operation is performed based on the pixel point change state of the entire image, so that all moving subjects in the whole image can achieve motion compensation, ensuring the best effect of motion compensation. Through detailed motion analysis and compensation image generation, the overall image quality of the video is improved. And this method is applicable to a variety of different scenes, whether it is sports events or other dynamic content, it can provide good results. Reduce motion blur and enhance the viewing experience, especially on large screens or high-definition display devices.

[0092] In one embodiment, Figure 4 As shown, step S301 includes the following steps S321 to S324:

[0093] Step S321: Identify the moving subject in the video.

[0094] In applications, image processing techniques can be used to identify moving subjects in video frames whose motion trends are relatively stable and concentrated in a certain area. This usually includes background subtraction, optical flow, feature point detection (such as SIFT, SURF or ORB) and deep learning methods (such as convolutional neural networks (CNNs). These technologies can accurately locate and track moving objects in videos to ensure the accuracy of subsequent processing.

[0095] Step S322: If the activity range of the moving subject is within a local area in the picture, then compare the local areas of each adjacent frame image in the video data to obtain the pixel point change state in the local area between two adjacent frame images.

[0096] In the application, when the moving subject is identified to be concentrated in a local area of ​​the picture, the system only needs to perform a detailed comparison on the local area. Figure 5 As shown, when the moving subject is concentrated on the left side of the picture, only a part of the area on the left side of the center line of the picture is compared, and the right side area is not compensated. When making comparisons, a differential image can be generated by calculating the pixel difference between adjacent frames in the local area, or the optical flow vector between adjacent frames in the local area can be calculated to represent the movement direction and speed of each pixel. The local area can also be divided into small blocks, and the position changes of these blocks between adjacent frames can be compared. In the above manner, the pixel change state in the local area between two adjacent frames can be obtained.

[0097] Step S323: Based on the pixel change state, obtain the motion trajectory of the moving subject in the local area between two adjacent frame images.

[0098] In the application, the pixel change state of the local area is obtained to further analyze the specific motion path of each moving subject. Specifically, the motion vector clustering method can be used to cluster similar optical flow vectors to form the motion trajectory of each moving subject. Alternatively, the Kalman filter method can be used to smooth and predict the motion trajectory to improve the stability of trajectory estimation. The multi-target tracking method can also be used to track the position of each moving subject in different frames in combination with time continuity. Through the above methods, the motion trajectory of the moving subject in the local area between two adjacent frames can be obtained.

[0099] Step S324: Using the standard frame rate as the target frame rate for motion compensation, insert at least one compensation image frame between every two image frames to compensate for a local area in the picture.

[0100] In the application, according to the motion trajectory of each moving subject in the local area, the system generates an intermediate compensation frame and inserts it into the original video sequence. Linear interpolation can be used to generate new frames based on the position and speed of adjacent frames. Nonlinear interpolation can also be used to use more complex interpolation methods, such as B-spline curves or cubic spline interpolation, to obtain a smoother effect. New frames can also be synthesized, that is, based on the motion trajectory and image content, a completely new frame is synthesized instead of simply interpolating. The synthesized new frame only performs corresponding operations and motion compensation in the local area, while the data of the previous frame is maintained for the part outside the local area.

[0101] The embodiment of the present application identifies the moving subject in the video; if the range of the moving subject is within a local area in the picture, then the local areas of each adjacent frame image in the video data are compared to obtain the pixel point change state in the local area between the two adjacent frames; based on the pixel point change state, the motion trajectory of the moving subject in the local area between the two adjacent frames is obtained; with the standard frame rate as the target frame rate of motion compensation, at least one frame of compensation image is inserted between every two frames to compensate the local area in the picture. When the range of the moving subject is relatively concentrated, only the moving subject in the local area is compensated, thereby ensuring the premise of better motion compensation effect, there is no need to perform motion compensation calculation on the data in the rest of the picture area, reducing the amount of data to be calculated and reducing the computing load. This method avoids unnecessary processing of the rest of the static parts of the picture, thereby improving processing efficiency and resource utilization. This localized processing strategy is particularly suitable for scenes where the moving subject is concentrated in a small range, such as news broadcasts, talk shows, etc., and can optimize the performance of the system while maintaining a high-quality visual experience.

[0102] In one embodiment, Figure 6 As shown, step S301 includes the following steps S331 to S334:

[0103] Step S331: Identify the moving subject in the video.

[0104] In applications, image processing techniques can be used to identify single moving subjects in video frames. This usually includes background subtraction, optical flow, feature point detection (such as SIFT, SURF or ORB), and deep learning methods (such as convolutional neural networks (CNNs)), which can accurately locate and track moving objects in videos to ensure the accuracy of subsequent processing.

[0105] Step S332: If there is only one moving subject, then compare the moving subjects of each adjacent frame image in the video data to obtain the pixel point change state of the moving subject between two adjacent frame images.

[0106] In applications, such as Figure 7 As shown, once it is identified that there is only one main moving subject in the video, such as a fast-moving vehicle, the system only focuses on the changes of the moving subject (i.e., the vehicle) between adjacent frames, without compensating for parts outside the moving subject. By comparing pixel by pixel, the system calculates the pixel change state of the moving subject in two adjacent frames. Specifically, differential image analysis and optical flow algorithms can be used. These methods can capture subtle changes in the moving subject, including position movement, shape changes, etc., thereby providing detailed information for the subsequent generation of accurate motion trajectories.

[0107] Step S333: based on the pixel change state, obtain the motion trajectory of the moving subject between two adjacent frames of images.

[0108] In the application, after obtaining the pixel change state, further analyze and determine the specific motion trajectory of the single moving subject between two adjacent frames. Motion vector clustering, Kalman filtering or multi-target tracking techniques can be used to smooth and predict the motion trajectory and improve the stability of trajectory estimation. Through these methods, the continuous path of the moving subject between different frames can be accurately depicted, ensuring that the generated compensated image can truly reflect the actual motion of the moving subject.

[0109] Step S334: Using the standard frame rate as the target frame rate for motion compensation, insert at least one compensation image frame between every two frames of images to compensate for the moving subject in the picture.

[0110] In the application, according to the determined motion trajectory, at least one compensation image is inserted between every two frames to achieve a predetermined standard frame rate. The generation of compensation images can be done by linear interpolation, nonlinear interpolation (such as B-spline curve or cubic spline interpolation) or synthesis of new frames. These compensation images can fill the gaps between the original frames, so that the moving subject can show a smoother and smoother motion effect in the final output video, thereby improving the overall viewing experience. The compensation image only performs motion compensation for the moving subject, and for the rest of the moving subject, the data of the previous frame is used.

[0111] The embodiment of the present application identifies the moving subject in the video; if there is only one moving subject, then the moving subjects of each adjacent frame image in the video data are compared to obtain the pixel point change state of the moving subject between two adjacent frames of images; based on the pixel point change state, the motion trajectory of the moving subject between two adjacent frames of images is obtained; with the standard frame rate as the target frame rate for motion compensation, at least one compensation image is inserted between every two frames of images to compensate for the moving subject in the picture. When there is only one moving subject, only the moving subject is calculated and compensated, thereby ensuring that the best motion compensation effect is achieved without the need to perform motion compensation calculations on the remaining data other than the moving subject, thereby reducing the amount of data that needs to be calculated and reducing the computing load. For single-person action clips, game recordings, etc., this method can optimize system performance and resource utilization while maintaining the best visual experience.

[0112] In one embodiment, it further includes:

[0113] When n frames of compensation images are inserted between every two frames of images, the i-th frame of compensation image is obtained based on the i-th n+1 equally divided point of the motion trajectory;

[0114] Wherein, 0<i≤n, i is an integer, and n is an integer not less than 1.

[0115] In the application, assuming that n frames of compensation images need to be inserted between two adjacent frames, the entire motion trajectory will be divided into n+1 segments. For example, if n=2, the motion trajectory will be divided into 3 segments, corresponding to three equally divided points. For each equally divided point, the position of the point can be determined by linear interpolation or nonlinear interpolation (such as B-spline curve). Specifically, if the motion trajectory is represented by a series of coordinate points, the coordinates of each equally divided point can be calculated by interpolation between these points.

[0116] In the application, when the target frame rate is 60Hz and the initial frame rate of the video data is 30Hz, only one compensation image needs to be inserted between two adjacent frames. When the target frame rate is 90Hz and the initial frame rate of the video data is 30Hz, two compensation images need to be inserted between two adjacent frames.

[0117] Let's take a specific scenario to illustrate:

[0118] There are two original images: frame A (starting frame) and frame B (ending frame). Two compensation images (i.e., n=2) are inserted between these two frames, so that the final sequence is A→C1→C2→B, and the moving object moves from the position of frame A to the position of frame B. Assuming that the motion trajectory is a straight line, find the positions of three equally divided points on this line: the first equally divided point (the position of C1) is located at 1 / 3 between A and B, and the second equally divided point (the position of C2) is located at 2 / 3 between A and B. If the position coordinates of A are (x A ,y A ), the position coordinates of B are (x B ,y B ), then the positions of C1 and C2 can be calculated by a simple linear interpolation formula: C1 position coordinates = ((2x A +x B ) / 3,(2y A +y B ) / 3), the position coordinates of C2 = ((x A +2x B ) / 3,(y A +2y B) / 3). After calculating the positions of C1 and C2, new images C1 and C2 are synthesized based on these two position information and the contents of frames A and B. At this time, for the static background part, the corresponding area in frame A or frame B can be directly copied. For the moving subject, it is rendered based on its state at the positions of C1 and C2. Shape deformation, texture mapping and other technologies can be combined to ensure that the moving object looks like it is moving naturally along the predetermined trajectory. In the end, a series of continuous image sequences are obtained: A→C1→C2→B. The viewer will see a smoother and smoother motion effect because the newly added compensation frames fill the visual gaps between the original frames and reduce the sense of jump.

[0119] In the embodiment of the present application, when n frames of compensation images are inserted between every two frames of images, the i-th frame of compensation image is obtained based on the i-th n+1 equally divided points of the motion trajectory; wherein 0<i≤n, i is an integer, and n is an integer not less than 1. The compensation image used in the interpolation process is located in the middle of the motion trajectory, which can achieve a more natural transition visually and further improve the effect of motion compensation on improving image quality.

[0120] In one embodiment, after receiving the video data sent by the main control module, the method further includes:

[0121] Perform image quality processing and playback control on video data.

[0122] In the application, the image quality processing and playback control processing of the video data include but are not limited to scaling the video to adapt to the resolution of different display devices, adjusting the color temperature and brightness of the image to optimize the visual effect, and playback control such as fast forward, pause and other functions.

[0123] The embodiment of the present application first performs image quality processing and playback control processing on the video data after receiving the video data sent by the main control module. This preprocessing step ensures that the video has reached the best image quality before motion compensation is performed, so that the final output video is not only smooth but also has a high-quality visual experience. By performing image quality processing first, unnecessary image quality loss introduced during the motion compensation process can be avoided. At the same time, the optimized video data is also easier to perform efficient motion estimation and compensation calculations, thereby improving the overall processing efficiency. In addition, the playback control processing enables users to control video playback more flexibly, enhancing the user experience. While ensuring high-quality output, this method can also effectively utilize system resources, improve processing speed and responsiveness, and is particularly suitable for application scenarios that require real-time processing and high-quality output.

[0124] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0125] The present application also provides a motion compensation device for executing the steps in the above motion compensation method embodiment. The motion compensation device can be a virtual appliance in the display device, which is run by a processor of a main control board of the display device, or it can be the display device itself.

[0126] like Figure 8 As shown, the motion compensation device 100 provided in the embodiment of the present application includes:

[0127] The data receiving module 101 is used to receive the video data sent by the main control module;

[0128] The motion compensation module 102 performs motion compensation on the video data based on the standard frame rate to obtain video data with a frame rate of the standard frame rate if the initial frame rate of the video data is less than the standard frame rate of the interface conversion module;

[0129] The data sending module 103 transmits the video data obtained by motion compensation to the interface conversion module.

[0130] The embodiment of the present application sets a motion compensation device, including a data receiving module for receiving video data sent by the main control module; a motion compensation module, if the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, the video data is motion compensated based on the standard frame rate to obtain video data with a frame rate of the standard frame rate; a data sending module, the video data obtained by the motion compensation is transmitted to the interface conversion module. For video data whose initial frame rate is less than the standard frame rate of the interface conversion module, motion compensation is performed before the interface conversion module, and the goal of the compensation is to change the frame rate of the video data to the standard frame rate, so that it can be adapted to the interface conversion module, so that the display device using the interface conversion module that only supports a certain fixed video frame rate can also use the motion compensation method to optimize the image quality, so that the display device has better image quality, eliminating low image quality problems such as motion jitter, motion tailing, and clarity. And a certain optimization work is completed directly before the data enters the display device, which reduces the pressure on the terminal hardware, helps to extend its service life and maintain a good performance state.

[0131] In applications, each module in the motion compensation device may be a software program module, or may be implemented by different logic circuits integrated in a processor, or may be implemented by multiple distributed processors.

[0132] The embodiment of the present application also provides a main control board of a display device, the main control board comprising a main control module and a motion compensation module;

[0133] The main control module is configured to send video data to the motion compensation module;

[0134] The motion compensation module is configured to execute the motion compensation method provided in the first aspect of the embodiment of the present application.

[0135] An embodiment of the present application further provides a display device, comprising a main control board and a display module as provided in the second aspect of the embodiment of the present application.

[0136] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0137] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0138] An embodiment of the present application provides a computer program product. When the computer program product runs on a main control board of a display device, the main control board can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0140] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0142] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0143] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0144] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A motion compensation method, characterized in that: Applied to a display device, the display device includes a main control module, an interface conversion module and a display module, the interface conversion module is used to convert video data sent by the main control module into video data adapted to the display module, the method includes: Receiving video data sent by the main control module; If the initial frame rate of the video data is less than the standard frame rate of the interface conversion module, motion compensation is performed on the video data based on the standard frame rate to obtain video data with a frame rate of the standard frame rate; The video data obtained by motion compensation is transmitted to the interface conversion module.

2. The motion compensation method according to claim 1, characterized in that: After receiving the video data sent by the main control module, the method further includes: If the initial frame rate of the video data is equal to the standard frame rate of the interface conversion module, the video data is transmitted to the interface conversion module at the initial frame rate.

3. The motion compensation method according to claim 1, wherein: The performing motion compensation on the video data based on the standard frame rate includes: Based on the motion compensation condition satisfied by the video data, corresponding motion compensation is performed on the video data based on the standard frame rate.

4. The motion compensation method according to claim 3, characterized in that: The performing corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation condition satisfied by the video data includes: Identify moving subjects in the video; If there are multiple moving subjects and their distribution areas are scattered, then each adjacent frame image in the video data is compared to obtain the pixel point change state between two adjacent frame images; Based on the pixel point change state, a motion trajectory of each moving subject between two adjacent frames of images is obtained; generating a compensation image based on the motion trajectory of each moving subject; Taking the standard frame rate as the target frame rate of the motion compensation, at least one compensation image is inserted between every two adjacent frames of images to compensate the entire picture.

5. The motion compensation method according to claim 3, characterized in that: The performing corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation condition satisfied by the video data includes: Identify moving subjects in the video; If the activity range of the moving subject is within a local area in the picture, then the local areas of each adjacent frame image in the video data are compared to obtain the pixel point change state in the local area between two adjacent frames of image; Based on the pixel point change state, a motion trajectory of a moving subject in a local area between two adjacent frames of images is obtained; The standard frame rate is used as the target frame rate of the motion compensation, and at least one compensation image is inserted between every two frames of images to compensate for a local area in the picture.

6. The motion compensation method according to claim 3, characterized in that: The performing corresponding motion compensation on the video data based on the standard frame rate based on the motion compensation condition satisfied by the video data includes: Identify moving subjects in the video; If the moving subject is one, then comparing the moving subjects of each adjacent frame image in the video data to obtain the pixel point change state of the moving subject between two adjacent frame images; Based on the pixel point change state, a motion trajectory of the moving subject between two adjacent frames of images is obtained; The standard frame rate is used as the target frame rate for motion compensation, and at least one compensation image is inserted between every two frames of images to compensate for the moving subject in the picture.

7. The motion compensation method according to any one of claims 4 to 6, characterized in that: Also includes: When n frames of compensation images are inserted between every two frames of images, the compensation image of the i-th frame is obtained based on the i-th n+1 equally divided point of the motion trajectory; Wherein, 0<i≤n, i is an integer, and n is an integer not less than 1.

8. The motion compensation method according to claim 1, characterized in that: After receiving the video data sent by the main control module, the method further includes: The video data is subjected to image quality processing and playback control processing.

9. A main control board of a display device, characterized in that: The main control board includes a main control module and a motion compensation module; The main control module is configured to send video data to the motion compensation module; The motion compensation module is configured to execute the motion compensation method according to any one of claims 1 to 8.

10. A display device, characterized in that: It comprises the main control board and the display module as claimed in claim 9.

Images