Video resolution improving method and device, equipment and storage medium

Abstract

The invention relates to a video resolution improving method and device, equipment and a storage medium. The method comprises the following steps: receiving low-resolution video data sent by a server; for each video frame in the video data, whether the video frame is a target video frame is determined, and the target video frame is determined every interval frame number; when the video frame is the target video frame, determining a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model; and performing image rendering based on the high-resolution video frame and the non-target video frame. The method can reduce the loss of the client and improve the operation efficiency of the client.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for improving video resolution. Background Art

[0002] In real life, the clarity of a video is related to its resolution, that is, the clearer the video, the higher its corresponding resolution. High-resolution videos are clearer and bring a different experience to users.

[0003] In the related art, when a user wants to play a super-resolution video on a client, the user needs to trigger the super-resolution video playback operation on the client, so that the client sends a video request message to the server, and the video request message is used to request the server to send high-resolution video data. When the server receives the video request message, the server processes the pre-stored low-resolution video to obtain high-resolution video data, and sends the video data to the client so that the client can play it based on the video data.

[0004] Since high-resolution videos are very large in size, they will greatly occupy network bandwidth resources during data transmission. If the client is required to process low-resolution video data, the processing involves a large amount of data processing, which will cause high power consumption and low client operation efficiency. Therefore, a method for improving video high resolution is urgently needed. Summary of the invention

[0005] The present application provides a method, apparatus, device and storage medium for improving video resolution, which can reduce the loss of the client and improve the operating efficiency of the client.

[0006] In a first aspect, the present application provides a method for improving video resolution, the method comprising:

[0007] Receive low-resolution video data sent by the server;

[0008] For each video frame in the video data, determining whether the video frame is a target video frame, wherein the target video frame is determined by the number of frames per interval;

[0009] When the video frame is the target video frame, determining a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model;

[0010] Image rendering is performed based on the high-resolution video frame and the non-target video frame.

[0011] Optionally, determining a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model includes:

[0012] Input the target video frame into a trained residual calculation model to obtain a residual result;

[0013] The residual result is added to the target video frame to determine a high-resolution video frame corresponding to the video frame.

[0014] Optionally, performing image rendering based on the high-resolution video frame and the non-target video frame includes:

[0015] Performing image processing on the high-resolution video frames and the non-target video frames to obtain processed video frames, wherein the image processing includes color enhancement and sharpening processing;

[0016] Performing image rendering based on the processed video frame;

[0017] Optionally, before determining, for each video frame in the video data, whether the video frame is a target video frame, the method further includes:

[0018] Get the related properties related to the interval frame number;

[0019] According to the preset corresponding relationship between the relevant attribute and the interval frame number, determining the interval frame number corresponding to the relevant attribute;

[0020] In the video frame data, video frames of each interval frame number are determined as target video frames, and other video frames are determined as non-target video frames.

[0021] Optionally, for each video frame in the video data, determining whether the video frame is a target video frame includes:

[0022] For each video frame in the video data, preprocess the video frame to obtain a processed video frame, and determine whether the processed video frame is a target video frame.

[0023] Optionally, the training steps of the residual calculation model are:

[0024] Acquire a sample to be used, wherein the sample includes a low-resolution image and a high-resolution image;

[0025] Inputting the low-resolution image into a residual calculation model to obtain a prediction residual result;

[0026] Determining whether the residual calculation model meets a preset condition according to the high-resolution image, the low-resolution image and the prediction residual result;

[0027] When the residual calculation model does not meet the preset conditions, adjusting the residual calculation model according to the high-resolution image, the low-resolution image and the predicted residual result to obtain an adjusted residual calculation model, and repeatedly performing the determining step based on other samples;

[0028] When the residual calculation model meets the preset conditions, the residual calculation model is determined as a trained residual calculation model.

[0029] Optionally, the low-resolution image is obtained by blurring a high-resolution image in the same sample.

[0030] In a second aspect, the present application provides a device for improving video resolution, the device comprising:

[0031] A receiving unit, used for receiving low-resolution video data sent by a server;

[0032] A first determination unit, configured to determine, for each video frame in the video data, whether the video frame is a target video frame, wherein the target video frame is determined per interval frame number;

[0033] A second determining unit is used to determine, when the video frame is the target video frame, a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model;

[0034] The rendering unit is used to perform image rendering based on the high-resolution video frame and the non-target video frame.

[0035] Optionally, the second determining unit is configured to:

[0036] Input the target video frame into a trained residual calculation model to obtain a residual result;

[0037] The residual result is added to the target video frame to determine a high-resolution video frame corresponding to the video frame.

[0038] Optionally, the rendering unit is used to:

[0039] Performing image processing on the high-resolution video frames and the non-target video frames to obtain processed video frames, wherein the image processing includes color enhancement and sharpening processing;

[0040] Performing image rendering based on the processed video frame;

[0041] Optionally, the device further includes a third determining unit, and the third determining unit is configured to:

[0042] Get the related properties related to the interval frame number;

[0043] According to the preset corresponding relationship between the relevant attribute and the interval frame number, determining the interval frame number corresponding to the relevant attribute;

[0044] In the video frame data, video frames of each interval frame number are determined as target video frames, and other video frames are determined as non-target video frames.

[0045] Optionally, the first determining unit is configured to:

[0046] For each video frame in the video data, preprocess the video frame to obtain a processed video frame, and determine whether the processed video frame is a target video frame.

[0047] Optionally, the device further comprises a training unit, wherein the training unit is used to:

[0048] Acquire a sample to be used, wherein the sample includes a low-resolution image and a high-resolution image;

[0049] Inputting the low-resolution image into a residual calculation model to obtain a prediction residual result;

[0050] Determining whether the residual calculation model meets a preset condition according to the high-resolution image, the low-resolution image and the prediction residual result;

[0051] When the residual calculation model does not meet the preset conditions, adjusting the residual calculation model according to the high-resolution image, the low-resolution image and the predicted residual result to obtain an adjusted residual calculation model, and repeatedly performing the determining step based on other samples;

[0052] When the residual calculation model meets the preset conditions, the residual calculation model is determined as a trained residual calculation model.

[0053] Optionally, the low-resolution image is obtained by blurring a high-resolution image in the same sample.

[0054] In a third aspect, the present application provides a video resolution enhancement device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to:

[0055] Receive low-resolution video data sent by the server;

[0056] For each video frame in the video data, determining whether the video frame is a target video frame, wherein the target video frame is determined by the number of frames per interval;

[0057] When the video frame is the target video frame, determining a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model;

[0058] Image rendering is performed based on the high-resolution video frame and the non-target video frame.

[0059] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned method for improving video resolution when executed by a processor.

[0060] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: In the present application, the client receives low-resolution video data, processes the target video frame in the low-resolution video data, obtains the high-resolution video frame corresponding to the target video frame, and then performs image rendering based on the high-resolution video frame and the non-target video frame. In summary, since the server sends low-resolution video data to the client instead of high-resolution data, and the volume of low-resolution video data is much smaller than that of high-resolution video data, the present application can reduce the network bandwidth occupied by video transmission. At the same time, the present application only processes the target video frame in the low-resolution video data instead of processing all video frames, which can reduce the loss of the client and improve the operating efficiency of the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0063] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0064] Figure 1 A schematic diagram of a flow chart of a method for improving video resolution provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of a flow chart of a method for determining a high-resolution video frame provided in an embodiment of the present application;

[0066] Figure 3 A flowchart of a method for training a residual calculation model provided in an embodiment of the present application;

[0067] Figure 4 A flowchart of a method for determining the number of interval frames provided in an embodiment of the present application;

[0068] Figure 5 A flowchart of another method for improving video resolution provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of a process flow of a device for improving video resolution provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of a video resolution enhancement device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0072] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0073] In a usage scenario involved in the present application, when a user wants to play a video on a client, if the video is set to a high-resolution video by default, the client will send a video request message to the server, and the video request message is used to request the server to send low-resolution video data. If the video is not set to a high-resolution video by default, the user can switch the definition of the current video to switch the current video to a high-resolution video. At this time, the client will receive a switching instruction triggered by the user, and send a video request message to the server based on the switching instruction, and the video request message is used to request the server to send low-resolution video data. After receiving the low-resolution video data, the server searches for the low-resolution video data required by the client and sends it to the client. After the client receives the low-resolution video data sent by the server, it can determine whether each video frame in the video data is a target video frame in turn, and if so, generate a high-resolution video frame corresponding to the video frame. Finally, image rendering is performed on all video frames. Since the present application only processes the target video frame in the low-resolution video data, rather than processing all video frames, the loss of the client can be reduced and the operation efficiency of the client can be improved.

[0074] In summary, the embodiment of the present application provides a method for improving video resolution, which can reduce the loss of the client and improve the operating efficiency of the client. Figure 1 As shown, the specific steps include:

[0075] Step 101: Receive low-resolution video data sent by a server.

[0076] The resolution of a video is related to the number of pixels. When the number of pixels in a video is more, the resolution of the video is higher. When the number of pixels in a video is less, the resolution of the video is lower. In this application, a video with a number of pixels less than a preset value can be determined as a low-resolution video, and a video with a number of pixels greater than a preset value can be determined as a high-resolution video. The high-resolution video described in this application includes not only the high-resolution video described in the prior art, but also ultra-high-resolution video.

[0077] In this step, when the user wants to play a video on the client, if the video is a high-resolution video by default, the client will send a video request message to the server, and the video request message is used to request the server to send low-resolution video data. After receiving the low-resolution video data, the server searches for the low-resolution video data required by the client and sends it to the client, so that the client receives the low-resolution video data sent by the server.

[0078] If the video is not a high-resolution video by default, the user can switch the definition of the current video to switch the current video to a high-resolution video. Therefore, if the video is not a high-resolution video by default, the user can trigger a definition switching instruction so that the client responds to the instruction and sends video request information to the server based on the switching instruction. The video request information is used to request the server to send low-resolution video data. After receiving the low-resolution video data, the server searches for the low-resolution video data required by the client and sends it to the client so that the client receives the low-resolution video data sent by the server.

[0079] Step 102: for each video frame in the video data, determine whether the video frame is a target video frame.

[0080] The video data includes multiple video frames, and a video frame is essentially an image. The target video frame is determined by the number of frames per interval. For example, if the video data has 10 video frames and the preset number of frames is 3, the 1st video frame, the 5th video frame, and the 9th video frame can be determined as the target video frame. Of course, the 2nd video frame, the 6th video frame, and the 10th video frame can also be determined as the target video frame, and this step is not limited to this, as long as the target video frame meets the condition of being determined by the number of frames per interval.

[0081] Step 103: When the video frame is a target video frame, a high-resolution video frame corresponding to the target video frame is determined according to the target video frame and the trained residual calculation model.

[0082] The residual calculation model is a machine learning model, which is used to predict the corresponding residual result based on the low-resolution image. The residual result in this step is the difference between the high-resolution image and the low-resolution image.

[0083] In this step, when the current video frame is the target video frame, a high-resolution video frame of the video frame needs to be obtained. At this time, the high-resolution video frame corresponding to the target video frame can be determined based on the target video frame and the trained residual calculation model.

[0084] For example, assume that the video data has 10 video frames and the preset frame number is 3. If the first video frame is the target video frame, the first video frame needs to perform the steps described in step 103. After the first video frame is processed, the second video frame needs to be processed. Since the interval between the second video frame and the first video frame is not 3, it is determined that the second video frame is not the target video frame, and the steps described in step 103 do not need to be performed.

[0085] Step 104 , performing image rendering based on the high-resolution video frame and the non-target video frame.

[0086] In an embodiment of the present application, in order to play the video in real time, image rendering can be performed each time a video frame is obtained. Specifically, after receiving the low-resolution video data sent by the server, each time a video frame in the video data is obtained, it is determined whether the video frame is a target video frame. When the video frame is the target video frame, the high-resolution video frame corresponding to the video frame is determined, and image rendering is performed based on the high-resolution video frame. When the video frame is not the target video frame, that is, when the current video frame is a non-target video frame, image rendering can be performed directly based on the non-target video frame.

[0087] Alternatively, in an embodiment of the present application, after obtaining high-resolution video frames corresponding to all target video frames in the video data, rendering may be performed based on the obtained high-resolution video frames and non-target video frames.

[0088] That is, in this step, rendering can be performed based on the video frame after obtaining a high-resolution video frame or determining that the current video frame is a non-target video frame, or rendering can be performed based on the obtained high-resolution video frame and non-target video frame after obtaining high-resolution video frames corresponding to all target video frames in the video data. In this step, other processes for image rendering based on high-resolution video frames and non-target video frames can also be used, and this is not limited.

[0089] In the present application, the client receives low-resolution video data, processes the target video frame in the low-resolution video data, obtains the high-resolution video frame corresponding to the target video frame, and then performs image rendering based on the high-resolution video frame and the non-target video frame. In summary, since the server sends low-resolution video data to the client instead of high-resolution data, and the volume of low-resolution video data is much smaller than that of high-resolution video data, the present application can reduce the network bandwidth occupied by video transmission. At the same time, the present application only processes the target video frame in the low-resolution video data instead of processing all video frames, which can reduce the loss of the client and improve the operating efficiency of the client.

[0090] The present application embodiment also provides a method for determining a high-resolution video frame. The method is a specific limitation of step 103. The specific steps are as follows: Figure 2 As shown, including:

[0091] Step 21, input the target video frame into the trained residual calculation model to obtain a residual result.

[0092] In this step, when the client determines that the current video frame is the target video frame, the client directly inputs the target video frame into the trained residual calculation model to obtain a residual result.

[0093] Step 22: Add the residual result to the target video frame to determine a high-resolution video frame corresponding to the target video frame.

[0094] In this step, since the residual result is the difference between the high-resolution image and the low-resolution image, the residual result and the target video frame may be added to determine the high-resolution video frame corresponding to the target video frame.

[0095] The present application also provides a method for training a residual calculation model, which can obtain a trained residual calculation model. The specific steps are as follows: Figure 3 As shown, including:

[0096] Step 31, obtaining the sample to be used.

[0097] The samples include low-resolution images and high-resolution images. The low-resolution images are obtained by blurring the high-resolution images in the same sample.

[0098] In this step, the technician may pre-set a sample set, which includes a plurality of samples. Therefore, when it is necessary to train the residual calculation model, a sample may be randomly selected from the sample set to obtain a sample to be used.

[0099] Step 32: input the low-resolution image into the residual calculation model to obtain a prediction residual result.

[0100] In this step, a low-resolution image is obtained from the samples involved in step 31, the low-resolution image is input into the residual calculation model, and the predicted residual result is output.

[0101] Step 33, determining whether the residual calculation model meets preset conditions according to the high-resolution image, the low-resolution image and the prediction residual result.

[0102] In this step, a high-resolution image is obtained from the sample involved in step 31. First, a reference residual result is calculated based on the high-resolution image and the low-resolution image. Then, a difference value between the reference residual result and the predicted residual result is calculated. If the difference value is greater than a preset value, it is determined that the residual calculation model does not meet the preset conditions. If the difference value is less than or equal to the preset value, it is determined that the residual calculation model meets the preset conditions.

[0103] Step 34, when the residual calculation model does not meet the preset conditions, the residual calculation model is adjusted according to the high-resolution image, the low-resolution image and the predicted residual result to obtain an adjusted residual calculation model, and the determination step is repeated based on other samples.

[0104] In this step, when the residual calculation model does not meet the preset conditions, it means that the predicted residual result is still inaccurate and the model needs to be further adjusted. Therefore, when the residual calculation model does not meet the preset conditions, the residual calculation model can be adjusted according to the difference value determined in step 33 to obtain an adjusted residual calculation model. After that, other samples are selected from the sample set, and the low-resolution images in the other samples are input into the adjusted residual calculation model to obtain the predicted residual result, and then based on the high-resolution images, low-resolution images and predicted residual results in other samples, it is determined whether the residual calculation model meets the preset conditions. If it still does not meet the preset conditions, continue to adjust the adjusted residual calculation model, and repeat the above steps until a residual calculation model that meets the preset conditions is obtained.

[0105] Step 35: when the residual calculation model meets the preset conditions, the residual calculation model is determined as the trained residual calculation model.

[0106] In this step, when the residual calculation model meets the preset conditions, it means that the residual calculation model can accurately output the residual result. Therefore, the residual calculation model can be determined as a trained residual calculation model.

[0107] In the embodiment of the present application, in order to make the displayed image clearer, image processing may be performed on the high-resolution video frame and the non-target video frame, and then rendering may be performed based on the image-processed video frame. The specific steps include:

[0108] Perform image processing on the high-resolution video frame and the non-target video frame to obtain processed video frames, and perform image rendering based on the processed video frames.

[0109] Among them, image processing includes color enhancement and sharpening.

[0110] In this step, in order to make the displayed image clearer, the video frame can also be color enhanced and sharpened. Specifically, the high-resolution video frame and the non-target video frame can be color enhanced by LUT (look up table), and then the color enhanced video frame can be subjected to USM sharpening to obtain the processed video frame. Finally, image rendering is performed based on the processed video frame.

[0111] In the embodiment of the present application, since the format of the received video data may be incompatible with the residual calculation model or the video requirements of the client, the video data may be preprocessed so that the processed video data meets the requirements. Therefore, in the embodiment of the present application, after the client receives the video data, the client may also preprocess the video data. The specific steps are:

[0112] For each video frame in the video frame data, the video frame is preprocessed to obtain a processed video frame, and it is determined whether the processed video frame is a target video frame.

[0113] The preprocessing includes format conversion, video frame cropping and other processing, so that the preprocessed video frames are compatible with the residual calculation model or compatible with the video requirements of the client.

[0114] In this step, the requirements of the client for video data and the requirements of the residual calculation model for video data are first determined, which are about video format, video size, etc. According to these requirements, each video frame in the received video data is preprocessed to obtain a processed video frame, and finally it is determined whether the processed video frame is the target video frame.

[0115] In the embodiment of the present application, since different videos have different requirements for clarity, for videos with poor clarity requirements, a larger number of video frames may be used to determine the target video frame, and for videos with high clarity requirements, a smaller number of video frames may be used to determine the target video frame. Therefore, before step 102, the embodiment of the present application provides a method for determining the number of interval frames, and the specific steps of the method are as follows: Figure 4 As shown, including:

[0116] Step 41, obtaining relevant attributes related to the interval frame number.

[0117] Among them, different types of videos have different definition requirements. For example, the definition requirements of movies are often very high, while the definition requirements of animations are often very low. Therefore, the relevant attributes can include the video type. Different users also have different requirements for definition. For example, some users like to watch videos with very high definition. Therefore, the relevant attributes include the user definition requirements. Of course, the relevant attributes in this step can also include the video type and the user definition requirements at the same time, or can be other relevant attributes related to video definition.

[0118] Step 42, according to the preset corresponding relationship between the relevant attribute and the interval frame number, determine the interval frame number corresponding to the relevant attribute.

[0119] In this step, the technician can preset the corresponding relationship between the relevant attribute and the interval frame number and store it in the client. In this way, after executing step 41, the interval frame number corresponding to the relevant attribute can be determined according to the preset corresponding relationship between the relevant attribute and the interval frame number.

[0120] For example, when the relevant attribute is video type, the video can be a costume type, a movie type, or an animation type. The interval frame number corresponding to the animation type is set to 4, the interval frame number corresponding to the costume type is set to 2, and the interval frame number corresponding to the movie texture type is set to 1.

[0121] Step 43: In the video frame data, determine the video frames of each interval frame number as target video frames, and determine the other video frames as non-target video frames.

[0122] In this step, after determining the number of interval frames, the target video frame and the non-target video frame can be determined. Specifically, starting from the first frame in the video frame data, the video frames of each interval frame number are determined as target video frames, and the other video frames are determined as non-target video frames.

[0123] In an embodiment of the present application, a method for improving video resolution is also provided. Figure 5 As shown, specifically including:

[0124] Step 501: Receive low-resolution video data sent by a server.

[0125] Step 502 : For each video frame in the video data, pre-process the video frame to obtain a processed video frame, and determine whether the processed video frame is a target video frame.

[0126] Step 503: when the processed video frame is a target video frame, the target video frame is input into a trained residual calculation model to obtain a residual result.

[0127] Step 504 , adding the residual result to the target video frame to determine a high-resolution video frame corresponding to the video frame.

[0128] Step 505: perform color enhancement and sharpening processing on the high-resolution video frame, and perform image rendering on the processed video frame.

[0129] Among them, image processing includes color enhancement and sharpening;

[0130] Step 506: When the processed video frame is a non-target video frame, color enhancement and sharpening processing are performed on the non-target video frame, and image rendering is performed on the processed video frame.

[0131] In the present application, the client receives low-resolution video data, processes the target video frame in the low-resolution video data, obtains the high-resolution video frame corresponding to the target video frame, and then performs image rendering based on the high-resolution video frame and the non-target video frame. In summary, since the server sends low-resolution video data to the client instead of high-resolution data, and the volume of low-resolution video data is much smaller than that of high-resolution video data, the present application can reduce the network bandwidth occupied by video transmission. At the same time, the present application only processes the target video frame in the low-resolution video data instead of processing all video frames, which can reduce the loss of the client and improve the operating efficiency of the client.

[0132] In the embodiment of the present application, a video resolution improvement device is also provided, which corresponds to the method embodiment, specifically as follows: Figure 6 As shown, including:

[0133] The receiving unit 601 is used to receive low-resolution video data sent by the server;

[0134] A first determining unit 602 is used to determine, for each video frame in the video data, whether the video frame is a target video frame, wherein the target video frame is determined per interval frame number;

[0135] A second determining unit 603 is configured to determine, when the video frame is the target video frame, a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model;

[0136] The rendering unit 604 is configured to perform image rendering based on the high-resolution video frame and the non-target video frame.

[0137] Optionally, the second determining unit 603 is configured to:

[0138] Input the target video frame into a trained residual calculation model to obtain a residual result;

[0139] The residual result is added to the target video frame to determine a high-resolution video frame corresponding to the video frame.

[0140] Optionally, the rendering unit 604 is used to:

[0141] Performing image processing on the high-resolution video frames and the non-target video frames to obtain processed video frames, wherein the image processing includes color enhancement and sharpening processing;

[0142] Performing image rendering based on the processed video frame;

[0143] Optionally, the apparatus further includes a third determining unit 605, wherein the third determining unit 605 is configured to:

[0144] Get the related properties related to the interval frame number;

[0145] According to the preset corresponding relationship between the relevant attribute and the interval frame number, determining the interval frame number corresponding to the relevant attribute;

[0146] In the video frame data, video frames of each interval frame number are determined as target video frames, and other video frames are determined as non-target video frames.

[0147] Optionally, the first determining unit 602 is configured to:

[0148] For each video frame in the video data, preprocess the video frame to obtain a processed video frame, and determine whether the processed video frame is a target video frame.

[0149] Optionally, the apparatus further includes a training unit 606, wherein the training unit 606 is configured to:

[0150] Acquire a sample to be used, wherein the sample includes a low-resolution image and a high-resolution image;

[0151] Inputting the low-resolution image into a residual calculation model to obtain a prediction residual result;

[0152] Determining whether the residual calculation model meets a preset condition according to the high-resolution image, the low-resolution image and the prediction residual result;

[0153] When the residual calculation model does not meet the preset conditions, adjusting the residual calculation model according to the high-resolution image, the low-resolution image and the predicted residual result to obtain an adjusted residual calculation model, and repeatedly performing the determining step based on other samples;

[0154] When the residual calculation model meets the preset conditions, the residual calculation model is determined as a trained residual calculation model.

[0155] Optionally, the low-resolution image is obtained by blurring a high-resolution image in the same sample.

[0156] like Figure 7 As shown, the embodiment of the present application provides a video resolution improvement device, including a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0157] Memory 703, used for storing computer programs;

[0158] In one embodiment of the present application, the processor 701 is used to execute the program stored in the memory 703 to implement the video resolution improvement method provided by any of the above method embodiments, including:

[0159] Receive low-resolution video data sent by the server;

[0160] For each video frame in the video data, determining whether the video frame is a target video frame, wherein the target video frame is determined by the number of frames per interval;

[0161] When the video frame is the target video frame, determining a high-resolution video frame corresponding to the video frame according to the target video frame and a trained residual calculation model;

[0162] Image rendering is performed based on the high-resolution video frame and the non-target video frame.

[0163] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for improving video resolution provided in any of the aforementioned method embodiments are implemented.

[0164] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0166] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0167] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for improving video resolution, characterized in that: The method comprises: Receive low-resolution video data sent by the server; For each video frame in the video data, determining whether the video frame is a target video frame, wherein the target video frame is determined by the number of frames per interval; When the video frame is the target video frame, determining a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model; Image rendering is performed based on the high-resolution video frame and the non-target video frame.

2. The method according to claim 1, characterized in that: The step of determining a high-resolution video frame corresponding to the target video frame according to the target video frame and the trained residual calculation model includes: Input the target video frame into a trained residual calculation model to obtain a residual result; The residual result is added to the target video frame to determine a high-resolution video frame corresponding to the video frame.

3. The method according to claim 1, characterized in that: The performing image rendering based on the high-resolution video frame and the non-target video frame comprises: Performing image processing on the high-resolution video frames and the non-target video frames to obtain processed video frames, wherein the image processing includes color enhancement and sharpening processing; Image rendering is performed based on the processed video frame.

4. The method according to claim 1, characterized in that: Before determining, for each video frame in the video data, whether the video frame is a target video frame, the method further includes: Get the related properties related to the interval frame number; According to the preset corresponding relationship between the relevant attribute and the interval frame number, determining the interval frame number corresponding to the relevant attribute; In the video frame data, video frames of each interval frame number are determined as target video frames, and other video frames are determined as non-target video frames.

5. The method according to claim 1, characterized in that: The step of determining, for each video frame in the video data, whether the video frame is a target video frame comprises: For each video frame in the video data, preprocess the video frame to obtain a processed video frame, and determine whether the processed video frame is a target video frame.

6. The method according to claim 1, characterized in that: The training steps of the residual calculation model are: Acquire a sample to be used, wherein the sample includes a low-resolution image and a high-resolution image; Inputting the low-resolution image into a residual calculation model to obtain a prediction residual result; Determining whether the residual calculation model meets a preset condition according to the high-resolution image, the low-resolution image and the prediction residual result; When the residual calculation model does not meet the preset conditions, adjusting the residual calculation model according to the high-resolution image, the low-resolution image and the predicted residual result to obtain an adjusted residual calculation model, and repeatedly performing the determining step based on other samples; When the residual calculation model meets the preset conditions, the residual calculation model is determined as a trained residual calculation model.

7. The method according to claim 6, characterized in that: The low-resolution image is obtained by blurring the high-resolution image in the same sample.

8. A video resolution enhancement device, characterized in that: The device comprises: A receiving unit, used for receiving low-resolution video data sent by a server; A first determination unit, configured to determine, for each video frame in the video data, whether the video frame is a target video frame, wherein the target video frame is determined per interval frame number; A second determining unit is used to determine, when the video frame is the target video frame, a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model; The rendering unit is used to perform image rendering based on the high-resolution video frame and the non-target video frame.

9. A video resolution enhancement device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to: Receive low-resolution video data sent by the server; For each video frame in the video data, determining whether the video frame is a target video frame, wherein the target video frame is determined by the number of frames per interval; When the video frame is the target video frame, determining a high-resolution video frame corresponding to the target video frame according to the target video frame and a trained residual calculation model; Image rendering is performed based on the high-resolution video frame and the non-target video frame.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video resolution improvement method according to any one of claims 1 to 7 is implemented.

Images