Video high-resolution processing method and device, equipment and storage medium

Abstract

The invention relates to a video high-resolution processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a frame image and a plurality of image processing models sent by a server, the frame image being a low-resolution image, and the image processing models being used for outputting a high-resolution image; according to each image processing model, the frame image is segmented to obtain an input sub-image corresponding to each image processing model, and the sum of all the input sub-images is the frame image; for each input subimage, inputting the input subimage into a corresponding image processing model to obtain an output subimage corresponding to the input subimage; and performing image rendering according to the output subimage corresponding to each input subimage. According to the method, the terminal can obtain the high-resolution image in a short time, and the video watching experience of a user is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for high-resolution video processing. Background Art

[0002] In real life, the clarity of a video is related to its resolution, that is, the clearer the video, the higher its corresponding resolution. Therefore, users generally like to watch videos with high resolution.

[0003] In the related art, when a user wants to play a high-resolution video on a terminal, the terminal sends video request information to the server according to the user's trigger operation, and the video request information is used to request the server to send low-resolution video data. The server receives the video request information and sends the low-resolution video data to the terminal. After the terminal receives the low-resolution video data, it obtains high-resolution video data based on the low-resolution video data and the trained image processing model, and then renders based on it. The image processing model is used to output a high-resolution image.

[0004] In the process of processing low-resolution video data by the terminal, the entire processing time is mainly determined by the processing time of the image processing model. Since the image processing model processes a large amount of data, it takes a long time to complete the data processing, which results in a long processing time. This also causes the terminal to take a long time to complete the data processing process, reducing the user's video viewing experience. Therefore, a high-resolution video processing method is urgently needed. Summary of the invention

[0005] The present application provides a video high-resolution processing method, apparatus, device and storage medium, which can enable a terminal to obtain a high-resolution image in a shorter time, thereby improving the user's video viewing experience.

[0006] In a first aspect, the present application provides a method for high-resolution video processing, the method comprising:

[0007] Acquire a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used to output a high-resolution image;

[0008] According to each image processing model, the frame image is segmented to obtain an input sub-image corresponding to each image processing model, and the sum of all the input sub-images is the frame image;

[0009] For each input sub-image, input the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image;

[0010] Image rendering is performed according to the output sub-image corresponding to each input sub-image.

[0011] Optionally, the method further includes:

[0012] receiving at least one model set sent by a server, wherein different model sets contain different numbers of models, and the model set includes a plurality of image processing models;

[0013] According to the number of threads of the terminal, a model set having the same number of models as the number of threads is selected, wherein the terminal includes a plurality of threads;

[0014] The image processing models in the model set whose model number is equal to the thread number are determined as the image processing models used in the image segmentation step.

[0015] Optionally, inputting the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image includes:

[0016] For each thread of the terminal, the image processing model is used to run the image processing process of the input sub-image to obtain the output sub-image corresponding to the input sub-image.

[0017] Optionally, the method further includes:

[0018] When there is a thread to be tested in an image processing state, periodically detecting whether the thread to be tested is in the image processing state, the image processing state indicating that the thread is running the image processing process;

[0019] When a thread to be tested that is not in an image processing state is detected, detecting whether there is a next frame image;

[0020] When it is detected that there is no next frame image, releasing the thread to be tested;

[0021] When it is detected that there is a next frame image, the next frame image is obtained to repeatedly run the image processing process based on the thread to be tested.

[0022] Optionally, segmenting the frame image according to each image processing model to obtain an input sub-image corresponding to each image processing model includes:

[0023] According to each image processing model, determining an input area corresponding to each image processing model in the frame image;

[0024] The frame image is segmented according to the input area corresponding to each image processing model in the frame image to obtain the input sub-image corresponding to each image processing model.

[0025] Optionally, the image processing model includes a residual module and a summation module, and the step of inputting the input sub-image into the corresponding image processing model to obtain an output sub-image corresponding to the input sub-image includes:

[0026] Inputting the input sub-image into the residual module to obtain a residual result;

[0027] The residual result and the input sub-image are input into a summing module to obtain an output sub-image corresponding to the input sub-image.

[0028] Optionally, performing image rendering according to the output sub-image corresponding to each input sub-image includes:

[0029] Obtaining an image region corresponding to each input sub-image in the frame image;

[0030] According to the image region corresponding to each input sub-image in the frame image, splicing the output sub-image corresponding to each input sub-image to obtain a high-resolution image;

[0031] Performing color enhancement and sharpening processing on the high-resolution image to obtain a processed high-resolution image;

[0032] Rendering is performed based on the processed high-resolution image.

[0033] In a second aspect, the present application provides a video high-resolution processing device, the device comprising:

[0034] An acquisition unit, used for acquiring a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used for outputting a high-resolution image;

[0035] A segmentation unit, used for segmenting the frame image according to each image processing model to obtain an input sub-image corresponding to each image processing model, wherein the sum of all input sub-images is the frame image;

[0036] An input unit, for inputting each input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image;

[0037] The rendering unit is used to perform image rendering according to the output sub-image corresponding to each input sub-image.

[0038] Optionally, the device further comprises a screening unit, wherein the screening unit is used to:

[0039] receiving at least one model set sent by a server, wherein different model sets contain different numbers of models, and the model set includes a plurality of image processing models;

[0040] According to the number of threads of the terminal, a model set having the same number of models as the number of threads is selected, wherein the terminal includes a plurality of threads;

[0041] The image processing models in the model set whose model number is equal to the thread number are determined as the image processing models used in the image segmentation step.

[0042] Optionally, the input unit includes:

[0043] For each thread of the terminal, the image processing model is used to run the image processing process of the input sub-image to obtain the output sub-image corresponding to the input sub-image.

[0044] Optionally, the device further comprises a detection unit, wherein the detection unit is used to:

[0045] When there is a thread to be tested in an image processing state, periodically detecting whether the thread to be tested is in the image processing state, the image processing state indicating that the thread is running the image processing process;

[0046] When a thread to be tested that is not in an image processing state is detected, detecting whether there is a next frame image;

[0047] When it is detected that there is no next frame image, releasing the thread to be tested;

[0048] When it is detected that there is a next frame image, the next frame image is obtained to repeatedly run the image processing process based on the thread to be tested.

[0049] Optionally, the segmentation unit is used to:

[0050] According to each image processing model, determining an input area corresponding to each image processing model in the frame image;

[0051] The frame image is segmented according to the input area corresponding to each image processing model in the frame image to obtain the input sub-image corresponding to each image processing model.

[0052] Optionally, the image processing model includes a residual module and a summation module, and the input unit is used to:

[0053] Inputting the input sub-image into the residual module to obtain a residual result;

[0054] The residual result and the input sub-image are input into a summing module to obtain an output sub-image corresponding to the input sub-image.

[0055] Optionally, the rendering unit is used to:

[0056] Obtaining an image region corresponding to each input sub-image in the frame image;

[0057] According to the image region corresponding to each input sub-image in the frame image, splicing the output sub-image corresponding to each input sub-image to obtain a high-resolution image;

[0058] Performing color enhancement and sharpening processing on the high-resolution image to obtain a processed high-resolution image;

[0059] Rendering is performed based on the processed high-resolution image.

[0060] In a third aspect, an embodiment of the present application provides a video high-resolution processing device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to:

[0061] Acquire a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used to output a high-resolution image;

[0062] According to each image processing model, the frame image is segmented to obtain an input sub-image corresponding to each image processing model, and the sum of all the input sub-images is the frame image;

[0063] For each input sub-image, input the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image;

[0064] Image rendering is performed according to the output sub-image corresponding to each input sub-image.

[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned high-resolution video processing method is implemented.

[0066] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art: in the embodiment of the present application, after receiving the frame image sent by the server, the frame image is segmented according to multiple image processing models to obtain the input sub-image corresponding to each image processing model, wherein the sum of all input sub-images is the frame image. Then each input sub-image is input into its corresponding image processing model to obtain the output sub-image corresponding to each input sub-image. Since the image output by the image processing model is a high-resolution image, the output sub-image output by the image processing model is also a high-resolution image. Finally, the image is rendered according to these high-resolution images, and the clarity of the image can be improved. In other words, the present application uses multiple image processing models to process different regions of the frame image respectively to obtain high-resolution images of different regions, and then renders based on these high-resolution images. Since the present application uses multiple image processing models to process different regions of the frame image, its essence is to distribute the pressure of data processing to multiple image processing models, and the data that each image processing model needs to process will be reduced, thereby reducing the time for the image processing model to process data, shortening the time for the terminal to obtain a high-resolution image, so that the terminal can obtain a high-resolution image in a shorter time, and improving the user's experience of watching videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0069] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0070] Figure 1 A schematic diagram of a process flow of a high-resolution video processing method provided in an embodiment of the present application;

[0071] Figure 2 A flowchart of a method for determining an image processing model provided in an embodiment of the present application;

[0072] Figure 3 A schematic diagram of a detection method provided in an embodiment of the present application;

[0073] Figure 4 A schematic diagram of a flow chart of an image rendering method provided in an embodiment of the present application;

[0074] Figure 5 A schematic diagram of a process flow of a high-resolution video processing device provided in an embodiment of the present application;

[0075] Figure 6 A schematic diagram of a high-resolution video processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0077] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0078] In a usage scenario involved in the present application, when a user wants to play a video on a terminal, if the video is a high-resolution video by default, the client will send a video request message to the server, and the video request message is used to request the server to send low-resolution video data. If the video is not a high-resolution video by default, the user can switch the clarity of the current video to switch the current video to a high-resolution video. At this time, the terminal will receive a switching instruction triggered by the user, and send a video request message to the server based on the switching instruction, and the video request message is used to request the server to send low-resolution video data. After receiving the low-resolution video data, the server searches for the low-resolution video data required by the terminal and sends it to the terminal. After the terminal receives the low-resolution video data sent by the server, it uses multiple image processing models to process different areas of the frame image respectively to obtain high-resolution images of different areas, and then renders based on these high-resolution images. Since the present application uses multiple image processing models to process different areas of the frame image, its essence is to distribute the pressure of data processing to multiple image processing models. The data that each image processing model needs to process will be reduced, thereby reducing the time for the image processing model to process data, shortening the time for the terminal to obtain a high-resolution image, enabling the terminal to obtain a high-resolution image in a shorter time, thereby improving the user's video viewing experience.

[0079] In summary, the embodiments of the present application provide a method for processing high-resolution video, which enables a terminal to obtain a high-resolution image in a shorter time, thereby improving the user's experience of watching videos. Figure 1 As shown, the specific steps include:

[0080] Step 101, obtaining a frame image and multiple image processing models sent by a server.

[0081] Among them, the low-resolution video data includes a series of frame images corresponding to time points, and the frame images are low-resolution images. The resolution of an image is related to the number of pixels. When the number of pixels of an image is more, the resolution of the image is higher, and when the number of pixels of an image is fewer, the resolution of the image is lower. In the present application, an image with a number of pixels less than a preset value can be determined as a low-resolution image, and an image with a number of pixels greater than a preset value can be determined as a high-resolution image. The high-resolution images described in the present application include not only the high-resolution images described in the prior art, but also ultra-high-resolution images. The image processing model is used to process low-resolution images to obtain high-resolution images. In other words, the input of the image processing model is a low-resolution image, and the output is a high-resolution image.

[0082] In this step, when the terminal receives the low-resolution video data sent by the server, since the low-resolution video data includes a series of frame images corresponding to time points, the terminal can obtain the frame images to be processed in the video data. At the same time, the client also obtains multiple pre-stored image processing models for subsequent processing.

[0083] In this step, after obtaining the frame image sent by the server, the frame image may not meet the requirements, and the frame image needs to be preprocessed to obtain the preprocessed frame image so as to segment the preprocessed frame image. Specifically, when the frame image is in RGB data format, it is converted into YUV data format every other frame to obtain a frame image after format conversion. The frame image is cropped to obtain a frame image whose size meets the requirements. The resolution in the frame image is verified to obtain a verified frame image.

[0084] Step 102, segmenting the frame image according to each image processing model to obtain an input sub-image corresponding to each image processing model.

[0085] Among them, the image processing model can correspond to one input sub-image or multiple input sub-images. The input image size of each image processing model can be the same or different. In the process of video data processing, in order to make the image processing model complete the processing at the same time as much as possible, each image processing model is set to correspond to an input sub-image, and the size of the input sub-image corresponding to each image processing model is the same. Since the frame image is a low-resolution image, and the input sub-image is a partial image in the frame image, the input sub-image is also a low-resolution image. In order not to miss each area of ​​the frame image, the sum of all input sub-images should be the frame image, that is, all input sub-images are spliced ​​to obtain the frame image.

[0086] In this step, the frame image can be segmented according to the input image size of the image processing model to obtain the input sub-image corresponding to each image processing model. For example, assuming that the width and height of the frame image are W*H, there are two image processing models, and the input image sizes of the two image processing models are the same, and the width and height are both W / 2*H. Based on the above assumptions, the frame image can be segmented into the left half image and the right half image, and each image is used as the input sub-image of the image processing model, thereby obtaining the input sub-image corresponding to each image processing model.

[0087] Alternatively, in this step, the technician may also pre-set the input area corresponding to each image processing model in the frame image, and then segment the frame image according to the input area corresponding to each image processing model in the frame image to obtain the input sub-image corresponding to each image processing model. For example, assuming that there are two image processing models, the first image processing model corresponds to the left half of the frame image, and the second image processing model corresponds to the right half of the frame image. According to the input area corresponding to the above image processing models in the frame image, the frame image is divided into the left half and the right half, and then the left half is used as the input sub-image of the first image processing model, and the right half is used as the input sub-image of the second image processing model.

[0088] Step 103: for each input sub-image, input the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image.

[0089] Among them, the image processing model is a trained machine learning model, and its training process is similar to the training process in the prior art, which will not be described here one by one.

[0090] In this step, if the image processing model corresponds to only one input sub-image, the input sub-image is directly input into the image processing model, and the output sub-image corresponding to the input sub-image is output. If the image processing model corresponds to multiple input sub-images, each input sub-image can be input into the image processing model in turn to obtain the output sub-image corresponding to each input sub-image.

[0091] It should be noted that these image processing models run in parallel, that is, the image processing model processes its corresponding input sub-image almost simultaneously. For example, in the above example, when the left half of the image is input into the first image processing model, the right half of the image is input into the second image processing model almost simultaneously.

[0092] Step 104 , performing image rendering according to the output sub-image corresponding to each input sub-image.

[0093] In this step, each output sub-image may be spliced ​​to obtain a spliced ​​image, and then image rendering may be performed based on the spliced ​​image. Alternatively, image rendering may be performed directly based on each output sub-image.

[0094] In an embodiment of the present application, after receiving a frame image sent by a server, the frame image is segmented according to a plurality of image processing models to obtain an input sub-image corresponding to each image processing model, wherein the sum of all input sub-images is a frame image. Then each input sub-image is input into its corresponding image processing model to obtain an output sub-image corresponding to each input sub-image. Since the image output by the image processing model is a high-resolution image, the output sub-image output by the image processing model is also a high-resolution image. Finally, by rendering the image according to these high-resolution images, the clarity of the image can be improved. In other words, the present application uses a plurality of image processing models to process different regions of the frame image respectively to obtain high-resolution images of different regions, and then renders based on these high-resolution images. Since the present application uses a plurality of image processing models to process different regions of the frame image, its essence is to distribute the pressure of data processing to a plurality of image processing models, and the data that each image processing model needs to process will be reduced, thereby reducing the time for the image processing model to process data, shortening the time for the terminal to obtain a high-resolution image, enabling the terminal to obtain a high-resolution image in a shorter time, and improving the user's experience of watching videos.

[0095] In the embodiment of the present application, the image processing model involved in step 102 may be sent in advance by the server to the terminal, or may be determined by the terminal based on a certain method. Therefore, the present application provides a method for determining an image processing model, which can determine the image processing model used in step 102, such as Figure 2 As shown, the specific steps are:

[0096] Step 21: Receive at least one model set sent by the server.

[0097] Among them, the number of models in different model sets is different, and the model set includes multiple image processing models. Preferably, the model set includes multiple image processing models with the same input size. In each model set, the sum of the input image sizes of all image processing models is less than or equal to the size of the frame image. When the two are equal, it means that each image processing model corresponds to only one input sub-image. When the two are not equal, it means that there are image processing models corresponding to multiple input sub-images. Preferably, the input image size of each image processing model in the model set is equal to the size of the frame image, that is, each image processing model corresponds to only one input sub-image.

[0098] In this step, the technician can make at least one model set and upload it to the server. After the server receives the at least one model set uploaded by the technician, the server can send it to the terminal.

[0099] Step 22, according to the number of threads of the terminal, filter out a model set whose number of models is equal to the number of threads.

[0100] The terminal includes multiple threads. The multiple threads of the terminal run in parallel, which is related to the core of the terminal. Specifically, the threads and the cores of the terminal are in a one-to-one correspondence. For example, when there are two cores in the terminal, the terminal has two parallel threads.

[0101] In order to process the frame image as quickly as possible, the image processing model used in the image segmentation step can be determined at any time point before the frame image is received, and then the subsequent steps are performed based on these image processing models, wherein the above-mentioned image segmentation step refers to step 102. Preferably, after executing step 21, the number of cores of the terminal is obtained, and the number of parallel threads in the terminal is determined according to the number of cores, and then the model set whose number of models is equal to the number of threads is screened out according to the number of threads.

[0102] Step 23, determining the image processing model in the model set whose number of models is equal to the number of threads as the image processing model used in the image segmentation step.

[0103] In this step, the image processing models in the model set whose number of models is equal to the number of threads are stored in the terminal, so that when the frame image sent by the server is obtained, these image processing models are obtained in the terminal, and the frame image is segmented according to each image processing model to obtain the input sub-image corresponding to each image processing model.

[0104] In the embodiment of the present application, after obtaining multiple image processing models based on the method described in steps 21 to 23, step 103 can be further refined, and the specific steps are:

[0105] For each thread of the terminal, the image processing model is used to run the image processing process of the input sub-image using the thread to obtain the output sub-image corresponding to the input sub-image.

[0106] In this step, one thread is used to run one image processing process, that is, one thread is used to run one image processing model to process the input sub-image. Since the number of image processing models is equal to the number of threads, this step can fully utilize the performance of the device to run the image processing process, thus reducing the image processing time.

[0107] In the embodiment of the present application, since the video data includes multiple frame images, the terminal needs to process multiple frame images continuously. In this process, due to the different busyness of the threads and the complexity of the images to be processed, some threads will be processed first and some threads will be processed later. For the threads that are processed first, if they are asked to wait for other threads, the operating resources of the terminal will be wasted. In order to solve this problem, the present application provides a detection method that can make full use of the operating resources of the terminal, such as Figure 3 As shown, specifically including:

[0108] Step 31: When there is a thread to be tested in the image processing state, periodically detect whether the thread to be tested is in the image processing state.

[0109] The image processing state indicates that the thread is running the image processing process. The thread to be tested is a thread that needs to be periodically tested. As long as the thread starts to run the image processing process involved in this application, it will be used as the thread to be tested.

[0110] In this step, after the image segmentation step is completed, all threads can be used to run the image processing process, so that all threads are in the image processing state. However, the busy state of each thread is different, which causes some threads to not be in the image processing state immediately, that is, only some threads are threads to be tested. Therefore, this step needs to monitor in real time whether the thread is in the image processing state. When a thread in the image processing state is detected, the thread is used as the thread to be tested, and periodic detection is performed to determine whether the thread to be tested is in the image processing state.

[0111] Step 32: When a thread to be tested that is not in the image processing state is detected, it is detected whether there is a next frame image.

[0112] In this step, when a thread to be tested that is not in the image processing state is detected, it means that the thread to be tested has completed the image processing process, that is, the thread to be tested is currently in an idle state, and whether there is a next frame image can be detected to determine whether to release the thread to be tested.

[0113] Step 33: when it is detected that there is no next frame image, the thread to be tested is released.

[0114] In this step, when it is detected that there is no next frame image, it means that the thread does not need to continue image processing, and the thread can be released to perform other tasks.

[0115] Step 34 , when it is detected that there is a next frame image, the next frame image is obtained to repeatedly run the image processing process based on the thread to be tested.

[0116] In this step, when the existence of the next frame image is detected, it means that the thread still needs to perform image processing to fully utilize the terminal operation resources. Therefore, when the existence of the next frame image is detected, the next frame image is obtained, and based on the thread to be tested, the image processing model is used to perform image processing on the input sub-image obtained based on the frame image.

[0117] In the implementation of the present application, an image segmentation method is provided, which is a further limitation of step 102, and specifically includes:

[0118] According to each image processing model, the input area corresponding to each image processing model in the frame image is determined; according to the input area corresponding to each image processing model in the frame image, the frame image is segmented to obtain the input sub-image corresponding to each image processing model.

[0119] In this step, the terminal not only stores multiple image processing models, but also stores the input area corresponding to each image processing model in the frame image. In this way, after determining the image processing model to be used, the input area corresponding to each image processing model in the frame image can be found in the terminal according to each image processing model. After that, the frame image can be segmented according to the input area corresponding to each image processing model in the frame image to obtain the input sub-image corresponding to each image processing model.

[0120] In an embodiment of the present application, when the image processing model includes a residual module and a summation module, an embodiment of the present application provides an image processing method, which is a further limitation of step 103, specifically including:

[0121] The input sub-image is input into the residual module to obtain a residual result; the residual result and the input sub-image are input into the summation module to obtain an output sub-image corresponding to the input sub-image.

[0122] The residual module is used to predict the residual result corresponding to the input sub-image. The residual result is the difference between the high-resolution image and the low-resolution image. The summation module is used to add the residual result and the low-resolution image to obtain the high-resolution image.

[0123] In the embodiment of the present application, an image rendering method is also provided, which is a further limitation of step 104. The specific method is as follows: Figure 4 As shown, including:

[0124] Step 41, obtaining the image area corresponding to each input sub-image in the frame image.

[0125] In this step, since the input sub-image is a sub-image of the frame image, the input sub-image and the frame image can be directly compared to obtain the image area corresponding to each input sub-image in the frame image. Alternatively, the image area corresponding to each input sub-image in the frame image can be determined based on the image area corresponding to each image processing model in the frame image involved in the above content and the corresponding relationship between the input sub-image and the image processing model.

[0126] Step 42 , according to the image area corresponding to each input sub-image in the frame image, the output sub-image corresponding to each input sub-image is spliced ​​to obtain a high-resolution image.

[0127] For example, there are two input sub-images a and b, input sub-image a corresponds to the left half of the image, and input sub-image b corresponds to the right half of the image. Input sub-image a corresponds to output sub-image A, and input sub-image b corresponds to output sub-image B. Since input sub-image a corresponds to the left half of the image, and input sub-image b corresponds to the left half of the image, output sub-image A can be used as the left half of the high-resolution image, and output sub-image B can be used as the right half of the high-resolution image, and then they can be spliced ​​to obtain a high-resolution image.

[0128] Step 43, performing color enhancement and sharpening processing on the high-resolution image to obtain a processed high-resolution image.

[0129] In this step, in order to make the high-resolution image clearer, the high-resolution image may be color enhanced to obtain a color-enhanced high-resolution image. After that, the color-enhanced high-resolution image may be sharpened to obtain a processed high-resolution image.

[0130] Step 44, rendering is performed based on the processed high-resolution image.

[0131] In this step, after obtaining the processed high-resolution image, the image can be rendered on the screen.

[0132] In the above process, the output sub-images may be firstly subjected to color enhancement and sharpening processing to obtain processed output sub-images, and then the processed output sub-images may be spliced ​​to obtain a spliced ​​high-resolution image, and then rendering may be performed based on the image.

[0133] like Figure 5 As shown, the embodiment of the present application also provides a video high-resolution processing device, the device comprising:

[0134] An acquisition unit 501 is used to acquire a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image and the image processing model is used to output a high-resolution image;

[0135] A segmentation unit 502 is used to segment the frame image according to each image processing model to obtain an input sub-image corresponding to each image processing model, and the sum of all input sub-images is the frame image;

[0136] An input unit 503 is used to input each input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image;

[0137] The rendering unit 504 is used to perform image rendering according to the output sub-image corresponding to each input sub-image.

[0138] Optionally, the device further includes a screening unit 505, wherein the screening unit 505 is used to:

[0139] receiving at least one model set sent by a server, wherein different model sets contain different numbers of models, and the model set includes a plurality of image processing models;

[0140] According to the number of threads of the terminal, a model set having the same number of models as the number of threads is selected, wherein the terminal includes a plurality of threads;

[0141] The image processing models in the model set whose model number is equal to the thread number are determined as the image processing models used in the image segmentation step.

[0142] Optionally, the input unit 503 includes:

[0143] For each thread of the terminal, the image processing model is used to run the image processing process of the input sub-image to obtain the output sub-image corresponding to the input sub-image.

[0144] Optionally, the device further includes a detection unit 506, and the detection unit 506 is used to:

[0145] When there is a thread to be tested in an image processing state, periodically detecting whether the thread to be tested is in the image processing state, the image processing state indicating that the thread is running the image processing process;

[0146] When a thread to be tested that is not in an image processing state is detected, detecting whether there is a next frame image;

[0147] When it is detected that there is no next frame image, releasing the thread to be tested;

[0148] When it is detected that there is a next frame image, the next frame image is obtained to repeatedly run the image processing process based on the thread to be tested.

[0149] Optionally, the segmentation unit 502 is used to:

[0150] According to each image processing model, determining an input area corresponding to each image processing model in the frame image;

[0151] The frame image is segmented according to the input area corresponding to each image processing model in the frame image to obtain the input sub-image corresponding to each image processing model.

[0152] Optionally, the image processing model includes a residual module and a summation module, and the input unit 503 is used to:

[0153] Inputting the input sub-image into the residual module to obtain a residual result;

[0154] The residual result and the input sub-image are input into a summing module to obtain an output sub-image corresponding to the input sub-image.

[0155] Optionally, the rendering unit 504 is used to:

[0156] Obtaining an image region corresponding to each input sub-image in the frame image;

[0157] According to the image region corresponding to each input sub-image in the frame image, splicing the output sub-image corresponding to each input sub-image to obtain a high-resolution image;

[0158] Performing color enhancement and sharpening processing on the high-resolution image to obtain a processed high-resolution image;

[0159] Rendering is performed based on the processed high-resolution image.

[0160] like Figure 6 As shown, the embodiment of the present application provides a video high-resolution processing device, including a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0161] Memory 603, used for storing computer programs;

[0162] In one embodiment of the present application, the processor 601 is used to execute the program stored in the memory 603 to implement the video high-resolution processing method provided by any of the above method embodiments, including:

[0163] Acquire a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used to output a high-resolution image;

[0164] According to each image processing model, the frame image is segmented to obtain an input sub-image corresponding to each image processing model, and the sum of all the input sub-images is the frame image;

[0165] For each input sub-image, input the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image;

[0166] Image rendering is performed according to the output sub-image corresponding to each input sub-image.

[0167] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the video high-resolution processing method provided in any of the aforementioned method embodiments are implemented.

[0168] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0170] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0171] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A video high-resolution processing method, characterized in that: The method comprises: Acquire a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used to output a high-resolution image; According to each image processing model, the frame image is segmented to obtain an input sub-image corresponding to each image processing model, and the sum of all the input sub-images is the frame image; For each input sub-image, input the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image; Image rendering is performed according to the output sub-image corresponding to each input sub-image.

2. The method according to claim 1, characterized in that: The method further comprises: receiving at least one model set sent by a server, wherein different model sets contain different numbers of models, and the model set includes a plurality of image processing models; According to the number of threads of the terminal, a model set having the same number of models as the number of threads is selected, wherein the terminal includes a plurality of threads; The image processing models in the model set whose model number is equal to the thread number are determined as the image processing models used in the image segmentation step.

3. The method according to claim 2, characterized in that: The step of inputting the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image includes: For each thread of the terminal, the image processing model is used to run the image processing process of the input sub-image to obtain the output sub-image corresponding to the input sub-image.

4. The method according to claim 3, characterized in that: The method further comprises: When there is a thread to be tested in an image processing state, periodically detecting whether the thread to be tested is in the image processing state, the image processing state indicating that the thread is running the image processing process; When a thread to be tested that is not in an image processing state is detected, detecting whether there is a next frame image; When it is detected that there is no next frame image, releasing the thread to be tested; When it is detected that there is a next frame image, the next frame image is obtained to repeatedly run the image processing process based on the thread to be tested.

5. The method according to claim 1, characterized in that: The step of segmenting the frame image according to each image processing model to obtain an input sub-image corresponding to each image processing model includes: According to each image processing model, determining an input area corresponding to each image processing model in the frame image; The frame image is segmented according to the input area corresponding to each image processing model in the frame image to obtain the input sub-image corresponding to each image processing model.

6. The method according to claim 1, characterized in that: The image processing model includes a residual module and a summation module, and the input sub-image is input into the corresponding image processing model to obtain an output sub-image corresponding to the input sub-image, including: Inputting the input sub-image into the residual module to obtain a residual result; The residual result and the input sub-image are input into a summing module to obtain an output sub-image corresponding to the input sub-image.

7. The method according to claim 1, characterized in that: The image rendering is performed according to the output sub-image corresponding to each input sub-image, including: Obtaining an image region corresponding to each input sub-image in the frame image; According to the image region corresponding to each input sub-image in the frame image, splicing the output sub-image corresponding to each input sub-image to obtain a high-resolution image; Performing color enhancement and sharpening processing on the high-resolution image to obtain a processed high-resolution image; Rendering is performed based on the processed high-resolution image.

8. A video high-resolution processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used for outputting a high-resolution image; A segmentation unit, used for segmenting the frame image according to each image processing model to obtain an input sub-image corresponding to each image processing model, wherein the sum of all input sub-images is the frame image; An input unit, for inputting each input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image; The rendering unit is used to perform image rendering according to the output sub-image corresponding to each input sub-image.

9. A video high-resolution processing device, characterized in that: include: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; at least one memory connected to the at least one bus, wherein the processor is configured to: Acquire a frame image and a plurality of image processing models sent by a server, wherein the frame image is a low-resolution image, and the image processing model is used to output a high-resolution image; According to each image processing model, the frame image is segmented to obtain an input sub-image corresponding to each image processing model, and the sum of all the input sub-images is the frame image; For each input sub-image, input the input sub-image into a corresponding image processing model to obtain an output sub-image corresponding to the input sub-image; Image rendering is performed according to the output sub-image corresponding to each input sub-image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the high-resolution video processing method according to any one of claims 1 to 7 is implemented.

Images