Video processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a video processing method and device, electronic equipment and a storage medium, and belongs to the technical field of video processing. The method comprises the following steps: determining the first N first frame images in a first frame image set as second frame images, and moving the second frame images from the first frame image set to a storage space of the electronic equipment; and if the number of the second frame images in the storage space reaches N, carrying out super-division processing on the second frame images in the storage space to obtain a target frame image. Therefore, before the video is subjected to super-division processing, a plurality of first frame images of the video can serve as second frame images in advance and are stored in the storage space of the electronic equipment, and when the number of the second frame images in the storage space reaches N, the video is subjected to super-division processing, that is, N frames are delayed for super-division processing; particularly, the problem of video multi-speed playing lagging caused by insufficient computing power of the mobile terminal when the super-resolution algorithm is deployed in the mobile terminal is avoided, and the super-resolution processing efficiency of the video is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method, device, electronic device and storage medium. Background Art

[0002] At present, super-resolution processing, also known as super-resolution processing, helps improve image quality and is widely used in medical image reconstruction, face image reconstruction, drone monitoring and other fields. However, the super-resolution processing methods in related technologies have the problem of long super-resolution processing time. For example, in the scenario of video speed playback, if the video is still super-resolution processed, it may cause video playback to freeze. Summary of the invention

[0003] The present disclosure provides a video processing method, device, electronic device, computer-readable storage medium, and computer program product to at least solve the problem of long super-resolution processing time in the related art. The technical solution of the present disclosure is as follows:

[0004] According to a first aspect of an embodiment of the present disclosure, a video processing method is provided, comprising: determining the first N first frame images in a first frame image set of a video as second frame images, and moving the second frame images from the first frame image set to a storage space of an electronic device, wherein the plurality of first frame images are sorted from early to late according to display time; if the number of second frame images in the storage space reaches N, performing super-resolution processing on the second frame images in the storage space to obtain a target frame image, wherein N is a positive integer greater than 1.

[0005] In one embodiment of the present disclosure, the method further includes: if the super-resolution processing of the first-order second frame image is completed, taking out the first-order second frame image from the storage space, wherein the plurality of second frame images are sorted from early to late according to the display time; if the first frame images in the first frame image set have not been completely taken out, determining the first-order first frame image in the first frame image set as the second frame image, and moving the second frame image from the first frame image set to the storage space; updating the sorting of the plurality of second frame images in the storage space, and returning to execute the super-resolution processing of the second frame images in the storage space and its subsequent steps until the first frame images in the first frame image set are completely taken out.

[0006] In one embodiment of the present disclosure, the method also includes: determining the undisplayed target frame image as the third frame image, and sorting the multiple third frame images from early to late according to the display time; and displaying the multiple third frame images in sequence according to the sorting of the multiple third frame images.

[0007] In one embodiment of the present disclosure, the super-resolution processing of multiple second frame images is performed in parallel.

[0008] In one embodiment of the present disclosure, N is the number of cores of the first processor of the electronic device, and the super-resolution processing of the second frame image in the storage space to obtain the target frame image includes: inputting the second frame image ranked i-th into the i-th core of the first processor, and super-resolution processing of the second frame image ranked i-th by the i-th core to obtain the target frame image ranked i-th, wherein i is a positive integer not greater than N.

[0009] In one embodiment of the present disclosure, the N cores of the first processor are respectively deployed with super-resolution models, and the super-resolution processing of the second frame image of the i-th order is performed by the i-th core to obtain the i-th target frame image, including: inputting the i-th second frame image of the i-th order to the super-resolution model deployed by the i-th core, and super-resolution processing of the second frame image of the i-th order is performed by the super-resolution model deployed by the i-th core to obtain the i-th target frame image.

[0010] In one embodiment of the present disclosure, the method also includes: obtaining a first super-resolution processing time of the i-th kernel for the second frame image of the i-th order; if the first super-resolution processing time is greater than a set threshold, inputting the j-th second frame image into a second processor of the electronic device, and performing super-resolution processing on the j-th second frame image by the second processor to obtain the j-th target frame image, wherein j is greater than i, and j is a positive integer not greater than N.

[0011] In one embodiment of the present disclosure, super-resolution processing is performed on the second frame image of the jth order by the second processor to obtain the target frame image of the jth order, including: interpolation processing is performed on the second frame image of the jth order by the second processor to obtain the target frame image of the jth order.

[0012] In one embodiment of the present disclosure, the method also includes: obtaining a second super-resolution processing duration of the second processor for the second frame image of the jth order; based on the first super-resolution processing duration and the second super-resolution processing duration, identifying whether a super-resolution end condition is met; if the super-resolution end condition is met, stopping super-resolution processing on the second frame image in the storage space.

[0013] In one embodiment of the present disclosure, after stopping the super-resolution processing of the second frame image in the storage space, it also includes: determining an undisplayed target frame image, a second frame image that has not completed the super-resolution processing, and a first frame image in the first frame image set as candidate frame images; sorting the multiple candidate frame images from early to late according to the display time; and displaying the multiple candidate frame images in sequence according to the sorting of the multiple candidate frame images.

[0014] In one embodiment of the present disclosure, before determining the first N first frame images in the first frame image set of the video as the second frame image, the method further includes: receiving a speed-up playback instruction for the video.

[0015] According to a second aspect of an embodiment of the present disclosure, a video processing device is provided, comprising: a determination module, configured to determine the first N first frame images in a first frame image set of a video as second frame images, and move the second frame images from the first frame image set to a storage space of an electronic device, wherein the plurality of first frame images are sorted from early to late according to display time; and a processing module, configured to perform super-resolution processing on the second frame images in the storage space to obtain a target frame image if the number of second frame images in the storage space reaches N, wherein N is a positive integer greater than 1.

[0016] In one embodiment of the present disclosure, the processing module is further configured to execute: if the super-resolution processing of the first-order second frame image is completed, taking out the first-order second frame image from the storage space, wherein the plurality of second frame images are sorted from early to late according to the display time; if the first frame images in the first frame image set have not been completely taken out, determining the first-order first frame image in the first frame image set as the second frame image, and moving the second frame image from the first frame image set to the storage space; updating the sorting of the plurality of second frame images in the storage space, and returning to execute the super-resolution processing of the second frame images in the storage space and its subsequent steps until the first frame images in the first frame image set are completely taken out.

[0017] In one embodiment of the present disclosure, the processing module is further configured to execute: determining the undisplayed target frame image as the third frame image, and sorting the multiple third frame images from early to late according to the display time; and displaying the multiple third frame images in sequence according to the sorting of the multiple third frame images.

[0018] In one embodiment of the present disclosure, the super-resolution processing of multiple second frame images is performed in parallel.

[0019] In one embodiment of the present disclosure, N is the number of cores of the first processor of the electronic device, and the processing module is further configured to execute: inputting the second frame image of the i-th order into the i-th core of the first processor, and performing super-resolution processing on the second frame image of the i-th order through the i-th core to obtain the i-th target frame image, wherein i is a positive integer not greater than N.

[0020] In one embodiment of the present disclosure, the N cores of the first processor are respectively deployed with super-resolution models, and the processing module is further configured to execute: inputting the second frame image of the i-th ranking into the super-resolution model deployed by the i-th core, and performing super-resolution processing on the second frame image of the i-th ranking through the super-resolution model deployed by the i-th core to obtain the target frame image of the i-th ranking.

[0021] In one embodiment of the present disclosure, the processing module is further configured to execute: obtaining a first super-resolution processing time of the i-th kernel for the second frame image of the i-th order; if the first super-resolution processing time is greater than a set threshold, inputting the j-th second frame image into a second processor of the electronic device, and performing super-resolution processing on the j-th second frame image through the second processor to obtain the j-th target frame image, wherein j is greater than i, and j is a positive integer not greater than N.

[0022] In one embodiment of the present disclosure, the processing module is further configured to execute: performing interpolation processing on the second frame image of the jth order through the second processor to obtain the target frame image of the jth order.

[0023] In one embodiment of the present disclosure, the processing module is further configured to execute: obtaining a second super-resolution processing duration of the second processor for the second frame image of the jth order; based on the first super-resolution processing duration and the second super-resolution processing duration, identifying whether a super-resolution end condition is met; if the super-resolution end condition is met, stopping super-resolution processing of the second frame image in the storage space.

[0024] In one embodiment of the present disclosure, after stopping the super-resolution processing of the second frame image in the storage space, the processing module is further configured to execute: determining the undisplayed target frame image, the second frame image that has not completed the super-resolution processing, and the first frame image in the first frame image set as candidate frame images; sorting the multiple candidate frame images from early to late according to the display time; and displaying the multiple candidate frame images in sequence according to the sorting of the multiple candidate frame images.

[0025] In one embodiment of the present disclosure, before determining the first N first frame images in the first frame image set of the video as the second frame image, the determination module is further configured to execute: receiving a speed-up playback instruction for the video.

[0026] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, comprising a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the steps of the method described in the first aspect of the embodiment of the present disclosure.

[0027] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method described in the first aspect of the embodiment of the present disclosure are implemented.

[0028] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor of an electronic device, the computer program implements the steps of the method described in the first aspect of the embodiment of the present disclosure.

[0029] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: before super-resolution processing of a video, multiple first frame images of the video can be pre-set as second frame images and stored in the storage space of an electronic device, and when the number of second frame images in the storage space reaches N, super-resolution processing of the video is started, that is, super-resolution processing is delayed by N frames, so that in the process of super-resolution processing of the video, multiple frame images to be super-resolution processed can be directly obtained from the storage space, which can save the time for obtaining the frame images to be super-resolution processed, effectively avoiding the problem of long super-resolution processing time and video playback jamming due to long acquisition time of the frame images to be super-resolution processed, especially avoiding the problem of video playback jamming due to insufficient computing power of the mobile terminal when the super-resolution algorithm is deployed on the mobile terminal, thereby improving the efficiency of video super-resolution processing.

[0030] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0032] Figure 1 The figure is a flowchart of a video processing method according to an exemplary embodiment.

[0033] Figure 2 It is a schematic diagram of a video processing method according to an exemplary embodiment.

[0034] Figure 3 is a flowchart of a video processing method according to another exemplary embodiment.

[0035] Figure 4 is a flowchart of a video processing method according to another exemplary embodiment.

[0036] Figure 5 is a flowchart of a video processing method according to another exemplary embodiment.

[0037] Figure 6 The invention is a block diagram of a video processing device according to an exemplary embodiment.

[0038] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0039] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0041] The acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations.

[0042] Figure 1 is a flowchart of a video processing method according to an exemplary embodiment. Figure 1 As shown, the video processing method of the embodiment of the present disclosure includes the following steps.

[0043] S101, determining the first N first frame images in a first frame image set of a video as second frame images, and moving the second frame images from the first frame image set to a storage space of an electronic device, wherein the plurality of first frame images are sorted from early to late according to display time.

[0044] It should be noted that the execution subject of the video processing method of the embodiment of the present disclosure is an electronic device, and the electronic device includes a mobile phone, a notebook, a desktop computer, a vehicle-mounted terminal, a smart home appliance, a wearable device, etc. Among them, the wearable device may include a wrist-worn device (such as a smart watch, a smart bracelet), a head-worn device, a foot-worn device, etc. The video processing method of the embodiment of the present disclosure can be executed by the video processing device of the embodiment of the present disclosure, and the video processing device of the embodiment of the present disclosure can be configured in any electronic device to execute the video processing method of the embodiment of the present disclosure.

[0045] It should be noted that there are no excessive restrictions on videos, for example, videos on video APPs (Applications) may be included. The first frame image set includes a large number of first frame images, where the first frame image refers to a frame image to be super-resolved of a video, and the first frame image set refers to a set consisting of a large number of first frame images.

[0046] It should be noted that the first frame image set can be acquired in advance or in real time. For example, the method also includes receiving a super-resolution processing instruction and / or a playback instruction (such as a double-speed playback instruction) for the video, preprocessing the original frame image of the video to obtain the first frame image, and adding the first frame image to the first frame image set. It should be noted that the preprocessing can be implemented by using any frame image preprocessing method in the relevant technology, and there is no excessive limitation here. For example, it can include decoding, etc.

[0047] It should be noted that moving the second frame image from the first frame image set to the storage space of the electronic device refers to taking the second frame image from the first frame image set and moving the taken second frame image to the storage space of the electronic device. Moving the second frame image from the first frame image set to the storage space of the electronic device in step S101 refers to taking the first N first frame images from the first frame image set and moving the taken first N first frame images to the storage space of the electronic device.

[0048] It should be noted that there are no excessive restrictions on the storage space, for example, it may include the memory of the electronic device. There are no excessive restrictions on N, for example, N may be 4, 6, etc.

[0049] For example, the first frame image set of the video includes frame image A 1 To A 10 , for frame image A 1 To A 10 Sort by display time from early to late, and the resulting sorting result is frame image A. 1 To A 10 , that is, frame image A 1 The display time of frame image A is the earliest. 10 The latest display time.

[0050] Taking N=4 as an example, we can select frame image A from the first frame image set. 1 To A 4 Determine it as the second frame image, and set frame image A 1 To A 4 The first frame image set is moved to the memory of the electronic device. The updated first frame image set includes frame image A 5 To A 10 , the memory of the updated electronic device includes frame image A 1 To A 4 .

[0051] Taking N=6 as an example, we can select frame image A from the first frame image set. 1 To A 6 Determine it as the second frame image, and set frame image A 1 To A 6 The first frame image set is moved to the memory of the electronic device. The updated first frame image set includes frame image A 7 To A 10 , the memory of the updated electronic device includes frame image A 1 To A 6 .

[0052] In one implementation, before determining the first N first frame images in the first frame image set of the video as the second frame image, the method further includes receiving a super-resolution processing instruction and / or a playback instruction for the video.

[0053] In one embodiment, before the first N first frames of the first frame images of the video are sorted and determined as the second frame images, it also includes receiving a speed-up playback instruction for the video. Therefore, in the method, after receiving the speed-up playback instruction for the video, the video processing method of the present disclosure can be executed to perform super-resolution processing on the video, that is, the video processing method of the present disclosure is applicable to the speed-up playback scenario of the video, and can achieve the parallel speed-up playback and super-resolution processing of the video, that is, in the speed-up playback scenario of the video, the super-resolution processing can still be performed on the video, which helps to improve the image quality of the video in the speed-up playback scenario.

[0054] It should be noted that there are no excessive restrictions on the generation methods of super-resolution processing instructions, double-speed playback instructions, and playback instructions. For example, taking the super-resolution processing instructions as an example, taking the electronic device as a mobile phone as an example, the user can click the icon of video A on the page of the video APP deployed on the mobile phone, and set the playback mode of video A to double-speed playback on the playback page of video A. Correspondingly, the video APP can generate a double-speed playback instruction for video A based on the above user operation.

[0055] S102, if the number of second frame images in the storage space reaches N, super-resolution processing is performed on the second frame images in the storage space to obtain a target frame image, wherein N is a positive integer greater than 1.

[0056] It should be noted that the storage space is used to store multiple second frame images, and the second frame images correspond to the target frame images one by one. The second frame images can be super-resolved by any super-resolved processing method in the related art, which is not limited here.

[0057] In an embodiment of the present disclosure, in the process of moving the second frame image from the first frame image set to the storage space of the electronic device in step S101, it can be identified whether the number of second frame images in the storage space reaches N. If the number of second frame images in the storage space reaches N, it indicates that the first N first frame images in the sorting order have all been moved to the storage space, that is, the storage space now stores N second frame images, and the N second frame images can be directly obtained from the storage space, which can avoid the problem in the related technology that when the super-resolution processing of the current frame image is completed, the next frame image has not been obtained, that is, it is necessary to wait for a certain period of time to obtain the next frame image, such as waiting for the decoding of the original frame image to be completed to generate the next frame image, resulting in a long super-resolution processing time and video playback jams.

[0058] On the contrary, if the number of second frame images in the storage space does not reach N, it indicates that at least one first frame image among the first N first frame images in the sorting has not been moved to the storage space.

[0059] It is understandable that if the number of second frame images in the storage space does not reach N, the second frame images are not super-resolution processed at this time, and the second frame images are returned to be moved from the first frame image set to the storage space of the electronic device and the subsequent steps until the number of second frame images in the storage space reaches N. That is, when the number of second frame images in the storage space does not reach N, the second frame images are not super-resolution processed, and the second frame images continue to be moved to the storage space, and when the number of second frame images in the storage space reaches N, the second frame images are super-resolution processed.

[0060] It is understandable that it takes time to move the first N frames of the sorted images to the storage space. For example, if it takes 30ms (milliseconds) to move the first N frames of the sorted images to the storage space, then in this solution, it is necessary to wait for at least 30ms before starting the super-resolution processing of the video. Taking the double-speed playback scenario as an example, in this solution, after receiving the double-speed playback instruction for the video, it is necessary to wait for at least 30ms before starting the super-resolution processing of the video.

[0061] In one implementation, the super-resolution processing of the plurality of second frame images is performed in parallel. Therefore, in this solution, the plurality of second frame images can be super-resolution processed in parallel, which helps to improve the efficiency of the super-resolution processing.

[0062] In one embodiment, a plurality of second frame images are sorted from earliest to latest according to display time, N is the number of cores of the first processor of the electronic device, and super-resolution processing is performed on the second frame images in the storage space to obtain a target frame image, including inputting the second frame image of the ith sorting into the ith core of the first processor, and super-resolution processing is performed on the second frame image of the ith sorting by the ith core to obtain the target frame image of the ith sorting. Therefore, in this solution, super-resolution processing of N second frame images can be implemented in parallel through the N cores of the first processor.

[0063] It should be noted that there are not too many restrictions on the first processor. For example, it may include an APU (Accelerated Processing Unit), and the APU may include a VPU (Vision Processing Unit), MDLA (MediaTek Deep Learning Accelerator), etc.

[0064] For example, Figure 2 As shown, the second frame image stored in the storage space includes frame image A 1 , A 2 To A N , for frame image A 1 To A N Sort by display time from early to late, and the resulting sorting result is frame image A. 1 To A N , that is, frame image A 1 The display time of frame image A is the earliest. N The display time of the i-th ranked second frame image is frame image A. i .

[0065] The frame image A can be i Input to the i-th core of the first processor, and the frame image A is processed by the i-th core i Perform super-resolution processing to obtain frame image B i The target frame image of the i-th order is frame image B i Frame image B i Including frame image B 1 , B 2 To B N .

[0066] In some examples, the N cores of the first processor are respectively deployed with super-resolution models, and the i-th core performs super-resolution processing on the second frame image of the i-th order to obtain the i-th target frame image, including inputting the i-th second frame image to the super-resolution model deployed by the i-th core, and performing super-resolution processing on the second frame image of the i-th order by the super-resolution model deployed by the i-th core to obtain the i-th target frame image. Therefore, in this solution, super-resolution processing of N second frame images can be achieved in parallel through the super-resolution models deployed on the N cores of the first processor.

[0067] It should be noted that there are no excessive restrictions on super-resolution models, for example, they may include deep learning models. The deployment of super-resolution models can be implemented by using any super-resolution model deployment method in the relevant technology, and there are no excessive restrictions here. For example, the super-resolution model can be deployed on the APU of the mobile terminal.

[0068] For example, continue with Figure 2 For example, the N cores of the first processor are respectively deployed with super-resolution models (not shown in the figure), and the frame image A i Input to the super-resolution model deployed by the i-th kernel, and use the super-resolution model deployed by the i-th kernel to analyze the frame image A i Perform super-resolution processing to obtain frame image B i .

[0069] In one embodiment, the method further includes determining the undisplayed target frame image as the third frame image, and sorting the plurality of third frame images from early to late according to the display time, and sequentially displaying the plurality of third frame images according to the sorting of the plurality of third frame images. Thus, the plurality of target frame images can be sequentially displayed from early to late according to the display time, and the target frame images can be displayed in sequence.

[0070] For example, the target frame images that are not displayed include frame images 11 to 13. Frame images 11 to 13 may be determined as the third frame images, and frame images 11 to 13 may be sorted from earliest to latest according to display time. The sorting result obtained is frame images 11 to 13, that is, frame image 11 has the earliest display time, and frame image 13 has the latest display time. Frame images 11 to 13 may be displayed in sequence according to the sorting of frame images 11 to 13.

[0071] The video processing method provided by the embodiment of the present disclosure determines the first N first frame images in the first frame image set of the video as the second frame image, and moves the second frame image from the first frame image set to the storage space of the electronic device, and sorts the multiple first frame images from early to late according to the display time. If the number of second frame images in the storage space reaches N, the second frame images in the storage space are super-resolution processed to obtain the target frame image, where N is a positive integer greater than 1. Therefore, before super-resolution processing of the video, multiple first frame images of the video can be pre-set as second frame images and stored in the storage space of the electronic device, and the super-resolution processing of the video will not be started until the number of second frame images in the storage space reaches N, that is, the super-resolution processing is delayed by N frames. Therefore, in the process of super-resolution processing of the video, multiple frame images to be super-resolution processed can be directly obtained from the storage space, which can save the time for obtaining the frame images to be super-resolution processed, and effectively avoid the problem of long super-resolution processing time and video playback jamming caused by long acquisition time of the frame images to be super-resolution processed. In particular, when the super-resolution algorithm is deployed on a mobile terminal, the problem of video playback jamming caused by insufficient computing power of the mobile terminal is avoided, thereby improving the efficiency of video super-resolution processing.

[0072] Figure 3 is a flowchart of a video processing method according to another exemplary embodiment. Figure 3 As shown, the video processing method of the embodiment of the present disclosure includes the following steps.

[0073] S301, determining the first N first frame images in a first frame image set of a video as second frame images, and moving the second frame images from the first frame image set to a storage space of an electronic device, wherein the multiple first frame images are sorted from early to late according to display time.

[0074] S302: If the number of second frame images in the storage space reaches N, super-resolution processing is performed on the second frame images in the storage space to obtain a target frame image, where N is a positive integer greater than 1.

[0075] For the relevant contents of steps S301 - S302 , please refer to the above embodiments and will not be described again here.

[0076] S303, if the super-resolution processing of the first second frame image is completed, taking out the first second frame image from the storage space, wherein the plurality of second frame images are sorted from early to late according to the display time.

[0077] S304: If all first frame images in the first frame image set have not been retrieved, determine the first first frame image in the first frame image set as the second frame image, and move the second frame image from the first frame image set to the storage space.

[0078] S305, updating the order of the plurality of second frame images in the storage space, and returning to execute super-resolution processing on the second frame images in the storage space and subsequent steps, until all the first frame images in the first frame image set are taken out.

[0079] It should be noted that moving the second frame image from the first frame image set to the storage space in step S304 refers to taking the first-ranked first frame image from the first frame image set and moving the taken-out first-ranked first frame image to the storage space.

[0080] It should be noted that, in the process of performing super-resolution processing on the second frame image in the storage space, steps S303 to S305 may be executed.

[0081] It can be understood that if all first frame images in the first frame image set are taken out, the video super-resolution processing is completed.

[0082] For example, the first frame image set of the video includes frame images 1 to 10. Frame images 1 to 10 are sorted from early to late according to display time, and the sorting result is frame images 1 to 10, that is, frame image 1 is displayed the earliest and frame image 10 is displayed the latest.

[0083] Taking N=4 as an example, frame images 1 to 4 in the first frame image set can be determined as the second frame images, and frame images 1 to 4 can be moved from the first frame image set to the memory of the electronic device. The updated first frame image set includes frame images 5 to 10, and the updated memory of the electronic device includes frame images 1 to 4.

[0084] If the number of second frame images in the memory of the electronic device reaches 4, super-resolution processing is performed on frame images 1 to 4.

[0085] In response to the generation of frame image 11, wherein frame image 11 is a frame image after super-resolution processing of frame image 1, frame image 1 is the second frame image ranked first in the memory, and frame image 11 is the target frame image ranked first, frame image 1 can be taken out from the memory of the electronic device, and the first frame image ranked first in the first frame image set, that is, frame image 5, is determined as the second frame image, and frame image 5 is moved from the first frame image set to the memory of the electronic device. The updated first frame image set includes frame images 6 to 10, and the updated memory of the electronic device includes frame images 2 to 5.

[0086] Super-resolution processing can be performed on frame images 2 to 5. In response to the generation of frame image 12, wherein frame image 12 is a frame image after super-resolution processing of frame image 2, frame image 2 is the second frame image ranked first in the memory, and frame image 12 is the target frame image ranked first, frame image 2 can be taken out from the memory of the electronic device, and the first frame image ranked first in the first frame image set, that is, frame image 6 is determined as the second frame image, and frame image 6 is moved from the first frame image set to the memory of the electronic device. The updated first frame image set includes frame images 7 to 10, and the updated memory of the electronic device includes frame images 3 to 6.

[0087] Frame images 3 to 6 may be super-resolution processed. In response to the generation of frame image 13, frame image 13 is a frame image after super-resolution processing of frame image 3, frame image 3 is the second frame image ranked first in the memory, and frame image 13 is the target frame image ranked first. Frame image 3 may be taken out from the memory of the electronic device, and the first frame image ranked first in the first frame image set, that is, frame image 7, may be determined as the second frame image, and frame image 7 may be moved from the first frame image set to the memory of the electronic device. The updated first frame image set includes frame images 8 to 10, and the updated memory of the electronic device includes frame images 4 to 7.

[0088] It should be noted that the processing process of frame images 4 to 10 can refer to the processing process of frame images 1 to 3, and will not be repeated here.

[0089] The video processing method provided by the embodiment of the present disclosure is as follows: if the super-resolution processing of the second frame image ranked first is completed, the second frame image ranked first is taken out from the storage space; if the first frame image in the first frame image set has not been completely taken out, the first frame image ranked first in the first frame image set is determined as the second frame image, and the second frame image is moved from the first frame image set to the storage space, the order of multiple second frame images in the storage space is updated, and the super-resolution processing of the second frame image in the storage space and subsequent steps are returned to execute until the first frame image in the first frame image set is completely taken out. Therefore, in the process of super-resolution processing of the video, the real-time update of the second frame image in the storage space can be realized until the first frame image in the first frame image set is completely taken out, that is, the video super-resolution processing is completed.

[0090] Figure 4 is a flowchart of a video processing method according to another exemplary embodiment. Figure 4 As shown, the video processing method of the embodiment of the present disclosure includes the following steps.

[0091] S401, determining the first N first frame images in a first frame image set of a video as second frame images, and moving the second frame images from the first frame image set to a storage space of an electronic device, wherein the multiple first frame images are sorted from early to late according to display time.

[0092] S402, if the number of second frame images in the storage space reaches N, the second frame image ranked i-th is input to the i-th core of the first processor, and the i-th second frame image ranked i-th is super-resolution processed by the i-th core to obtain the i-th target frame image ranked i-th.

[0093] For the relevant contents of steps S401 - S402 , please refer to the above embodiment and will not be described again here.

[0094] S403 , obtaining a first super-resolution processing duration of the i-th kernel for the second frame image of the i-th sort.

[0095] S404, if the first super-resolution processing time is greater than the set threshold, the second frame image of the jth order is input into the second processor of the electronic device, and the second processor performs super-resolution processing on the second frame image of the jth order to obtain the target frame image of the jth order, wherein j is greater than i, and j is a positive integer not greater than N.

[0096] It should be noted that there are no excessive restrictions on the set threshold, for example, it may include 30ms.

[0097] It should be noted that the display time of the second frame image of the jth order is later than that of the second frame image of the ith order. There are no excessive restrictions on the second processor, for example, it may include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc.

[0098] For example, the first frame image set of the video includes frame images 1 to 10. Frame images 1 to 10 are sorted from early to late according to display time, and the sorting result is frame images 1 to 10, that is, frame image 1 is displayed the earliest and frame image 10 is displayed the latest.

[0099] Taking N=2 as an example, frame images 1 to 2 in the first frame image set can be determined as the second frame image, and frame images 1 to 2 are moved from the first frame image set to the memory of the electronic device. The updated first frame image set includes frame images 3 to 10, and the updated memory of the electronic device includes frame images 1 to 2.

[0100] If the number of second frame images in the memory of the electronic device reaches 2, super-resolution processing is performed on frame images 1 to 2. For example, frame image 1 is input to the first core of the first processor, and super-resolution processing is performed on frame image 1 through the first core, and frame image 2 is input to the second core of the first processor, and super-resolution processing is performed on frame image 2 through the second core.

[0101] In response to the generation of frame image 11, wherein frame image 11 is a frame image after super-resolution processing of frame image 1, frame image 1 is the second frame image ranked first in the memory, and frame image 11 is the target frame image ranked first, frame image 1 can be taken out from the memory of the electronic device, and the first frame image ranked first in the first frame image set, that is, frame image 3 is determined as the second frame image, and frame image 3 is moved from the first frame image set to the memory of the electronic device. The updated first frame image set includes frame images 4 to 10, and the updated memory of the electronic device includes frame images 2 to 3.

[0102] The frame images 2 to 3 may be subjected to super-resolution processing, which may include the following possible implementations:

[0103] Method 1: If the first super-resolution processing time of the first kernel for frame image 1 is greater than 30ms, super-resolution processing is performed on frame image 2 through the second kernel, frame image 3 is input to the second processor of the electronic device, and super-resolution processing is performed on frame image 3 through the second processor.

[0104] Method 2: if the first super-resolution processing time of the first kernel for frame image 1 is longer than 30 ms, frame images 2 and 3 are input to the second processor of the electronic device, and frame images 2 and 3 are super-resolution processed by the second processor.

[0105] Method 3: If the first super-resolution processing time of the first kernel for frame image 1 is less than or equal to 30ms, super-resolution processing is performed on frame image 2 through the second kernel, frame image 3 is input to the first kernel of the first processor, and super-resolution processing is performed on frame image 3 through the first kernel.

[0106] In one embodiment, the second processor performs super-resolution processing on the second frame image of the jth order to obtain the target frame image of the jth order, including performing interpolation processing on the second frame image of the jth order to obtain the target frame image of the jth order. Thus, the second processor can perform interpolation processing on the second frame image to achieve super-resolution processing of the second frame image, and the interpolation processing has a small amount of calculation and a fast speed, which helps to improve the efficiency of super-resolution processing.

[0107] It should be noted that the interpolation process can be implemented by using any interpolation method in the relevant technology, without too much limitation here, for example, it can include nearest neighbor interpolation, bilinear interpolation, etc.

[0108] S405, taking out the first-ranked second frame image from the storage space, determining the first-ranked first frame image in the first frame image set as the second frame image, and moving the second frame image from the first frame image set to the storage space.

[0109] S406, updating the order of the plurality of second frame images in the storage space, and returning to execute super-resolution processing on the second frame image in the ith order and subsequent steps, until all the first frame images in the first frame image set are taken out.

[0110] For the relevant contents of steps S405-S406, please refer to the above embodiment and will not be repeated here.

[0111] It should be noted that there are no excessive restrictions on the execution order of steps S403-S406. For example, the execution order of steps S403-S404 is earlier or later than that of steps S405-S406, or steps S403-S404 and steps S405-S406 are executed in parallel.

[0112] In the video processing method provided by the embodiment of the present disclosure, if the first super-resolution processing time is longer than the set threshold, the second frame image of the jth order is input to the second processor of the electronic device, and the second processor performs super-resolution processing on the second frame image of the jth order to obtain the target frame image of the jth order, wherein j is greater than i, and j is a positive integer not greater than N. Therefore, when the first super-resolution processing time of the first processor is long, that is, when the super-resolution processing speed of the first processor is slow, the second processor can also perform super-resolution processing on part of the frame images, and the first processor and the second processor can be used to perform super-resolution processing on the video at the same time, which helps to speed up the super-resolution processing speed of the video and avoid the problem of video playback freeze.

[0113] Figure 5 is a flowchart of a video processing method according to another exemplary embodiment. Figure 5 As shown, the video processing method of the embodiment of the present disclosure includes the following steps.

[0114] S501, determining the first N first frame images in a first frame image set of a video as second frame images, and moving the second frame images from the first frame image set to a storage space of an electronic device, wherein the multiple first frame images are sorted from early to late according to display time.

[0115] S502, if the number of second frame images in the storage space reaches N, the second frame image ranked i-th is input to the i-th core of the first processor, and the i-th second frame image ranked i-th is super-resolution processed by the i-th core to obtain the i-th target frame image ranked i-th.

[0116] S503 , obtaining a first super-resolution processing duration of the i-th kernel for the second frame image of the i-th sort.

[0117] S504, if the first super-resolution processing time is greater than the set threshold, the second frame image of the jth order is input into the second processor of the electronic device, and the second processor performs super-resolution processing on the second frame image of the jth order to obtain the target frame image of the jth order, wherein j is greater than i, and j is a positive integer not greater than N.

[0118] For the relevant contents of steps S501 - S504 , please refer to the above embodiment and will not be described again here.

[0119] S505: Obtain a second super-resolution processing duration of the second processor for the second frame image of the jth order.

[0120] S506: Based on the first super-resolution processing time and the second super-resolution processing time, identify whether a super-resolution end condition is met.

[0121] S507: If the super-resolution end condition is met, stop super-resolution processing on the second frame image in the storage space.

[0122] It should be noted that there are no excessive restrictions on the super-resolution termination conditions. For example, the average value of the first super-resolution processing time and the second super-resolution processing time is greater than a set threshold, and / or the weighted sum of the first super-resolution processing time and the second super-resolution processing time is greater than the set threshold, and / or the average value of the third super-resolution processing time of multiple consecutive second frame images is greater than the set threshold, as the super-resolution termination conditions. Among them, the third super-resolution processing time is the first super-resolution processing time or the second super-resolution processing time.

[0123] It should be noted that stopping super-resolution processing on the second frame image in the storage space refers to stopping super-resolution processing on the second frame image that has not completed super-resolution processing. The second frame image that has not completed super-resolution processing may include a second frame image that has not been super-resolution processed and is in the process of super-resolution processing.

[0124] In one embodiment, after stopping the super-resolution processing of the second frame image in the storage space, it also includes determining the target frame image that has not been displayed, the second frame image that has not completed the super-resolution processing, and the first frame image in the first frame image set as candidate frame images, sorting the multiple candidate frame images from early to late according to the display time, and displaying the multiple candidate frame images in sequence according to the sorting of the multiple candidate frame images. In this way, multiple candidate frame images can be displayed in sequence from early to late according to the display time, and the candidate frame images include the target frame image that has not been displayed, the second frame image that has not completed the super-resolution processing, and the first frame image in the first frame image set, so that the candidate frame images can be displayed in sequence.

[0125] For example, the target frame images that are not displayed include frame images 11 to 13, the second frame images that have not completed the super-resolution processing include frame images 4 to 7, and the first frame images in the first frame image set include frame images 8 to 10. Frame images 4 to 13 can be determined as candidate frame images, and frame images 4 to 13 can be sorted from early to late according to the display time. The sorting result is frame images 11, 12, 13, 4 to 10, that is, the display time of frame image 11 is the earliest, and the display time of frame image 10 is the latest. According to the order of frame images 11, 12, 13, 4 to 10, frame images 11, 12, 13, 4 to 10 can be displayed in sequence.

[0126] The video processing method provided by the embodiment of the present disclosure obtains the second super-resolution processing time of the second processor for the second frame image of the jth order, and identifies whether the super-resolution end condition is met based on the first super-resolution processing time and the second super-resolution processing time. If the super-resolution end condition is met, the super-resolution processing of the second frame image is stopped. Therefore, the first super-resolution processing time and the second super-resolution processing time can be comprehensively considered to identify whether the super-resolution end condition is met. When the super-resolution end condition is met, the super-resolution processing of the video is stopped, which can avoid the problem of video playback freeze caused by slow super-resolution processing speed.

[0127] Figure 6 is a block diagram of a video processing device according to an exemplary embodiment. Figure 6 The video processing device 100 of the embodiment of the present disclosure includes: a determination module 110 and a processing module 120.

[0128] The determining module 110 is configured to determine the first N first frame images in the first frame image set of the video as the second frame image, and move the second frame image from the first frame image set to a storage space of the electronic device, wherein the first frame images are sorted from early to late according to display time;

[0129] The processing module 120 is configured to perform super-resolution processing on the second frame images in the storage space to obtain a target frame image if the number of the second frame images in the storage space reaches N, where N is a positive integer greater than 1.

[0130] In one embodiment of the present disclosure, the processing module 120 is further configured to execute: if the super-resolution processing of the first-order second frame image is completed, taking out the first-order second frame image from the storage space, wherein the plurality of second frame images are sorted from early to late according to the display time; if the first frame images in the first frame image set have not been completely taken out, determining the first-order first frame image in the first frame image set as the second frame image, and moving the second frame image from the first frame image set to the storage space; updating the sorting of the plurality of second frame images in the storage space, and returning to execute the super-resolution processing of the second frame images in the storage space and its subsequent steps until the first frame images in the first frame image set are completely taken out.

[0131] In one embodiment of the present disclosure, the processing module 120 is further configured to execute: determining the undisplayed target frame image as the third frame image, and sorting the multiple third frame images from early to late according to the display time; and displaying the multiple third frame images in sequence according to the sorting of the multiple third frame images.

[0132] In one embodiment of the present disclosure, the super-resolution processing of multiple second frame images is performed in parallel.

[0133] In one embodiment of the present disclosure, N is the number of cores of the first processor of the electronic device, and the processing module 120 is further configured to execute: inputting the second frame image of the i-th order into the i-th core of the first processor, and performing super-resolution processing on the second frame image of the i-th order through the i-th core to obtain the i-th target frame image of the i-th order, wherein i is a positive integer not greater than N.

[0134] In one embodiment of the present disclosure, the N cores of the first processor are respectively deployed with super-resolution models, and the processing module 120 is further configured to execute: inputting the second frame image of the i-th ranking into the super-resolution model deployed by the i-th core, and performing super-resolution processing on the second frame image of the i-th ranking through the super-resolution model deployed by the i-th core to obtain the target frame image of the i-th ranking.

[0135] In one embodiment of the present disclosure, the processing module 120 is further configured to execute: obtaining a first super-resolution processing time of the i-th kernel for the second frame image of the i-th order; if the first super-resolution processing time is greater than a set threshold, inputting the j-th second frame image into a second processor of the electronic device, and performing super-resolution processing on the j-th second frame image through the second processor to obtain the j-th target frame image, wherein j is greater than i, and j is a positive integer not greater than N.

[0136] In one embodiment of the present disclosure, the processing module 120 is further configured to execute: performing interpolation processing on the second frame image of the jth order through the second processor to obtain the target frame image of the jth order.

[0137] In one embodiment of the present disclosure, the processing module 120 is further configured to execute: obtaining a second super-resolution processing duration of the second processor for the second frame image of the jth order; based on the first super-resolution processing duration and the second super-resolution processing duration, identifying whether a super-resolution end condition is met; if the super-resolution end condition is met, stopping super-resolution processing on the second frame image in the storage space.

[0138] In one embodiment of the present disclosure, after stopping the super-resolution processing of the second frame image in the storage space, the processing module 120 is further configured to execute: determining the undisplayed target frame image, the second frame image that has not completed the super-resolution processing, and the first frame image in the first frame image set as candidate frame images; sorting the multiple candidate frame images from early to late according to the display time; and displaying the multiple candidate frame images in sequence according to the sorting of the multiple candidate frame images.

[0139] In one embodiment of the present disclosure, before determining the first N first frame images in the first frame image set of the video as the second frame image, the determination module 110 is further configured to execute: receiving a speed-up playback instruction for the video.

[0140] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0141] The video processing device provided by the embodiments of the present disclosure can, before super-resolution processing of the video, pre-set multiple first frame images of the video as second frame images and store them in the storage space of the electronic device, and start super-resolution processing of the video when the number of second frame images in the storage space reaches N, that is, delay super-resolution processing by N frames, so that in the process of super-resolution processing of the video, multiple frame images to be super-resolution processed can be directly obtained from the storage space, which can save the time for obtaining the frame images to be super-resolution processed, effectively avoid the problem of long super-resolution processing time and video playback jamming caused by long acquisition time of the frame images to be super-resolution processed, especially avoid the problem of video playback jamming caused by insufficient computing power of the mobile terminal when the super-resolution algorithm is deployed on the mobile terminal, thereby improving the efficiency of video super-resolution processing.

[0142] Figure 7 is a block diagram of an electronic device 200 according to an exemplary embodiment.

[0143] like Figure 7 As shown, the electronic device 200 includes:

[0144] The memory 210 and the processor 220, a bus 230 connecting different components (including the memory 210 and the processor 220), the memory 210 stores a computer program, and when the processor 220 executes the program, the video processing method described in the embodiment of the present disclosure is implemented.

[0145] Bus 230 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0146] The electronic device 200 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 200, including volatile and non-volatile media, removable and non-removable media.

[0147] The memory 210 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 240 and / or cache memory 250. The electronic device 200 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 260 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 230 via one or more data medium interfaces. The memory 210 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0148] A program / utility 280 having a set (at least one) of program modules 270 may be stored, for example, in the memory 210, such program modules 270 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 270 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0149] The electronic device 200 may also communicate with one or more external devices 290 (e.g., keyboards, pointing devices, displays 291, etc.), one or more devices that enable a user to interact with the electronic device 200, and / or any device that enables the electronic device 200 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 292. Furthermore, the electronic device 200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 293. Figure 7 As shown, the network adapter 293 communicates with other modules of the electronic device 200 via the bus 230. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0150] The processor 220 executes various functional applications and data processing by running the programs stored in the memory 210 .

[0151] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the video processing method of the embodiment of the present disclosure, which will not be repeated here.

[0152] The electronic device provided by the embodiment of the present disclosure can execute the video processing method as described above. Before super-resolution processing is performed on the video, multiple first frame images of the video can be pre-set as second frame images and stored in the storage space of the electronic device. When the number of second frame images in the storage space reaches N, super-resolution processing of the video is started, that is, super-resolution processing is delayed by N frames. Therefore, in the process of super-resolution processing of the video, multiple frame images to be super-resolution processed can be directly obtained from the storage space, which can save the time for obtaining the frame images to be super-resolution processed, effectively avoiding the problem of long super-resolution processing time and video playback jamming caused by long acquisition time of the frame images to be super-resolution processed, especially avoiding the problem of video playback jamming caused by insufficient computing power of the mobile terminal when the super-resolution algorithm is deployed on the mobile terminal, thereby improving the efficiency of video super-resolution processing.

[0153] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the video processing method provided by the present disclosure are implemented.

[0154] Alternatively, the computer readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0155] In order to implement the above embodiments, the present disclosure further provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor of an electronic device, the video processing method as described above is implemented.

[0156] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0157] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that: include: Determine the first N first frame images in the first frame image set of the video as the second frame image, and move the second frame image from the first frame image set to the storage space of the electronic device, wherein the first frame images are sorted from early to late according to display time; If the number of the second frame images in the storage space reaches N, super-resolution processing is performed on the second frame images in the storage space to obtain a target frame image, wherein N is a positive integer greater than 1.

2. The method according to claim 1, characterized in that The method further comprises: If the super-resolution processing of the first second frame image is completed, taking out the first second frame image from the storage space, wherein the plurality of second frame images are sorted from early to late according to display time; If all first frames of images in the first frame of image set have not been taken out, determining the first first frame of image in the first frame of image set as the second frame of image, and moving the second frame of image from the first frame of image set to the storage space; The sorting of the plurality of second frame images in the storage space is updated, and the super-resolution processing of the second frame images in the storage space and subsequent steps thereof are returned to be executed until all the first frame images in the first frame image set are taken out.

3. The method according to claim 1, characterized in that The method further comprises: Determine the target frame image that is not displayed as the third frame image, and sort the plurality of third frame images from early to late according to display time; The plurality of third frame images are displayed in sequence according to the order of the plurality of third frame images.

4. The method according to claim 1, characterized in that: The super-resolution processing of multiple second frame images is performed in parallel.

5. The method according to claim 2, characterized in that: N is the number of cores of the first processor of the electronic device, and the super-resolution processing of the second frame image in the storage space to obtain the target frame image includes: The second frame image of the i-th order is input to the i-th core of the first processor, and the i-th core performs super-resolution processing on the second frame image of the i-th order to obtain the i-th target frame image, wherein i is a positive integer not greater than N.

6. The method according to claim 5, characterized in that The N cores of the first processor are respectively deployed with super-resolution models, and the super-resolution processing of the second frame image ranked i-th by the i-th core to obtain the target frame image ranked i-th includes: The second frame image of the i-th sorting is input into the super-resolution model deployed by the i-th kernel, and the second frame image of the i-th sorting is super-resolution processed by the super-resolution model deployed by the i-th kernel to obtain the target frame image of the i-th sorting.

7. The method according to claim 5, characterized in that The method further comprises: Obtaining a first super-resolution processing duration of the i-th kernel for the second frame image of the i-th sort; If the first super-resolution processing time is greater than a set threshold, the second frame image of the jth order is input into the second processor of the electronic device, and the second processor performs super-resolution processing on the second frame image of the jth order to obtain the target frame image of the jth order, wherein j is greater than i and j is a positive integer not greater than N.

8. The method according to claim 7, characterized in that The step of performing super-resolution processing on the second frame image of the jth order by the second processor to obtain the target frame image of the jth order includes: The second processor performs interpolation processing on the second frame image of the jth order to obtain the target frame image of the jth order.

9. The method according to claim 7, characterized in that: The method further comprises: Obtaining a second super-resolution processing duration of the second processor for the second frame image of the jth order; Based on the first super-resolution processing duration and the second super-resolution processing duration, identifying whether a super-resolution end condition is met; If the super-resolution end condition is met, the super-resolution processing of the second frame image in the storage space is stopped.

10. The method according to claim 9, characterized in that After stopping the super-resolution processing on the second frame image in the storage space, the method further includes: Determine the undisplayed target frame image, the second frame image that has not completed super-resolution processing, and the first frame image in the first frame image set as candidate frame images; Sorting the plurality of candidate frame images from early to late according to display time; The plurality of candidate frame images are displayed in sequence according to their order.

11. The method according to any one of claims 1 to 10, characterized in that Before determining the first N first frames of the video as the second frame of the video, the method further includes: Receive a speed-up playback instruction for the video.

12. A video processing device, characterized in that: include: A determination module is configured to determine the first N first frame images in the first frame image set of the video as the second frame image, and move the second frame image from the first frame image set to a storage space of the electronic device, wherein the first frame images are sorted from early to late according to display time; The processing module is configured to perform super-resolution processing on the second frame images in the storage space to obtain a target frame image if the number of the second frame images in the storage space reaches N, wherein N is a positive integer greater than 1.

13. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 11 are implemented.

Images