Super-resolution processing method of target video, storage medium and electronic equipment

Abstract

The invention discloses a super-resolution processing method and device for a target video, a storage medium and electronic equipment. The method comprises the following steps: in response to a super-resolution processing request executed on a target video, performing super-resolution processing on the target video by using a first super-resolution model to obtain a first video, the super-resolution processing request being used for requesting to perform intelligent super-resolution processing on the target video, a corresponding relation exists between the first super-division model and the first multiple speed of the first video; and in response to a speed-multiplying adjustment request executed on the first video, a second super-division model is used to perform super-division processing on the first video after speed-multiplying adjustment to obtain a second video, the speed-multiplying adjustment request is used to request to adjust the first speed-multiplying speed to the second speed-multiplying speed, and the second super-division model and the second speed-multiplying speed have a corresponding relationship. According to the invention, the technical problem of low super-resolution processing accuracy of the target video is solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method, device, storage medium and electronic device for super-resolution processing of a target video. Background Art

[0002] When the player plays a video, if the user's local network is poor, or if the user chooses to play the video in standard definition or ultra-high definition to save traffic, or if the video source is not clear, the player will perform super-resolution processing when the video is decoded to enhance the image quality. This way, you can experience the viewing effect of ultra-high-definition video with a low-definition video stream.

[0003] However, the amount of calculation required for super-resolution processing is relatively large, especially when the playback multiples change. Using super-resolution processing will result in frame loss (poor picture continuity experience) or freeze problems.

[0004] The reason is that the super-resolution processing mechanism lacks flexible adaptability to changes in playback multiples. Since the processing strategy is not adjusted according to different playback multiples, super-resolution processing, while improving image quality, also introduces problems in terms of coherence and fluency, which in turn leads to the problem of low super-resolution processing accuracy of the target video. Therefore, there is a problem of low super-resolution processing accuracy of the target video.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present application provide a method, device, storage medium and electronic device for super-resolution processing of a target video, so as to at least solve the technical problem of low accuracy of super-resolution processing of the target video.

[0007] According to one aspect of an embodiment of the present application, a super-resolution processing method for a target video is provided, comprising: in response to a super-resolution processing request for a target video, using a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing of the target video, and there is a correspondence between the first super-resolution model and the first speed of the first video; in response to a speed adjustment request for the first video, using a second super-resolution model to perform super-resolution processing on the first video after the speed adjustment to obtain a second video, wherein the speed adjustment request is used to request adjustment of the first speed to a second speed, and there is the above-mentioned correspondence between the second super-resolution model and the second speed.

[0008] According to another aspect of an embodiment of the present application, a super-resolution processing device for a target video is also provided, including: a first super-resolution unit, for responding to a super-resolution processing request for the target video, using a first super-resolution model, to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing of the target video, and there is a correspondence between the first super-resolution model and the first speed of the first video; a second super-resolution unit, for responding to a speed adjustment request for the first video, using a second super-resolution model, to perform super-resolution processing on the first video after the speed adjustment, to obtain a second video, wherein the speed adjustment request is used to request adjustment of the first speed to a second speed, and there is the above-mentioned correspondence between the second super-resolution model and the second speed.

[0009] As an optional solution, the above-mentioned device also includes: an acquisition unit, which is used to obtain at least one second super-resolution model having the above-mentioned corresponding relationship with the above-mentioned second speed from at least two candidate super-resolution models before using the second super-resolution model to super-resolution the above-mentioned first video after adjusting the speed to obtain the second video, wherein the above-mentioned at least two candidate super-resolution models include the above-mentioned first super-resolution model and the above-mentioned at least one second super-resolution model.

[0010] As an optional solution, the above-mentioned acquisition unit includes: a first acquisition module, used to acquire at least one second super-resolution model having the above-mentioned corresponding relationship with the above-mentioned second speed from the portrait super-resolution model, the subtitle super-resolution model, and the logo super-resolution model, wherein the above-mentioned portrait super-resolution model is used to perform intelligent super-resolution processing on the portrait area in the above-mentioned target video, the above-mentioned subtitle super-resolution model is used to perform intelligent super-resolution processing on the subtitle area in the above-mentioned target video, and the above-mentioned logo super-resolution model is used to perform intelligent super-resolution processing on the logo area in the above-mentioned target video.

[0011] As an optional solution, the above-mentioned acquisition unit includes: a second acquisition module, used to obtain at least one second super-resolution model having the above-mentioned corresponding relationship with the above-mentioned second speed from the super-resolution models of at least two precisions, wherein the above-mentioned at least two candidate super-resolution models include the super-resolution models of at least two precisions.

[0012] As an optional solution, the above-mentioned acquisition unit includes: a third acquisition module, which is used to acquire at least one second super-resolution model having the above-mentioned corresponding relationship with the above-mentioned second speed from at least two portrait super-resolution models with at least two precisions, at least two subtitle super-resolution models with at least two precisions, and at least two logo super-resolution models with at least two precisions, when the above-mentioned at least two candidate super-resolution models include the above-mentioned super-resolution models with at least two precisions.

[0013] As an optional solution, the third acquisition module includes: an execution submodule, which is used to execute the following steps until the at least one second super-resolution model is obtained: determining a portrait super-resolution model of a first accuracy from the portrait super-resolution models of the at least two accuracy, determining a subtitle super-resolution model of a second accuracy from the subtitle super-resolution models of the at least two accuracy, and determining a logo super-resolution model of a third accuracy from the logo super-resolution models of the at least two accuracy; the first super-resolution processing time corresponding to the portrait super-resolution model of the first accuracy, the second super-resolution processing time corresponding to the subtitle super-resolution model of the second accuracy, and the third super-resolution processing time corresponding to the logo super-resolution model of the third accuracy. If the sum of the time consumed between the first super-resolution processing time, the second super-resolution processing time and the third super-resolution processing time does not satisfy the super-resolution processing time condition, the first precision portrait super-resolution model, the second precision subtitle super-resolution model and the third precision logo super-resolution model are adjusted until the at least one second super-resolution model is obtained.

[0014] As an optional solution, the execution submodule includes at least one of the following: a first adjustment subunit, used to adjust the portrait super-resolution model of the first precision to a portrait super-resolution model of the fourth precision, wherein the at least two portrait super-resolution models of the fourth precision include the portrait super-resolution model of the fourth precision; a second adjustment subunit, used to adjust the subtitle super-resolution model of the second precision to a subtitle super-resolution model of the fifth precision, wherein the at least two subtitle super-resolution models of the fifth precision include the subtitle super-resolution model of the fifth precision; a third adjustment subunit, used to adjust the logo super-resolution model of the third precision to a logo super-resolution model of the sixth precision, wherein the at least two logo super-resolution models of the sixth precision include the logo super-resolution model of the sixth precision; a fourth adjustment subunit, used to adjust the portrait super-resolution model of the first precision to empty; a fifth adjustment subunit, used to adjust the subtitle super-resolution model of the second precision to empty; a sixth adjustment subunit, used to adjust the logo super-resolution model of the third precision to empty.

[0015] As an optional solution, the second super-resolution unit includes: an increase module, which is used to respond to the speed increase request executed on the first video to obtain the first video after the speed is increased, wherein the speed adjustment request includes the speed increase request; the first super-resolution module is used to use the first model to perform super-resolution processing on the first video after the speed is increased to obtain the second video, wherein the first video after the speed adjustment includes the first video after the speed is increased, the second super-resolution model includes the first model, and the super-resolution processing time corresponding to the first model is less than or equal to that of the first super-resolution model The corresponding super-resolution processing time; or, a reduction module, used to respond to a speed reduction request executed on the above-mentioned first video, to obtain the first video after the speed is reduced, wherein the above-mentioned speed adjustment request includes the above-mentioned speed reduction request; a second super-resolution module, used to use a second model to super-resolution the above-mentioned first video after the speed is reduced, to obtain the above-mentioned second video, wherein the above-mentioned first video after the speed adjustment includes the above-mentioned first video after the speed is reduced, the above-mentioned second super-resolution model includes the above-mentioned second model, and the super-resolution processing time corresponding to the above-mentioned second model is greater than or equal to the super-resolution processing time corresponding to the above-mentioned first super-resolution model.

[0016] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the super-resolution processing method of the target video as described above.

[0017] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the super-resolution processing method of the target video through the computer program.

[0018] In an embodiment of the present application, in response to a super-resolution processing request for a target video, a first super-resolution model is used to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing of the target video, and there is a correspondence between the first super-resolution model and the first speed of the first video; in response to a speed adjustment request for the first video, a second super-resolution model is used to perform super-resolution processing on the first video after the speed adjustment to obtain a second video, wherein the speed adjustment request is used to request adjustment of the first speed to a second speed, and there is the above-mentioned correspondence between the second super-resolution model and the second speed.

[0019] When receiving a super-resolution processing request for a target video, this embodiment will first use the first super-resolution model to process the video. The selection of this model is not random, but is closely related to the initial playback speed (first speed) of the video, ensuring that the processing process matches the playback state. In this way, this embodiment can optimize the effect of super-resolution processing for a specific playback speed, thereby improving the enhancement effect of image quality.

[0020] Further, when the user adjusts the video playback multiple, the present embodiment will respond to this change and switch to the second super-resolution model that is adapted to the new playback speed (second speed) for further processing. The key to this dynamic adjustment strategy is that an accurate correspondence is established between different super-resolution models and different playback speeds. This means that no matter how the playback speed changes, the present embodiment can always find the super-resolution model that best suits the current state for processing, thereby ensuring the accuracy and efficiency of the super-resolution processing.

[0021] From a technical point of view, the effectiveness of this embodiment lies in making full use of the flexibility and configurability of the super-resolution model. By customizing a special super-resolution model for different playback speeds, the computational load and resource allocation in the processing process can be more accurately controlled, thereby effectively avoiding the occurrence of frame loss and jamming problems. This not only improves the accuracy of super-resolution processing, but also significantly improves the user's viewing experience, thereby achieving the purpose of customizing a special super-resolution model for different playback speeds and more accurately controlling the computational load and resource allocation in the processing process, thereby achieving the technical effect of improving the accuracy of super-resolution processing of the target video, thereby solving the technical problem of low accuracy of super-resolution processing of the target video. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] Figure 1 is a schematic diagram of an application environment of an optional super-resolution processing method of a target video according to an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of a process of an optional super-resolution processing method of a target video according to an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of an optional super-resolution processing method of a target video according to an embodiment of the present application;

[0026] Figure 4 is a schematic diagram of another optional super-resolution processing method of a target video according to an embodiment of the present application;

[0027] Figure 5 is a schematic diagram of another optional super-resolution processing method of a target video according to an embodiment of the present application;

[0028] Figure 6 is a schematic diagram of another optional super-resolution processing method of a target video according to an embodiment of the present application;

[0029] Figure 7 is a schematic diagram of another optional super-resolution processing method of a target video according to an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of an optional super-resolution processing device for a target video according to an embodiment of the present application;

[0031] Fig. 9 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] According to one aspect of the embodiments of the present application, a method for super-resolution processing of a target video is provided. Optionally, as an optional implementation, the method for super-resolution processing of the target video can be applied to, but is not limited to, Figure 1In the environment shown, the environment may include, but is not limited to, a user device 102 and a server 112 . The user device 102 may include, but is not limited to, a display 104 , a processor 106 , and a memory 108 . The server 112 includes a database 114 and a processing engine 116 .

[0035] The specific process can be as follows:

[0036] Step S102, the user device 102 obtains relevant requests triggered on the target video, such as a request for super-resolution processing performed on the target video and / or a request for speed adjustment performed on the first video;

[0037] Step S104, sending the relevant request to the server 112 via the network 110;

[0038] Steps S106-S108, the server 112, through the processing engine 116, responds to a request for super-resolution processing on the target video, uses a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, and / or responds to a request for speed adjustment on the first video, uses a second super-resolution model to perform super-resolution processing on the first video after speed adjustment to obtain a second video;

[0039] Step S110 , sending the first video and / or the second video to the user device 102 via the network 110 , displaying the first video and / or the second video on the display 104 , and storing the first video and / or the second video in the memory 108 .

[0040] remove Figure 1 In addition to the examples shown, the terminal device can be a terminal device configured with a target client, which can include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, etc. The target client can be a video client, an instant messaging client, a browser client, an education client, etc. The network can include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The server can be a single server, or a server cluster composed of multiple servers, or a cloud server. The above is only an example, and no limitation is made to this in this embodiment.

[0041] Optionally, as an optional implementation, as Figure 2 As shown, the super-resolution processing method of the target video can be performed by an electronic device, and the electronic device can be, for example, Figure 1The user device or server shown in the figure comprises:

[0042] S202, in response to a super-resolution processing request for a target video, using a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing on the target video, and there is a corresponding relationship between the first super-resolution model and a first speed of the first video;

[0043] S204, in response to a speed adjustment request for the first video, super-resolution is performed on the first video after the speed adjustment using a second super-resolution model to obtain a second video, wherein the speed adjustment request is used to request that the first speed be adjusted to a second speed, and there is a corresponding relationship between the second super-resolution model and the second speed.

[0044] In an optional embodiment, the super-resolution processing request refers to an instruction issued by the system requesting intelligent super-resolution processing of the target video.

[0045] In an optional embodiment, the first super-resolution model refers to the first algorithm model used to perform super-resolution processing on the target video, and its selection and application are intended to optimize the video quality at a specific playback speed (first speed).

[0046] In an optional embodiment, the first video refers to a video output with improved resolution and clarity after being processed by the first super-resolution model.

[0047] In an optional embodiment, the corresponding relationship refers to the matching and optimization relationship between the first super-resolution model and the first speed of the first video, or the matching and optimization relationship between the second super-resolution model and the second speed, to ensure maximization of video quality at a specific playback speed.

[0048] Specifically, when the system of the present embodiment receives a super-resolution processing request for a target video, it uses a pre-set first super-resolution model to perform super-resolution processing on the video. This processing is intended to improve the resolution and clarity of the video, thereby obtaining a first video with higher quality. Importantly, this super-resolution processing request is issued specifically for the purpose of performing intelligent super-resolution processing on the target video, which means that the processing process is not a simple amplification, but a complex algorithm is used to enhance the video details. At the same time, there is a matching relationship between the first super-resolution model selected for use and the playback speed (first speed) of the first video obtained after the processing, that is, the model is selected and applied on the basis of optimizing the video performance effect at a specific playback speed.

[0049] As an example, let's assume that there is an old home video with a resolution of 480p, and the user wants to upscale it to 1080p so that it can be viewed on a modern HDTV. The user sends a super-resolution processing request through the system, and the system responds to this request by processing the video using a first super-resolution model optimized for a standard playback speed (e.g., 1x speed). After processing, the user obtains a first video with a resolution of 1080p and significantly improved clarity, and the video performs best when played at 1x speed.

[0050] In an optional embodiment, the speed adjustment request is an instruction issued by the user to the system, requesting to change the playback speed of the first video from a first speed to a second speed.

[0051] In an optional embodiment, the second super-resolution model is a super-resolution processing algorithm model specially selected to meet the quality requirements of the video when it is played at the second speed.

[0052] In an optional embodiment, the second video: after being processed by a second super-resolution model, a high-quality video output adapted for second-speed playback is produced.

[0053] Specifically, when the user wishes to adjust the speed of the first video that has been processed by super-resolution, the system of this embodiment will respond to this request and adjust the playback speed of the video from the first speed to the second speed. However, the speed adjustment may affect the clarity and fluency of the video. In order to maintain the quality of the video, after the speed adjustment, the system will again use the second super-resolution model corresponding to the second speed to perform super-resolution processing on the video, thereby obtaining a second video that still maintains high quality at the adjusted speed.

[0054] To further illustrate, let's assume that a user watches a documentary that has been super-resolution processed and wants to quickly browse some of the content at 1.5 times the speed. The user sends a speed adjustment request through the system interface, and the system then increases the playback speed of the video to 1.5 times. In order to maintain the video clarity at this new playback speed, the system uses a second super-resolution model optimized for 1.5 times the speed to process the video again. In the end, the user gets a second video that has a faster playback speed while maintaining high resolution and smoothness.

[0055] It should be noted that this embodiment describes a continuous video processing flow. First, the system responds to a super-resolution processing request for a target video, and uses a first super-resolution model that matches the original playback speed of the video (first speed) to improve the resolution of the video, thereby obtaining a first video. Subsequently, if the user adjusts the speed of the first video, such as speeding up or slowing down the playback, the system will respond to this speed adjustment request, and use a second super-resolution model that is compatible with the adjusted playback speed (second speed) to perform super-resolution processing on the video again, so as to ensure that the video still maintains a high-quality display effect at a changing speed, and finally obtain a second video.

[0056] To further illustrate, optionally assume that the user has a low-resolution lecture video and wants to improve its clarity and speed up the playback to save time. First, the user requests the system to super-resolution the video, and the system processes the video using a first super-resolution model designed for normal playback speed to obtain a first video with enhanced clarity. Then, the user requests to double the video playback speed, and the system responds to this request and processes the first video again using a second super-resolution model optimized for double-speed playback, and finally the user obtains a second video that is both clear and fast.

[0057] In practical applications, the design and optimization of super-resolution models are key. Different playback speeds may require different algorithms to handle interpolation and detail enhancement between video frames to ensure that the smoothness and clarity of the video are not affected by speed changes. In addition, with the development of deep learning technology, super-resolution models can more intelligently adapt to different video content and playback conditions, providing a higher quality viewing experience.

[0058] Through the embodiment provided by the present application, when a super-resolution processing request for a target video is received, the present embodiment will first use the first super-resolution model to process the video. The selection of this model is not random, but is closely related to the initial playback speed (first speed) of the video, ensuring that the processing process matches the playback state. In this way, the present embodiment can optimize the effect of super-resolution processing for a specific playback speed, thereby improving the enhancement effect of image quality.

[0059] Further, when the user adjusts the video playback multiple, the present embodiment will respond to this change and switch to the second super-resolution model that is adapted to the new playback speed (second speed) for further processing. The key to this dynamic adjustment strategy is that an accurate correspondence is established between different super-resolution models and different playback speeds. This means that no matter how the playback speed changes, the present embodiment can always find the super-resolution model that best suits the current state for processing, thereby ensuring the accuracy and efficiency of the super-resolution processing.

[0060] From a technical point of view, the effectiveness of this embodiment lies in making full use of the flexibility and configurability of the super-resolution model. By customizing a special super-resolution model for different playback speeds, the computational load and resource allocation in the processing process can be more accurately controlled, thereby effectively avoiding the occurrence of frame loss and jamming problems. This not only improves the accuracy of super-resolution processing, but also significantly improves the user's viewing experience, thereby achieving the purpose of customizing a special super-resolution model for different playback speeds, more accurately controlling the computational load and resource allocation in the processing process, thereby achieving the technical effect of improving the accuracy of super-resolution processing of the target video.

[0061] As an optional solution, before using the second super-resolution model to perform super-resolution processing on the first video after the speed adjustment to obtain the second video, the method further includes:

[0062] At least one second super-resolution model having a corresponding relationship with the second speed is obtained from at least two candidate super-resolution models, wherein the at least two candidate super-resolution models include a first super-resolution model and at least one second super-resolution model.

[0063] In an optional embodiment, candidate super-resolution models refer to multiple super-resolution processing models available for selection, which may be optimized for different playback speeds or other conditions.

[0064] In an optional embodiment, the second speed refers to the video playback speed adjusted as requested by the user.

[0065] It should be noted that in the content described in this embodiment, a step is added before selecting and using the second super-resolution model. This step is to select at least one second super-resolution model that has a corresponding relationship with the adjusted second speed from at least two candidate super-resolution models. This means that after the speed adjustment, the system does not directly apply a fixed super-resolution model, but selects the most suitable one from multiple models according to the adjusted speed for super-resolution processing.

[0066] To further illustrate, let's assume that a video platform provides multiple speed playback options, and has prepared corresponding optimized super-resolution models for each speed. When a user adjusts a video from normal speed to 1.25 times speed, the system will select the model corresponding to 1.25 times speed from multiple candidate super-resolution models including the normal speed model and the 1.25 times speed model for super-resolution processing. In this way, no matter which playback speed the user chooses, the system can ensure the best video quality.

[0067] By introducing the step of selecting from multiple candidate super-resolution models, the system can more accurately match the video processing requirements at different playback speeds. This refined processing method not only improves the clarity and smoothness of the video at various playback speeds, but also provides users with a more consistent and high-quality viewing experience. At the same time, it also demonstrates the development trend of video processing technology in terms of intelligence and personalization.

[0068] As an optional solution, obtaining at least one second super-resolution model having a corresponding relationship with the second speed from at least two candidate super-resolution models includes:

[0069] From the portrait super-resolution model, the subtitle super-resolution model, and the logo super-resolution model, at least one second super-resolution model having a corresponding relationship with the second speed is obtained, wherein the portrait super-resolution model is used to perform intelligent super-resolution processing on the portrait area in the target video, the subtitle super-resolution model is used to perform intelligent super-resolution processing on the subtitle area in the target video, and the logo super-resolution model is used to perform intelligent super-resolution processing on the logo area in the target video.

[0070] In an optional embodiment, a portrait super-resolution model: a model specifically used to perform intelligent super-resolution processing on the portrait area in the video, aiming to improve the clarity and details of the portrait.

[0071] In an optional embodiment, a subtitle super-resolution model: a model specifically used to perform intelligent super-resolution processing on the subtitle area in the video to ensure that the subtitles remain clear and readable at different playback speeds.

[0072] In an optional embodiment, a logo super-resolution model: a model specifically used to perform intelligent super-resolution processing on logo areas in a video, aiming to enhance the visibility and recognizability of the logo.

[0073] It should be noted that this embodiment describes the process of selecting at least one second super-resolution model corresponding to the second speed from multiple candidate super-resolution models. These candidate models include super-resolution models specifically used to process specific areas in the video (such as portraits, subtitles, and logos). According to the adjustment of the video playback speed, the system will select the most suitable model for super-resolution processing to ensure that the specific area can still maintain high quality at a changing speed.

[0074] To further illustrate, let's assume a video of a sports game, which contains key information such as the portraits of athletes, game scores, and team logos. When the user chooses to play this video at a faster speed, the system will select those models that match the current playback speed from the portrait super-resolution model, subtitle super-resolution model, and logo super-resolution model to process the video. In this way, even when playing at a fast speed, the user can clearly see the athletes' movements, the game scores, and the logos of each team.

[0075] By selecting a model that matches the second speed from multiple super-resolution models for specific areas, the system can ensure that key areas in the video (such as portraits, subtitles, and logos) remain high quality and clear after adjusting the playback speed. This processing method not only improves the video viewing experience, but also allows users to accurately obtain key information in the video at different playback speeds.

[0076] As an optional solution, obtaining at least one second super-resolution model having a corresponding relationship with the second speed from at least two candidate super-resolution models includes:

[0077] At least one second super-resolution model having a corresponding relationship with the second speed is obtained from the super-resolution models of at least two accuracies, wherein the at least two candidate super-resolution models include the super-resolution models of at least two accuracies.

[0078] In an optional embodiment, super-resolution models of different precisions refer to super-resolution processing models with different clarity or resolution enhancement capabilities. These models may be optimized for different precision requirements to provide different levels of video quality improvement.

[0079] It should be noted that this embodiment describes the process of selecting at least one second super-resolution model corresponding to the second speed from at least two super-resolution models of different precision. The "precision" here refers to the degree of clarity or resolution improvement that the super-resolution model can achieve when processing the video. According to the adjustment of the video playback speed, the system will select a super-resolution model that matches the adjusted speed and meets the specific precision requirements for processing.

[0080] To further illustrate, let's assume that a video editing software provides two super-resolution models with different precisions: a high-definition model and a standard model. When the user adjusts the video playback speed to twice the speed, the software will select a high-definition model that matches the double-speed playback from the two models based on the current requirements (such as the output quality set by the user). In this way, even in the case of accelerated playback, the user can watch a relatively higher-definition video.

[0081] By selecting a model that matches the second speed from multiple super-resolution models of different precisions, the system can ensure that the video quality reaches the precision level expected by the user while maintaining the flexibility of adjusting the video playback speed. This processing method provides users with a more delicate and personalized video viewing experience, meeting the diverse needs of different users for video quality and playback speed.

[0082] As an optional solution, when the at least two candidate super-resolution models include super-resolution models of at least two precisions, obtaining at least one second super-resolution model having a corresponding relationship with the second speed from the at least two candidate super-resolution models includes:

[0083] At least one second super-resolution model having a corresponding relationship with the second speed is obtained from at least two portrait super-resolution models with precision, at least two subtitle super-resolution models with precision, and at least two logo super-resolution models with precision.

[0084] In an optional embodiment, at least two precision portrait super-resolution models are provided: a model for performing super-resolution processing on the portrait area in the video, each model providing a different clarity or resolution improvement level.

[0085] In an optional embodiment, at least two subtitle super-resolution models with different precision are provided: models specifically used to improve the clarity of subtitle areas in videos, and these models have different precision levels.

[0086] In an optional embodiment, at least two precision logo super-resolution models are provided: a super-resolution model for optimizing the logo area in the video, each model providing different levels of detail enhancement according to the precision requirement.

[0087] It should be noted that the description of this embodiment further refines the process of selecting a second super-resolution model corresponding to the second speed from the candidate super-resolution models. Here, the candidate super-resolution models are not only classified by function (such as portraits, subtitles, logos), but also provide at least two models of different precision in each functional category. The system will select the most suitable model from these super-resolution models of portraits, subtitles, and logos for video processing based on the adjusted playback speed (second speed) and the required precision.

[0088] To further illustrate, let's assume that an online education platform provides two super-resolution models, HD and standard, to process course videos. When a user chooses to watch a video containing a lecturer's portrait, course subtitles, and textbook logos at a faster speed, the platform will select the model that best matches the current playback speed and meets the user's clarity requirements from the HD and standard precision portrait super-resolution models, subtitle super-resolution models, and logo super-resolution models to process the video. In this way, even in fast playback mode, users can clearly see the lecturer's expression, course subtitles, and textbook logos.

[0089] By combining super-resolution models of different feature types (portraits, subtitles, logos) and different precisions, the system can adjust the video playback speed while ensuring that the clarity and recognition of key areas meet the user's expectations. This processing method that takes into account the playback speed and precision requirements significantly improves the flexibility and overall quality of video viewing, bringing users a more personalized visual experience.

[0090] As an optional solution, obtaining at least one second super-resolution model having a corresponding relationship with the second speed from at least two portrait super-resolution models with precision, at least two subtitle super-resolution models with precision, and at least two logo super-resolution models with precision includes:

[0091] Perform the following steps until at least one second super-resolution model is obtained:

[0092] S1-1, determining a portrait super-resolution model of first precision from at least two portrait super-resolution models of precision, determining a subtitle super-resolution model of second precision from at least two subtitle super-resolution models of precision, and determining a logo super-resolution model of third precision from at least two logo super-resolution models of precision;

[0093] S1-2, when the sum of the first super-resolution processing time corresponding to the portrait super-resolution model of the first precision, the second super-resolution processing time corresponding to the subtitle super-resolution model of the second precision, and the third super-resolution processing time corresponding to the logo super-resolution model of the third precision satisfies the super-resolution processing time condition corresponding to the second speed, the portrait super-resolution model of the first precision, the subtitle super-resolution model of the second precision, and the logo super-resolution model of the third precision are determined as second super-resolution models, and at least one second super-resolution model is obtained;

[0094] S1-3, when the total time of the first super-resolution processing time, the second super-resolution processing time and the third super-resolution processing time does not meet the super-resolution processing time condition, the first precision portrait super-resolution model, the second precision subtitle super-resolution model and the third precision logo super-resolution model are adjusted until at least one second super-resolution model is obtained.

[0095] In an optional embodiment, the first / second / third precision portrait / subtitle / logo super-resolution model refers to the super-resolution model with specific precision selected by the system when processing portrait, subtitle, and logo areas, respectively.

[0096] In an optional embodiment, the first / second / third super-resolution processing time refers to the time required to process a video using a first / second / third precision super-resolution model, respectively.

[0097] In an optional embodiment, the super-resolution processing time condition refers to the maximum super-resolution processing time that the system can tolerate under the second speed playback.

[0098] It should be noted that this embodiment describes the process of selecting at least one second super-resolution model corresponding to the second speed from multiple portrait super-resolution models, subtitle super-resolution models and logo super-resolution models of different precisions. This process involves determining the precision of the super-resolution model of each functional type (portrait, subtitle, logo), and considering whether the sum of the time consumed by these models when processing the video meets the processing time requirement at a specific playback speed (second speed). If not, the system will adjust the precision of the selected model until a model combination that meets the conditions is found.

[0099] To further illustrate, optionally assume that a video editing system provides three super-resolution models with high, medium and low precision to process the portrait, subtitle and logo areas in the video. When the user adjusts the video playback speed to 1.5 times, the system will first try to select a high-precision portrait super-resolution model, a medium-precision subtitle super-resolution model and a low-precision logo super-resolution model. If the total processing time of these three models exceeds the maximum processing time allowed at 1.5 times the speed, the system may adjust to select the super-resolution model accuracy of portraits and subtitles, and maintain the low-precision logo super-resolution model to meet the time-consuming conditions.

[0100] By comprehensively considering the super-resolution model accuracy and processing time of different functional areas (portraits, subtitles, logos), the system can ensure that the clarity and processing efficiency of key video areas are maintained while adjusting the video playback speed. This method of dynamically adjusting the model accuracy not only improves the flexibility and efficiency of video processing, but also provides users with a smoother and more personalized video viewing experience.

[0101] As an optional solution, the portrait super-resolution model of the first precision, the subtitle super-resolution model of the second precision, and the logo super-resolution model of the third precision are adjusted, including at least one of the following:

[0102] Adjusting the portrait super-resolution model of the first precision to a portrait super-resolution model of the fourth precision, wherein the at least two portrait super-resolution models of the fourth precision include the portrait super-resolution model of the fourth precision;

[0103] Adjusting the subtitle super-resolution model of the second precision to a subtitle super-resolution model of the fifth precision, wherein at least two subtitle super-resolution models of the fifth precision include the subtitle super-resolution model of the fifth precision;

[0104] Adjusting the third precision logo super-resolution model to a sixth precision logo super-resolution model, wherein at least two of the second precision logo super-resolution models include the sixth precision logo super-resolution model;

[0105] Adjust the portrait super-resolution model of the first accuracy to empty;

[0106] Adjust the second-precision subtitle super-resolution model to empty;

[0107] Adjust the third precision logo super-resolution model to empty.

[0108] In an optional embodiment, the fourth / fifth / sixth precision portrait / subtitle / logo super-resolution model refers to super-resolution models with different precisions that may be selected for use during the system adjustment process, and these models belong to part of at least two pre-defined precision levels.

[0109] In an optional embodiment, the model is adjusted to empty: this means that in certain cases, the system may decide not to use a certain type of super-resolution model (such as a portrait, subtitle, or logo super-resolution model) to save processing time or meet other conditions.

[0110] It should be noted that this embodiment illustrates the specific operations that the system can take when adjusting the super-resolution model to meet the specific playback speed and processing time requirements. The adjustment process includes changing the accuracy of the super-resolution model of portraits, subtitles, and logos, or setting the accuracy of certain models to null (i.e., not using the model). These adjustments are intended to ensure that the combination of the selected models can maintain the clarity of the key areas of the video as much as possible while meeting the processing speed requirements.

[0111] To further illustrate, it is optional to assume that the system initially selects a high-precision portrait super-resolution model, a medium-precision subtitle super-resolution model, and a low-precision logo super-resolution model. If the total processing time of these models exceeds the maximum allowed time, the system may make one of the following adjustments: adjust the high-precision portrait super-resolution model to medium-precision, or adjust the medium-precision subtitle super-resolution model to low-precision, or even adjust the low-precision logo super-resolution model to empty (i.e., do not process the logo area).

[0112] By flexibly adjusting the accuracy of portrait, subtitle, and logo super-resolution models or setting certain models to null, the system is able to maintain efficient video processing capabilities under different playback speeds and processing requirements. This dynamic adjustment strategy not only improves the adaptability and flexibility of the system, but also ensures that key video areas maintain optimal clarity when possible, thereby providing users with a smoother and more personalized video playback experience.

[0113] As an optional solution, in response to a speed adjustment request for a first video, a second super-resolution model is used to perform super-resolution processing on the first video after the speed adjustment to obtain a second video, including:

[0114] S2-1, in response to a speed increase request executed on a first video, obtaining the first video after the speed is increased, wherein the speed adjustment request includes a speed increase request;

[0115] S2-2, using the first model, super-resolution processing is performed on the first video after the speed is increased to obtain a second video, wherein the first video after the speed is adjusted includes the first video after the speed is increased, the second super-resolution model includes the first model, and the super-resolution processing time corresponding to the first model is less than or equal to the super-resolution processing time corresponding to the first super-resolution model; or,

[0116] S3-1, in response to a speed reduction request executed on a first video, obtaining the first video after the speed is reduced, wherein the speed adjustment request includes a speed reduction request;

[0117] S3-2, using the second model, super-resolution the first video after the speed is reduced to obtain a second video, wherein the first video after the speed is adjusted includes the first video after the speed is reduced, the second super-resolution model includes the second model, and the super-resolution processing time corresponding to the second model is greater than or equal to the super-resolution processing time corresponding to the first super-resolution model.

[0118] In an optional embodiment, the speed increase / decrease request refers to a request issued by a user or a system to change the video playback speed, which is to speed up the playback speed (increase the speed) and slow down the playback speed (decrease the speed).

[0119] In an optional embodiment, the first / second model: the first model and the second model are two different super-resolution processing models, and their selection depends on the speed adjustment of the video. The first model is suitable for video processing after the speed is increased, and its processing time is less than or equal to the time of the original first super-resolution model; the second model is suitable for video processing after the speed is reduced, and its processing time is greater than or equal to the time of the original first super-resolution model.

[0120] It should be noted that the description of this embodiment involves adjusting the speed of a video (increasing or decreasing it), and then using a corresponding super-resolution model (a first model or a second model) to perform super-resolution processing on the video after the speed adjustment to obtain a second video with improved quality. According to the direction of the speed adjustment (increasing or decreasing), the system will select different super-resolution models to process the video, and the selection of these models is based on the comparison of their respective processing time with the processing time of the original first super-resolution model.

[0121] To further illustrate, optionally assume that a user is watching an online video and decides to speed up the playback to save time. In response to the speed increase request, the system increases the video playback speed to 1.5 times the original speed. Subsequently, the system uses the first model to super-resolution the accelerated video to ensure that the video can still maintain a high clarity when it is played quickly. Since the first model has a high processing efficiency (short time consumption), it can complete the processing quickly and provide the user with a smooth and clear accelerated video playback experience.

[0122] Conversely, if the user chooses to slow down the video playback to watch the content more carefully, the system will use the second model to super-resolution the slowed down video. Although the second model may take a little longer to process, it can provide more subtle image quality enhancement to adapt to the reduced playback speed.

[0123] By dynamically selecting the appropriate super-resolution processing model (first model or second model) according to the speed adjustment of the video, the system can provide a consistent and high-quality video viewing experience at different playback speeds. This flexibility and adaptability enables the video processing system to better meet the personalized needs of users, providing excellent video quality whether browsing content quickly or appreciating details carefully.

[0124] As an optional solution, for ease of understanding, the above-mentioned super-resolution processing method of the target video is applied to the video playback scenario. During video playback, the player faces many challenges, such as poor user network environment, traffic usage restrictions, or clarity issues of the video source itself. In order to meet these challenges and improve user experience, the player introduces artificial intelligence (AI) super-resolution technology in the video decoding stage, aiming to convert low-definition video streams into ultra-high-definition viewing effects. However, the implementation of AI super-resolution technology also has certain problems.

[0125] First, AI super-resolution processing is computationally intensive, especially when distinguishing characteristic areas (such as explicit features of portraits, subtitle areas, and station logos) for super-resolution, which places high demands on device performance. In fact, only a few high-end devices can support more than 2x AI super-resolution image quality enhancement, while most mid- and low-end devices may experience frame drops or freezes when using AI super-resolution technology when playing at high magnifications, seriously affecting viewing continuity and user experience.

[0126] To solve this problem, this embodiment proposes a dynamic adaptive mechanism in operation. This mechanism estimates the optimal AI super-resolution model parameter strategy under different playback multiples by real-time statistics of the average time consumption of the AI ​​super-resolution model. This strategy can give full play to the performance of the device chip, ensure the optimization of AI super-resolution image quality enhancement under the premise of smooth playback without frame loss, thereby greatly improving the user's viewing experience, as follows:

[0127] Real-time statistics and adaptive selection: By real-time statistics of the running time of each precision model, this embodiment can accurately estimate the AI ​​super-resolution time of the current frame at a specific multiple, and adaptively select the most appropriate precision model for AI super-resolution image quality enhancement. This method not only fully utilizes the hardware performance, but also maximizes the effect of AI image quality enhancement.

[0128] Dynamic adjustment of model selection parameters: This embodiment presets model selection parameter caches at 1x, 2x, and 3x playback, and dynamically adjusts during operation to achieve the optimal model selection strategy after stabilization. This design effectively improves the parameter instability problem that may occur when the multiple is frequently switched, ensuring that the optimal state is quickly reached.

[0129] Intelligent decision-making when switching multiples: When switching playback multiples, the system will determine whether the AI ​​super-resolution calculation time requirements are met based on the model selection parameters under the current multiples. If met, the AI ​​super-resolution function will continue; otherwise, the function will be temporarily disabled until it is automatically re-enabled when switching back to a supported multiple. This intelligent decision-making mechanism ensures smooth playback and efficient use of the AI ​​super-resolution function.

[0130] In summary, through the application of dynamic adaptive mechanism, this embodiment successfully solves the problem that AI super-resolution image quality enhancement is difficult to run smoothly and stably under high-multiple playback on low-end and mid-end devices. This technological innovation not only improves the quality of video playback, but also saves traffic for users, while reducing the bandwidth cost and pressure on the server, achieving multi-faceted benefit improvement.

[0131] In an optional embodiment, the overall process is as follows Figure 3 As shown in the figure, it covers multiple key links from video frame decoding to final rendering to ensure efficient video playback and optimized image quality enhancement. The following is a detailed description of the process:

[0132] S4-1, video frame decoding and feature extraction:

[0133] The player first decodes the video stream sent by the server and converts it into continuous video image data. During this process, the system also extracts a set of feature interval sequences in the image. These feature intervals are the basis for subsequent super-resolution calculations and are crucial to improving image quality. After processing with a special fusion algorithm, the system obtains the final set of regions to be super-resolution calculated, mainly including the portrait explicit feature area, subtitle area, and station logo area.

[0134] S4-2, multiple playback mode and model parameter selection:

[0135] When the user adjusts the multiple playback mode, the system triggers an optimization process to test and determine the optimal selection parameters for each model under the current multiple playback mode. This step ensures that at different playback speeds, the system can select the most appropriate model for image quality enhancement, thereby maintaining high definition and smoothness of the video.

[0136] S4-3, Enhanced computational decision making based on model parameters:

[0137] Based on the model parameters selected in the current multiple playback mode, the system will determine whether to perform AI super-resolution image quality enhancement calculations. If the model parameters are empty, no enhancement processing will be performed to save computing resources and maintain smooth playback. This decision-making mechanism ensures efficient use of resources while avoiding unnecessary computing overhead.

[0138] S4-4, perform AI super-resolution image quality enhancement calculation:

[0139] After determining to perform AI super-resolution image quality enhancement calculation, the system will perform accurate super-resolution calculation based on the model parameters and final area set selected in the current multiple mode. This step uses advanced AI technology to improve the image quality of key areas in the video, thereby significantly improving the user's viewing experience.

[0140] S4-5, real-time feedback updates and estimate adjustments:

[0141] After executing AI super-resolution image quality enhancement calculations, the system will update the real-time statistical values ​​of the precision model currently selected for calculation based on the actual time consumed. This feedback mechanism ensures that the system can dynamically adjust the model parameters to adapt to different device performance and playback conditions, thereby continuously optimizing the image quality enhancement effect.

[0142] S4-6, Rendering and Output:

[0143] Finally, the image frames processed by AI super-resolution image quality enhancement calculation will replace the original frames and enter the rendering queue for image rendering. This step ensures that the enhanced video frames can be seamlessly integrated into the playback process, providing users with a high-quality video viewing experience.

[0144] In summary, the overall process achieves an optimal balance between video playback and image quality enhancement through designed steps and dynamic adjustment mechanisms. From a technical perspective, the process makes full use of advanced AI super-resolution technology and real-time feedback mechanisms to ensure an excellent video viewing experience on different devices and playback conditions.

[0145] In an optional embodiment, the final region set of the current frame is generated in the server-side video encoding stage and further processed by a fusion algorithm. This set mainly covers three key regions: a portrait explicit feature region, a subtitle region, and a station logo region.

[0146] To further illustrate, the optional Figure 4As shown, the current video frame is decoded, and the final three major region sets, namely the human portrait dominant region set, the subtitle region set, and the logo region set, are obtained through fusion calculation. They are respectively subjected to AI super-resolution calculation with the human portrait model, subtitle model, and logo model of the currently selected precision version to obtain the super-resolved images, and then fused with the remaining image regions of the original image to obtain the AI super-resolved image. Note: When the selected human portrait model, subtitle model, and logo model may be empty (the three models will not be empty at the same time), the AI super-resolution calculation for the corresponding regions will not be performed.

[0147] In an optional embodiment, the AI super-resolution technology mainly uses three major types of models: the human portrait dominant model, the subtitle model, and the logo model. Each type of model is further divided into three versions: high-precision, medium-precision, and low-precision according to the precision requirements. This classification method aims to adapt to different scenarios and computing resources to achieve the best image quality enhancement effect.

[0148] In terms of computational complexity, the human portrait dominant model has the largest computational amount, followed by the subtitle model, and finally the logo model. At the same time, the higher the precision of the model, the greater its computational amount. Specifically, the computational amount of the high-precision model is greater than that of the medium-precision model, and the medium-precision model is greater than that of the low-precision model. The computational amounts of the models with different precisions are in a proportional relationship. For example, if the computational time of the low-precision model is n, then the computational time of the medium-precision model is x times that of n, and the high-precision model is y times that of n (where x < y). This proportional relationship is used as the initial default value for calculation when the initial model statistical value is 0 and will be updated in real time with subsequent actual applications.

[0149] For further illustration by example, optionally Figure 5 As shown, when there is a selected model for AI super-resolution calculation in the human portrait dominant region, the statistical value g (representing one of g1, g2, g3) of the currently selected precision version model will be updated. The update formula is g = (g * (n - 1) + tn / sn) / n. The initial value of g is 0 when starting to play, otherwise it is obtained from the conversion of the relationship between the models with different precisions. Here, n is the number of times the human portrait dominant model is calculated. If no model is selected for AI super-resolution calculation, no update will be made. Similarly, the update of the subtitle model statistical value and the update of the logo model statistical value are similar to the update of the human portrait dominant model statistical value.

[0150] In an optional embodiment, in order to accurately control the AI ​​super-resolution process, the system needs to count the time consumption of various models in real time, including the portrait explicit model time consumption (g1, g2, g3), the subtitle model time consumption (g4, g5, g6) and the logo model time consumption (g7, g8, g9). These statistical values ​​are only updated when the model actually participates in the calculation, in milliseconds per second (ms / s), indicating the processing time per unit area. These real-time data are crucial for dynamically adjusting model selection and parameters, ensuring the efficiency and adaptability of AI super-resolution processing.

[0151] To further illustrate, the optional Figure 6 As shown in the figure, when the model is optimally adjusted under the current multiple playback, the AI ​​super-resolution time consumption will be estimated for the current frame. The estimated time consumption of the portrait explicit area is t1, s1 is the area of ​​the portrait explicit area of ​​the current frame, if the current portrait explicit model is not empty, then the estimated value of t1 is one of s1*g1, s1*g2, s1*g3, if the current portrait explicit model is empty, then the estimated value of t1 is 0. Similarly, the estimated time consumption t2 for subtitles and t3 for station logos can be calculated. The total time consumption t=t1+t2+t3. (Since the calculation of the remaining area is relatively simple, and the time consumption is very small relative to t1, t2, and t3, it can be ignored).

[0152] In an optional embodiment, in the AI ​​super-resolution processing, the selection of the model follows certain priorities and logic. Since the portrait explicit model has the largest amount of computation, when playing at high magnifications, in order to meet real-time requirements, the system will tend to select a subtitle model or a station logo model with less computation, as well as a low-precision version of the model for enhanced calculations. When there is no suitable model available to meet real-time performance requirements, the system will turn off the AI ​​super-resolution function until the user reduces the playback magnification and will not re-enable it. This strategy ensures that the system can provide a smooth and high-quality video playback experience under different playback conditions.

[0153] To further illustrate, the optional Figure 7 As shown, the specific steps are as follows:

[0154] S5-1, when the multiple playback changes, the model adjustment strategy logic of the changed multiple playback will be triggered. If the model parameters under the changed multiple playback are empty, the model parameters under the changed multiple playback are used for initialization. If they are not empty, the model parameters under the changed multiple playback are used. Transfer to S5-2.

[0155] S5-2, judging whether the multiple playback is changed this time, whether it is increased or decreased. If the multiple playback is increased, jump to S5-3, otherwise jump to S5-4.

[0156] S5-3. First, calculate the time consumption t1 for AI super-resolution using the current model parameters. Then, starting from the logo log model, subtitle model, and portrait dominance model in sequence, gradually reduce the precision of each model. When all models reach the lowest precision, cancel the selection of each model in this order and calculate the estimated time consumptions t2, t3, and t4 (t1 > t2 > t3 > t4), etc. Then, determine if the estimated value t is less than the changed multiple playback threshold M (empirical values: 10 ms for 3x speed, 15 ms for 2x speed, 30 ms for 1x speed). When the t value is 0, it means that the model selection parameters for the changed multiple playback are all empty, and subsequently, the AI super-resolution function for the changed multiple playback will be turned off. If not empty, select the model selection parameters at the current t value as the optimal AI super-resolution parameters for the changed multiple playback. Jump to S5-5.

[0157] S5-4. First, calculate the time consumption t1 for AI super-resolution statistics using the current model parameters. Then, starting from the portrait dominance model, subtitle model, and logo log model in sequence, gradually increase the precision of each model. If there is a situation where a model is empty, re-select the low-precision model sequence for sequential increase and calculate the estimated values t2, t3, t4 (t1 < t2 < t3 < t4), etc. Then, repeatedly determine if the t value is greater than the changed multiple playback threshold M (empirical values: 10 ms for 3x speed, 15 ms for 2x speed, 30 ms for 1x speed), that is, repeatedly calculate until the first estimated t value is greater than the threshold. Then, select the previously calculated threshold t (at this time, t is less than the threshold M). If the t value is 0, it means that the model selection parameters for the changed multiple playback are all empty, and subsequently, the AI super-resolution function for the changed multiple playback will be turned off. If not 0, it is the optimal AI super-resolution parameter for the changed multiple playback. Go to S5-5.

[0158] S5-5. If all the selected models are empty, turn off the AI super-resolution function. If not all are empty, turn on the AI super-resolution function and end the strategy adjustment.

[0159] Through the embodiments provided in this application, a dynamic adaptive mechanism is proposed. This mechanism estimates the optimal super-resolution model parameter strategy for devices at different playback multiples by real-time monitoring the average processing time of the AI super-resolution model. This can not only make full use of the chip performance of the device but also ensure the smoothness of the video during high-multiple playback and avoid frame loss. At the same time, this mechanism can also maximize the effect of AI super-resolution in image quality enhancement, thereby significantly improving the user's viewing experience.

[0160] It can be understood that in the specific implementation of this application, data related to user information, etc., is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0161] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0162] According to another aspect of the embodiment of the present application, a target video super-resolution processing device for implementing the target video super-resolution processing method is also provided. Figure 8 As shown, the device comprises:

[0163] A first super-resolution unit 802 is configured to, in response to a super-resolution processing request for a target video, use a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing on the target video, and there is a corresponding relationship between the first super-resolution model and the first speed of the first video;

[0164] The second super-resolution unit 804 is used to respond to a speed adjustment request for the first video, use a second super-resolution model, and super-resolution the first video after the speed adjustment to obtain a second video, wherein the speed adjustment request is used to request to adjust the first speed to a second speed, and there is a corresponding relationship between the second super-resolution model and the second speed.

[0165] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0166] As an optional solution, the device also includes:

[0167] The acquisition unit is used to perform super-resolution processing on the first video after adjusting the speed using the second super-resolution model, and before obtaining the second video, obtain at least one second super-resolution model having a corresponding relationship with the second speed from at least two candidate super-resolution models, wherein the at least two candidate super-resolution models include the first super-resolution model and at least one second super-resolution model.

[0168] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0169] As an optional solution, obtain the unit, including:

[0170] The first acquisition module is used to acquire at least one second super-resolution model having a corresponding relationship with the second speed from the portrait super-resolution model, the subtitle super-resolution model, and the logo super-resolution model, wherein the portrait super-resolution model is used to perform intelligent super-resolution processing on the portrait area in the target video, the subtitle super-resolution model is used to perform intelligent super-resolution processing on the subtitle area in the target video, and the logo super-resolution model is used to perform intelligent super-resolution processing on the logo area in the target video. For specific embodiments, reference may be made to the example shown in the super-resolution processing method of the target video, which will not be repeated in this example.

[0171] As an optional solution,

[0172] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0173] As an optional solution, obtain the unit, including:

[0174] The second acquisition module is used to acquire at least one second super-resolution model having a corresponding relationship with the second speed from the super-resolution models of at least two accuracies, wherein the at least two candidate super-resolution models include the super-resolution models of at least two accuracies.

[0175] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0176] As an optional solution, obtain the unit, including:

[0177] The third acquisition module is used to acquire at least one second super-resolution model having a corresponding relationship with the second speed from the portrait super-resolution models with at least two precisions, the subtitle super-resolution models with at least two precisions, and the logo super-resolution models with at least two precisions when the at least two candidate super-resolution models include super-resolution models with at least two precisions.

[0178] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0179] As an optional solution, the third acquisition module includes:

[0180] The execution submodule is used to execute the following steps until at least one second super-resolution model is obtained:

[0181] Determine a portrait super-resolution model of first precision from at least two portrait super-resolution models of precision, determine a subtitle super-resolution model of second precision from at least two subtitle super-resolution models of precision, and determine a logo super-resolution model of third precision from at least two logo super-resolution models of precision;

[0182] When the sum of the first super-resolution processing time corresponding to the portrait super-resolution model of the first precision, the second super-resolution processing time corresponding to the subtitle super-resolution model of the second precision, and the third super-resolution processing time corresponding to the logo super-resolution model of the third precision satisfies the super-resolution processing time condition corresponding to the second speed, the portrait super-resolution model of the first precision, the subtitle super-resolution model of the second precision, and the logo super-resolution model of the third precision are determined as second super-resolution models to obtain at least one second super-resolution model;

[0183] When the total time of the first super-resolution processing time, the second super-resolution processing time, and the third super-resolution processing time does not meet the super-resolution processing time condition, the first precision portrait super-resolution model, the second precision subtitle super-resolution model, and the third precision logo super-resolution model are adjusted until at least one second super-resolution model is obtained.

[0184] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0185] As an optional solution, the execution submodule includes at least one of the following:

[0186] A first adjustment subunit is used to adjust the portrait super-resolution model of the first precision to a portrait super-resolution model of the fourth precision, wherein the at least two portrait super-resolution models of the fourth precision include the portrait super-resolution model of the fourth precision;

[0187] A second adjustment subunit is used to adjust the subtitle super-resolution model of the second precision to a subtitle super-resolution model of the fifth precision, wherein the at least two subtitle super-resolution models of the fifth precision include the subtitle super-resolution model of the fifth precision;

[0188] a third adjustment subunit, configured to adjust the third precision logo super-resolution model to a sixth precision logo super-resolution model, wherein the at least two precision logo super-resolution models include the sixth precision logo super-resolution model;

[0189] A fourth adjustment subunit, configured to adjust the portrait super-resolution model of the first precision to be empty;

[0190] A fifth adjustment subunit, used for adjusting the subtitle super-resolution model of the second precision to be empty;

[0191] The sixth adjustment subunit is used to adjust the third precision landmark super-resolution model to be empty.

[0192] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0193] As an optional solution, the second super-resolution unit 804 includes:

[0194] An increasing module, configured to obtain the first video after the speed is increased in response to a speed increase request executed on the first video, wherein the speed adjustment request includes a speed increase request;

[0195] A first super-resolution module is used to use a first model to perform super-resolution processing on the first video after the speed is increased to obtain a second video, wherein the first video after the speed is adjusted includes the first video after the speed is increased, the second super-resolution model includes the first model, and the super-resolution processing time corresponding to the first model is less than or equal to the super-resolution processing time corresponding to the first super-resolution model; or

[0196] A reduction module, configured to obtain the first video after the speed is reduced in response to a speed reduction request executed on the first video, wherein the speed adjustment request includes a speed reduction request;

[0197] The second super-resolution module is used to use the second model to super-resolution the first video after the speed is reduced to obtain a second video, wherein the first video after the speed is adjusted includes the first video after the speed is reduced, the second super-resolution model includes the second model, and the super-resolution processing time corresponding to the second model is greater than or equal to the super-resolution processing time corresponding to the first super-resolution model.

[0198] For a specific embodiment, reference may be made to the example shown in the above-mentioned method for super-resolution processing of the target video, which will not be described in detail in this example.

[0199] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned target video super-resolution processing method is also provided. The electronic device can be but is not limited to Figure 1 The user device 102 or the server 112 shown in FIG. 1 is used as an example to illustrate the electronic device as the user device 102. Fig. 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.

[0200] In an optional embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0201] In an optional embodiment, the processor may be configured to perform the following steps through a computer program:

[0202] S1, in response to a super-resolution processing request for a target video, using a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing on the target video, and there is a corresponding relationship between the first super-resolution model and a first speed of the first video;

[0203] S2, in response to a speed adjustment request for the first video, uses a second super-resolution model to super-resolution the first video after the speed adjustment to obtain a second video, wherein the speed adjustment request is used to request that the first speed be adjusted to a second speed, and there is a corresponding relationship between the second super-resolution model and the second speed.

[0204] Alternatively, a person skilled in the art may understand that: Fig. 9 The structure shown is for illustration only. Fig. 9 The structure of the electronic device is not limited. Fig. 9 More or fewer components (such as network interfaces, etc.) as shown in, or with Fig. 9 Different configurations are shown.

[0205] Among them, the memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the super-resolution processing method and device of the target video in the embodiment of the present application. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, to realize the above-mentioned super-resolution processing method of the target video. The memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include a memory remotely arranged relative to the processor 904, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks and combinations thereof. Among them, the memory 902 can be specifically but not limited to being used to store information such as a first super-resolution model, a first video, a second super-resolution model, and a second video. As an example, such as Fig. 9 As shown, the memory 902 may include, but is not limited to, the first super-resolution unit 802 and the second super-resolution unit 804 in the super-resolution processing device of the target video. In addition, it may also include, but is not limited to, other module units in the super-resolution processing device of the target video, which will not be repeated in this example.

[0206] Optionally, the transmission device 906 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired network and a wireless network. In one example, the transmission device 906 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 906 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0207] In addition, the above-mentioned electronic device also includes: a display 908 for displaying information such as the above-mentioned first super-resolution model, the first video, the second super-resolution model and the second video; and a connection bus 910 for connecting the various module components in the above-mentioned electronic device.

[0208] In other embodiments, the user device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. Among them, a peer-to-peer network may be formed between the nodes, and any form of computing device, such as a server, user device, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0209] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program / instruction, the computer program / instruction comprising a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions provided by the embodiments of the present application are executed.

[0210] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0211] It should be noted that the computer system of the electronic device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0212] The computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) or the program loaded from the storage part to the random access memory (RAM). In the random access memory, various programs and data required for system operation are also stored. The central processing unit, the read-only memory and the random access memory are connected to each other through a bus. The input / output interface (I / O interface) is also connected to the bus.

[0213] The following components are connected to the input / output interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a local area network card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0214] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit, various functions defined in the system of the present application are executed.

[0215] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0216] In an optional embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0217] S1, in response to a super-resolution processing request for a target video, using a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing on the target video, and there is a corresponding relationship between the first super-resolution model and a first speed of the first video;

[0218] S2, in response to a speed adjustment request for the first video, uses a second super-resolution model to super-resolution the first video after the speed adjustment to obtain a second video, wherein the speed adjustment request is used to request that the first speed be adjusted to a second speed, and there is a corresponding relationship between the second super-resolution model and the second speed.

[0219] Optionally, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0220] In an optional embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the electronic device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0221] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0222] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers or network devices, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0223] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0224] In the several embodiments provided in the present application, it should be understood that the disclosed user equipment can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0225] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0226] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0227] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A super-resolution processing method for a target video, characterized in that: include: In response to a super-resolution processing request for a target video, use a first super-resolution model to perform super-resolution processing on the target video to obtain a first video, wherein the super-resolution processing request is used to request intelligent super-resolution processing on the target video, and there is a corresponding relationship between the first super-resolution model and a first speed of the first video; In response to a speed adjustment request for the first video, a second super-resolution model is used to super-resolution the first video after the speed adjustment to obtain a second video, wherein the speed adjustment request is used to request that the first speed be adjusted to a second speed, and there is a corresponding relationship between the second super-resolution model and the second speed.

2. The method according to claim 1, characterized in that Before using the second super-resolution model to perform super-resolution processing on the first video after the speed adjustment to obtain the second video, the method further includes: At least one second super-resolution model having the corresponding relationship with the second speed is obtained from at least two candidate super-resolution models, wherein the at least two candidate super-resolution models include the first super-resolution model and the at least one second super-resolution model.

3. The method according to claim 2, characterized in that The acquiring, from the at least two candidate super-resolution models, at least one second super-resolution model having the corresponding relationship with the second speed comprises: From the portrait super-resolution model, the subtitle super-resolution model, and the logo super-resolution model, at least one second super-resolution model having the corresponding relationship with the second speed is obtained, wherein the portrait super-resolution model is used to perform intelligent super-resolution processing on the portrait area in the target video, the subtitle super-resolution model is used to perform intelligent super-resolution processing on the subtitle area in the target video, and the logo super-resolution model is used to perform intelligent super-resolution processing on the logo area in the target video.

4. The method according to claim 2, characterized in that: The acquiring, from the at least two candidate super-resolution models, at least one second super-resolution model having the corresponding relationship with the second speed comprises: At least one second super-resolution model having the corresponding relationship with the second speed is obtained from the super-resolution models of at least two accuracies, wherein the at least two candidate super-resolution models include the super-resolution models of the at least two accuracies.

5. The method according to claim 4, characterized in that In a case where the at least two candidate super-resolution models include the super-resolution models of the at least two accuracies, acquiring at least one second super-resolution model having the corresponding relationship with the second speed from the at least two candidate super-resolution models comprises: From at least two portrait super-resolution models with precision, at least two subtitle super-resolution models with precision, and at least two logo super-resolution models with precision, at least one second super-resolution model having the corresponding relationship with the second speed is obtained, wherein the portrait super-resolution model is used to perform intelligent super-resolution processing on the portrait area in the target video, the subtitle super-resolution model is used to perform intelligent super-resolution processing on the subtitle area in the target video, and the logo super-resolution model is used to perform intelligent super-resolution processing on the logo area in the target video.

6. The method according to claim 5, characterized in that The step of acquiring at least one second super-resolution model having the corresponding relationship with the second speed from at least two portrait super-resolution models with precision, at least two subtitle super-resolution models with precision, and at least two logo super-resolution models with precision comprises: Perform the following steps until the at least one second super-resolution model is obtained: Determine a portrait super-resolution model of first precision from the at least two portrait super-resolution models of precision, determine a subtitle super-resolution model of second precision from the at least two subtitle super-resolution models of precision, and determine a logo super-resolution model of third precision from the at least two logo super-resolution models of precision; When the sum of the first super-resolution processing time corresponding to the portrait super-resolution model with the first precision, the second super-resolution processing time corresponding to the subtitle super-resolution model with the second precision, and the third super-resolution processing time corresponding to the logo super-resolution model with the third precision satisfies the super-resolution processing time condition corresponding to the second speed, the portrait super-resolution model with the first precision, the subtitle super-resolution model with the second precision, and the logo super-resolution model with the third precision are determined as the second super-resolution models, to obtain the at least one second super-resolution model; When the sum of the first super-resolution processing time, the second super-resolution processing time, and the third super-resolution processing time does not meet the super-resolution processing time condition, the first precision portrait super-resolution model, the second precision subtitle super-resolution model, and the third precision logo super-resolution model are adjusted until the at least one second super-resolution model is obtained.

7. The method according to claim 6, characterized in that The adjusting the portrait super-resolution model of the first precision, the subtitle super-resolution model of the second precision, and the logo super-resolution model of the third precision includes at least one of the following: Adjusting the portrait super-resolution model of the first precision to a portrait super-resolution model of a fourth precision, wherein the at least two portrait super-resolution models of precision include the portrait super-resolution model of the fourth precision; Adjusting the subtitle super-resolution model of the second precision to a subtitle super-resolution model of the fifth precision, wherein the at least two subtitle super-resolution models of precision include the subtitle super-resolution model of the fifth precision; Adjusting the third precision logo super-resolution model to a sixth precision logo super-resolution model, wherein the at least two precision logo super-resolution models include the sixth precision logo super-resolution model; Adjust the portrait super-resolution model with the first precision to be empty; Adjust the subtitle super-resolution model of the second precision to be empty; The third precision landmark super-resolution model is adjusted to be empty.

8. The method according to any one of claims 1 to 7, characterized in that In response to the speed adjustment request for the first video, using a second super-resolution model to perform super-resolution processing on the first video after the speed adjustment to obtain a second video, comprising: In response to a speed increase request executed on the first video, obtaining the first video after the speed is increased, wherein the speed adjustment request includes the speed increase request; Using the first model, super-resolution processing is performed on the first video after the speed is increased to obtain the second video, wherein the first video after the speed is adjusted includes the first video after the speed is increased, the second super-resolution model includes the first model, and the super-resolution processing time corresponding to the first model is less than or equal to the super-resolution processing time corresponding to the first super-resolution model; or, In response to a speed reduction request executed on the first video, obtaining the first video after the speed is reduced, wherein the speed adjustment request includes the speed reduction request; Using the second model, super-resolution is performed on the first video after the speed is reduced to obtain the second video, wherein the first video after the speed is adjusted includes the first video after the speed is reduced, the second super-resolution model includes the second model, and the super-resolution processing time corresponding to the second model is greater than or equal to the super-resolution processing time corresponding to the first super-resolution model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 8 when executed by an electronic device.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 8 through the computer program.

Images