Super-resolution video quality analysis model generation, video quality analysis method and device

Abstract

The present disclosure is about a super-resolution video quality analysis model generation, video quality analysis method and device, and relates to the field of artificial intelligence technology, including: obtaining multiple sample super-resolution videos corresponding to each sample video and their corresponding label quality information; the multiple sample super-resolution videos of each sample video are obtained by performing multiple super-resolution processes on each sample video; the label quality information is determined based on the comparison quality information of the multiple sample super-resolution videos; based on a preset machine learning model, the multiple sample super-resolution videos are subjected to video quality analysis to obtain the sample quality information of the multiple sample super-resolution videos; based on the label quality information and the sample quality information, the target loss information is determined; based on the target loss information, the preset machine learning model is trained to obtain the target quality analysis model. The embodiment of the present disclosure can improve the accuracy of the target quality analysis model for the video quality analysis of the super-resolution video.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a super-resolution video quality analysis model generation, a video quality analysis method and a device. Background Art

[0002] Artificial intelligence is a technology that simulates human intelligence and aims to create intelligent machines that can learn and process information autonomously. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Specifically, computer vision technology (Computer Vision, CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instrument detection. Computer vision technology usually includes technologies such as video processing, video semantic understanding, video content / behavior recognition, and three-dimensional object reconstruction.

[0003] In the field of video processing, existing video quality analysis methods mainly focus on the pros and cons of coding and decoding models, communication transmission systems, and image enhancement. For the quality analysis of super-resolution videos, the current practice is to use the above-mentioned existing video quality analysis models to perform quality analysis on super-resolution videos. However, since the analysis dimensions of video quality analysis models for different scenarios are different, for example, the quality analysis of video coding requires analysis of the clarity, stability, and block effects of the video, while the quality analysis of super-resolution videos mainly analyzes the detail texture richness of the video frames and the degree of coherence between the previous and next frames. Therefore, using the above-mentioned existing video quality analysis methods to perform quality analysis on super-resolution videos will lead to inaccurate quality analysis results. Summary of the invention

[0004] In view of the above-mentioned technical problems, the present disclosure proposes a super-resolution video quality analysis model generation, a video quality analysis method and a device.

[0005] According to one aspect of an embodiment of the present disclosure, a method for generating a super-resolution video quality analysis model is provided, comprising:

[0006] Acquire multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos; the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processing on each sample video; the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs, and any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents a quality comparison result obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair;

[0007] Based on a preset machine learning model, perform video quality analysis on the multiple sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos;

[0008] Determining target loss information based on the label quality information and the sample quality information;

[0009] Based on the target loss information, the preset machine learning model is trained to obtain a target quality analysis model for the super-resolution video.

[0010] According to another aspect of an embodiment of the present disclosure, a video quality analysis method is provided, including:

[0011] Obtain the super-resolution video to be analyzed;

[0012] Inputting the super-resolution video to be analyzed into the target quality analysis model to perform video quality analysis, and obtaining a target quality analysis result corresponding to the super-resolution video to be analyzed;

[0013] Wherein, the target quality analysis model is obtained based on the above-mentioned super-resolution video quality analysis model generation method.

[0014] According to another aspect of an embodiment of the present disclosure, there is provided a device for generating a super-resolution video quality analysis model, comprising:

[0015] An information acquisition module is used to acquire multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos; the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processes on each sample video; the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs, and any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents a quality comparison result obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair;

[0016] A first quality analysis module, configured to perform video quality analysis on the plurality of sample super-resolution videos based on a preset machine learning model, to obtain sample quality information corresponding to each of the plurality of sample super-resolution videos;

[0017] A loss determination module, configured to determine target loss information based on the label quality information and the sample quality information;

[0018] A training module is used to train the preset machine learning model based on the target loss information to obtain a target quality analysis model for the super-resolution video.

[0019] According to another aspect of an embodiment of the present disclosure, there is provided a video quality analysis device, including:

[0020] A super-resolution video acquisition module is used to acquire the super-resolution video to be analyzed;

[0021] The second quality analysis module is used to input the super-resolution video to be analyzed into the target quality analysis model for video quality analysis to obtain the target quality analysis result corresponding to the super-resolution video to be analyzed; wherein the target quality analysis model is obtained based on the above-mentioned super-resolution video quality analysis model generation method.

[0022] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the above-mentioned super-resolution video quality analysis model generation method or the above-mentioned video quality analysis method.

[0023] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above-mentioned super-resolution video quality analysis model generation method or the above-mentioned video quality analysis method.

[0024] According to another aspect of an embodiment of the present disclosure, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the above-mentioned super-resolution video quality analysis model generation method or the above-mentioned video quality analysis method.

[0025] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0026] By obtaining multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos, wherein the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processing on each sample video, the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs, and any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents the quality comparison result obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair, the collection of sample data sets for super-resolution videos and the generation of corresponding label quality information can be achieved; then, based on a preset machine learning model, video quality analysis is performed on the multiple sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos, target loss information is determined based on the label quality information and the sample quality information, and based on the target loss information, the preset machine learning model is trained to obtain a target quality analysis model for the super-resolution video, so that the target quality analysis model can achieve quality analysis of the super-resolution effect of the super-resolution video, thereby improving the accuracy of the video quality analysis for the super-resolution video.

[0027] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0029] Figure 1 is a schematic diagram of an application system according to an exemplary embodiment;

[0030] Figure 2 is a flow chart of a method for generating a super-resolution video quality analysis model according to an exemplary embodiment;

[0031] Figure 3 is a flow chart of a video quality analysis method according to an exemplary embodiment;

[0032] Figure 4 is a block diagram of a device for generating a super-resolution video quality analysis model according to an exemplary embodiment;

[0033] Figure 5 is a block diagram of a video quality analysis device according to an exemplary embodiment;

[0034] Figure 6 is a block diagram of an electronic device for generating a target quality analysis model or performing video quality analysis on a super-resolution video to be analyzed according to an exemplary embodiment;

[0035] Figure 7 It is a block diagram of another electronic device for generating a target quality analysis model or performing video quality analysis on a super-resolution video to be analyzed according to an exemplary embodiment. DETAILED DESCRIPTION

[0036] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0037] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0038] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0039] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0040] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0041] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing so that computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0042] Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, allowing various application systems to obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0043] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0044] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0045] Cloud storage is a new concept that extends and develops from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0046] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0047] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disk), the physical storage space is divided into stripes in advance. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0048] In short, a database can be seen as an electronic filing cabinet - a place where electronic files are stored, and users can add, query, update, delete, and other operations on the data in the files. The so-called "database" is a collection of data that is stored together in a certain way, can be shared with multiple users, has as little redundancy as possible, and is independent of the application.

[0049] A database management system (DBMS) is a computer software system designed for managing databases. It generally has basic functions such as storage, retrieval, security, and backup. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language used, such as SQL (Structured Query Language), XQuery; or according to performance focus, such as maximum scale, maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMS can cross categories, for example, supporting multiple query languages ​​at the same time.

[0050] In recent years, with the research and progress of artificial intelligence technology, artificial intelligence technology has been widely used in many fields. The solution provided in the embodiment of the present application involves technologies such as computer vision, which is specifically described by the following embodiments:

[0051] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an application system according to an exemplary embodiment. The application system can be used in the super-resolution video quality analysis model generation method or video quality analysis method of the present application. Figure 1 As shown, the application system may at least include a server 01 and a terminal 02 .

[0052] In the embodiment of the present application, the server 01 can be used to generate a target quality analysis model. Specifically, the server 01 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0053] In the embodiment of the present application, the terminal 02 can be used to generate the super-resolution video to be analyzed. The terminal 02 may include physical devices such as smart phones, desktop computers, tablet computers, laptops, smart speakers, vehicle terminals, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, etc., and may also include software running in physical devices, such as applications. The operating system running on the terminal 02 in the embodiment of the present application may include but is not limited to Android system, IOS system, Linux, Windows, etc.

[0054] In addition, it should be noted that Figure 1 What is shown is only one application environment provided by the present disclosure. In actual application, other application environments may also be included. For example, the generation process of the target quality analysis model may also be implemented on the terminal 02.

[0055] In the embodiments of this specification, the terminal 02 and the server 01 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0056] It should be noted that the following figure shows a possible sequence of steps, which is not actually limited to strictly follow this sequence. Some steps can be executed in parallel without relying on each other.

[0057] Specifically, Figure 2 FIG. 1 is a flow chart of a method for generating a super-resolution video quality analysis model according to an exemplary embodiment. Figure 2 As shown, the super-resolution video quality analysis model generation method can be used in electronic devices such as terminals or servers, and specifically may include the following steps:

[0058] S201: Acquire multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos.

[0059] In a specific embodiment, the sample video may refer to a video whose corresponding picture quality is lower than a preset picture quality. The sample video may include a preset low-quality video, etc. Specifically, the model developer may generate a sample video by inputting a sample video to the above terminal.

[0060] In a specific embodiment, the at least one sample video may further include at least one second sample video.

[0061] In a specific embodiment, when at least one sample video includes at least one second sample video, the at least one sample video may be obtained in the following manner:

[0062] Acquire at least one second preset video;

[0063] The at least one second preset video is degraded to obtain at least one second sample video.

[0064] In a specific embodiment, any second preset video may refer to a video whose preset corresponding resolution is greater than the preset resolution. The preset resolution may be used to distinguish high-definition videos. Optionally, the preset resolution may include 720P or 1080I, etc. Specifically, the preset resolution may be set according to actual application needs, and the present disclosure does not limit this. It is understood that the resolution of each second preset video is greater than the above preset resolution.

[0065] In a specific embodiment, each second preset video may be degraded to obtain a second sample video corresponding to each second preset video; accordingly, at least one second sample video may be obtained.

[0066] In a specific embodiment, performing quality degradation processing on at least one second preset video to obtain at least one second sample video may include:

[0067] Performing image blurring processing on each preset video frame in each second preset video from at least one image blurring dimension to obtain a blurred video frame corresponding to each preset video frame;

[0068] At least one second sample video is generated based on blurred video frames corresponding to a plurality of preset video frames in each second preset video.

[0069] In a specific embodiment, the blurred video frame may refer to a blurred video frame.

[0070] In a specific embodiment, image blurring processing may be performed on any preset video frame from any image blurring dimension among a plurality of image blurring dimensions to obtain a blurred video frame corresponding to any of the preset video frames.

[0071] In a specific embodiment, any image blur dimension may correspond to an image blur method. At least one image blur dimension may include one or more of multiple image blur dimensions such as Gaussian blur dimension, downsampling and interpolation dimension, and Gaussian noise dimension. Among them, the image blur processing corresponding to the Gaussian blur dimension can be used to achieve image blur of the video frame by performing image blur filtering on the video frame. The image blur processing corresponding to the downsampling and interpolation dimension can be used to achieve image blur of the video frame by downsampling and interpolating and amplifying the video frame. The image blur processing corresponding to the Gaussian noise dimension can be used to achieve image blur of the video frame by adding Gaussian noise to the video frame.

[0072] In a specific embodiment, the video frame can be input into an image blur filter for Gaussian blur processing to achieve blur processing of the video frame under the Gaussian blur dimension. Specifically, each pixel in the video frame can be Gaussian blurred to obtain Gaussian blurred pixel information corresponding to each pixel; accordingly, the blurred video frame can be generated based on the Gaussian blurred pixel information corresponding to each pixel. Among them, the Gaussian blurred pixel information corresponding to any pixel can represent the pixel value of any of the above pixels after blur processing. Further, in the process of Gaussian blurring for any pixel, the blur area corresponding to any pixel can be determined based on a preset blur radius; based on a normal distribution, the weight corresponding to each pixel in the blur area corresponding to any of the above pixels is determined; based on the weight of each pixel in the blur area, the pixel value of each pixel in the blur area corresponding to any of the above pixels is weighted averaged to obtain the Gaussian blurred pixel information of any of the above pixels. Among them, the preset blur radius can belong to the degradation indication information under the Gaussian blur dimension. Specifically, the preset blur radius can be set according to the actual application needs; optionally, the preset blur radius can be any odd number from 1 to 21.

[0073] In a specific embodiment, the blurring of the video frame in the downsampling and interpolation dimension can be achieved by first downsampling the video frame and then interpolating the downsampled video frame. Specifically, the video frame can be downsampled by a preset number of downsampling times to obtain a reduced video frame; then the reduced video frame can be interpolated to obtain an enlarged video frame; accordingly, the enlarged video frame can be used as a blurred video frame. Further, the size of the enlarged video frame can be half of the size of the video frame before the downsampling process; accordingly, it can be subsequently enlarged to the size of the video frame before the process by super-resolution processing. Among them, the preset number of downsampling times can refer to the number of downsampling processes performed during the image blurring process of the video frame. The preset number of downsampling times can belong to the degradation indication information in the downsampling and interpolation dimension. Optionally, the value range of the preset number of downsampling times can be [2,4]; illustratively, the preset number of downsampling times can be 3. Specifically, in the process of interpolating the reduced video frame, any dimension from a plurality of preset interpolation dimensions may be randomly selected to interpolate the reduced video frame; optionally, the plurality of preset difference dimensions may include a bilinear interpolation dimension, a bicubic interpolation dimension, and a nearest neighbor interpolation dimension, etc. The interpolation processing corresponding to the bilinear interpolation dimension may refer to a bilinear interpolation processing; the interpolation processing corresponding to the bicubic interpolation dimension may refer to a bicubic interpolation processing; and the interpolation processing corresponding to the nearest neighbor interpolation dimension may refer to a nearest neighbor interpolation processing.

[0074] In a specific embodiment, the video frame can be blurred in the Gaussian noise dimension by adding preset Gaussian noise to the video frame. The preset Gaussian noise may refer to a type of noise whose corresponding probability density obeys the Gaussian distribution. The preset Gaussian noise may be determined based on a preset noise mean. The preset noise mean may be degradation indication information under the Gaussian noise dimension. Specifically, the preset noise mean may be set according to actual application needs. Optionally, the value range of the preset noise mean may be [1,20]; exemplarily, the preset noise mean may be 10.

[0075] In a specific embodiment, image blur processing can be performed on any preset video frame from multiple image blur dimensions to obtain a blurred video frame corresponding to any of the preset video frames. Exemplarily, image blur processing can be performed on any preset video frame from a downsampling and re-interpolation dimension and a Gaussian noise dimension to obtain a blurred video frame corresponding to any of the preset video frames; for example, in the case where at least one of the image blur dimensions is a downsampling and re-interpolation dimension and a Gaussian noise dimension, blur processing corresponding to the downsampling and re-interpolation dimension can be performed on any of the preset video frames to obtain a first video frame, and then blur processing corresponding to the Gaussian noise dimension can be performed on the first video frame to obtain a blurred video frame corresponding to any of the preset video frames; or blur processing corresponding to the downsampling and re-interpolation dimension can be performed on any of the preset video frames to obtain a first video frame, and then blur processing corresponding to the Gaussian noise dimension can be performed on the first video frame to obtain a second video frame, and then blur processing corresponding to the downsampling and re-interpolation dimension can be performed on the second video frame to obtain a blurred video frame corresponding to any of the preset video frames, wherein, in the image blur processing process for any of the preset video frames, the total number of downsampling processes can be the preset number of downsampling processes. It is understandable that in the process of performing image blur processing from multiple image blur dimensions, the number of blur processing for each image blur dimension may be multiple times. The present disclosure does not limit the order of processing methods corresponding to different image blur dimensions.

[0076] In a specific embodiment, a second sample video corresponding to any second preset video may be generated based on blurred video frames corresponding to each of a plurality of preset video frames in any second preset video and a time sequence of a plurality of preset video frames in any second preset video.

[0077] In the above embodiment, by acquiring at least one second preset video and performing quality degradation processing on the at least one second preset video respectively, at least one second sample video is obtained, so that rapid expansion of the sample video can be achieved.

[0078] In a specific embodiment, the sample super-resolution video may refer to a video obtained by super-resolution processing. The multiple sample super-resolution videos corresponding to any sample video may be obtained by performing multiple different super-resolution processing on the above-mentioned any sample video. It is understandable that the super-resolution processing methods corresponding to the multiple sample super-resolution videos corresponding to any sample video are different, but the source of the above-mentioned multiple sample super-resolution videos is the same, which is any of the above-mentioned sample videos.

[0079] In a specific embodiment, the multiple sample super-resolution videos corresponding to each sample video may be obtained in the following manner:

[0080] A plurality of different super-resolution processes are performed on each sample video to obtain a plurality of sample super-resolution videos corresponding to each sample video.

[0081] In a specific embodiment, multiple sample super-resolution videos corresponding to any sample video can be obtained by performing multiple super-resolution processing on any sample video. Specifically, the multiple super-resolution processing can include interpolation processing, reconstruction processing, shallow learning processing, or processing based on a deep learning model.

[0082] In a specific embodiment, the label quality information of any sample super-resolution video can be used to provide a reference for the training of the target quality analysis model. The label quality information can be determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs. Among them, any sample super-resolution video pair can be any two of the multiple sample super-resolution videos corresponding to the same sample video. The comparison quality information corresponding to any sample super-resolution video pair can characterize the quality comparison results obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair. Exemplarily, assuming that a variety of different super-resolution processing is applied to the sample video V to obtain sample super-resolution video V1, sample super-resolution video V2 and sample super-resolution video V3, accordingly, it can be determined that the multiple sample super-resolution video pairs corresponding to the above sample video V may include sample super-resolution video pair (V1, V2), sample super-resolution video pair (V2, V3) and sample super-resolution video pair (V1, V3).

[0083] In a specific embodiment, the label quality information corresponding to each of the plurality of sample super-resolution videos may be obtained in the following manner:

[0084] Show multiple sample super-resolved video pairs;

[0085] In response to the comparison quality input instruction, obtaining the comparison quality information corresponding to each sample super-resolution video pair;

[0086] Based on the comparison quality information corresponding to multiple sample super-resolution video pairs, quality data analysis is performed on each sample super-resolution video to obtain label quality information corresponding to each sample super-resolution video.

[0087] In a specific embodiment, multiple sample super-resolution video pairs can be displayed on the analysis terminals corresponding to the multiple analysis objects. Specifically, multiple sample super-resolution video pairs can be displayed on the analysis terminal of each analysis object. Furthermore, each sample super-resolution video pair in the multiple sample super-resolution video pairs can be displayed in sequence on the analysis terminal of any analysis object; or the multiple sample super-resolution video pairs can be displayed synchronously on the analysis terminal of any analysis object. In the process of displaying any sample super-resolution video pair, two super-resolution videos in any one of the sample super-resolution video pairs can be displayed synchronously; specifically, two super-resolution videos in any one of the sample super-resolution video pairs can be displayed in full size at the same time in two different preset areas, or two super-resolution videos in any one of the sample super-resolution video pairs can be displayed synchronously through a slider playback method, which is not limited in the present disclosure.

[0088] In a specific embodiment, the comparison quality information corresponding to any sample super-resolution video pair may include target comparison quality information corresponding to each of the multiple analysis objects. The target comparison quality information of any sample super-resolution video pair corresponding to any analysis object may represent the quality comparison result obtained by quality comparison of any sample super-resolution video pair by any analysis object.

[0089] In a specific embodiment, each analysis object can perform a comparison quality input operation through the analysis terminal corresponding to the analysis object to achieve the input of the target comparison quality information for the above-mentioned sample super-resolution video pair. Specifically, any of the above-mentioned analysis objects can observe any of the displayed sample super-resolution video pairs, and then use the analysis terminal of any of the above-mentioned analysis objects to input the target option in the preset options. Correspondingly, the above-mentioned analysis terminal can generate the target comparison quality information corresponding to any of the above-mentioned analysis objects based on the above-mentioned target option. Among them, the target option can be used to characterize the sample super-resolution video with better quality in any of the above-mentioned sample super-resolution video pairs selected by any of the above-mentioned analysis objects.

[0090] In a specific embodiment, based on the comparison quality information corresponding to a plurality of sample super-resolution video pairs, quality data analysis is performed on each sample super-resolution video to obtain label quality information corresponding to each sample super-resolution video, which may include:

[0091] Determining a target comparison number corresponding to at least one sample video based on target comparison quality information of a plurality of sample super-resolution video pairs;

[0092] Determine at least one first super-resolution video pair corresponding to each sample super-resolution video;

[0093] Determine a target quality quantity corresponding to each sample super-resolution video based on target comparison quality information of at least one first super-resolution video pair corresponding to each sample super-resolution video;

[0094] Based on the number of target comparisons and the number of target qualities corresponding to each sample super-resolution video, a target quality ratio analysis is performed on each sample super-resolution video to obtain the label quality information corresponding to each sample super-resolution video.

[0095] In a specific embodiment, the target comparison number may refer to the quality comparison number of multiple sample super-resolution videos corresponding to at least one sample video. Among them, the quality comparison number of any sample super-resolution video may refer to the number of times quality comparison is performed on any of the above sample super-resolution videos. Exemplarily, taking the example that the analysis objects include 20 analysis objects, and the multiple sample super-resolution video pairs corresponding to the above at least one sample video include 3 sample super-resolution video pairs, it can be determined that each analysis object needs to perform quality comparison on the above 3 different sample super-resolution video pairs respectively, and accordingly, it can be determined that the target comparison number corresponding to the above at least one sample video is 60.

[0096] In a specific embodiment, the target comparison number corresponding to the at least one sample video can be determined based on the number of target comparison quality information of the multiple sample super-resolution video pairs. Specifically, the number of target comparison quality information of the multiple sample super-resolution video pairs can be used as the target comparison number corresponding to the at least one sample video.

[0097] In a specific embodiment, the at least one first super-resolution video pair corresponding to any sample super-resolution video may be a sample super-resolution video pair including any sample super-resolution video among a plurality of sample super-resolution video pairs corresponding to any sample super-resolution video.

[0098] In a specific embodiment, a sample super-resolution video pair including any sample super-resolution video among a plurality of sample super-resolution video pairs corresponding to any sample super-resolution video may be used as at least one first super-resolution video pair corresponding to the above-mentioned sample super-resolution video. The plurality of sample super-resolution video pairs corresponding to any of the above-mentioned sample super-resolution videos may refer to a plurality of sample super-resolution video pairs corresponding to a target sample video. The target sample video corresponding to any of the above-mentioned sample super-resolution videos may refer to a sample video before super-resolution processing of any of the above-mentioned sample super-resolution videos. The plurality of sample super-resolution video pairs corresponding to the target sample video may refer to sample super-resolution video pairs generated based on the plurality of sample super-resolution videos corresponding to the above-mentioned target sample video. The plurality of sample super-resolution videos corresponding to the above-mentioned target sample video may refer to sample super-resolution videos obtained by performing a plurality of different super-resolution processing on the above-mentioned target sample video.

[0099] In a specific embodiment, the target quality quantity corresponding to each sample super-resolution video may be the quantity belonging to the first quality information among the multiple target comparison quality information corresponding to each sample super-resolution video. The first quality information of any sample super-resolution video pair corresponding to any sample super-resolution video may be used to indicate that the quality of any sample super-resolution video is better than another sample super-resolution video in any sample super-resolution video pair.

[0100] In a specific embodiment, at least one third super-resolution video pair corresponding to any sample super-resolution video can be determined based on the target comparison quality information of at least one first super-resolution video pair corresponding to any sample super-resolution video; and then based on the number of at least one third super-resolution video pair corresponding to any sample super-resolution video, the target quality number corresponding to any sample super-resolution video can be determined. Among them, at least one third super-resolution video pair corresponding to any sample super-resolution video can refer to a video pair whose corresponding target comparison quality information belongs to the first quality information in at least one first super-resolution video pair corresponding to any sample super-resolution video. Specifically, in the case where the target comparison quality information corresponding to any first super-resolution video pair in at least one first super-resolution video pair corresponding to any sample super-resolution video belongs to the first quality information, it can be determined that the quality of any sample super-resolution video is better than the quality of another sample super-resolution video in the first super-resolution video pair, and accordingly, any first super-resolution video pair can be used as the third super-resolution video pair. Then, the number of at least one third super-resolution video pair corresponding to any sample super-resolution video can be used as the target quality number corresponding to any sample super-resolution video.

[0101] In a specific embodiment, based on the target comparison times and the target quality quantity corresponding to any sample super-resolution video, the target quality ratio analysis can be performed on any sample super-resolution video to obtain the label quality information corresponding to any sample super-resolution video. Specifically, the label quality information corresponding to any sample super-resolution video can be obtained by the following formula:

[0102]

[0103] Among them, n i is the target quality number corresponding to the i-th sample super-resolution video; N is the number of target comparisons; S i is the label quality information corresponding to the i-th sample super-resolution video.

[0104] In the above embodiment, by displaying multiple sample super-resolution video pairs, in response to the comparison quality input instruction, the comparison quality information corresponding to each sample super-resolution video pair is obtained, and based on the comparison quality information corresponding to the multiple sample super-resolution video pairs, quality data analysis is performed on each sample super-resolution video to obtain the label quality information corresponding to each sample super-resolution video. This can achieve accurate acquisition of the label quality information corresponding to each sample super-resolution video, thereby improving the accuracy of the target quality analysis model.

[0105] In a specific embodiment, the above method may further include:

[0106] Based on the label quality information corresponding to the multiple sample super-resolution videos, quality distribution analysis is performed on the multiple sample super-resolution videos to obtain quality distribution indicator data;

[0107] When the quality distribution indicator data does not satisfy a preset uniform distribution condition, each of the at least one first preset video is degraded to obtain at least one first sample video;

[0108] Performing a plurality of different super-resolution processes on each first sample video to obtain a plurality of first sample super-resolution videos corresponding to each first sample video;

[0109] Accordingly, the video quality analysis of the plurality of sample super-resolution videos is performed based on the preset machine learning model to obtain the sample quality information corresponding to each of the plurality of sample super-resolution videos, which may include:

[0110] Based on a preset machine learning model, video quality analysis is performed on multiple sample super-resolution videos and multiple first sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos.

[0111] In a specific embodiment, the quality distribution index data may characterize the degree of uniformity of distribution of label quality information corresponding to multiple sample super-resolution videos. The quality distribution index data may include peak quality intervals and quality extreme value difference information. The peak quality interval may refer to the interval with the largest number of corresponding interval quality quantities corresponding to multiple different preset quality intervals. The quality extreme value difference information may refer to the difference information between the maximum number and the minimum number of interval quality quantities corresponding to multiple different preset quality intervals. Among them, multiple different preset quality intervals may be multiple intervals divided based on the distributable range of target comparison quality information. The distributable range of target comparison quality information may refer to the range between the minimum target comparison quality information that can be generated and the maximum target comparison quality information. The interval quality quantity corresponding to any preset quality interval may refer to the number of quality information that falls into any of the above-mentioned preset quality intervals in the label quality information corresponding to multiple sample super-resolution videos. Illustratively, the plurality of different preset quality intervals may include [0, 0.1], (0.1, 0.2], (0.2, 0.3], (0.3, 0.4], (0.4, 0.5], (0.5, 0.6], (0.6, 0.7], (0.7, 0.8], (0.8, 0.9], (0.9, 1].

[0112] In a specific embodiment, based on the label quality information corresponding to multiple sample super-resolution videos, the interval quality quantity corresponding to each preset quality interval can be determined; based on the interval quality quantity corresponding to each preset quality interval, the peak quality interval can be determined; then, based on the interval quality quantity corresponding to multiple preset quality intervals, the quality extreme value difference information can be determined. Specifically, the quantity of quality information falling into any preset quality interval in the label quality information corresponding to multiple sample super-resolution videos can be used as the interval quality quantity corresponding to any of the above preset quality intervals, and accordingly, the interval quality quantities corresponding to multiple different preset quality intervals can be obtained; then, the preset quality interval with the largest interval quality quantity corresponding to the interval quality quantities corresponding to the multiple different preset quality intervals can be used as the peak quality interval; then, the maximum interval quality quantity and the minimum interval quality quantity can be determined from the interval quality quantities corresponding to the multiple different preset quality intervals, and accordingly, the quality extreme value difference information can be determined based on the maximum interval quality quantity and the minimum interval quality quantity.

[0113] In a specific embodiment, the preset uniform distribution condition may be a condition for determining whether the label quality information corresponding to multiple sample super-resolution videos is uniformly distributed. The preset uniform distribution condition may include that the peak quality interval is a preset peak quality interval, and the quality extreme value difference information does not exceed a preset deviation amount. Specifically, the preset peak quality interval may be set according to actual application needs; optionally, the preset peak quality interval may be any one of multiple intervals belonging to the middle of multiple different preset quality intervals. Exemplarily, when multiple different preset quality intervals include [0, 0.1], (0.1, 0.2], (0.2, 0.3], (0.3, 0.4], (0.4, 0.5], (0.5, 0.6], (0.6, 0.7], (0.7, 0.8], (0.8, 0.9], (0.9, 1]), the preset peak quality interval may be (0.4, 0.5]. Specifically, the preset deviation number may be set according to actual application needs; optionally, the preset deviation number may be a preset multiple of the minimum number of interval quality numbers corresponding to each of the multiple different preset quality intervals. Optionally, the preset multiple may range from 2 to 4; exemplarily, the preset multiple may be 3.

[0114] In a specific embodiment, when the quality distribution index data satisfies the preset uniform distribution condition, it can be determined that the content of the above-mentioned multiple sample super-resolution videos covers a wide range and is very diverse. Accordingly, the above-mentioned multiple sample super-resolution videos can be used as sample data sets to train the target quality analysis model. Specifically, when the peak quality interval is the preset peak quality interval, and the quality extreme value difference information does not exceed the preset deviation number, it can be determined that the quality distribution index data satisfies the preset uniform distribution condition. It can be understood that when the quality distribution index data satisfies the preset uniform distribution condition, it can be determined that the content of the above-mentioned multiple sample super-resolution videos covers a wide range, is very diverse, and can clearly express quality information close to the subjective perception of the human eye.

[0115] In a specific embodiment, any first preset video may refer to a video having a preset resolution greater than a preset resolution.

[0116] In a specific embodiment, when the peak quality interval is not a preset peak quality interval, or the extreme value difference information of the quality exceeds the preset deviation number, it can be determined that the quality distribution index data does not meet the preset uniform distribution condition, and accordingly, it can be determined that the content coverage of the above-mentioned multiple sample super-resolution videos is not wide enough, that is, further sample data expansion is required; accordingly, each first preset video in at least one first preset video can be degraded to obtain at least one first sample video to achieve sample data expansion. Specifically, any first preset video can be degraded to obtain a first sample video corresponding to any first preset video.

[0117] In a specific embodiment, when the quality distribution indicator data does not satisfy a preset uniform distribution condition, degrading each first preset video in at least one first preset video to obtain at least one first sample video may include:

[0118] When the quality distribution indicator data does not meet the preset uniform distribution condition, quality missing distribution information is determined based on label quality information corresponding to multiple sample super-resolution videos;

[0119] Based on the quality loss distribution information, updating the preset quality degradation indication information;

[0120] Based on the updated preset degradation indication information, degradation processing is performed on each first preset video in the at least one first preset video to obtain at least one first sample video.

[0121] In a specific embodiment, the quality missing distribution information may characterize the distribution missing range of the label quality information corresponding to the multiple sample super-resolution videos. The quality missing distribution information may include at least one missing quality interval. The at least one missing quality interval may be a preset quality interval whose corresponding interval quality quantity among the multiple preset quality intervals is lower than the preset interval quality quantity.

[0122] In a specific embodiment, when the quality distribution indicator data does not meet the preset uniform distribution condition, the interval quality quantities corresponding to multiple different preset quality intervals can be determined based on the label quality information corresponding to multiple sample super-resolution videos, and accordingly, at least one missing quality interval can be determined based on the interval quality quantities corresponding to the multiple different preset quality intervals. Specifically, the preset quality interval whose corresponding interval quality quantity among the multiple preset quality intervals is lower than the preset interval quality quantity can be used as at least one missing quality interval.

[0123] In a specific embodiment, the preset degradation indication information can be used to indicate the degree of degradation during the degradation process. The preset degradation indication information can include one or more degradation indication information under multiple image blur dimensions such as a preset blur radius, a preset number of downsampling times, and a preset noise mean.

[0124] In a specific embodiment, the preset degradation indication information may be updated based on the at least one missing quality interval. Specifically, when there are continuous quality intervals in the at least one missing quality interval, and the number of intervals of the continuous quality intervals is greater than the preset number of intervals, the preset degradation indication information may be updated based on the continuous quality intervals. The continuous quality intervals may refer to multiple continuous quality intervals in the at least one missing quality interval. Specifically, when there are continuous quality intervals in the at least one missing quality interval, and the number of intervals of the continuous quality intervals is greater than the preset number of intervals, the interval positions corresponding to the continuous quality intervals may be determined; when the interval positions corresponding to the continuous quality intervals belong to the first interval positions, the first degradation indication information may be used as the updated preset degradation indication information; when the interval positions corresponding to the continuous quality intervals belong to the second interval positions, the second degradation indication information may be used as the updated preset degradation indication information. The first interval position may refer to a position close to the minimum quality interval among the multiple preset quality intervals; and the second interval position may refer to a position close to the maximum quality interval among the multiple preset quality intervals. The degradation degree corresponding to the first degradation indication information may be greater than the degradation degree corresponding to the second degradation indication information. Exemplarily, the preset degradation indication information before updating includes a preset blur radius of any odd number from 1 to 21, a preset downsampling number in the range of [2, 4], and a preset noise mean in the range of [1, 20]. The first degradation indication information may include a preset blur radius of any odd number from 11 to 21, a preset downsampling number in the range of [3, 4], and a preset noise mean in the range of [10, 20]. Accordingly, the second degradation indication information may include a preset blur radius of any odd number from 1 to 9, a preset downsampling number in the range of [2, 3], and a preset noise mean in the range of [1, 10].

[0125] In a specific embodiment, each video frame in any first preset video can be degraded based on the updated preset degradation indication information to obtain a first sample video corresponding to any of the first preset videos. Specifically, each video frame in any first preset video can be image blurred from at least one image blur dimension based on the updated preset degradation indication information to obtain a first sample video corresponding to any of the first preset videos. Further, in the case where the image blur dimension includes a Gaussian blur dimension, the video frame can be Gaussian blurred based on the updated preset blur radius in the updated preset degradation indication information; in the case where the image blur dimension includes a downsampling and interpolation dimension, the video frame can be downsampled based on the updated preset downsampling times in the updated preset degradation indication information, and then the downsampled video frame can be interpolated; in the case where the image blur dimension includes a Gaussian noise dimension, the updated preset Gaussian noise can be determined based on the updated preset noise mean in the updated preset degradation indication information, and accordingly, the updated preset Gaussian noise can be added to the video frame.

[0126] In the above embodiment, when the quality distribution indicator data does not meet the preset uniform distribution condition, the quality missing distribution information is determined based on the label quality information corresponding to multiple sample super-resolution videos, and the preset degradation indication information is updated based on the quality missing distribution information. Based on the updated preset degradation indication information, each first preset video in at least one first preset video is degraded, which can achieve control of the degree of degradation, thereby achieving targeted generation of missing sample videos, and then achieving targeted expansion of sample data, thereby improving the generation efficiency of sample data.

[0127] In a specific embodiment, by performing a variety of different super-resolution processing on any first sample video, a plurality of first sample super-resolution videos corresponding to any first sample video can be obtained. Furthermore, a plurality of fourth super-resolution video pairs can be generated based on the plurality of first sample super-resolution videos corresponding to any of the above-mentioned first sample videos; accordingly, the plurality of fourth super-resolution video pairs can be displayed; in response to the comparison quality input instruction for the plurality of fourth super-resolution video pairs, the comparison quality information corresponding to each fourth super-resolution video pair can be obtained; based on the comparison quality information corresponding to the plurality of fourth super-resolution video pairs, quality data analysis is performed on each first sample super-resolution video, and label quality information corresponding to each first sample super-resolution video can be obtained.

[0128] In a specific embodiment, based on the label quality information corresponding to the plurality of second sample super-resolution videos, the quality distribution analysis of the plurality of second sample super-resolution videos can be performed to obtain the quality distribution index data corresponding to the plurality of second sample super-resolution videos. When the quality distribution index data corresponding to the plurality of second sample super-resolution videos meet the preset uniform distribution condition, the video quality analysis of the plurality of second sample super-resolution videos can be performed based on the preset machine learning model to obtain the sample quality information corresponding to each of the plurality of second sample super-resolution videos; then, the target loss information is determined by combining the sample quality information corresponding to each of the plurality of second sample super-resolution videos and the label quality information corresponding to the plurality of second sample super-resolution videos; then, the preset machine learning model can be trained based on the target loss information to obtain the target quality analysis model for the super-resolution video. Among them, the plurality of second sample super-resolution videos can include the plurality of sample super-resolution videos and the plurality of first sample super-resolution videos. Further, when the quality distribution index data corresponding to the plurality of second sample super-resolution videos do not meet the preset uniform distribution condition, the sample data can be expanded until the quality distribution index data meets the preset uniform distribution condition.

[0129] In the above embodiment, based on the label quality information corresponding to the multiple sample super-resolution videos, the quality distribution analysis of the multiple sample super-resolution videos is performed to obtain quality distribution index data. When the quality distribution index data does not meet the preset uniform distribution condition, each first preset video in the at least one first preset video is degraded to obtain at least one first sample video. A variety of different super-resolution processing is performed on each first sample video to obtain multiple first sample super-resolution videos corresponding to each first sample video. Sample data expansion can be achieved, which is convenient for improving the diversity of the sample data set, and further, the generation of a data set with wide content coverage and great diversity can be achieved.

[0130] In a specific embodiment, the above method may further include:

[0131] Performing video frame analysis on the video to be extracted to obtain first video index data corresponding to each of the plurality of video frames to be extracted in the video to be extracted, and second video index data corresponding to each of the plurality of video frames to be extracted;

[0132] Based on the first video indicator data corresponding to each of the plurality of video frames to be extracted and the second video indicator data corresponding to each of the plurality of video frames to be extracted, performing continuous picture segment analysis on the video to be extracted to obtain segment extraction indication information;

[0133] Based on the segment extraction indication information, perform video segment extraction processing on each of the multiple sample super-resolution videos corresponding to the video to be extracted, to obtain multiple first video segments corresponding to the video to be extracted;

[0134] From a plurality of first video segments corresponding to the video to be extracted, any two first video segments corresponding to the same video sequence are selected as a second super-resolution video pair.

[0135] In a specific embodiment, the video to be extracted may be any sample video or any sample super-resolution video corresponding to any sample video. The multiple video frames to be extracted may refer to multiple video frames contained in the video to be extracted.

[0136] In a specific embodiment, the first video indicator data corresponding to any video frame to be extracted can be used to characterize the degree of change between any video frame to be extracted and any adjacent video frame of any video frame to be extracted. The second video indicator data corresponding to any video frame to be extracted can be used to characterize whether the picture information contained in any video frame to be extracted is greater than the preset picture information.

[0137] In a specific embodiment, performing video frame analysis on the video to be extracted to obtain first video indicator data corresponding to each of the plurality of video frames to be extracted in the video to be extracted and second video indicator data corresponding to each of the plurality of video frames to be extracted may include:

[0138] From the video to be extracted, obtain the current video frame and the current associated video frame corresponding to the current video frame;

[0139] Performing grayscale processing on the current video frame and the current associated video frame to obtain a first grayscale image corresponding to the current video frame and a second grayscale image corresponding to the current associated video frame;

[0140] Performing texture feature extraction processing on a first grayscale image corresponding to a current video frame and a second grayscale image corresponding to a current associated video frame to obtain first texture feature information corresponding to the current video frame and second texture feature information corresponding to the current associated video frame;

[0141] Based on the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame, a frame difference analysis is performed on the current video frame to obtain a current frame difference image corresponding to the current video frame;

[0142] Based on the current frame difference image corresponding to the current video frame, performing picture change analysis on the current video frame to obtain first video indicator data corresponding to the current video frame;

[0143] Based on the first texture feature information, a picture content analysis is performed on the current video frame to obtain second video indicator data corresponding to the current video frame.

[0144] In a specific embodiment, the current video frame may be any frame except the first frame in the video to be extracted. The current associated video frame may refer to a previous video frame of the current video frame.

[0145] In a specific embodiment, the first grayscale image may refer to the current video frame after grayscale. The second grayscale image may refer to the current associated video frame after grayscale.

[0146] In a specific embodiment, the first grayscale image can be obtained by grayscale processing the current video frame. The second grayscale image can be obtained by grayscale processing the current associated video frame. Specifically, the first grayscale image or the second grayscale image can be obtained by the following formula:

[0147] Gray i =R i ×0.299+G i ×0.587+B i ×0.114

[0148] Among them, Gray i is the first grayscale image or the second grayscale image; R i G is the red channel brightness of the current video frame or the current associated video frame; i B is the green channel brightness of the current video frame or the current associated video frame; i The blue channel brightness of the current video frame or the currently associated video frame.

[0149] In a specific embodiment, the first texture feature information may represent the texture feature in the current video frame. The second texture feature information may represent the texture feature in the current associated video frame.

[0150] In a specific embodiment, the first texture feature information can be obtained by performing texture extraction processing on the first grayscale image in multiple preset extraction directions. Specifically, the first grayscale image can be filtered in multiple preset extraction directions to obtain the first filtered images corresponding to the above multiple preset extraction directions; then, the first filtered images corresponding to the above multiple preset extraction directions can be fused to obtain the above first texture feature information. The preset extraction direction can be a preset texture extraction direction; optionally, the preset extraction direction can include 0 degrees, 90 degrees, 180 degrees or 270 degrees. Furthermore, the first grayscale image can be input into a preset linear filter to perform filtering processing in any preset extraction direction to obtain the first filtered image corresponding to any preset extraction direction. Specifically, the preset linear filter can be a Gabor filter.

[0151] In a specific embodiment, the second texture feature information can be obtained by performing texture extraction processing on the second grayscale image in multiple preset extraction directions. Specifically, the second grayscale image can be filtered in multiple preset extraction directions to obtain the second filtered images corresponding to the multiple preset extraction directions; then, the second filtered images corresponding to the multiple preset extraction directions can be fused to obtain the second texture feature information. Furthermore, the second grayscale image can be input into a preset linear filter to perform filtering processing in any preset extraction direction to obtain the second filtered image corresponding to any preset extraction direction.

[0152] In the above embodiment, by performing texture extraction processing on the first grayscale image or the second grayscale image in multiple preset extraction directions, texture information extraction in multiple directions can be achieved, and local structural information corresponding to spatial frequency, spatial position and directional selectivity can be well described. Further, by performing filtering processing by the above preset linear filter, texture information in multiple directions can be extracted, and it is insensitive to illumination, that is, the extracted images are nearly consistent under different illuminations of the same scene image. In addition, the above preset linear filter has the characteristic of achieving optimal localization in both the spatial domain and the frequency domain, and can well describe local structural information corresponding to spatial frequency, spatial position and directional selectivity.

[0153] In a specific embodiment, the first grayscale image and the second grayscale image can be downsampled to obtain a reduced first grayscale image and a reduced second grayscale image. Accordingly, the reduced first grayscale image and the reduced second grayscale image can be subjected to texture feature extraction to obtain first texture feature information corresponding to the current video frame and second texture feature information corresponding to the current associated video frame.

[0154] In the above embodiment, by downsampling the first grayscale image and the second grayscale image respectively, the amount of calculation can be reduced while ensuring the accuracy of the first video index data and the second video index data, thereby improving the efficiency of sample data generation.

[0155] In a specific embodiment, the current frame difference image may represent the degree of difference between the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame.

[0156] In a specific embodiment, the current frame difference image can be obtained by the following formula:

[0157] D i =|F i -F i-1 |

[0158] Among them, D i is the current frame difference image; F i is the first texture feature information; F i-1 is the second texture feature information.

[0159] In a specific embodiment, the first video indicator data may be obtained by the following formula:

[0160]

[0161] Among them, Ave i is the first video index data; sum(D i (x, y)) is the sum of the pixel values ​​of multiple pixels in the current frame difference image; w is the width value of the current frame difference image; h is the height value of the current frame difference image.

[0162] In a specific embodiment, the second video indicator data may be obtained by the following formula:

[0163]

[0164] Among them, Ave Fi is the second video index data; sum(F i (x, y)) is the sum of the feature information corresponding to the multiple pixels in the first texture feature information; w is the width value of the current frame difference image; h is the height value of the current frame difference image.

[0165] In a specific embodiment, after analyzing and obtaining the first video indicator data corresponding to the current video frame and the second video indicator data corresponding to the current video frame, the current video frame can be updated based on the next video frame of the current video frame, and the currently associated video frame can be updated accordingly, and based on the updated current video frame and the updated currently associated video frame, the above grayscale processing of the current video frame and the currently associated video frame is repeated to obtain the first grayscale image corresponding to the current video frame and the second grayscale image corresponding to the currently associated video frame, and the above picture content analysis of the current video frame based on the first texture feature information is performed to obtain the second video indicator data corresponding to the current video frame. Specifically, before updating the current video frame, the first texture feature information corresponding to the current video frame can be cached, so that after updating the current video frame, it can be directly read and used as the second texture feature information corresponding to the updated currently associated video frame, thereby reducing the amount of calculation and improving the analysis efficiency of the video frame picture analysis.

[0166] In the above embodiment, by obtaining the current video frame and the current associated video frame corresponding to the current video frame from the video to be extracted, grayscale processing is performed on the current video frame and the current associated video frame to obtain a first grayscale image corresponding to the current video frame and a second grayscale image corresponding to the current associated video frame, texture feature extraction processing is performed on the first grayscale image corresponding to the current video frame and the second grayscale image corresponding to the current associated video frame to obtain first texture feature information corresponding to the current video frame and second texture feature information corresponding to the current associated video frame, frame difference analysis is performed on the current video frame based on the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame to obtain a current frame difference image corresponding to the current video frame, picture change analysis is performed on the current video frame based on the current frame difference image corresponding to the current video frame to obtain first video indicator data corresponding to the current video frame, and picture content analysis is performed on the current video frame based on the first texture feature information to obtain second video indicator data corresponding to the current video frame, thereby realizing video frame picture analysis of the video to be extracted and accurately obtaining first video indicator data and second video indicator data corresponding to multiple video frames to be extracted in the video to be extracted.

[0167] In a specific embodiment, the segment extraction indication information may be used to indicate at least one video segment in the video to be extracted that meets a preset video segment condition.

[0168] In a specific embodiment, based on the first video indicator data corresponding to each of the plurality of video frames to be extracted and the second video indicator data corresponding to each of the plurality of video frames to be extracted, performing continuous picture segment analysis on the video to be extracted to obtain segment extraction indication information may include:

[0169] Based on the first video indicator data corresponding to each of the plurality of video frames to be extracted and the second video indicator data corresponding to each of the plurality of video frames to be extracted, segmenting the video to be extracted to obtain a plurality of second video segments;

[0170] Determining at least one third video segment from the plurality of second video segments;

[0171] Based on at least one third video segment, segment extraction indication information is generated.

[0172] In a specific embodiment, the first video index data corresponding to each frame in each second video clip may be greater than the first preset index data and less than the second preset index data. The second video index data corresponding to each frame in each second video clip may be greater than the third preset index data. Specifically, the first preset index data, the second preset index data and the third preset index data can be set according to actual application needs; optionally, the value range of the first preset index data can be 10 to 15, the value range of the second preset index data can be 40 to 45, and the value range of the third preset index data can be 20 to 25. Optionally, the first preset index data can be 10, the second preset index data can be 45, and the third preset index data can be 20.

[0173] In a specific embodiment, multiple segmented video frames can be determined based on the first video index data corresponding to each of the multiple video frames to be extracted, and the second video index data corresponding to each of the multiple video frames to be extracted; then, the multiple segmented video frames are sorted according to the time sequence of the multiple segmented video frames in the video to be extracted to obtain a segmented video frame sequence; then, based on the positions of any two adjacent segmented video frames in the segmented video frame sequence in the video to be extracted, the video to be extracted is segmented to obtain multiple second video clips. Among them, any segmented video frame can be any video frame in the multiple video frames to be extracted whose corresponding first video index data is less than or equal to the first preset index data or greater than or equal to the second preset index data, or whose corresponding second video index data is less than or equal to the third preset index data. Specifically, the first video indicator data corresponding to any video frame to be extracted can be compared with the first preset indicator data and the second preset indicator data to obtain a first comparison result; the second video indicator data corresponding to any of the above-mentioned video frames to be extracted can be compared with the third preset indicator data to obtain a second comparison result; when the first comparison result indicates that the first video indicator data corresponding to any of the above-mentioned video frames to be extracted is less than or equal to the first preset indicator data or greater than or equal to the second preset indicator data, and the second comparison result indicates that the second video indicator data corresponding to any of the above-mentioned video frames to be extracted is less than or equal to the third preset indicator data, it can be determined that any of the above-mentioned video frames to be extracted is a segmented video frame.

[0174] In a specific embodiment, at least one third video segment may be a video segment of the plurality of second video segments whose corresponding video duration is greater than a preset duration. Specifically, the preset duration may be set according to actual application needs, and the present disclosure does not limit this.

[0175] In a specific embodiment, the video length of each second video segment among multiple second video segments can be determined first; then, at least one corresponding video segment whose video length is greater than a preset length can be selected from the multiple second video segments as at least one third video segment.

[0176] In a specific embodiment, the segment extraction indication information may be generated based on the segment timing information of at least one third video segment. Specifically, the segment timing information of at least one third video segment may be used as the segment extraction indication information. The segment timing information of any third video segment may represent the position of any third video segment in the video to be extracted.

[0177] In the above embodiment, based on the first video indicator data corresponding to each of the multiple video frames to be extracted, and the second video indicator data corresponding to each of the multiple video frames to be extracted, the video to be extracted is segmented to obtain multiple second video segments, and at least one third video segment is determined from the multiple second video segments. Based on the at least one third video segment, segment extraction indication information is generated, which can facilitate the accurate extraction of video segments with picture motion, more picture textures, and continuous picture content without scene cuts.

[0178] In a specific embodiment, the multiple sample super-resolution videos corresponding to the video to be extracted may refer to multiple super-resolution videos obtained by performing multiple different super-resolution processing on the same sample video. Specifically, in the case where the video to be extracted is a sample video, the multiple sample super-resolution videos corresponding to the video to be extracted may refer to multiple super-resolution videos obtained by performing multiple different super-resolution processing on the video to be extracted; in the case where the video to be extracted is a sample super-resolution video, the multiple sample super-resolution videos corresponding to the video to be extracted may refer to multiple super-resolution videos obtained by performing multiple different super-resolution processing on the sample video corresponding to the video to be extracted; wherein, the sample video corresponding to the video to be extracted may refer to the sample video used to generate the above-mentioned video to be extracted.

[0179] In a specific embodiment, the first video segment may refer to a video segment in which a plurality of sample super-resolution videos corresponding to the video to be extracted have picture motion, more picture textures, and continuous picture content without scene cuts.

[0180] In a specific embodiment, based on the fragment timing information of at least one third video fragment in the fragment extraction indication information, the video fragment corresponding to each fragment timing information can be extracted from each sample super-resolution video of the multiple sample super-resolution videos corresponding to the video to be extracted, so as to obtain multiple first video fragments.

[0181] In a specific embodiment, the second super-resolution video pair may be any one of a plurality of sample super-resolution video pairs corresponding to the video to be extracted.

[0182] In a specific embodiment, the multiple first video segments corresponding to the video to be extracted may include first video segments with the same video sequence, and may also include first video segments with different video sequences.

[0183] In a specific embodiment, video timing matching can be performed on any two first video segments among the multiple first video segments corresponding to the video to be extracted to obtain a video timing matching result; the above video timing matching result can indicate any two first video segments belonging to the same video timing as the second super-resolution video pair. Exemplarily, assuming that the multiple sample super-resolution videos corresponding to the video to be extracted include a sample super-resolution video V1 and a sample super-resolution video V2, the first video segment V3 corresponding to video timing 1 and the first video segment V4 corresponding to video timing 2 are extracted from the above sample super-resolution video V1, and the first video segment V5 corresponding to video timing 1 and the first video segment V6 corresponding to video timing 2 are extracted from the above sample super-resolution video V2, and the second super-resolution video pair including (V3, V5) and (V4, V6) can be generated.

[0184] In the above embodiment, by performing video frame analysis on the video to be extracted, first video indicator data corresponding to each of the multiple video frames to be extracted in the video to be extracted and second video indicator data corresponding to each of the multiple video frames to be extracted are obtained; based on the first video indicator data corresponding to each of the multiple video frames to be extracted and the second video indicator data corresponding to each of the multiple video frames to be extracted, continuous picture segment analysis is performed on the video to be extracted to obtain segment extraction indication information; based on the segment extraction indication information, video segment extraction processing is performed on each of the multiple sample super-resolution videos corresponding to the video to be extracted to obtain multiple first video segments corresponding to the video to be extracted, thereby realizing the extraction of video segments with picture motion, more picture texture, and continuous picture content without scene cuts; and then, from the multiple first video segments corresponding to the video to be extracted, any two first video segments corresponding to the same video sequence are selected as the second super-resolution video pair, thereby generating a video pair with picture motion, more picture texture, and continuous picture content without scene cuts, thereby facilitating the improvement of the accuracy of the label quality information, and further improving the accuracy of the quality analysis of the target quality analysis model.

[0185] S203: Based on a preset machine learning model, perform video quality analysis on the multiple sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos.

[0186] In a specific embodiment, the preset machine learning model may refer to a target quality analysis model to be trained.

[0187] In a specific embodiment, the sample quality information corresponding to any sample super-resolution video can represent the quality of any sample super-resolution video. Further, the sample quality information corresponding to any sample super-resolution video can be used to represent the super-resolution effect of any sample super-resolution video.

[0188] In a specific embodiment, each sample super-resolution video may be input into a preset machine learning model for video quality analysis to obtain sample quality information corresponding to each sample super-resolution video.

[0189] S205: Determine target loss information based on the label quality information and the sample quality information.

[0190] In a specific embodiment, the target loss information may represent the degree of difference between the label quality information and the sample quality information.

[0191] In a specific embodiment, the label quality information corresponding to each sample super-resolution video and the sample quality information corresponding to each sample super-resolution video can be differenced to obtain the quality difference information corresponding to each sample super-resolution video; and then the quality difference information corresponding to multiple sample super-resolution videos can be superimposed to obtain the target loss information.

[0192] S207: Based on the target loss information, a preset machine learning model is trained to obtain a target quality analysis model for the super-resolution video.

[0193] In a specific embodiment, the target quality analysis model may be used to perform video quality analysis on the super-resolution video to be analyzed.

[0194] In a specific embodiment, after determining the target loss information, the preset machine learning model can be updated according to the target loss information to obtain an updated preset machine learning model; then, based on the updated preset machine learning model, repeat the above steps of obtaining multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos, until the preset machine learning model is updated according to the target loss information to obtain the updated model training step of the preset machine learning model, until the preset convergence condition is met; accordingly, the preset machine learning model obtained when the above preset convergence condition is met can be used as the target quality analysis model. Among them, the preset convergence condition can include that the target loss information is less than the preset loss information, the number of training iterations meets the preset number of iterations, or the target loss information converges unchanged. Specifically, the model parameters in the preset machine learning model can be updated according to stochastic gradient descent based on the target loss information.

[0195] In the above embodiment, multiple sample super-resolution videos corresponding to each sample video in at least one sample video and label quality information corresponding to each of the multiple sample super-resolution videos are obtained, wherein the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processing on each sample video, and the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs. Any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents the comparison quality information of the two sample super-resolution videos in any sample super-resolution video pair by multiple analysis objects. The quality comparison result obtained by performing quality comparison can realize the collection of sample data sets for super-resolution videos and the generation of corresponding label quality information; then, based on the preset machine learning model, video quality analysis is performed on multiple sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos, and the target loss information is determined based on the label quality information and the sample quality information. Based on the target loss information, the preset machine learning model is trained to obtain a target quality analysis model for the super-resolution video, which can enable the target quality analysis model to realize quality analysis of the super-resolution effect of the super-resolution video, thereby improving the accuracy of video quality analysis for the super-resolution video.

[0196] Specifically, Figure 3 FIG. 1 is a flow chart of a video quality analysis method according to an exemplary embodiment. Figure 3 As shown, based on the target quality analysis model obtained by the above-mentioned super-resolution video quality analysis model generation method, the present disclosure also provides a video quality analysis method, which can be used in electronic devices such as terminals and servers, and can include the following steps:

[0197] S301: Obtaining a super-resolution video to be analyzed;

[0198] S303: Input the super-resolution video to be analyzed into the target quality analysis model to perform video quality analysis, and obtain a target quality analysis result corresponding to the super-resolution video to be analyzed.

[0199] Figure 4 FIG. 1 is a block diagram of a device for generating a super-resolution video quality analysis model according to an exemplary embodiment. Figure 4 As shown, the device may include:

[0200] The information acquisition module 410 can be used to obtain multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos; the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processes on each sample video; the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs, and any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents the quality comparison result obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair;

[0201] The first quality analysis module 420 may be used to perform video quality analysis on a plurality of sample super-resolution videos based on a preset machine learning model to obtain sample quality information corresponding to each of the plurality of sample super-resolution videos;

[0202] The loss determination module 430 may be used to determine target loss information based on the label quality information and the sample quality information;

[0203] The training module 440 can be used to train a preset machine learning model based on target loss information to obtain a target quality analysis model for super-resolution video.

[0204] In a specific embodiment, the above device may further include:

[0205] The display module can be used to display multiple sample super-resolution video pairs;

[0206] A comparison quality acquisition module may be used to obtain the comparison quality information corresponding to each sample super-resolution video pair in response to a comparison quality input instruction;

[0207] The quality data analysis module can be used to perform quality data analysis on each sample super-resolution video based on the comparison quality information corresponding to multiple sample super-resolution video pairs, and obtain label quality information corresponding to each sample super-resolution video.

[0208] In a specific embodiment, the quality data analysis module may include:

[0209] The comparison times acquisition module can be used to determine the target comparison times corresponding to at least one sample video based on the target comparison quality information of multiple sample super-resolution video pairs; the target comparison times is the quality comparison times of multiple sample super-resolution videos corresponding to the at least one sample video;

[0210] The first video pair acquisition module can be used to determine at least one first super-resolution video pair corresponding to each sample super-resolution video; the at least one first super-resolution video pair corresponding to any sample super-resolution video is a sample super-resolution video pair including any sample super-resolution video among the multiple sample super-resolution video pairs corresponding to any sample super-resolution video;

[0211] The quality data acquisition module can be used to determine the target quality quantity corresponding to each sample super-resolution video based on the target comparison quality information of at least one first super-resolution video pair corresponding to each sample super-resolution video; the target quality quantity corresponding to each sample super-resolution video is the quantity belonging to the first quality information among the multiple target comparison quality information corresponding to each sample super-resolution video;

[0212] The quality ratio analysis module can be used to perform target quality ratio analysis on each sample super-resolution video based on the target comparison times and the target quality quantity corresponding to each sample super-resolution video, and obtain the label quality information corresponding to each sample super-resolution video.

[0213] In a specific embodiment, the above device may further include:

[0214] The quality distribution analysis module can be used to perform quality distribution analysis on multiple sample super-resolution videos based on the label quality information corresponding to the multiple sample super-resolution videos to obtain quality distribution index data; the quality distribution index data represents the distribution uniformity of the label quality information corresponding to the multiple sample super-resolution videos;

[0215] A first quality degradation processing module may be used to, when the quality distribution indicator data does not satisfy a preset uniform distribution condition, perform quality degradation processing on each of the at least one first preset video to obtain at least one first sample video; the resolution of each first preset video is greater than a preset resolution;

[0216] The first super-resolution processing module can be used to perform a variety of different super-resolution processing on each first sample video to obtain a plurality of first sample super-resolution videos corresponding to each first sample video;

[0217] Accordingly, the first quality analysis module 420 may include:

[0218] The second quality analysis module can be used to perform video quality analysis on multiple sample super-resolution videos and multiple first sample super-resolution videos based on a preset machine learning model to obtain sample quality information corresponding to each of the multiple sample super-resolution videos.

[0219] In a specific embodiment, the first degradation processing module may include:

[0220] The missing distribution acquisition module can be used to determine the quality missing distribution information based on the label quality information corresponding to the multiple sample super-resolution videos when the quality distribution indicator data does not meet the preset uniform distribution condition; the quality missing distribution information represents the distribution missing range of the label quality information corresponding to the multiple sample super-resolution videos;

[0221] The degradation indication updating module may be used to update preset degradation indication information based on the quality loss distribution information; the preset degradation indication information is used to indicate the degree of degradation during the degradation process;

[0222] The second degradation processing module can be used to perform degradation processing on each first preset video in the at least one first preset video based on the updated preset degradation indication information to obtain at least one first sample video.

[0223] In a specific embodiment, the above device may further include:

[0224] The video frame picture analysis module can be used to perform video frame picture analysis on the video to be extracted, and obtain first video index data corresponding to each of the multiple video frames to be extracted in the video to be extracted, and second video index data corresponding to each of the multiple video frames to be extracted; the video to be extracted is any sample video or any sample super-resolution video corresponding to any sample video; the first video index data corresponding to any video frame to be extracted is used to characterize the degree of change between any video frame to be extracted and any adjacent video frame of any video frame to be extracted; the second video index data corresponding to any video frame to be extracted is used to characterize whether the picture information contained in any video frame to be extracted is greater than the preset picture information;

[0225] The continuous picture analysis module can be used to perform continuous picture segment analysis on the video to be extracted based on the first video indicator data corresponding to each of the multiple video frames to be extracted and the second video indicator data corresponding to each of the multiple video frames to be extracted, so as to obtain segment extraction indication information; the segment extraction indication information is used to indicate at least one video segment in the video to be extracted that meets the preset video segment condition;

[0226] The video segment extraction module can be used to perform video segment extraction processing on each of the multiple sample super-resolution videos corresponding to the video to be extracted based on the segment extraction indication information, so as to obtain multiple first video segments corresponding to the video to be extracted;

[0227] The second video pair acquisition module can be used to select any two first video segments corresponding to the same video sequence from multiple first video segments corresponding to the video to be extracted as the second super-resolution video pair; the second super-resolution video pair is any one of the multiple sample super-resolution video pairs corresponding to the video to be extracted.

[0228] In a specific embodiment, the video frame analysis module may include:

[0229] The video frame acquisition module can be used to obtain the current video frame and the current associated video frame corresponding to the current video frame from the video to be extracted; the current video frame is any frame except the first frame in the video to be extracted; the current associated video frame is the previous video frame of the current video frame;

[0230] A grayscale processing module can be used to perform grayscale processing on the current video frame and the current associated video frame to obtain a first grayscale image corresponding to the current video frame and a second grayscale image corresponding to the current associated video frame;

[0231] The feature extraction module can be used to perform texture feature extraction processing on the first grayscale image corresponding to the current video frame and the second grayscale image corresponding to the current associated video frame to obtain first texture feature information corresponding to the current video frame and second texture feature information corresponding to the current associated video frame;

[0232] The frame difference analysis module can be used to perform frame difference analysis on the current video frame based on the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame to obtain a current frame difference image corresponding to the current video frame; the current frame difference image represents the degree of difference between the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame;

[0233] The picture change analysis module can be used to perform picture change analysis on the current video frame based on the current frame difference image corresponding to the current video frame to obtain the first video indicator data corresponding to the current video frame;

[0234] The picture content analysis module can be used to perform picture content analysis on the current video frame based on the first texture feature information to obtain second video indicator data corresponding to the current video frame.

[0235] In a specific embodiment, the above-mentioned continuous picture analysis module may include:

[0236] The segmentation processing module can be used to perform segmentation processing on the video to be extracted based on the first video index data corresponding to each of the multiple video frames to be extracted and the second video index data corresponding to each of the multiple video frames to be extracted, so as to obtain multiple second video segments; the first video index data corresponding to each frame in each second video segment is greater than the first preset index data and less than the second preset index data, and the second video index data corresponding to each frame in each second video segment is greater than the third preset index data;

[0237] The video segment acquisition module can be used to determine at least one third video segment from the plurality of second video segments; the at least one third video segment is a video segment in the plurality of second video segments whose corresponding video duration is greater than a preset duration;

[0238] The extraction indication acquisition module may be configured to generate segment extraction indication information based on at least one third video segment.

[0239] In a specific embodiment, the above device may further include:

[0240] The second super-resolution processing module can be used to perform a variety of super-resolution processing on each sample video to obtain a plurality of sample super-resolution videos corresponding to each sample video.

[0241] In a specific embodiment, the above device may further include:

[0242] The preset video acquisition module can be used to acquire at least one second preset video, and the resolution of each second preset video is greater than the preset resolution;

[0243] The third quality reduction processing module can be used to perform quality reduction processing on at least one second preset video to obtain at least one second sample video.

[0244] In a specific embodiment, the third degradation processing module may include:

[0245] The image blur processing module may be used to perform image blur processing on each preset video frame in each second preset video from at least one image blur dimension to obtain a blurred video frame corresponding to each preset video frame;

[0246] The sample video generation module can be used to generate at least one second sample video based on blurred video frames corresponding to multiple preset video frames in each second preset video.

[0247] Figure 5 FIG. 1 is a block diagram of a video quality analysis device according to an exemplary embodiment. Figure 5 As shown, the device may include:

[0248] The super-resolution video acquisition module 510 can be used to acquire the super-resolution video to be analyzed;

[0249] The second quality analysis module 520 can be used to input the super-resolution video to be analyzed into the target quality analysis model for video quality analysis to obtain the target quality analysis result corresponding to the super-resolution video to be analyzed; wherein the target quality analysis model is obtained based on the above-mentioned super-resolution video quality analysis model generation method.

[0250] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0251] Figure 6 is a block diagram of an electronic device for generating a target quality analysis model or performing video quality analysis on a super-resolution video to be analyzed according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a super-resolution video quality analysis model generation method or a video quality analysis method is implemented.

[0252] Figure 7 is a block diagram of another electronic device for generating a target quality analysis model or performing video quality analysis on a super-resolution video to be analyzed according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a super-resolution video quality analysis model generation method or a video quality analysis method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0253] Those skilled in the art will understand that Figure 6 or Figure 7 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0254] In an exemplary embodiment, an electronic device is also provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a super-resolution video quality analysis model generation method or a video quality analysis method as in an embodiment of the present disclosure.

[0255] In an exemplary embodiment, a computer-readable storage medium is also provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the super-resolution video quality analysis model generation method or the video quality analysis method in the embodiment of the present disclosure.

[0256] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer executes the super-resolution video quality analysis model generation method or the video quality analysis method in the embodiment of the present disclosure.

[0257] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0258] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0259] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0260] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating a super-resolution video quality analysis model, characterized in that: The method comprises: Acquire multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos; the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processing on each sample video; the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs, and any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents a quality comparison result obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair; Based on a preset machine learning model, perform video quality analysis on the multiple sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos; Determining target loss information based on the label quality information and the sample quality information; Based on the target loss information, the preset machine learning model is trained to obtain a target quality analysis model for the super-resolution video.

2. The method according to claim 1, characterized in that The label quality information corresponding to each of the plurality of sample super-resolution videos is obtained by: Displaying the plurality of sample super-resolution video pairs; In response to the comparison quality input instruction, obtaining the comparison quality information corresponding to each sample super-resolution video pair; Based on the comparison quality information corresponding to the multiple sample super-resolution video pairs, quality data analysis is performed on each of the sample super-resolution videos to obtain label quality information corresponding to each of the sample super-resolution videos.

3. The method according to claim 2, characterized in that The comparison quality information corresponding to any sample super-resolution video pair includes target comparison quality information corresponding to each of the multiple analysis objects, and any target comparison quality information corresponding to any sample super-resolution video pair represents a quality comparison result obtained by any analysis object performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair; the quality data analysis of each sample super-resolution video based on the comparison quality information corresponding to the multiple sample super-resolution video pairs to obtain label quality information corresponding to each sample super-resolution video includes: Based on the target comparison quality information of the multiple sample super-resolution video pairs, determine the target comparison number corresponding to the at least one sample video; the target comparison number is the quality comparison number of the multiple sample super-resolution videos corresponding to the at least one sample video; Determine at least one first super-resolution video pair corresponding to each sample super-resolution video; the at least one first super-resolution video pair corresponding to any sample super-resolution video is a sample super-resolution video pair including the any sample super-resolution video among the multiple sample super-resolution video pairs corresponding to the any sample super-resolution video; Based on the target comparison quality information of at least one first super-resolution video pair corresponding to each sample super-resolution video, determine the target quality quantity corresponding to each sample super-resolution video; the target quality quantity corresponding to each sample super-resolution video is the quantity belonging to the first quality information among the multiple target comparison quality information corresponding to each sample super-resolution video; Based on the target comparison times and the target quality quantity corresponding to each sample super-resolution video, a target quality ratio analysis is performed on each sample super-resolution video to obtain label quality information corresponding to each sample super-resolution video.

4. The method according to claim 2, characterized in that: The method further comprises: Based on the label quality information corresponding to the multiple sample super-resolution videos, quality distribution analysis is performed on the multiple sample super-resolution videos to obtain quality distribution index data; the quality distribution index data represents the distribution uniformity of the label quality information corresponding to the multiple sample super-resolution videos; When the quality distribution indicator data does not satisfy a preset uniform distribution condition, each of the at least one first preset video is degraded to obtain at least one first sample video; the resolution of each first preset video is greater than a preset resolution; Performing a plurality of different super-resolution processes on each first sample video to obtain a plurality of first sample super-resolution videos corresponding to each first sample video; The performing video quality analysis on the plurality of sample super-resolution videos based on a preset machine learning model to obtain sample quality information corresponding to each of the plurality of sample super-resolution videos includes: Based on the preset machine learning model, video quality analysis is performed on the multiple sample super-resolution videos and the multiple first sample super-resolution videos to obtain sample quality information corresponding to each of the multiple sample super-resolution videos.

5. The method according to claim 4, characterized in that When the quality distribution indicator data does not satisfy a preset uniform distribution condition, performing quality degradation processing on each of the at least one first preset video to obtain at least one first sample video includes: In the case where the quality distribution indicator data does not satisfy the preset uniform distribution condition, quality missing distribution information is determined based on the label quality information corresponding to the multiple sample super-resolution videos; the quality missing distribution information represents the distribution missing range of the label quality information corresponding to the multiple sample super-resolution videos; Based on the quality loss distribution information, updating preset degradation indication information; the preset degradation indication information is used to indicate the degree of degradation in the degradation process; Based on the updated preset degradation indication information, degradation processing is performed on each first preset video in the at least one first preset video to obtain the at least one first sample video.

6. The method according to claim 1, characterized in that The method further comprises: Performing video frame analysis on the video to be extracted, obtaining first video index data corresponding to each of the multiple video frames to be extracted in the video to be extracted, and second video index data corresponding to each of the multiple video frames to be extracted; the video to be extracted is any sample video or any sample super-resolution video corresponding to any sample video; the first video index data corresponding to any video frame to be extracted is used to characterize the degree of change between any video frame to be extracted and an adjacent video frame of any video frame to be extracted; the second video index data corresponding to any video frame to be extracted is used to characterize whether the picture information contained in any video frame to be extracted is greater than the preset picture information; Based on the first video indicator data corresponding to each of the plurality of video frames to be extracted and the second video indicator data corresponding to each of the plurality of video frames to be extracted, performing continuous picture segment analysis on the video to be extracted to obtain segment extraction indication information; the segment extraction indication information is used to indicate at least one video segment in the video to be extracted that meets a preset video segment condition; Based on the segment extraction indication information, perform video segment extraction processing on each of the multiple sample super-resolution videos corresponding to the video to be extracted, to obtain multiple first video segments corresponding to the video to be extracted; From the multiple first video segments corresponding to the video to be extracted, any two first video segments corresponding to the same video sequence are selected as a second super-resolution video pair; the second super-resolution video pair is any one of the multiple sample super-resolution video pairs corresponding to the video to be extracted.

7. The method according to claim 6, characterized in that The video frame analysis is performed on the video to be extracted to obtain first video indicator data corresponding to each of the plurality of video frames to be extracted in the video to be extracted, and second video indicator data corresponding to each of the plurality of video frames to be extracted, including: From the video to be extracted, obtain a current video frame and a currently associated video frame corresponding to the current video frame; the current video frame is any frame except the first frame in the video to be extracted; the currently associated video frame is a video frame previous to the current video frame; Performing grayscale processing on the current video frame and the current associated video frame to obtain a first grayscale image corresponding to the current video frame and a second grayscale image corresponding to the current associated video frame; Performing texture feature extraction processing on the first grayscale image corresponding to the current video frame and the second grayscale image corresponding to the current associated video frame to obtain first texture feature information corresponding to the current video frame and second texture feature information corresponding to the current associated video frame; Based on the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame, a frame difference analysis is performed on the current video frame to obtain a current frame difference image corresponding to the current video frame; the current frame difference image represents the degree of difference between the first texture feature information corresponding to the current video frame and the second texture feature information corresponding to the current associated video frame; Based on the current frame difference image corresponding to the current video frame, performing picture change analysis on the current video frame to obtain first video indicator data corresponding to the current video frame; Based on the first texture feature information, the current video frame is subjected to a picture content analysis to obtain second video indicator data corresponding to the current video frame.

8. The method according to claim 6, characterized in that The method of performing continuous picture segment analysis on the video to be extracted based on the first video indicator data corresponding to each of the plurality of video frames to be extracted and the second video indicator data corresponding to each of the plurality of video frames to be extracted to obtain segment extraction indication information includes: Based on the first video index data corresponding to each of the multiple video frames to be extracted and the second video index data corresponding to each of the multiple video frames to be extracted, the video to be extracted is segmented to obtain multiple second video segments; the first video index data corresponding to each frame in each second video segment is greater than the first preset index data and less than the second preset index data, and the second video index data corresponding to each frame in each second video segment is greater than the third preset index data; Determine at least one third video segment from the plurality of second video segments; the at least one third video segment is a video segment in the plurality of second video segments whose corresponding video duration is greater than a preset duration; The segment extraction indication information is generated based on the at least one third video segment.

9. The method according to claim 1, characterized in that: The multiple sample super-resolution videos corresponding to each sample video are obtained by: A plurality of different super-resolution processes are performed on each sample video to obtain a plurality of sample super-resolution videos corresponding to each sample video.

10. The method according to any one of claims 1 to 9, characterized in that: The at least one sample video includes at least one second sample video; the at least one sample video is obtained in the following manner: Acquire at least one second preset video, where the resolution of each second preset video is greater than the preset resolution; The at least one second preset video is degraded to obtain the at least one second sample video.

11. The method according to claim 10, characterized in that The step of respectively performing quality degradation processing on the at least one second preset video to obtain the at least one second sample video includes: Performing image blurring processing on each preset video frame in each second preset video from at least one image blurring dimension to obtain a blurred video frame corresponding to each preset video frame; The at least one second sample video is generated based on blurred video frames corresponding to a plurality of preset video frames in each second preset video.

12. A video quality analysis method, characterized in that: The method comprises: Obtain the super-resolution video to be analyzed; Inputting the super-resolution video to be analyzed into the target quality analysis model to perform video quality analysis, and obtaining a target quality analysis result corresponding to the super-resolution video to be analyzed; Wherein, the target quality analysis model is obtained based on the super-resolution video quality analysis model generation method described in any one of claims 1-11.

13. A super-resolution video quality analysis model generation device, characterized in that: The device comprises: An information acquisition module is used to acquire multiple sample super-resolution videos corresponding to each sample video in at least one sample video, and label quality information corresponding to each of the multiple sample super-resolution videos; the multiple sample super-resolution videos corresponding to each sample video are obtained by performing multiple different super-resolution processes on each sample video; the label quality information is determined based on the comparison quality information corresponding to each of the multiple sample super-resolution video pairs, and any sample super-resolution video pair is any two of the multiple sample super-resolution videos corresponding to the same sample video, and the comparison quality information corresponding to any sample super-resolution video pair represents a quality comparison result obtained by multiple analysis objects performing quality comparison on two sample super-resolution videos in any sample super-resolution video pair; A first quality analysis module, configured to perform video quality analysis on the plurality of sample super-resolution videos based on a preset machine learning model, to obtain sample quality information corresponding to each of the plurality of sample super-resolution videos; A loss determination module, configured to determine target loss information based on the label quality information and the sample quality information; A training module is used to train the preset machine learning model based on the target loss information to obtain a target quality analysis model for the super-resolution video.

14. A video quality analysis device, characterized in that: The device comprises: A super-resolution video acquisition module is used to acquire the super-resolution video to be analyzed; The second quality analysis module is used to input the super-resolution video to be analyzed into the target quality analysis model for video quality analysis to obtain the target quality analysis result corresponding to the super-resolution video to be analyzed; wherein the target quality analysis model is obtained based on the super-resolution video quality analysis model generation method described in any one of claims 1-11.

15. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the executable instructions to implement the super-resolution video quality analysis model generation method described in any one of claims 1 to 11 or the video quality analysis method described in claim 12.

Images