A resolution detection method, system, storage medium, and terminal device

Abstract

This invention discloses a sharpness detection method, system, storage medium, and terminal device, applied in the field of information processing technology. The sharpness detection system first trains a feature extraction module using a first training sample, determining that the initial sharpness recognition model includes the trained feature extraction module and sharpness module. Then, it adjusts the initial sharpness recognition model using a second training sample to obtain a final trained sharpness recognition model capable of acquiring the sharpness information of any target object. Because the sharpness discrimination criteria differ for different types of sample images when annotating them to obtain second annotation information in the second training sample, the trained sharpness recognition model associates the sharpness information with the actual type of the target object during the sharpness information determination process. This makes the determined sharpness information closer to the user's perception and the quality assessment of the target object more objective.

Description

Technical Field

[0001] This invention relates to the field of information processing technology based on artificial intelligence, and in particular to a resolution detection method, system, storage medium, and terminal device. Background Technology

[0002] In the era of self-media, information dissemination has shifted from the traditional "point-to-surface" to a more equitable "point-to-point" approach. Self-media content formats include text and images, videos, and especially short videos, which have experienced rapid growth in recent years. With the rapid development of the internet and the lowering of barriers to content production, the volume of various types of content has increased exponentially. This content originates from various content creation organizations, such as self-media and institutional user-generated content (UGC), and information flow services like public accounts. Various types of content (especially video content) are distributed through algorithms, resulting in rapid growth as feeds, including original materials, news feeds, information provision, submissions, summaries, news subscriptions, and web sources.

[0003] For all types of content (especially video content), the cover image (or first frame) directly impacts the user's initial impression, much like the content's packaging, and ultimately influences click-through rates. Therefore, the actual video content, loading speed, and clarity become core user requirements when consuming video content. Video clarity is a crucial indicator of video quality, especially for films, TV dramas, and animation. High-definition videos significantly enhance the user experience. Thus, accurately judging video clarity and identifying high-definition video content is invaluable for users.

[0004] Currently, video content clarity can be assessed using deep learning-based methods. These methods combine deep learning neural network models with video resolution and bitrate to score the clarity of the video content. However, in practical applications, numerous factors influence the clarity of any given video content, including various subjective and objective factors. Different categories and content types make it difficult to establish a unified measurement and evaluation system for video content clarity. Furthermore, consistent standards for defining and evaluating samples are hard to achieve. As a result, deep learning neural network models struggle to learn the true factors, necessitating manual correction and verification, leading to unsatisfactory results. Summary of the Invention

[0005] This invention provides a sharpness detection method, system, storage medium, and terminal device, which realizes sharpness detection based on the type of the target object.

[0006] One embodiment of the present invention provides a sharpness detection method, comprising:

[0007] Obtain a first training sample, which includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar.

[0008] A feature extraction module is trained based on the first training sample. The feature extraction module is used to extract features of any target object containing an image.

[0009] A sharpness recognition initial model is determined, the sharpness recognition initial model including the feature extraction module and the sharpness module;

[0010] Obtain a second training sample, which includes: multiple types of sample images and second annotation information for each sample image. The second annotation information includes a sharpness score, wherein the sharpness discrimination criteria for different types of sample images are different.

[0011] The initial sharpness recognition model is adjusted based on the second training sample to obtain the final sharpness recognition model, which is used to extract feature information of any target object and determine the sharpness information of the target object based on the feature information.

[0012] One embodiment of the present invention provides a sharpness detection system, comprising:

[0013] The first sample unit is used to acquire the first training sample, which includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar.

[0014] A feature training unit is used to train a feature extraction module based on the first training sample. The feature extraction module is used to extract features of any target object containing an image.

[0015] A model determination unit is used to determine an initial model for sharpness recognition, wherein the initial model for sharpness recognition includes the feature extraction module and the sharpness module;

[0016] The second sample unit is used to acquire the second training sample, which includes: multiple types of sample images and the second annotation information of each sample image. The second annotation information includes a sharpness score, wherein the sharpness discrimination criteria of different types of sample images are different.

[0017] The adjustment unit is used to adjust the initial model of sharpness recognition according to the second training sample to obtain the final sharpness recognition model, and to extract the feature information of any target object and determine the sharpness information of the target object according to the feature information.

[0018] Another aspect of the present invention provides a computer-readable storage medium storing a plurality of computer programs adapted for loading by a processor and executing the sharpness detection method as described in one aspect of the present invention.

[0019] Another embodiment of the present invention provides a terminal device, including a processor and a memory;

[0020] The memory is used to store multiple computer programs, which are loaded by a processor and executed as the sharpness detection method according to one aspect of the present invention; the processor is used to implement each of the multiple computer programs.

[0021] As can be seen, in the method of this embodiment, the sharpness detection system first trains the feature extraction module using the first training samples, and determines that the initial sharpness recognition model includes the trained feature extraction module and sharpness module. Then, it adjusts the initial sharpness recognition model using the second training samples to obtain the final trained sharpness recognition model, which can acquire the sharpness information of any target object. Since the sharpness discrimination benchmarks of different types of sample images are different when annotating each sample image in the second training samples to obtain the second annotation information, the trained sharpness recognition model associates the sharpness information with the actual type of the target object in the process of determining the sharpness information, making the determined sharpness information closer to the user's perception and the quality assessment of the target object more objective. Furthermore, since the feature extraction module in the sharpness recognition model has been trained using the first training samples, only fine-tuning of the feature extraction module is needed when training the sharpness recognition model using the second training samples. Thus, a smaller number of second training samples are required to train the sharpness recognition model. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a sharpness detection method provided in an embodiment of the present invention;

[0024] Figure 2 This is a flowchart of a sharpness detection method provided in one embodiment of the present invention;

[0025] Figure 3This is a schematic diagram of a sample image with a canvas in one embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of a scenario in which the sharpness detection method of this invention is applied in one application embodiment;

[0027] Figure 5 This is a schematic diagram of a similarity discrimination model determined in one application embodiment of the present invention;

[0028] Figure 6 This is a schematic diagram illustrating the determination of a high-resolution image in one application embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the initial model for sharpness recognition determined in one application embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of a distributed system to which the sharpness detection method is applied in another application embodiment of the present invention;

[0031] Figure 9 This is a schematic diagram of the block structure in another application embodiment of the present invention;

[0032] Figure 10 This is a schematic diagram of the logical structure of a sharpness detection system provided in an embodiment of the present invention;

[0033] Figure 11 This is a schematic diagram of the logical structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0036] This invention provides a sharpness detection method, which mainly detects the sharpness of any target object (such as a video or an image) containing an image, specifically as follows: Figure 1 As shown, the sharpness detection system can detect sharpness by following these steps:

[0037] Obtain a first training sample, which includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar.

[0038] A feature extraction module is trained based on the first training sample. The feature extraction module is used to extract features of any target object containing an image.

[0039] A sharpness recognition initial model is determined, the sharpness recognition initial model including the feature extraction module and the sharpness module;

[0040] Obtain a second training sample, which includes: multiple types of sample images and second annotation information for each sample image. The second annotation information includes a sharpness score, wherein the sharpness discrimination criteria for different types of sample images are different.

[0041] The initial sharpness recognition model is adjusted based on the second training sample to obtain the final sharpness recognition model, which is used to extract feature information of any target object and determine the sharpness information of the target object based on the feature information.

[0042] The aforementioned clarity recognition model is a machine learning model based on artificial intelligence (AI). AI is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0043] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0044] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0045] In this way, since the sharpness discrimination benchmarks of different types of sample images are different when annotating each sample image in the second training samples to obtain the second annotation information, the sharpness recognition model trained in this way associates the sharpness information with the actual type of the target object in the process of determining the sharpness information. This makes the determined sharpness information closer to the user's perception and the quality assessment of the target object more objective. Furthermore, since the feature extraction module in the sharpness recognition model has already been trained with the first training samples, the feature extraction module can be fine-tuned when training the sharpness recognition model with the second training samples. Thus, the sharpness recognition model can be trained with a smaller number of second training samples.

[0046] This invention provides a sharpness detection method, mainly executed by a sharpness detection system, the flowchart of which is shown below. Figure 2As shown, it includes:

[0047] Step 101: Obtain the first training sample. The first training sample includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar.

[0048] The sample pairs can include positive and negative sample pairs. The two sample images in a positive sample pair are similar, while the two sample images in a negative sample pair are dissimilar.

[0049] Specifically, when obtaining the first training sample, the following methods can be used, but are not limited to:

[0050] (1) Obtain from sample videos

[0051] First, acquire multiple sample videos, and extract two frames from any sample video to form a positive sample pair, and set the first annotation information of the positive sample pair to be the information that the two frames are similar; extract one frame from any two different sample videos to form a negative sample pair, and set the first annotation information of the negative sample pair to be the information that the two frames in the negative sample pair are dissimilar.

[0052] The acquired sample videos are deduplicated to ensure that the two frames in the resulting negative sample pairs are dissimilar. Furthermore, extracting images from the sample videos to form positive and negative sample pairs facilitates the setting of the first annotation information in the first training samples.

[0053] (2) Obtain from information search systems

[0054] In a statistical information search system, multiple labeled data are used. Two images under any label are selected to form a positive sample pair, and the first label information of the positive sample pair is set to indicate that the two images are similar. Two images under any two different labels are selected to form a negative sample pair, and the first label information of the negative sample pair is set to indicate that the two images in the negative sample pair are dissimilar.

[0055] In information search systems, users typically input a query term, which is then combined with tags from various data points within the system to search for tags that match the query term. The search results are the data points corresponding to the matching tags. The system can also deduplicate the tags of various streaming media data (images or videos) stored over a period of time (e.g., 200 days) and calculate the tag matching frequency to obtain multiple tags with high matching frequencies. Then, sample pairs from the first training sample are selected from these multiple tags.

[0056] (3) Obtain from existing classification label datasets

[0057] Existing datasets such as ImageNet or OpenImage include multiple classification label data. Each classification label data is labeled with the type label of the data. You can select two images with the same type label in the dataset to form a positive sample pair, and select two images with different type labels to form a negative sample pair.

[0058] Step 102: Train a feature extraction module based on the first training sample. The feature extraction module is used to extract features of any target object containing an image.

[0059] Specifically, in one scenario, when training the feature extraction module, the sharpness detection system can first determine the initial values ​​of the multi-layered structure and parameters in each layer of the initial feature extraction model. Then, it extracts feature information from each sample image in each sample pair of the first training samples using the initial feature extraction model. Next, based on the results obtained from the initial feature extraction model and the first annotation information in the first training samples, it calculates a first loss function related to the initial feature extraction model, and then adjusts the parameter values ​​in the initial feature extraction model based on the first loss function. Here, the parameter values ​​in the initial feature extraction model refer to the fixed values ​​of parameters used in the calculation process of each layer of the initial feature extraction model that do not need to be constantly reassigned, such as parameter size, number of network layers, and user vector length.

[0060] In this process, the parameter values ​​in the initial feature extraction model need to be adjusted multiple times so that the calculated first loss function reaches the preset range. The initial feature extraction model after adjusting the parameter values ​​is the trained feature extraction module.

[0061] The first loss function is used to identify whether the two sample images in each sample pair obtained by the initial model of feature extraction are similar, and the error between the result and whether the two sample images in the corresponding sample pair are actually similar (obtained from the first annotation information in the first training sample).

[0062] In another scenario, when training the feature extraction module, the sharpness detection system can first determine the initial values ​​of the multi-layer structure and parameters in each layer of a similarity discrimination model, and then use the similarity discrimination model to determine whether the two sample images in each sample pair of the first training sample are similar. Then, based on the results obtained from the similarity discrimination model and the first annotation information in the first training sample, the loss function related to the similarity discrimination model is calculated, and the parameter values ​​in the similarity discrimination model are adjusted based on the loss function.

[0063] In this process, the parameter values ​​in the similarity discrimination model need to be adjusted multiple times so that the calculated loss function reaches the preset range. The feature extraction module included in the similarity discrimination model after adjusting the parameter values ​​is the trained feature extraction module. The loss function related to the similarity discrimination model calculated in this process is used to identify whether the two sample images in each sample pair obtained by the similarity discrimination model are similar, and the error between the result and the actual similarity between the two sample images in the corresponding sample pair (obtained based on the first annotation information in the first training sample).

[0064] The similarity discrimination model can include a feature extraction module and a discrimination module. The feature extraction module is used to extract feature information of each sample image in any sample pair. The discrimination module is used to determine whether the two sample images in each sample pair are similar based on the feature information extracted by the feature extraction module. Specifically, the discrimination module can output the similarity between the two sample images. If the similarity is greater than a certain preset value, it means that the two sample images are similar.

[0065] Step 103: Determine the initial model for sharpness recognition. The initial model for sharpness recognition includes the feature extraction module and sharpness module trained in the above steps.

[0066] Specifically, the sharpness detection system can determine the structure of the initial model for sharpness recognition, which includes the aforementioned pre-trained feature extraction module and sharpness module. The feature extraction module is used to extract feature information of various types of sample images, and the sharpness module is used to determine the sharpness score of the sample images based on the feature information extracted by the feature extraction module.

[0067] Since the feature extraction module has been trained through the above steps, that is, the parameter values ​​involved in the feature extraction module have been determined, it is not necessary to determine the initial values ​​of the parameters in the feature extraction module when determining the initial model for sharpness recognition; only the initial values ​​of the parameters in the sharpness module need to be determined.

[0068] Step 104: Obtain the second training sample, which includes: multiple types of sample images and the second annotation information of each sample image. The second annotation information includes a sharpness score, wherein the sharpness benchmarks of different types of sample images are different.

[0069] Specifically, when acquiring the second training sample, multiple types of high-resolution images can be identified first, and the high-resolution images can be subjected to resolution degradation processing to obtain degraded images. The degraded images are then used as sample images in the second training sample; and the resolution score of the degraded images is set.

[0070] When identifying multiple types of high-definition images, the sharpness detection system can first identify multiple types of benchmark images and images to be labeled, where the sharpness of the multiple types of benchmark images belongs to the same level; the image to be labeled is compared with the corresponding type of benchmark image multiple times, and multiple comparison results are obtained; if the multiple comparison results are consistent, and the comparison result shows that the sharpness of the image to be labeled is higher than that of the benchmark image, the image to be labeled is identified as a high-definition image of the corresponding type.

[0071] It should be noted that the multiple types of benchmark images can be derived from attributes such as content or texture features within the image. Specifically, these can include, but are not limited to, the following types: people, objects, landscapes, game images, anime virtual images, historical photographs, and old movies. In this embodiment, the image sharpness score is associated with the image type. Therefore, when determining the sharpness score of different image types, different sharpness judgment criteria will be used due to the different image types. For example, generally, images from old movies are not very clear, and users have a higher tolerance and acceptance when viewing such images. Therefore, the sharpness judgment criterion for such images can be lowered, resulting in a higher sharpness score for these less clear images. Thus, if two images of different types have the same sharpness level, it does not mean that the two images are equally clear or equally blurry, but rather that the user accepts the same level of sharpness when viewing different types of images.

[0072] Specifically, in this embodiment, it is necessary to align the sharpness of multiple types of benchmark images to the same level. The benchmark images for this level of sharpness are high-resolution images of each type. For example, the sharpness can be divided into n levels, and each level can correspond to a certain range of sharpness scores. Here, aligning the sharpness to the same level means that the sharpness scores are within the same range of the same level.

[0073] Furthermore, when performing sharpness degradation processing on high-definition images, multiple levels of sharpness degradation processing can be applied to the high-definition images to obtain multiple degraded images. Then, when setting the sharpness score of the degraded images, the sharpness scores of the multiple degraded images will be set according to the level of sharpness degradation processing.

[0074] It should be noted that a level of sharpness degradation processing may include, but is not limited to, at least one of the following: Gaussian blur, Gaussian noise, high-frequency noise, impulse noise, quantization noise, tile occlusion, Joint Photographic Experts Group (JPEG) compression, JP2K compression, color shift, saturation change, and mixed Gaussian noise. Degraded images obtained after different levels of sharpness degradation processing belong to different levels of sharpness scores. For example, if two high-resolution images are both processed with Gaussian blur to obtain degraded images, these two degraded images belong to the same level of sharpness score; if one high-resolution image is processed with Gaussian blur to obtain degraded image 1, and the other high-resolution image is processed with both Gaussian blur and impulse noise to obtain degraded image 2, these two degraded images belong to different levels of sharpness score.

[0075] Furthermore, after determining the second training samples, each sample image in the second training samples can be preprocessed, and then the following step 105 can be performed on the preprocessed sample images. For example, it can be detected whether each sample image is an image with a canvas. If so, the canvas area in the sample image can be cropped, leaving only the main image in the sample image.

[0076] Since sample images with a canvas can affect the determination of sharpness information, the canvas here refers to a black area, other colors, static images, semi-transparent effects, or dynamic effects added to the boundary of the main image when displaying the sample image. For example... Figure 3 In the sample images shown, the main subject is located in the central area, while the canvas area is located above and below the main subject. Therefore, the clarity of the canvas can significantly interfere with determining the clarity of the main subject in the sample image. For example, a semi-transparent, blurry canvas will lower the overall clarity of the sample image, while a solid-color canvas, appearing high-definition, will increase the overall clarity. Furthermore, the canvas position and size vary across different sample images, and the types of canvas are diverse, severely impacting the determination of the overall clarity of the sample image. Therefore, in this embodiment, the canvas area in the sample image can be detected and cropped, eliminating the influence of the canvas when adjusting the initial clarity recognition model later.

[0077] Step 105: Adjust the initial sharpness recognition model based on the second training sample to obtain the final sharpness recognition model, which is used to extract feature information of any target object and determine the sharpness information of the target object based on the feature information.

[0078] Specifically, when adjusting the initial model for sharpness recognition, the sharpness detection system can first determine the sharpness information of each sample image in the second training sample through the initial model, which may be a sharpness level or score; then, based on the results obtained from the initial model and the second annotation information in the second training sample, calculate the second loss function related to the initial model for sharpness recognition, and then adjust the parameter values ​​in the initial model for sharpness recognition based on the second loss function.

[0079] The second loss function represents the error between the sharpness information of each sample image obtained from the initial sharpness recognition model and the actual sharpness information (obtained from the second annotation information) in each sample image of the second training samples. The training process of the sharpness recognition model aims to minimize the value of the second loss function. This training process continuously optimizes the parameter values ​​in the initial sharpness recognition model determined in step 103 using a series of mathematical optimization techniques such as backpropagation differentiation and gradient descent, thereby minimizing the calculated value of the second loss function.

[0080] It should be noted that step 105 involves adjusting the parameter values ​​in the initial sharpness recognition model based on the sharpness scores of each sample image obtained from the initial model. In practical applications, step 105 needs to be executed repeatedly until the parameter value adjustment meets certain stopping conditions. Therefore, after executing step 105 in the above embodiment, the sharpness detection system needs to determine whether the current parameter value adjustment meets the preset stopping conditions. If it does, the process ends, and the parameter values ​​obtained after adjustment in step 105 are used as the parameter values ​​of the finally trained sharpness recognition model. If it does not meet the conditions, the system returns to executing step 105 again for the initial sharpness recognition model after parameter value adjustment. The preset stopping conditions include, but are not limited to, any one of the following: the difference between the currently adjusted parameter value and the previously adjusted parameter value is less than a threshold, i.e., the adjusted parameter value has converged; and the number of parameter value adjustments is equal to the preset number, etc.

[0081] As can be seen, in the method of this embodiment, the sharpness detection system first trains the feature extraction module using the first training samples, and determines that the initial sharpness recognition model includes the trained feature extraction module and sharpness module. Then, it adjusts the initial sharpness recognition model using the second training samples to obtain the final trained sharpness recognition model, which can acquire the sharpness information of any target object. Since the sharpness discrimination benchmarks of different types of sample images are different when annotating each sample image in the second training samples to obtain the second annotation information, the trained sharpness recognition model associates the sharpness information with the actual type of the target object in the process of determining the sharpness information, making the determined sharpness information closer to the user's perception and the quality assessment of the target object more objective. Furthermore, since the feature extraction module in the sharpness recognition model has been trained using the first training samples, only fine-tuning of the feature extraction module is needed when training the sharpness recognition model using the second training samples. Thus, a smaller number of second training samples are required to train the sharpness recognition model.

[0082] It should be further explained that after training the sharpness recognition model through the above steps, the operating logic of the sharpness recognition model can be pre-installed in the system. This way, when sharpness detection is needed for any target object (image or video), the sharpness recognition model can be directly invoked to obtain the sharpness score. In one case, if the target object is a video, the sharpness detection system will obtain the sharpness score for that target object as follows:

[0083] First, determine the number of frames to extract images from the target object. Then, extract the corresponding number of images from the target object based on the determined number of frames. Use a sharpness recognition model to determine the sharpness score of each extracted image. Finally, obtain the overall sharpness of the target object based on the sharpness scores of the extracted images. For example, the average sharpness score of each extracted image can be used as the overall sharpness of the target object.

[0084] Specifically, when determining the number of frames for image extraction of a target object, the content-based type of the target object can be determined first, and a corresponding weight value can be set according to the type of the target object. Then, based on the weight value and the preset frame number constraint relationship based on the weight value, the number of frames for image extraction of the target object can be determined.

[0085] The preset frame number constraint relationship based on weight values ​​can be represented by the following formula 1:

[0086] (1)

[0087] in, This represents the complete set of videos. Typically, 30 frames are uniformly extracted from a video to form the complete set, i.e., N = 30. This represents a subset of the entire set, where k is the number of frames in the subset. The proportion of the subset to the whole set. This represents the standard deviation. In determining the number of frames in an image, it's necessary to minimize the value of the frame number constraint relationship, which requires... Make it as large as possible The larger the sample size, the more frames are extracted, and the smaller the standard deviation of the selected images (to eliminate severely blurry and low-scoring images that do not match the video clarity due to special effects, fast motion, etc.).

[0088] The above The weight value is assigned to the video. In practice, different weight values ​​can be determined for different video content types. For example, for ordinary videos of everyday life, a weight value of 2 can be used, which means extracting as many frames as possible while excluding occasional motion-blurred frames. For videos of street dance or K-pop, a weight value of 5 can be used, which means excluding images of frequently occurring severe motion blur and glare frames. Different weight values ​​can be configured for different types of videos, such as movie clips, concert performances, some animal videos, and anime clips, to ensure that each video type has the most suitable frame rate to video score. The mapping.

[0089] The following is a specific application example to illustrate the sharpness detection method of the present invention, which can be applied to, for example... Figure 4 In the scenario shown, in this scenario:

[0090] I. Content Production and Consumption 10

[0091] The content production and consumption terminal 10 is used to generate PGC, UGC, MCN, or PUG content. For example, it can shoot video content, during which local video content can be matched with music, filter templates, and video beautification functions. It can also write self-media articles or photo albums. Then, through communication with the upstream and downstream content interface server 11, it uploads the generated streaming media content. It is also used to obtain the index information of the streaming media content, download the corresponding streaming media content, and play it through a local player.

[0092] On the content production and consumption side, the system also records user behavior data during the upload and download process, such as reading speed, completion rate, reading time, buffering, loading time, and playback clicks, and reports this data to the server.

[0093] The content production and consumption end 10 is also used to provide a direct feedback entry point for streaming media content that is not clear enough or blurry. It directly connects with the manual review system for confirmation and review. The review results can be stored in the content metadata and database 12 to provide training samples for subsequent model training.

[0094] II. Uplink and Downlink Content Interface Server 11

[0095] The uplink and downlink content interface server 11 communicates directly with the content production and consumption end 10 to obtain streaming media content (such as videos or pictures) and its metadata. The metadata can typically include: title, publisher, summary, cover image, publication time, file format, content size and bitrate, etc., and writes the streaming media content and its metadata into the content metadata and database 12.

[0096] III. Content Meta-information and Database 12

[0097] The content metadata and database 12 are used to store the aforementioned streaming media content and its metadata, such as content size, cover image link, bitrate, file format, title, release time, author, whether it is original, possible categories and tags; it can also store the classification of the aforementioned streaming media content during the manual review process.

[0098] During the subsequent manual review process, the manual review service system 14 will read the streaming media content from the content metadata and database 12, and simultaneously send the review results and status back to the content metadata and database 12 for storage. In this embodiment, the manual review service system 14 can also return information such as the resolution level (e.g., a level from 1 to 10) of the streaming media content to the content metadata and database 12 for storage.

[0099] IV. Content Ingestion Service 13

[0100] The content entry service 13 is used to receive streaming media content through the uplink and downlink content interface server 11, and can also obtain the metadata of streaming media content from the content metadata and database 12; it is also used to schedule the corresponding image clarity scoring model and service 18 or video clarity fusion scoring service 19 to perform clarity detection for any target object (image or video).

[0101] The content loading service 13 is also used to call the video frame extraction service 20 to extract images of the corresponding number of frames in the video when the target object is a video.

[0102] 5. The manual review service system 14 is used to receive review tasks, review the tasks accordingly, and write the review results into the content metadata and database 12 for storage; it is also used to connect with the content production and consumption end 10, and after reviewing and confirming unclear streaming media content, it marks the clarity level of the streaming media content according to 10 levels, and returns the review results and markings to the content metadata and database 12 for storage.

[0103] The human review service system 14 can supplement the machine review capabilities to handle other quality issues (such as clickbait or legal issues) in streaming media content.

[0104] (3) Regarding the issue of content distribution based on clarity, the fine-grained clarity score can be used as a factor for sorting or content filtering based on the application scenario and the richness of content supply in different categories. During the review process, high-definition videos can be reviewed first, and high-definition versions can be retained when deduplicating videos. This can reduce unnecessary review costs and improve user experience.

[0105] VI. Content distribution library 15

[0106] The distribution content library 15 is a storage server deployed near the user. It can store streaming media content uploaded by content producers through the uplink and downlink content interface server 11, and allows content producers and consumers 10 to directly access and download the streaming media content stored in the distribution content library 15.

[0107] VII. Image Corpus and Scoring Sample Database 16

[0108] The image corpus and scoring sample library 16 are used to obtain images marked with clarity information during the manual review process from the content metadata and database 12, so as to serve as training samples in the subsequent model training process; it is also used to obtain the first training sample and the second training sample through various methods (such as obtaining them from the image corpus crawling and processing system 21) so as to train the feature extraction module and the clarity recognition model respectively.

[0109] 8. A large-scale image pre-training model 17 is used to obtain the first training samples from the image corpus and the scoring sample library 16 to train the feature extraction module.

[0110] 9. Image Clarity Scoring Model and Service 18: This model is used to obtain second training samples from the image corpus and scoring sample library 16, and to obtain a trained feature extraction module from the large-scale image pre-trained model 17 to train the clarity recognition model. It is also used to periodically obtain new second training samples based on unclear images reported by users, for example, on a weekly or daily basis, to update the clarity recognition model.

[0111] 10. Video Clarity Fusion Scoring Service 19 is used to pre-set the image clarity scoring model and the clarity recognition model trained by Service 18. When any target object is a video, the image extracted from the video is obtained from the video frame extraction service 20. The extracted image is processed by the clarity recognition model to obtain the overall clarity information of the target object.

[0112] 11. Video frame extraction service 20, used to extract images from video based on a certain number of frames to obtain multiple frames.

[0113] XIV. Image Corpus Crawling and Processing System 21, used to crawl image data from an information search system through information crawling methods, and statistically obtain images with the same or similar tags to form positive sample pairs in the first training sample, and obtain images with different tags to form negative sample pairs in the first training sample.

[0114] In the above scenario, the sharpness detection method in this embodiment may include the following parts: pre-training data acquisition and processing, self-supervised model training, benchmark sample data annotation and processing, fine-tuning model training, and video frame score fusion output. Wherein:

[0115] (a) Acquisition and processing of 16 pairs of pre-training data from the image corpus and scoring sample database, i.e., obtaining the first training samples.

[0116] Specifically, the sample pairs in the first training sample can be obtained in the following ways, including but not limited to: extracting the images included in each sample pair from multiple sample videos; obtaining the images included in each sample pair statistically from an information search system; and obtaining the images included in each sample pair from an existing classification label dataset.

[0117] (ii) Large-scale image pre-training model 17 performs self-supervised training, that is, the feature extraction module is trained based on the first training sample.

[0118] Specifically, the sample images in each sample pair of the first training sample can be preprocessed, such as cropping and resizing, color distortion, color gittering, and Gaussian blurring.

[0119] Then an initial feature extraction model can be determined, specifically as follows: Figure 5As shown, the process includes: Swing + Transformer + Contrastive Learning + Model Distillation (distillation is optional, as it reduces model size, facilitates deployment and model acceleration, and better meets the needs of actual business). The Self-Supervised Learning (SSL) method learns relevant features from the preprocessed first training samples and combines various general representations for a better understanding of the business domain. Here, the Swing Transformer is used as the backbone of the feature extraction module.

[0120] During the training of the above feature extraction model, a first loss function can be calculated to adjust the parameter values ​​in the initial feature extraction model, so that the value of the first loss function converges. The first loss function L can be expressed by the following formula 2:

[0121] (2)

[0122] Where d represents the Euclidean distance between the feature information of two sample images in a sample pair extracted by the initial feature extraction model, y is the first annotation information of whether two sample images in the first training sample are similar, y=1 indicates similarity, y=0 indicates dissimilarity, and margin is the set threshold.

[0123] (III) Benchmark sample data annotation and processing are performed on the image corpus and scoring sample library 16 to obtain the second training samples.

[0124] Specifically, the image corpus and scoring sample library 16 can first determine multiple types of benchmark images. These benchmark images are of the same level of clarity and are all high-definition images in their respective types. For example, benchmark images can be selected from 10 types such as people, objects, landscapes, game and anime virtual images. Multiple types of images to be labeled also need to be determined. One frame can be randomly selected from each type of video as the image to be labeled.

[0125] Then, the sharpness of each image to be labeled is compared multiple times with the corresponding type of benchmark image, and multiple comparison results are obtained. Specifically, in order to optimize the annotation of the second annotation information for the images to be labeled, a double-blind voting process can be performed on each image to be labeled and multiple types of benchmark images, i.e. Figure 6As shown, the annotation results of multiple annotators (annotators A and B are used as examples in the figure) comparing the sharpness of any image to be annotated can include whether the sharpness of the image to be annotated is higher than, lower than, or similar to that of a benchmark image of the corresponding type. If the sharpness comparison results of the annotators on the image to be annotated are consistent, the image to be annotated can be identified as a high-definition image of the corresponding type. If the sharpness comparison results of the annotators on the image to be annotated are inconsistent, the sharpness comparison between the image to be annotated and the benchmark image can be re-examined.

[0126] Furthermore, each high-resolution image undergoes multiple levels of sharpness degradation processing to obtain multiple degraded images, which serve as the sample images in the second training sample. Sharpness degradation processing can include Gaussian blur, Gaussian noise, high-frequency noise, impulse noise, quantization noise, tile occlusion, etc. Finally, sharpness scores are assigned to the multiple degraded images according to the level of sharpness degradation processing. By associating the set sharpness scores with multiple levels of sharpness degradation processing, sample images with different levels of sharpness under various types can be obtained, thereby enhancing the robustness and generalization ability of the sharpness recognition model subsequently trained using the second training sample.

[0127] Furthermore, the image corpus and scoring sample library 16 can also preprocess each sample image, such as detecting whether each sample image is an image with a canvas. If so, the canvas area in the sample image can be cropped, leaving only the main image in the sample image.

[0128] (iv) The image clarity scoring model and service 18 are fine-tuned during model training, that is, the clarity recognition model is trained based on the second training sample.

[0129] Specifically, the image sharpness scoring model and service 18 can determine an initial sharpness recognition model, which can be as follows: Figure 7 As shown, it includes: the feature extraction module and the sharpness module trained by the first training sample. Specifically, the feature extraction module can be a Swing Transformer structure, and the sharpness module specifically includes a classifier (such as an LR classifier) ​​and a Softmax activation function. This initial sharpness recognition model can extract the feature information of each sample image in the second training sample through the feature extraction module. After the feature information is processed by the classifier and the Softmax activation function, it can output the information of which sharpness level the sample image belongs to. In this embodiment, the sharpness level can be set to 10 levels.

[0130] Then, the image clarity scoring model and service 18 adjust the initial clarity recognition model based on the second training sample, and finally obtain the trained clarity recognition model.

[0131] (v) Video Clarity Fusion Scoring Service 19 When any target object is a video, the video frame score is fused and output, that is, the clarity score of the target object is output through the trained clarity recognition model.

[0132] Specifically, for a target object that is a video, the video frame extraction service 20 can first determine the content-based type of the video, and determine the weight value corresponding to the video based on the type. Combined with the above formula 1, it can obtain the number of frames to extract from the video, and extract the corresponding number of frames from the target object based on the obtained number of frames.

[0133] Then, the video clarity fusion scoring service 19 can determine the clarity information corresponding to the multiple images extracted by the video frame extraction service 20 through the clarity recognition model trained above. Specifically, it can be the information of which clarity level each image belongs to. Then, the average value of the clarity information corresponding to the multiple images can be used as the clarity information of the target object.

[0134] It should be noted that, due to the wide variety of video types and their significant stylistic differences, the user experience can vary considerably. For example, video A, with its intense and fast-paced action, results in a large proportion of motion blur and ghosting in the images regardless of the frame extraction method used. In contrast, most of the frame-extracted images from video B are relatively clear. Therefore, this embodiment extracts a corresponding number of frames based on the video type and fuses the clarity information of the extracted multiple frames, taking into account the differences in video content, thus making the obtained clarity information of the target object more accurate.

[0135] As can be seen, the sharpness detection method of this embodiment can achieve the following technical effects:

[0136] (1) During the training of the sharpness recognition model, when the second annotation information is added to the sample images in the second training sample, it is associated with the type of the sample image, so that the sharpness judgment benchmark of different types of sample images is different. This allows the trained sharpness recognition model to take into account the actual type of the target object when determining the sharpness information of the target object, making it more objective.

[0137] (2) When acquiring the second training sample, a high-definition image is obtained by comparing it with various types of benchmark images. Then, the high-definition image is processed to obtain sample images of various types. Thus, the trained clarity recognition model can distinguish video clarity of more granularities, adapt to different application scenarios, and improve the distribution efficiency and effect of information recommendation based on clarity information.

[0138] The following uses another specific application example to illustrate the sharpness detection method in this invention. The sharpness detection system in this embodiment is mainly a distributed system 100. The distributed system may include a client 300 and multiple nodes 200 (any form of computing device in the network, such as a server or user terminal). The client 300 and the nodes 200 are connected through network communication.

[0139] Taking a distributed system as an example, see blockchain system. Figure 8 This is an optional structural diagram of the distributed system 100 provided in this embodiment of the invention applied to a blockchain system. It consists of multiple nodes 200 (any form of computing device connected to the network, such as servers or user terminals) and clients 300. The nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In the distributed system, any machine, such as a server or terminal, can join and become a node. A node includes a hardware layer, a middleware layer, an operating system layer, and an application layer.

[0140] See Figure 8 The functions of each node in the blockchain system shown include:

[0141] 1) Routing: A basic function of nodes used to support communication between nodes.

[0142] In addition to routing capabilities, nodes can also have the following functions:

[0143] 2) Applications are deployed in the blockchain to implement specific business needs. They record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system. When other nodes successfully verify the source and integrity of the record data, they add the record data to a temporary block.

[0144] For example, the application's business logic includes code that implements a sharpness detection function, which mainly includes:

[0145] A first training sample is obtained, comprising multiple sample pairs, each sample pair including two sample images and a first annotation indicating whether the two sample images are similar; a feature extraction module is trained based on the first training sample, the feature extraction module being used to extract features of any target object containing an image; an initial sharpness recognition model is determined, the initial sharpness recognition model including the feature extraction module and the sharpness module; a second training sample is obtained, comprising multiple types of sample images and second annotation information for each sample image, the second annotation information including a sharpness score, wherein the sharpness discrimination benchmarks for different types of sample images are different; the initial sharpness recognition model is adjusted based on the second training sample to obtain a final sharpness recognition model, used to extract feature information of any target object, and the sharpness information of the target object is determined based on the feature information.

[0146] 3) A blockchain consists of a series of blocks that are sequentially generated. Once a new block is added to the blockchain, it will not be removed. The blocks contain the data submitted by the nodes in the blockchain system.

[0147] See Figure 9 This is an optional schematic diagram of the block structure provided in an embodiment of the present invention. Each block includes the hash value of the transaction records stored in this block (the hash value of this block) and the hash value of the previous block. The blocks are connected through their hash values ​​to form a blockchain. Additionally, the block may also include information such as a timestamp when it was generated. A blockchain is essentially a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains relevant information used to verify the validity of the information (anti-counterfeiting) and to generate the next block.

[0148] This invention also provides a sharpness detection system, the structural schematic of which is shown below. Figure 10 As shown, it can specifically include:

[0149] The first sample unit 30 is used to acquire a first training sample, which includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar.

[0150] The first sample unit 30 is specifically used to acquire multiple sample videos, extract two frames from any of the sample videos to form a positive sample pair, and set the first annotation information of the positive sample pair to be the information that the two frames are similar; extract one frame from any two different sample videos to form a negative sample pair, and set the first annotation information of the negative sample pair to be the information that the two frames in the negative sample pair are not similar.

[0151] The first sample unit 30 is also used to search multiple label data in the statistical information search system; select two images under any label in the multiple label data to form a positive sample pair, and set the first annotation information of the positive sample pair to be the information that the two images are similar; select an image under any two different labels to form a negative sample pair, and set the first annotation information of the negative sample pair to be the information that the two images in the negative sample pair are not similar.

[0152] The feature training unit 31 is used to train a feature extraction module based on the first training sample obtained by the first sample unit 30. The feature extraction module is used to extract features of any target object containing an image.

[0153] The model determination unit 32 is used to determine the initial model for sharpness recognition, which includes the feature extraction module and sharpness module trained by the feature training unit 31.

[0154] The second sample unit 33 is used to acquire a second training sample, which includes: multiple types of sample images and second annotation information for each sample image. The second annotation information includes a sharpness score, wherein the sharpness discrimination criteria for different types of sample images are different.

[0155] The second sample unit 33 is specifically used to determine multiple types of high-definition images; to perform sharpness degradation processing on the high-definition images to obtain degraded images, and to use the degraded images as sample images in the second training samples; and to set the sharpness score of the degraded images.

[0156] Specifically, when determining multiple types of high-definition images, the second sample unit 33 is used to determine multiple types of benchmark images and to determine the image to be labeled, wherein the multiple types of benchmark images belong to the same level of clarity; the image to be labeled is compared with the corresponding type of benchmark image multiple times in terms of clarity, and multiple comparison results are obtained respectively; if the multiple comparison results are consistent, and the comparison result is that the clarity of the image to be labeled is higher than that of the benchmark image, the image to be labeled is determined as a high-definition image of the corresponding type.

[0157] Furthermore, when the second sample unit 33 performs sharpness degradation processing on the high-definition image to obtain a degraded image, it is specifically used to perform multiple levels of sharpness degradation processing on the high-definition image to obtain multiple degraded images respectively; and when setting the sharpness score of the degraded image, it is specifically used to set the sharpness score of the multiple degraded images according to the level of sharpness degradation processing.

[0158] The adjustment unit 34 is used to adjust the initial sharpness recognition model determined by the model determination unit 32 according to the second training sample obtained by the second sample unit 33, so as to obtain the final sharpness recognition model, and is used to extract the feature information of any target object and determine the sharpness information of the target object according to the feature information.

[0159] Furthermore, the sharpness detection system may also include:

[0160] The clarity unit 35 is used to determine the number of frames for image extraction from the target object if the target object is a video; extract a corresponding number of images from the target object according to the determined number of frames; determine the clarity score of the extracted images respectively through the clarity recognition model; and obtain the overall clarity of the target object according to the clarity score of the extracted images.

[0161] Specifically, when determining the number of frames for image extraction of the target object, the clarity unit 35 is used to determine the content-based type of the target object; set a corresponding weight value according to the type of the target object; and determine the number of frames for image extraction of the target object according to the weight value and a preset frame number constraint relationship based on the weight value.

[0162] As can be seen, in the sharpness detection system of this embodiment, the feature training unit 31 first trains the feature extraction module using the first training samples, and the model determination unit 32 determines that the initial sharpness recognition model includes the trained feature extraction module and sharpness module. Then, the adjustment unit 34 adjusts the initial sharpness recognition model using the second training samples to obtain the final trained sharpness recognition model, which can obtain the sharpness information of any target object. Since the sharpness discrimination benchmarks of different types of sample images are different when annotating each sample image in the second training samples to obtain the second annotation information, the trained sharpness recognition model associates the sharpness information with the actual type of the target object in the process of determining the sharpness information, making the determined sharpness information closer to the user's perception and the quality assessment of the target object more objective. Furthermore, since the feature extraction module in the sharpness recognition model has been trained using the first training samples, only fine-tuning of the feature extraction module is needed when training the sharpness recognition model using the second training samples. Thus, a smaller number of second training samples are required to train the sharpness recognition model.

[0163] This invention also provides a terminal device, the structural schematic of which is shown below. Figure 11 As shown, the terminal device can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 40 (e.g., one or more processors) and memory 41, and one or more storage media 42 (e.g., one or more mass storage devices) for storing application programs 421 or data 422. The memory 41 and storage media 42 can be temporary or persistent storage. The program stored in the storage media 42 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the terminal device. Furthermore, the CPU 40 may be configured to communicate with the storage media 42 and execute the series of instruction operations in the storage media 42 on the terminal device.

[0164] Specifically, the application program 421 stored in the storage medium 42 includes a sharpness detection application program, which may include the first sample unit 30, feature training unit 31, model determination unit 32, second sample unit 33, adjustment unit 34, and sharpness unit 35 of the aforementioned sharpness detection system, which will not be elaborated here. Furthermore, the central processing unit 40 may be configured to communicate with the storage medium 42 and execute a series of operations corresponding to the sharpness detection application program stored in the storage medium 42 on the terminal device.

[0165] The terminal device may also include one or more power supplies 43, one or more wired or wireless network interfaces 44, one or more input / output interfaces 45, and / or one or more operating systems 423, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0166] The steps performed by the sharpness detection system in the above method embodiments can be based on this. Figure 11 The structure of the terminal device shown is illustrated.

[0167] Furthermore, in another aspect, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of computer programs adapted to be loaded by a processor and executed as described in the sharpness detection system above, for performing a sharpness detection method.

[0168] Another embodiment of the present invention provides a terminal device, including a processor and a memory;

[0169] The memory is used to store multiple computer programs, which are loaded by a processor and executed as described in the sharpness detection system above; the processor is used to implement each of the multiple computer programs.

[0170] Additionally, according to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the sharpness detection method provided in the various alternative implementations described above.

[0171] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0172] The above provides a detailed description of a resolution detection method, system, storage medium, and terminal device provided by embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A sharpness detection method, characterized in that, include: Obtain a first training sample, which includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar. A feature extraction module is trained based on the first training sample. The feature extraction module is used to extract features of any target object containing an image. A sharpness recognition initial model is determined, the sharpness recognition initial model including the feature extraction module and the sharpness module; Obtain a second training sample, which includes: multiple types of sample images and second annotation information for each sample image. The second annotation information includes a sharpness score, wherein the sharpness discrimination criteria for different types of sample images are different. The initial sharpness recognition model is adjusted based on the second training sample to obtain the final sharpness recognition model, which is used to extract feature information of any target object containing an image, and to determine the sharpness information of the target object based on the feature information.

2. The method as described in claim 1, characterized in that, The acquisition of the first training sample specifically includes: Multiple sample videos are acquired, and two frames are extracted from any of the sample videos to form a positive sample pair. The first annotation information of the positive sample pair is set as the information that the two frames are similar. Extract one frame from each of two different sample videos to form a negative sample pair, and set the first annotation information of the negative sample pair as the information that the two frames in the negative sample pair are not similar; Alternatively, obtaining the first training sample specifically includes: Multiple tag data in a statistical information search system; Select two images under any label from the multiple label data to form a positive sample pair, and set the first annotation information of the positive sample pair to be the information that the two images are similar; Under any two different labels, select an image to form a negative sample pair, and set the first label information of the negative sample pair to be the information that the two images in the negative sample pair are not similar.

3. The method according to any one of claims 1 to 2, characterized in that, The acquisition of the second training sample specifically includes: Identify multiple types of high-resolution images; The high-resolution image is subjected to a sharpening degradation process to obtain a degraded image, which is then used as a sample image in the second training sample. Set the sharpness score of the degraded image.

4. The method as described in claim 3, characterized in that, The determination of multiple types of high-definition images specifically includes: Multiple types of benchmark images are identified, and images to be labeled are determined, wherein the multiple types of benchmark images belong to the same level of sharpness; The image to be labeled is compared with the corresponding type of benchmark image in terms of sharpness multiple times, and the comparison results are obtained for each comparison. If the results of the multiple comparisons are consistent, and the comparison results indicate that the image to be labeled has a higher resolution than the benchmark image, then the image to be labeled is identified as a high-resolution image of the corresponding type.

5. The method as described in claim 3, characterized in that, The process of performing a sharpness degradation process on the high-definition image to obtain a degraded image specifically includes: performing multiple levels of sharpness degradation processing on the high-definition image to obtain multiple degraded images respectively; Setting the sharpness score of the degraded image specifically includes: setting the sharpness score of each of the multiple degraded images according to the level of sharpness degradation processing.

6. The method according to any one of claims 1 to 2, characterized in that, If the target object is a video, the method further includes: Determine the number of frames for image extraction of the target object; Extract a corresponding number of images from the target object based on the determined number of frames; The sharpness score of the extracted images is determined by the sharpness recognition model, and the overall sharpness of the target object is obtained based on the sharpness score of the extracted images.

7. The method as described in claim 6, characterized in that, The determination of the number of frames for image extraction of the target object specifically includes: Determine the content-based type of the target object; Set the corresponding weight value according to the type of the target object; Based on the weight values ​​and the preset frame count constraint relationship based on the weight values, the number of frames for image processing of the target object is determined.

8. A sharpness detection system, characterized in that, include: The first sample unit is used to acquire the first training sample, which includes multiple sample pairs. Each sample pair includes two sample images and first annotation information on whether the two sample images are similar. A feature training unit is used to train a feature extraction module based on the first training sample. The feature extraction module is used to extract features of any target object containing an image. A model determination unit is used to determine an initial model for sharpness recognition, wherein the initial model for sharpness recognition includes the feature extraction module and the sharpness module; The second sample unit is used to acquire the second training sample, which includes: multiple types of sample images and the second annotation information of each sample image. The second annotation information includes a sharpness score, wherein the sharpness discrimination criteria of different types of sample images are different. The adjustment unit is used to adjust the initial model of sharpness recognition according to the second training sample to obtain the final sharpness recognition model, and to extract feature information of any target object containing the image, and determine the sharpness information of the target object according to the feature information.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs adapted to be loaded by a processor and executed as described in any one of claims 1 to 7.

10. A terminal device, characterized in that, Including processor and memory; The memory is used to store a plurality of computer programs, the computer programs being loaded by a processor and executed as described in any one of claims 1 to 7; the processor is used to implement each of the plurality of computer programs.

11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the sharpness detection method as described in any one of claims 1 to 7.

Images