Image definition recognition method and device, and storage medium

Abstract

The application discloses an image definition recognition method and device and a storage medium, which can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and Internet of Vehicles. The method comprises the following steps: obtaining an image to be recognized; performing definition parameter recognition processing on the image to be recognized based on a definition parameter recognition model to obtain a target definition parameter of the image to be recognized; and a training method of the definition parameter recognition model comprises the following steps: obtaining a first sample image; performing definition parameter recognition training on an image feature extraction model based on the first sample image to adjust model parameters of the image feature extraction model until definition parameter labels output by the image feature extraction model are matched with labeled definition parameter labels; and taking the image feature extraction model corresponding to the model parameters when the output definition parameter labels are matched with the labeled definition parameter labels as the definition parameter recognition model. The application improves the recognition accuracy of image definition.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image sharpness recognition method, apparatus and storage medium. Background Technology

[0002] Video clarity is a crucial indicator of video quality, especially for films, TV dramas, and animations, where high-definition video significantly enhances the user experience. Many factors influence clarity, including blurriness, mosaic effects, noise levels, brightness, and encoding errors. These factors typically arise from compression, transmission, reshooting, and editing of the original video or image. While humans can accurately judge video clarity with their eyes, it's extremely difficult for computers to automatically assess it. This is because: many factors affect clarity, each difficult to quantify mathematically; computers struggle to distinguish between true and false blur (e.g., a clear foreground but a blurred background during filming), and blurriness caused by encoding and compression are easily identifiable by humans, as are mosaic effects and noise; and computers rely on quantified clarity to assess the clarity of unknown videos, a level of intelligence that cannot yet match human capabilities. For video services, user perception of video quality is subjective, diverse, and difficult to quantify and measure. Subjective testing is the gold standard for video quality analysis. Subjective testing is considered a relatively accurate method for evaluating video quality, but it has three drawbacks: (1) high time and money costs; (2) limited video types, test conditions, and number of viewers; and (3) it cannot be used for real-time evaluation. Existing image sharpness recognition methods are mainly based on features related to visual degradation or features of the original undamaged image / video, thus forming an estimate of perceived quality. The main solutions are blur (loss of spatial details and reduction of edge sharpness, using edge detectors to calculate image edge distortion values ​​to estimate the degree of blur), noise (dividing the image into smooth regions, where the variance above a certain threshold gives an estimate of noise), ringing (ringing is the form of ripples around high-contrast edges, which can be used to classify image regions by extracting certain features from edge mapping), and blocking (blocking is manifested as discontinuity of adjacent blocks in image and video frames, related to compression technology, which estimates the blockiness of the image domain by calculating the difference between adjacent blocks and the brightness around these blocks). However, traditional fuzzy image recognition algorithms handle JPEG compression patch noise and boundary blur well, but perform poorly on images with overall blurriness. In other words, traditional fuzzy image recognition algorithms cannot effectively describe these types of problems and can only provide coarse-grained options such as blurred, clear, and high-resolution, which is not conducive to implementing fine-grained strategies based on specific business scenarios. Furthermore, for images with blurred backgrounds or artistic styles, this type of noise involves semantic information, which simple traditional fuzzy image recognition algorithms cannot identify. Many such videos or images will be considered blurry, thus making it impossible to accurately determine the image's sharpness. Summary of the Invention

[0003] This application provides an image sharpness recognition method, apparatus, and storage medium, which can improve the accuracy of image sharpness recognition.

[0004] On the one hand, this application provides an image sharpness recognition method, the method comprising:

[0005] The image to be identified is processed by a sharpness parameter recognition model to obtain the target sharpness parameter of the image to be identified.

[0006] The training method for the sharpness parameter recognition model includes:

[0007] Obtain the first sample image labeled with the sharpness parameter.

[0008] Based on the first sample image, the image feature extraction model is trained to identify sharpness parameters, thereby adjusting the model parameters of the image feature extraction model until the sharpness parameter labels output by the image feature extraction model match the labeled sharpness parameter labels. The image feature extraction model is obtained by training a preset model to extract images based on a second sample image pair. The second sample image pair includes positive sample image pairs and negative sample image pairs. The similarity between two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between two negative sample images in the negative sample image pair is less than the first threshold.

[0009] The image feature extraction model corresponding to the model parameters when the output sharpness parameter labels are matched with the labeled sharpness parameter labels is used as the sharpness parameter recognition model.

[0010] In some embodiments, obtaining the preset image set includes:

[0011] Retrieve video and image sets from the information stream;

[0012] Determine the first text tag for each video in the video set to obtain the first text tag set;

[0013] Determine the second text label for each image in the image set to obtain the second text label set;

[0014] The target text tag is determined based on the first text tag set and the second text tag set;

[0015] The preset image set is obtained by using the target text tag as the search term.

[0016] In some embodiments, determining the target text tag based on the first text tag set and the second text tag set includes:

[0017] Based on the first text tag set and the second text tag set, a comprehensive text tag set is determined;

[0018] Determine the frequency of occurrence of each comprehensive text tag in the information stream from the comprehensive text tag set;

[0019] The target text tag is determined based on the frequency of occurrence of each comprehensive text tag.

[0020] In some embodiments, determining the target text tag based on the first text tag set and the second text tag set includes:

[0021] Based on the first text tag set, determine the first target text tag set;

[0022] Based on the second text tag set, determine the second target text tag set;

[0023] The tags in the first target text tag set and the second target text tag set are determined as target text tags.

[0024] On the other hand, an image sharpness recognition device is provided, the device comprising:

[0025] The image acquisition module is used to acquire the image to be recognized.

[0026] The target sharpness parameter determination module is used to perform sharpness parameter recognition processing on the image to be recognized based on the sharpness parameter recognition model to obtain the target sharpness parameter of the image to be recognized.

[0027] The model training module is used to train the sharpness parameter recognition model;

[0028] The model training module includes:

[0029] The first sample image acquisition submodule is used to acquire the first sample image labeled with the sharpness parameter.

[0030] The model parameter adjustment submodule is used to train the image feature extraction model on the first sample image to identify sharpness parameters, so as to adjust the model parameters of the image feature extraction model until the sharpness parameter label output by the image feature extraction model matches the labeled sharpness parameter label; the image feature extraction model is obtained by training a preset model on the second sample image pair, the second sample image pair includes positive sample image pairs and negative sample image pairs, the similarity between the two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between the two negative sample images in the negative sample image pair is less than the first threshold;

[0031] The sharpness parameter recognition model determination submodule is used to extract the image feature model corresponding to the model parameters when the output sharpness parameter label is matched with the labeled sharpness parameter label, and use it as the sharpness parameter recognition model.

[0032] On the other hand, an image sharpness recognition device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image sharpness recognition method as described above.

[0033] On the other hand, a computer storage medium is provided that stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the image sharpness recognition method as described above.

[0034] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image sharpness recognition method as described above.

[0035] The image sharpness recognition method, apparatus, and storage medium provided in this application have the following technical advantages:

[0036] This application acquires an image to be identified; performs sharpness parameter identification processing on the image to be identified based on a sharpness parameter identification model to obtain the target sharpness parameter of the image to be identified; wherein, the training method of the sharpness parameter identification model includes: acquiring a first sample image labeled with sharpness parameter tags; training an image feature extraction model for sharpness parameter identification based on the first sample image to adjust the model parameters of the image feature extraction model until the sharpness parameter tags output by the image feature extraction model match the labeled sharpness parameter tags; the image feature extraction model is obtained by training a preset model for image feature extraction based on a second sample image pair, the second sample image pair including a positive sample image pair and a negative sample image pair, the similarity between two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between two negative sample images in the negative sample image pair is less than the first threshold; the image feature extraction model corresponding to the model parameters when the output sharpness parameter tags match the labeled sharpness parameter tags is used as the sharpness parameter identification model. In the training process of the sharpness parameter recognition model, this application first obtains an image feature extraction model through unsupervised training. Based on this model, it then conducts supervised training using sample images labeled with sharpness parameters. This enables the modeling of combining low-level sharpness information with high-level semantic information, thereby allowing the obtained sharpness parameter recognition model to accurately identify the sharpness of images and improving the accuracy of image sharpness recognition. Attached Figure Description

[0037] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of an image sharpness recognition system provided in an embodiment of this application;

[0039] Figure 2 This is a flowchart illustrating an image sharpness recognition method provided in an embodiment of this application;

[0040] Figure 3 This is a flowchart illustrating a method for acquiring an image to be recognized, as provided in an embodiment of this application.

[0041] Figure 4 This is a flowchart illustrating a method for determining a first image to be recognized, provided in an embodiment of this application. Figure 1 ;

[0042] Figure 5 This is a flowchart illustrating a method for determining a first image to be recognized, provided in an embodiment of this application. Figure 2 ;

[0043] Figure 6 This is a schematic flowchart of a method for determining a second image to be recognized, provided in an embodiment of this application.

[0044] Figure 7 This is a flowchart illustrating a method for determining the sharpness parameters of a video to be identified based on the target sharpness parameters corresponding to each of the at least two images to be identified, according to an embodiment of this application.

[0045] Figure 8 This is a flowchart illustrating a training method for an image feature extraction model provided in an embodiment of this application;

[0046] Figure 9 This is a schematic diagram of the structure of a self-supervised model provided in an embodiment of this application;

[0047] Figure 10 This is a flowchart illustrating a method for constructing a contrastive loss function provided in an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of the structure of a sharpness parameter recognition model provided in an embodiment of this application.

[0049] Figure 12 This is a schematic diagram of the structure of an image sharpness recognition device provided in an embodiment of this application;

[0050] Figure 13 This is a schematic diagram of the structure of an image sharpness recognition system provided in an embodiment of this application;

[0051] Figure 14 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0053] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0054] Feeds: A news feed, also translated as source, delivery, information provision, summary, source, news subscription, or web feed (English: web feed, news feed, syndicated feed), is a data format that websites use to disseminate the latest information to users. It is usually arranged in a timeline format; timeline is the most original, direct, and basic form of feed display. A prerequisite for users to subscribe to a website is that the website provides news sources. The aggregation of feeds in one place is called aggregation, and the software used for aggregation is called an aggregator. For end users, an aggregator is software specifically designed for subscribing to websites, and is also commonly referred to as an RSS reader, feed reader, or news reader.

[0055] CNN: Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that incorporate convolutional computations and have a deep structure. They are one of the representative algorithms of deep learning. CNNs possess representation learning capabilities, enabling them to perform shift-invariant classification of input information according to their hierarchical structure. Therefore, they are also known as "Shift-Invariant Artificial Neural Networks (SIANNs)".

[0056] ImageNet is a large visualization database used for research on visual object recognition software. Over 14 million image URLs are manually annotated with ImageNet to indicate objects in the images; bounding boxes are also provided for at least one million images.

[0057] Deep Learning (DL) is a new research direction in the field of Machine Learning (ML). It was introduced into machine learning to bring it closer to its original goal—Artificial Intelligence (AI). Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, and to recognize data such as text, images, and sound.

[0058] Intelligent transportation fully utilizes next-generation information technologies such as the Internet of Things, spatial sensing, cloud computing, and mobile internet across the entire transportation sector. It comprehensively applies theories and tools from transportation science, systems methods, artificial intelligence, and knowledge mining. With the goals of comprehensive perception, deep integration, proactive service, and scientific decision-making, it builds a real-time dynamic information service system, deeply mines transportation-related data, forms problem analysis models, and enhances the industry's ability to optimize resource allocation, improve public decision-making capabilities, enhance industry management capabilities, and improve public service capabilities. This promotes safer, more efficient, more convenient, more economical, more environmentally friendly, and more comfortable operation and development of transportation, and drives the transformation and upgrading of transportation-related industries.

[0059] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0060] Many factors influence video content clarity, including blurriness, mosaic effects, noise levels, brightness, and encoding errors. These factors are typically caused by compression, transmission, reshooting, and editing of the original video or image. Humans can accurately judge the clarity of a video with their eyes, but it is extremely difficult for computers to automatically identify video clarity. This is because: many factors affect clarity, each difficult to quantify mathematically; computers struggle to distinguish between true and false blur (e.g., a clear foreground but a blurred background in a shot), and blurriness caused by encoding and compression can be quickly identified by humans, as can mosaics and noise; and computers rely on quantified clarity to identify the clarity of unknown videos, but such intelligent systems cannot yet match human intelligence. For video services, user perception of video quality is subjective, diverse, and difficult to quantify and measure. Subjective testing is the gold standard for video quality analysis. Subjective testing is considered a relatively accurate method for evaluating video quality, but it has three main drawbacks: (1) high time and money costs; (2) limited video types, testing conditions, and number of viewers; and (3) it cannot be used for real-time evaluation.

[0061] Existing image sharpness recognition methods include traditional methods and deep learning-based methods. The former primarily relies on features related to visual degradation or features of the original, undamaged image / video to estimate perceived quality. It mainly addresses blurring caused by various factors, including: loss of spatial detail and reduced edge sharpness (using edge detectors to calculate image edge distortion values ​​to estimate blur level), noise (dividing the image into smooth regions where variance above a certain threshold provides an estimate of noise), ringing (ringing is the form of ripples around high-contrast edges, which can be used to classify image regions by extracting features from edge mapping), and occlusion (occlusion manifests as discontinuities between adjacent blocks in image and video frames, related to compression techniques; the block size of the image domain is estimated by calculating the differences between adjacent blocks and the brightness around these blocks). However, traditional blur recognition algorithms handle JPEG compression patch noise and boundary blur well; they perform poorly on images with overall blur. In other words, traditional blur recognition algorithms cannot effectively describe these problems and only offer three coarse-grained options: blurry, sharp, and high-definition, which is not conducive to implementing fine-grained strategies based on specific business scenarios. Furthermore, for images with blurred backgrounds or artistic styles, this type of noise involves semantic information, which simple traditional fuzzy recognition algorithms cannot identify. Many such videos or images will be considered blurry. Another method based on deep learning involves manually combining the video content's resolution and bitrate to subjectively score the sharpness, constructing corresponding learning samples. Then, a deep learning neural network is used to input the entire video content to build a sharpness scoring model through video classification and regression methods. However, as mentioned above, because many factors affect sharpness, including numerous subjective and objective factors, it is impossible to accurately determine the sharpness of an image or video.

[0062] To improve the accuracy of image sharpness recognition, this application provides an image sharpness recognition method, apparatus, and storage medium.

[0063] Please see Figure 1 , Figure 1 This is a schematic diagram of an image sharpness recognition system provided in an embodiment of this application, such as... Figure 1 As shown, the image sharpness recognition system may include at least a server 01 and a client 02.

[0064] Specifically, in this embodiment, server 01 may include a standalone server, a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, and a memory, etc. Specifically, server 01 can be used to acquire an image to be recognized; perform sharpness parameter recognition processing on the image to be recognized based on a sharpness parameter recognition model to obtain the target sharpness parameters of the image to be recognized; and train the sharpness parameter recognition model.

[0065] Specifically, in this embodiment, the client 02 may include physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, smart wearable devices, smart speakers, in-vehicle terminals, and smart TVs. It may also include software running on the physical device, such as web pages provided to users by service providers, or applications provided by those service providers. Specifically, the client 02 can be used to view the sharpness parameters of the image to be recognized online.

[0066] The following describes an image sharpness recognition method according to this application. Figure 2 This is a flowchart illustrating an image sharpness recognition method provided in an embodiment of this application. This specification provides the operational steps of the method as described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or accompanying drawings. Specifically, as... Figure 2 As shown, the method may include:

[0067] In step S201, the image to be identified is acquired.

[0068] In the embodiments of this application, the image to be identified can be an image captured in real time or obtained from an image library, or it can be an image extracted from a video.

[0069] In the embodiments of this application, such as Figure 3 As shown, acquiring the image to be identified includes:

[0070] In step S20101, the video to be identified is acquired;

[0071] In this embodiment of the application, the video to be identified can be a video captured in real time or a video in an Internet database.

[0072] In step S20103, the video to be identified is divided into a first video to be identified and a second video to be identified according to its playback time; the playback time of the first video to be identified is earlier than the playback time of the second video to be identified.

[0073] In this embodiment, the first frame of the video is usually a cover image, and cover images typically have high resolution. Evaluating video resolution based solely on cover image resolution has a large margin of error. Therefore, the cover image of the video to be identified can be deleted before the video is decomposed to obtain a first video to be identified and a second video to be identified. The playback duration of the first video to be identified can be a preset duration threshold. The playback duration of the first video to be identified can be less than the playback duration of the second video to be identified. For example, the first three seconds of the video to be identified can be used as the first video to be identified, and the remaining video can be used as the second video to be identified.

[0074] In step S20105, a first number of video frames are extracted from the first video to be identified as the first image to be identified;

[0075] In the embodiments of this application, such as Figure 4 As shown, the step of extracting a first number of video frames from the first video to be identified as the first image to be identified includes:

[0076] In step S201051, the first video to be identified is parsed to obtain a first video frame set;

[0077] In step S201053, a first video frame with a playback time of a first preset time is extracted from the first video frame set and used as the first image to be identified; the first preset time includes the first number of time points.

[0078] In this embodiment of the application, the first preset time can be 0-3 seconds; the first quantity can be set according to the actual situation, for example, it can be set to 3. Specifically, the first video frame with a playback time of 1 second, 2nd second and 3rd second can be used as the first image to be identified, thereby obtaining multiple first images to be identified.

[0079] In the embodiments of this application, such as Figure 5 As shown, the step of extracting a first number of video frames from the first video to be identified as the first image to be identified includes:

[0080] In step S2010501, the first playback duration of the first video to be identified is determined;

[0081] In step S2010503, the first video to be identified is divided into the first number of first sub-videos according to the first playback duration;

[0082] In this embodiment of the application, the first video to be identified can be divided into multiple first sub-videos on an equal basis, and then the first image to be identified can be determined based on each first sub-video. The first image to be identified determined in this way can better represent the clarity of the video, thereby improving the accuracy of the clarity parameters of the video to be processed.

[0083] In step S2010505, a first sub-video frame is extracted from each first sub-video to obtain the first number of first sub-video frames; the first number of first sub-video frames are video frames with equal time intervals.

[0084] In this embodiment of the application, a first sub-video frame can be extracted from each first sub-video, and the time interval between two adjacent first sub-videos can be controlled to a fixed value, thereby achieving uniform extraction of video frames.

[0085] In step S2010507, the first number of first sub-video frames are used as the first image to be identified.

[0086] In the embodiments of this application, the first sub-video frames extracted from each first sub-video can more accurately characterize the video to be identified, thereby improving the accuracy of the clarity parameters of the video to be identified.

[0087] In step S20107, a second number of video frames are extracted from the second video to be identified, which are used as the second image to be identified;

[0088] In the embodiments of this application, such as Figure 6 As shown, the step of extracting a second number of video frames from the second video to be identified as the second image to be identified includes:

[0089] In step S201071, the second playback duration of the second video to be identified is determined;

[0090] In step S201073, the second video to be identified is divided into the second number of second sub-videos according to the second playback duration;

[0091] In this embodiment of the application, the second video to be identified can be divided into the second number of second sub-videos according to the second playback duration, and then video frames can be extracted from each second sub-video to obtain the second image to be identified, thereby improving the accuracy of the second image to be identified.

[0092] In step S201075, one second sub-video frame is extracted from each second sub-video to obtain the second number of second sub-video frames;

[0093] In this embodiment of the application, the second quantity can be set according to actual needs. For example, the second quantity can be set to 10, thereby extracting ten second sub-video frames.

[0094] In step S201077, the second number of second sub-video frames are used as the second image to be identified.

[0095] In this embodiment of the application, the second video to be identified is divided into the second number of second sub-videos according to the second playback duration, and the frames of the second sub-videos are used as the second images to be identified; thereby improving the accuracy of the second images to be identified and improving the accuracy of the clarity parameters of the video to be processed.

[0096] In step S20109, the first image to be identified and the second image to be identified are determined as the image to be identified.

[0097] In this embodiment of the application, a first image to be identified and a second image to be identified can be determined as the image to be identified, and the clarity parameters of the video to be identified can be determined based on the image to be identified.

[0098] In one specific embodiment, the first image to be identified can be three, the second image to be identified can be ten, and the resulting image to be identified can be thirteen. The clarity parameters of the video to be identified can be evaluated based on these thirteen images to be identified.

[0099] In this embodiment, video frame images can be extracted from the video as images to be identified, thereby further determining the clarity of the video to be identified. Strategies can be flexibly configured according to different categories of video content; for example, social media videos generally have lower clarity, while multi-channel networks (MCNs) are used specifically for producing videos with higher clarity.

[0100] In step S203, the image to be identified is processed for sharpness parameter recognition based on the sharpness parameter recognition model to obtain the target sharpness parameter of the image to be identified.

[0101] In this embodiment of the application, the step of performing sharpness parameter recognition processing on the image to be recognized based on the sharpness parameter recognition model to obtain the target sharpness parameter of the image to be recognized includes:

[0102] Based on the sharpness parameter recognition model, sharpness parameter recognition processing is performed on the first image to be recognized and the second image to be recognized to obtain the target sharpness parameters corresponding to the first image to be recognized and the second image to be recognized respectively.

[0103] In this embodiment of the application, the method may further include:

[0104] The clarity parameters of the video to be identified are determined based on the target sharpness parameters corresponding to the first image to be identified and the second image to be identified, respectively.

[0105] In this embodiment of the application, the target sharpness parameter corresponding to each image to be identified in the video to be identified can be determined by the sharpness parameter recognition model, thereby further determining the sharpness parameter of the video to be identified; this avoids directly determining the video sharpness based on the video to be identified, and improves the efficiency and accuracy of determining the sharpness parameter of the video to be identified.

[0106] In the embodiments of this application, such as Figure 7 As shown, determining the sharpness parameters of the video to be identified based on the target sharpness parameters corresponding to the first image to be identified and the second image to be identified includes:

[0107] In step S701, the sum of the first quantity and the second quantity is calculated to obtain the preset quantity;

[0108] In step S703, the product of the preset number of target sharpness parameters corresponding to the first image to be identified and the second image to be identified is calculated to obtain the sharpness results corresponding to the first image to be identified and the second image to be identified.

[0109] In step S705, the sum of the sharpness results corresponding to the first image to be identified and the second image to be identified is calculated to obtain the initial sharpness parameters of the video to be identified;

[0110] In step S707, the preset number root of the initial clarity parameter is calculated to obtain the clarity parameter of the video to be identified.

[0111] In this embodiment, considering the actual user's video consumption habits, the image clarity of the first few frames is particularly important. Also, the first frame of a video is usually a cover image, which has its own specific clarity score, so the first frame is skipped. Then, when extracting the first three frames of the video content, frames including the 1st, 2nd, and 3rd seconds must be selected. Then, in the remaining video content, the remaining duration is evenly divided into 10 parts, and 10 images are extracted as sampled video frames. The formula for calculating the clarity parameter of the video to be identified is as follows:

[0112]

[0113] Wherein, Score is the resolution parameter of the video to be identified, n is the number of video frames extracted from the video to be identified, n can be set according to the actual situation, xi is the resolution parameter of the i-th extracted video frame, and k is a control parameter. The larger k is, the closer the resolution parameter of the video to be identified is to the resolution parameter of the high-scoring video frame. In a specific embodiment, n can be 13 and k can be 3.

[0114] In this embodiment of the application, the video's sharpness parameters can be determined by the sharpness parameters of the video frame images extracted from the video, thereby improving the accuracy of video sharpness recognition.

[0115] In this embodiment of the application, the training method for the sharpness parameter recognition model includes:

[0116] In step S301, a first sample image labeled with a sharpness parameter is obtained;

[0117] In this embodiment of the application, to ensure the final effect, all example images below may be significantly reduced in size, which may lead to doubts about the scoring. They should be viewed using the same width principle as a mobile phone. Unless otherwise specified, the annotation follows the principle that "clarity scoring tends to be based on the visual center area of ​​the first eye, first perceive the overall picture, and then pay attention to the details."

[0118] In this embodiment of the application, the method for determining the sharpness parameter of the first sample image includes:

[0119] If the first sample image includes a border image, extract the non-border image from the first sample image;

[0120] Based on the non-bordered image, the sharpness parameter of the first sample image is determined.

[0121] In the embodiments of this application, there are a large number of bordered images in the information flow service. For bordered images, the borders are ignored, and only the clarity of the main subject in the middle part of the image is considered to give people a visual impression; for multiple nested borders, only the middle part of the image is considered.

[0122] In this embodiment of the application, the method for determining the sharpness parameter of the first sample image includes:

[0123] Obtain the first sharpness parameter of the edge image and the second sharpness parameter of the non-edge image in the first sample image;

[0124] Based on the first sharpness parameter and the first sharpness parameter, determine the sharpness comparison result between the edge image and the non-edge image;

[0125] If the sharpness of the edge image is higher than that of the non-edge image, the second sharpness parameter is determined as the sharpness parameter of the first sample image.

[0126] In this embodiment of the application, the sharpness parameter is marked according to the principle of prioritizing the middle; if there is perspective in the picture, or a large difference between near and far, the edge part of the picture has higher sharpness, such as 5 points or above, but most of the middle part of the picture has solid color or blurry problems, and the sharpness is less important, the sharpness score of the subject area in the middle of the picture is used as the standard.

[0127] In this embodiment of the application, the method for determining the sharpness parameter of the first sample image includes:

[0128] Obtain the dimensions of the image display device;

[0129] Based on the size of the image display device, the size of the first sample image is adjusted to obtain the adjusted image;

[0130] Based on the adjusted image, the sharpness parameter of the first sample image is determined.

[0131] In this embodiment, the annotation of the sharpness parameter follows the principle of equal width for mobile phones. When annotating, the original image must be viewed. However, original images vary in size, some being very large and others relatively small. The perception of image detail blur also differs at different scaling sizes. Generally, when scoring images around the 4-5 point passing mark, regardless of whether the image is portrait or landscape, the perceived sharpness when the image is scaled to the same width as the mobile phone is considered in portrait mode. This means that some original images with high length and width resolution may appear blurry at their original size, but when scaled down to the same width as the mobile phone, the blurriness is not easily noticeable, and the overall image appears relatively clear. Conversely, some original images with lower length and width resolution may appear very clear and detailed at their original size, but when scaled up to the same width as the mobile phone, the details become insufficient, resulting in a blurry appearance. In such cases, the sharpness score should be lowered accordingly.

[0132] In this embodiment of the application, the method for determining the sharpness parameter of the first sample image includes:

[0133] If the first sample image includes a target object, the sharpness parameter of the first sample image is determined based on the sharpness of the target object.

[0134] In this embodiment of the application, the target object can be a person or animal in the image. The annotation of the sharpness parameter follows the principle of prioritizing the foreground subject. When the image contains a clear subject such as a person or animal and is in the foreground, the sharpness of the subject should have a greater impact on the score than the sharpness of the background.

[0135] In this embodiment of the application, the method for determining the sharpness parameter of the first sample image includes:

[0136] If there are at least two target objects, determine the category of each target object;

[0137] The weight of each target object is determined based on its corresponding category.

[0138] In this embodiment of the application, when there are multiple entities in the image, the first priority subject occupies a smaller proportion of the image than the second priority subject, and the two have different levels of clarity. When scoring the whole image, it is necessary to take this into consideration and weigh the difference in clarity against the size of the proportion of the image. Generally speaking, more attention is paid to the clarity of the first priority subject, and a higher weight is given to it.

[0139] In this embodiment of the application, determining the weight of each target object based on the category corresponding to each target object includes:

[0140] The priority of each target object is determined according to a preset mapping relationship; the preset mapping relationship represents the correspondence between category and priority.

[0141] Sort the at least two target objects according to their respective priorities;

[0142] Based on the sorting results, the target objects for filtering are determined; the target object for filtering is the object with the highest priority among the at least two target objects.

[0143] Obtain the status information of the target object being filtered;

[0144] The weight of the target object is determined based on its category and status information.

[0145] In this embodiment, the weight of the sharpness parameter of each target object can be determined according to the category of each target object in the first sample image. For example, the weight of people can be set to be the highest, followed by animals, and the weight of other categories of objects can be set to be lower. If the target object is a person or an animal, the weight of the sharpness parameter also needs to be determined in combination with the corresponding state information. For example, the first priority subject in the first sample image is a monkey, and the secondary subject is a tire. Because the sharpness of the monkey is lower than that of the tire due to motion blur, the sharpness parameter score of the tire is 3-4 points, and the sharpness parameter score of the monkey is 2 or 3, but closer to 2. At this time, the sharpness parameter of the first sample image can be determined to be 3.

[0146] The sharpness parameters of the first sample image are determined based on the sharpness parameters and weights corresponding to each target object.

[0147] In this embodiment, the sharpness parameter corresponding to the first sample image can be a numerical value or level of the target quantity, and the target quantity can be set according to the actual situation; the first sample image can be divided into multiple sharpness levels according to the target quantity.

[0148] Specifically, in this embodiment, the target quantity can be ten, and the sharpness parameter can include ten parameters, such as 1-10 points. The larger the parameter value, the higher the sharpness. The ten parameters can correspond to three sharpness levels: the first level is unqualified images, corresponding to 1-4 points; the second level is qualified images, corresponding to 5-7 points; and the third level is high-quality images, corresponding to 8-10 points. The method for determining the sharpness parameter of the first sample image includes:

[0149] If there is no identifiable object in the first sample image, the sharpness parameter of the first sample image is determined as the first parameter.

[0150] In this embodiment, the first parameter can be 1 point; for images whose content is indistinguishable, whose images are too simple and meaningless; or whose main content is completely unclear, including images that are blurred, whose main outline can only be barely seen or whose outline cannot be seen at all, whose background is completely solid color or nearly solid color (with only one line of title), or images containing unclear text, images without foreground or whose foreground is a small amount of solid text, simple graphics, or whose screen occupancy is too small, the clarity parameter is set as the first parameter.

[0151] In this embodiment of the application, the method for determining the sharpness parameter of the first sample image includes:

[0152] If there is an identifiable object in the first sample image, but the local information of the object cannot be identified, the sharpness parameter of the first sample image is determined to be the second parameter.

[0153] In this embodiment, the second parameter can be 2 points. For content that is generally blurry, the main body is vaguely discernible, and the details cannot be identified, such as severe motion blur, Gaussian blur, or mosaic jagged blur; or for content where the main body can be identified, such as people, but the facial features and other details are severely indistinguishable due to blurring or motion, resulting in severe Gaussian blur, severe mosaic blur leading to obvious jagged edges, or a blurry mess, including unclear but all-encompassing text images, the clarity parameter is set as the second parameter.

[0154] In this application embodiment, for images where the main subject is identifiable but has a large amount of noise or moderate Gaussian / mosaic jagged blur; or where the main subject is identifiable but has a large amount of noise or jagged edges when enlarged to the same width as a mobile phone due to low resolution; or where overexposure, oversharpening, over-softening, abnormal lighting, etc., will also reduce the overall score, generally resulting in a decrease of 1 point in visual perception; or where the main subject is visible and there is no obvious blurring; or where the content is well identifiable and the text proportion is appropriate, the clarity parameter is set as the third parameter, for example, it can be 3 points.

[0155] In this embodiment, for images where the overall perception is unclear, the subject has obvious noise, or the overall feeling is unclear, but the subject is relatively clear with no visible blurring at the edges of subjects such as people, and there is no significant mosaic or jaggedness, but upon closer inspection, the sharpness and clarity of details are still relatively limited (especially for subjects such as people), and blurring or mosaic effects can still be seen relatively easily, with details such as facial features, hair, and clothing not being clear enough. Or, for images where close-up subjects have distinguishable facial features, and distant subjects have relatively clear shapes that can be clearly distinguished from the background, the sharpness parameter is set as the fourth parameter, for example, it can be 4 points.

[0156] In this embodiment, for images that are perceived as clear with a sharp subject, but still have jagged edges or noise in the details; or images that are generally clear but still have some jagged edges or noise (including aged effects) upon closer inspection, the clarity parameter is set as the fifth parameter, for example, a score of 5. Especially when there are subjects such as people or animals, the overall score needs to consider both the clarity of the background and the subject. When the overall image is clear, if the subject is severely obscured, the score should be reduced by 1 point.

[0157] In this embodiment of the application, for images where the main subject is clear, perceived as clear and without jagged edges or noise, but details are still defective; or images are generally clear with no obvious noise or jagged edges, but upon closer inspection there are still problems such as over-sharpening, over-softening, over-exposure and blurring, or the image content is too simple and lacks information; or images are clear overall, but there is significant and severe motion blur in some areas that makes it impossible to distinguish local details (e.g., motion blur makes it impossible to distinguish hands, etc.), the clarity parameter is set as the sixth parameter, for example, it can be 6 points.

[0158] In this application embodiment, for images with good clarity and generally no low-level issues such as noise, mosaic blur, etc., but still with some flaws; or images with almost no noise, mosaic, motion blur, etc., with clear outlines of people and objects, smooth edges, and no obvious noise, but with obvious flaws in detail resolution and large areas of blur due to issues such as lighting, color, tone, inaccurate focus, and halo, the clarity parameter is set as the seventh parameter, for example, 7 points. For artwork such as animation and comics, the resolution size is insufficient to withstand magnification to view details, and some noise can be seen at the edges of lines, and the line clarity and sharpness are insufficient. For full-screen screenshots of games (unless it is a close-up of a game character or other special cases), the highest score is 7 points if there are no clarity issues; for cases involving the composition of people and images, since the images are often ultra-high definition, the score needs to be adjusted according to the clarity of the people. If the people are clear and occupy a small area, a score of 7 is given; if they are not clear, the score is reduced accordingly. If the overall image is clear, but the main subject is obscured, the score should also be reduced appropriately.

[0159] In this application embodiment, for images that do not have any jagged edges, noise, excessive sharpening, excessive soft lighting, overexposure, or local blurring, and whose overall visual quality is excellent but still has a certain degree of distinguishability; or images whose detail resolution is discernible but not perfect, and for which there is verifiable evidence, the sharpness parameter is set to the eighth parameter, for example, 8 points. Other parameters depend on individual overall visual judgment and will take into account factors such as image layout, color, saturation, and softness that affect aesthetics.

[0160] In this embodiment of the application, for images that lack a well-designed composition, resulting in a detail richness that does not reach the 10-point level; or where the main body is slightly blurred due to motion blur, resulting in a slight lack of detail resolution; or where only different parts of the image have different clarity of the main body, with some parts possibly having a 10-point score but not enough of the image's proportion, and the remaining parts having low clarity, the clarity parameter is set to the ninth parameter, for example, it can be 9 points.

[0161] In this embodiment of the application, for images that are satisfactory in terms of overall picture detail and visual appeal, have a well-designed composition, are detailed, and have a strong expressive power of the subject, the clarity parameter is set to the tenth parameter, for example, it can be 10 points.

[0162] In the embodiments of this application, the sharpness parameter label of each first sample image can be accurately labeled using the above method; based on the labeled sample image, the accuracy of the sharpness parameter recognition model can be improved.

[0163] In step S303, based on the first sample image, the image feature extraction model is trained to identify sharpness parameters, so as to adjust the model parameters of the image feature extraction model until the sharpness parameter label output by the image feature extraction model matches the labeled sharpness parameter label; the image feature extraction model is obtained by training a preset model to extract images based on a second sample image pair, the second sample image pair includes positive sample image pairs and negative sample image pairs, the similarity between the two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between the two negative sample images in the negative sample image pair is less than the first threshold;

[0164] In the embodiments of this application, such as Figure 8 As shown, the training method for the image feature extraction model includes:

[0165] In step S801, two preset sample images with a similarity greater than or equal to the first threshold are taken as the positive sample image pair, and two preset sample images with a similarity less than the first threshold are taken as the negative sample image pair.

[0166] Specifically, in the embodiments of this application, the positive sample image pair may include two second sample images with high similarity. The first threshold can be set according to the actual situation and can be determined according to the required number of positive sample image pairs.

[0167] In step S803, the positive sample image pair and the negative sample image pair are determined as the second sample image pair;

[0168] In this embodiment of the application, the method further includes:

[0169] Obtain a preset sample image set; the resolution of the preset sample images in the preset sample image set is greater than a preset resolution threshold;

[0170] In this embodiment of the application, a preliminary screening image set can be obtained from a video library or an image library. The preliminary screening image set may include images of different resolutions. In order to improve the accuracy of positive and negative sample pairs in the second sample image pair, images with a resolution less than or equal to a preset resolution threshold can be deleted from the preliminary screening image set. The preset resolution threshold can be set according to the actual situation, for example, it can be set to 200*200.

[0171] In this embodiment of the application, obtaining the preset sample image set includes:

[0172] Retrieve video and image sets from the information stream;

[0173] Specifically, in this application embodiment, an image set labeled with text can be obtained through ImageNet or a public dataset (OpenImage).

[0174] Determine the first text tag for each video in the video set to obtain the first text tag set;

[0175] Specifically, in this embodiment, a first text tag corresponding to each video in the video set can be obtained through machine learning algorithms or manual annotation, resulting in a first text tag set. The first text tag can include entity nouns and motion words. Entity nouns can be nouns in the video frame, such as animal names, car names, building names, etc., describing a specific entity or object. Motion words mainly refer to descriptive phrases consisting of a verb and a noun, such as riding a bicycle, playing the piano, kicking a football, etc., simple descriptions of video frames. A second text tag is determined for each image in the image set, resulting in a second text tag set.

[0176] Specifically, in this embodiment of the application, the second text tag corresponding to each video in the image set can be obtained by machine learning algorithm or manual annotation, thus obtaining the second text tag set; the second text tag can include entity nouns and motion words. Entity nouns can be nouns in the image, such as animal names, car names, building names, etc., which describe a specific entity or object; motion words mainly refer to descriptive phrases consisting of verbs and nouns, such as riding a bicycle, playing the piano, playing football, etc., which are simple descriptive phrases of the image.

[0177] The target text tag is determined based on the first text tag set and the second text tag set;

[0178] Specifically, in this embodiment of the application, the tags in both the first text tag set and the second text tag set can be determined as target text tags; thereby, a preset sample image set can be selected based on the target text tags.

[0179] In this embodiment of the application, determining the target text tag based on the first text tag set and the second text tag set includes:

[0180] Based on the first text tag set and the second text tag set, a comprehensive text tag set is determined;

[0181] Specifically, in this embodiment of the application, the text tags in both the first text tag set and the second text tag set can be determined as comprehensive text tags, thereby determining the comprehensive text tag set.

[0182] Determine the frequency of occurrence of each comprehensive text tag in the information stream from the comprehensive text tag set;

[0183] Specifically, in the embodiments of this application, the probability of each comprehensive text tag appearing in the information stream can be determined, and target text tags with higher appearance probabilities can be selected based on the appearance probabilities.

[0184] The target text tag is determined based on the occurrence probability of each comprehensive text tag.

[0185] Specifically, in this embodiment, the comprehensive text tags can be sorted according to their occurrence probability, and a preset number of tags with higher occurrence probabilities can be determined as target text tags based on the sorting results; or tags with occurrence probabilities greater than a preset probability threshold can be determined as target text tags. The preset number and preset probability threshold can be set according to actual conditions; for example, the preset number can be determined based on the total number of tags in the comprehensive text tag set, and can be 10%, 20%, etc. of the total number; in a specific embodiment, for example, if the total number of tags in the comprehensive text tag set is 5.2 million, the preset number can be set to 250,000, that is, the top 250,000 tags with the highest occurrence frequency are extracted from the 5.2 million tags as target text tags.

[0186] In this embodiment of the application, determining the target text tag based on the first text tag set and the second text tag set includes:

[0187] Based on the first text tag set, determine the first target text tag set;

[0188] In this embodiment, a first target text tag set can be determined based on the first occurrence probability of each tag in the first text tag set; the first text tags can be sorted according to the occurrence probability corresponding to each first text tag, and a first preset number of tags with higher first occurrence probabilities can be determined as the first target text tags based on the sorting results; or tags with first occurrence probabilities greater than a first preset probability threshold can be determined as the first target text tags. The first preset number and the first preset probability threshold can be set according to actual conditions. For all video tags in the information flow recommendation library, the tags corresponding to all videos within 200 days can be statistically analyzed, and the occurrence frequency of the tags can be calculated after deduplication to obtain the first target text tags.

[0189] In this embodiment of the application, matching videos can be filtered from the information stream based on the first target text tag, thereby further determining the preset sample image.

[0190] Based on the second text tag set, determine the second target text tag set;

[0191] In this embodiment, a second target text tag set can be determined based on the second occurrence probability of each tag in the second text tag set; the second text tags can be sorted according to the occurrence probability corresponding to each second text tag, and a second preset number of tags with higher second occurrence probabilities can be determined as the second target text tags based on the sorting results; or tags with second occurrence probabilities greater than a second preset probability threshold can be determined as the second target text tags. The second preset number and the second preset probability threshold can be set according to actual conditions. For images within the information flow content library, the text tags corresponding to all images in the past year can be counted, duplicates removed, and their occurrence frequencies calculated and then sorted to determine the second target text tags.

[0192] In this embodiment of the application, matching images can be filtered from the information stream based on the second target text tag as preset sample images.

[0193] The tags in the first target text tag set and the second target text tag set are determined as target text tags.

[0194] Specifically, in this embodiment, text tags with high occurrence probability can be extracted from the video set and the image set respectively as target text tags; target text tags with high occurrence probability correspond to a larger number of information streams, thereby increasing the number of preset sample images selected.

[0195] Using the target text tag as the search term, the preset sample image set is obtained.

[0196] Specifically, in this embodiment of the application, an image or video search can be performed in a search engine based on the target text tag to determine a preset sample image set.

[0197] Determine the image category corresponding to each preset sample image in the preset sample image set;

[0198] Specifically, in the embodiments of this application, the image category can represent information such as the object category and object state category (such as riding a bicycle) in the image.

[0199] Based on the image category corresponding to each preset sample image, determine the image category similarity between any two preset sample images in the preset sample image set.

[0200] Specifically, in this embodiment of the application, the similarity between two preset sample images can be determined according to the image category, thereby further determining positive and negative sample image pairs.

[0201] Based on the image category similarity between any two preset sample images in the preset sample image set, the positive sample image pair and the negative sample image pair are determined.

[0202] In this embodiment of the application, determining the positive sample image pair and the negative sample image pair based on the image category similarity of any two preset sample images in the preset sample image set includes:

[0203] Two preset sample images with an image category similarity greater than or equal to a third threshold are identified to obtain two first preset sample images; the similarity between the two first preset sample images is greater than or equal to the first threshold.

[0204] The two first preset sample images are determined as the positive sample image pair;

[0205] Two preset sample images with a category similarity less than the third threshold are identified to obtain two second preset sample images; the similarity between the two second preset sample images is less than the first threshold.

[0206] The two second preset sample images are determined as the negative sample image pair.

[0207] Specifically, in this embodiment, the third threshold and the first threshold can be the same or different. When they are different, the third threshold is greater than the first threshold. Positive sample image pairs can be determined based on the image category similarity of two preset sample images. Two preset sample images with low image category similarity can be determined as negative sample image pairs.

[0208] In this embodiment of the application, positive sample image pairs can be determined by parsing the video and extracting video frames from the video.

[0209] Specifically, in this embodiment of the application, the method for constructing the positive sample image pair includes:

[0210] The first video is parsed to obtain a first video frame set; the first video in the first video frame set is the preset sample image.

[0211] In this embodiment of the application, the first video can be parsed to obtain a first video frame set, and the playback time of each first video frame can be determined.

[0212] Two first video frames are identified whose playback time difference is less than or equal to a preset time threshold, thus obtaining two first target video frames; the similarity between the two first target video frames is greater than or equal to the first threshold.

[0213] The two first target video frames are identified as the positive sample image pair.

[0214] In this embodiment, the preset time threshold can be set according to the actual situation. Two first video frames with a playback time difference less than or equal to the preset time threshold may include adjacent video frames and close video frames.

[0215] Specifically, in the embodiments of this application, the frame-by-frame images of the video content distributed in the information stream, for example, for a video, can be considered to be similar adjacent frame-by-frame images; for the information stream content library, using the enabled videos in the information content library, adjacent / near frames are extracted from each video and regarded as similar images, which can be used as positive sample image pairs.

[0216] In this embodiment of the application, the method for constructing the positive sample image pair includes:

[0217] Obtain the original image;

[0218] In this embodiment, the original image can be an image with a resolution greater than a preset resolution threshold, which can be set according to actual conditions. In videos, cover images typically have high clarity, and the original image can include the video's cover image.

[0219] The original image is subjected to image enhancement processing to obtain at least one processed image;

[0220] Specifically, in the embodiments of this application, image enhancement processing may include image cropping and resizing, color distortion, color jitter, Gaussian blur, etc., wherein color jitter processing may include processing of brightness, contrast, saturation, and hue.

[0221] The positive sample image pair is constructed based on the original image and the at least one processed image.

[0222] Specifically, in this embodiment of the application, any two images from the original image and the at least one processed image can be combined to obtain two combined images, wherein the similarity between the two combined images is greater than or equal to a first threshold; and the positive sample image pair is constructed based on the two combined images.

[0223] Specifically, in the embodiments of this application, negative sample image pairs can be constructed by extracting frames from different videos.

[0224] In this embodiment of the application, the method for constructing the negative sample image pair includes:

[0225] Obtain at least two second videos; the correlation parameter between any two of the at least two second videos is less than a preset parameter threshold.

[0226] In this embodiment of the application, the correlation parameter between any second video and the first video is less than the preset parameter threshold; the correlation parameter between the two videos can be determined according to the categories corresponding to the two second videos. For example, the category information of the video can be determined from the text title of the preset video or the text in the video frame, and the correlation parameter between the two videos can be determined according to the similarity of the category information, thereby determining at least two second videos.

[0227] Frames are extracted from each of the at least two second videos to obtain a second video frame corresponding to each second video.

[0228] In this embodiment of the application, frames can be extracted once or multiple times from the at least two second videos to obtain the second video frame corresponding to each second video.

[0229] The negative sample image pair is constructed based on the second video frame corresponding to each second video.

[0230] In this embodiment of the application, two second video frames from different second videos can be determined as the negative sample image pair.

[0231] The positive sample image pair and the negative sample image pair are determined as the second sample image pair.

[0232] In this embodiment of the application, the second sample image pair may include the positive sample image pair and the negative sample image pair, thereby improving the accuracy of the image feature extraction model.

[0233] In step S805, based on the second sample image pair, image feature extraction training is performed on the preset model to adjust the model parameters of the preset model until the similarity loss value of the preset model is less than a second threshold; the similarity loss value is determined based on the first similarity and the second similarity corresponding to the second sample image pair; the first similarity is the similarity between two second sample images in the second sample image pair; the second similarity is the similarity between two features of two second sample images in the second sample image feature pair; the second sample image feature pair includes image features extracted from the two second sample images of the second sample image pair based on the preset model;

[0234] In this embodiment, an image feature extraction model can be trained using self-supervised learning (SSL). The pre-trained model can be a self-supervised pre-trained model, which may include a Swin Transformer network and a contrastive learning network. The Swin Transformer network comprises four stages, each reducing the resolution of the input feature map, expanding the receptive field layer by layer, similar to a convolutional neural network (CNN). At the beginning of the input, a patch embedding is performed, cutting the image into patches and embedding them into the embedding layer. Each stage consists of patch merging and multiple blocks; the patch merging module primarily reduces the image resolution at the beginning of each stage. Using the Swin Transformer network as the backbone network not only generates hierarchical feature representations but also reduces the time complexity to a linear relationship with the image size, resulting in less training time. Its core is that the introduction of windows reduces complexity. Patch fusion and the introduction of shifted windows can extract multi-scale features. A shifted window is a sampling sliding window that can extract image texture, local borders to the overall outline, and the relationship and relative position of different borders.

[0235] In some embodiments, the trained image feature extraction model can also be distilled to reduce the model size, which facilitates model deployment and acceleration.

[0236] In a specific embodiment, such as Figure 9 As shown, Figure 9 This is a schematic diagram of the structure of a self-supervised model. The self-supervised model may include a backbone network. It takes crawled Internet data, business link domain images, video frame images, and enhanced images (enhanced images of cover images in video frames) as first sample images and inputs them into the backbone network. It performs self-supervised pretraining through contrastive learning to obtain an image feature extraction model. This model can extract hidden layer features of images through the SSL backbone network.

[0237] In this embodiment of the application, contrastive learning is employed during the self-supervised pre-training process. A contrastive loss function is used to measure the similarity loss value between the two images in the second sample image pair. Figure 10 As shown, Figure 10 The flowchart for constructing the contrastive loss function shows that the second sample image x is processed by an encoder. query Encode the image to obtain q; then use a momentum encoder to process the second sample image. ...encode to obtain k0, k1, k2, ..., then determine x by calculating the similarity between q and k0, k1, k2, ... query and ...the corresponding contrastive loss function.

[0238] In this embodiment, the formula for contrastive loss is as follows: Where d = ||a n -b n || represents the Euclidean distance between the features of two sample images, y is the matching label of the two sample images, y=1 means the two sample images are similar or matched, y=0 means the two sample images do not match, and margin is the set threshold. This loss function is mainly used in feature vector transformation, that is, samples that were originally similar will remain similar in the feature space after feature extraction and transformation; while samples that were originally dissimilar will remain dissimilar in the feature space after feature transformation.

[0239] In step S807, the preset model corresponding to the model parameters when the similarity loss value is less than the preset value is used as the image feature extraction model.

[0240] In this embodiment of the application, the preset value can be set and adjusted according to the actual situation, and the image feature extraction model can be used to extract the features of objects in the image.

[0241] In step S305, the image feature extraction model corresponding to the model parameters when the output sharpness parameter label matches the labeled sharpness parameter label is used as the sharpness parameter recognition model.

[0242] In this embodiment, an image feature extraction model can be used as a basis, and a supervised classifier, such as a regression (Logistic, LR) classifier, can be used to fine-tune the model; and a Softmax activation function can be used to score the sharpness of multiple classifications.

[0243] In a specific embodiment, such as Figure 11 As shown, Figure 11 This is a schematic diagram of a sharpness parameter recognition model. The model may include a self-supervised pre-trained model (i.e., an image feature extraction model), an LR classifier, and a Softmax activation function for classification. Different image categories correspond to different sharpness parameters. The self-supervised pre-trained model includes contrastive learning Transformers and a Swin Transformer network. Model training includes a pre-training phase and a fine-tuning phase. In the pre-training phase, the self-supervised pre-trained model is obtained by pre-training with input sample images. In the fine-tuning phase, the LR classifier and the Softmax activation function are combined to obtain the sharpness parameter level for each image, with one level corresponding to one sharpness parameter. Image 1 is level 1, image 2 is level 5, and image 3 is level 10.

[0244] In this embodiment, supervised training is performed on the image feature extraction model to fine-tune the model structure, resulting in a sharpness parameter recognition model. This allows for modeling by combining low-level sharpness information with high-level semantic information through a neural network. The low-level sharpness information can characterize the image's texture and edge information, while the high-level semantic information (image features) can include descriptive information about objects in the image.

[0245] This embodiment constructs a basic image sharpness evaluation model based on fine-grained image content scoring. For video content, it uses fine-grained content sharpness evaluation of local images as the foundation, and then comprehensively evaluates the video sharpness by fusing features from multiple video frames. Based on large-scale image data in the information stream, a basic model is constructed through transfer learning using a self-supervised learning pre-trained network. First, image sharpness is scored according to a 10-level fine-grained standard sharpness, and corresponding supervised data is constructed. Then, this small sample set is combined with the basic pre-trained model for fine-tuning to obtain the image sharpness scoring model. Finally, frames are extracted from the video content, and the results of the image frame scoring are aggregated onto the original video content, thereby providing a fine-grained characterization of the video content's sharpness. This invention avoids image blurring and blurring types involving semantic information that traditional methods cannot cover, enabling a deep understanding of semantic information in images. It combines low-level sharpness information with high-level semantic information through neural networks, providing finer-grained differentiation of video sharpness. This allows for more granular strategies to be determined for different video content sharpness application scenarios: for example, reducing the recommendation weight of lower-resolution videos, prioritizing the review of high-resolution videos during content deduplication, and retaining high-resolution versions to reduce unnecessary review costs and improve user experience. Furthermore, it increases the recommendation weight of high-resolution video content and improves the cold start speed of corresponding content, thereby enhancing the efficiency of recommendation distribution.

[0246] As can be seen from the technical solutions provided by the embodiments of this application above, the embodiments of this application acquire an image to be identified; perform sharpness parameter identification processing on the image to be identified based on a sharpness parameter identification model to obtain the target sharpness parameter of the image to be identified; wherein, the training method of the sharpness parameter identification model includes: acquiring a first sample image labeled with a sharpness parameter label; performing sharpness parameter identification training on an image feature extraction model based on the first sample image to adjust the model parameters of the image feature extraction model until the sharpness parameter label output by the image feature extraction model matches the labeled sharpness parameter label; the image feature extraction model is obtained by performing image feature extraction training on a preset model based on a second sample image pair, the second sample image pair includes a positive sample image pair and a negative sample image pair, the similarity between the two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between the two negative sample images in the negative sample image pair is less than the first threshold; the image feature extraction model corresponding to the model parameters when the output sharpness parameter label matches the labeled sharpness parameter label is used as the sharpness parameter identification model. In the training process of the sharpness parameter recognition model, this application first obtains an image feature extraction model through unsupervised training. Based on this model, it then conducts supervised training using sample images labeled with sharpness parameters. This enables the modeling of combining low-level sharpness information with high-level semantic information, thereby allowing the obtained sharpness parameter recognition model to accurately identify the sharpness of images and improving the accuracy of image sharpness recognition.

[0247] This application also provides an image sharpness recognition device, such as... Figure 12 As shown, the device includes:

[0248] Image acquisition module 1210 is used to acquire the image to be recognized;

[0249] The target sharpness parameter determination module 1220 is used to perform sharpness parameter recognition processing on the image to be recognized based on the sharpness parameter recognition model to obtain the target sharpness parameter of the image to be recognized.

[0250] Model training module 1230 is used to train the sharpness parameter recognition model;

[0251] The model training module 1230 includes:

[0252] The first sample image acquisition submodule 12310 is used to acquire the first sample image labeled with the sharpness parameter.

[0253] The model parameter adjustment submodule 12320 is used to train the image feature extraction model for sharpness parameter recognition based on the first sample image, so as to adjust the model parameters of the image feature extraction model until the sharpness parameter label output by the image feature extraction model matches the labeled sharpness parameter label; the image feature extraction model is obtained by training a preset model for image feature extraction based on a second sample image pair, the second sample image pair includes a positive sample image pair and a negative sample image pair, the similarity between the two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between the two negative sample images in the negative sample image pair is less than the first threshold;

[0254] The sharpness parameter recognition model determination submodule 12330 is used to extract the image feature model corresponding to the model parameters when the output sharpness parameter label is matched with the labeled sharpness parameter label, and use it as the sharpness parameter recognition model.

[0255] In some embodiments, the image acquisition module may include:

[0256] The video acquisition submodule is used to acquire the video to be recognized;

[0257] The video decomposition submodule is used to divide the video to be identified into a first video to be identified and a second video to be identified according to the playback time; the playback time of the first video to be identified is earlier than the playback time of the second video to be identified.

[0258] The first image to be identified determination submodule is used to extract a first number of video frames from the first video to be identified as the first image to be identified;

[0259] The second image to be identified determination submodule is used to extract a second number of video frames from the second video to be identified as the second image to be identified.

[0260] The image to be identified submodule is used to identify the first image to be identified and the second image to be identified as the image to be identified.

[0261] In some embodiments, the first image determination submodule may include:

[0262] The first video frame set determination unit is used to parse the first video to be identified to obtain the first video frame set;

[0263] The first image to be identified unit is used to extract a first video frame with a playback time of a first preset time from the first video frame set as the first image to be identified; the first preset time includes the first number of time points.

[0264] In some embodiments, the first image determination submodule may include:

[0265] The first playback duration determination unit is used to determine the first playback duration of the first video to be identified.

[0266] The first sub-video determining unit is used to divide the first video to be identified into the first number of first sub-videos according to the first playback duration.

[0267] The first sub-video frame determination unit is used to extract a first sub-video frame from each first sub-video to obtain the first number of first sub-video frames, wherein the first number of first sub-video frames are video frames with equal time intervals.

[0268] The first image acquisition unit is used to acquire the first number of first sub-video frames as the first image to be identified.

[0269] In some embodiments, the second image-to-be-identified submodule may include:

[0270] The second playback duration determination unit is used to determine the second playback duration of the second video to be identified;

[0271] The second sub-video determining unit is used to divide the second video to be identified into the second number of second sub-videos according to the second playback duration.

[0272] The second sub-video frame determination unit is used to extract a second sub-video frame from each second sub-video to obtain the second number of second sub-video frames.

[0273] The second image to be identified unit is used to determine the second number of second sub-video frames as the second image to be identified.

[0274] In some embodiments, the target sharpness parameter determination module may include:

[0275] The parameter recognition submodule is used to perform sharpness parameter recognition processing on the first image to be recognized and the second image to be recognized based on the sharpness parameter recognition model, so as to obtain the target sharpness parameters corresponding to the first image to be recognized and the second image to be recognized respectively.

[0276] The device further includes:

[0277] The sharpness parameter determination module is used to determine the sharpness parameter of the video to be identified based on the target sharpness parameters corresponding to the first image to be identified and the second image to be identified, respectively.

[0278] In some embodiments, the sharpness parameter determination module may include:

[0279] The preset quantity determination submodule is used to calculate the sum of the first quantity and the second quantity to obtain the preset quantity;

[0280] The sharpness result determination submodule is used to calculate the product of the preset number of target sharpness parameters corresponding to the first image to be identified and the second image to be identified, so as to obtain the sharpness result corresponding to the first image to be identified and the second image to be identified.

[0281] The initial sharpness parameter determination submodule is used to calculate the sum of the sharpness results corresponding to the first image to be identified and the second image to be identified, so as to obtain the initial sharpness parameter of the video to be identified;

[0282] The sharpness parameter determination submodule is used to calculate the preset number root of the initial sharpness parameter to obtain the sharpness parameter of the video to be identified.

[0283] In some embodiments, the apparatus may further include:

[0284] The positive and negative sample determination module is used to identify two preset sample images with a similarity greater than or equal to the first threshold as the positive sample image pair, and to identify two preset sample images with a similarity less than the first threshold as the negative sample image pair.

[0285] The second sample image pair determination module is used to determine the positive sample image pair and the negative sample image pair as the second sample image pair;

[0286] The parameter adjustment module is used to train a preset model by image feature extraction based on the second sample image pair, so as to adjust the model parameters of the preset model until the similarity loss value of the preset model is less than a second threshold; the similarity loss value is determined based on the first similarity and the second similarity corresponding to the second sample image pair; the first similarity is the similarity between two second sample images in the second sample image pair; the second similarity is the similarity between the features of two second sample images in the second sample image feature pair; the second sample image feature pair includes image features extracted from the two second sample images of the second sample image pair based on the preset model;

[0287] The model determination module is used to select the preset model corresponding to the model parameters when the similarity loss value is less than the preset value as the image feature extraction model.

[0288] In some embodiments, the apparatus may further include:

[0289] The first video parsing unit is used to parse the first video to obtain a first video frame set; the first video in the first video frame set is the preset sample image.

[0290] The playback time difference determination unit is used to calculate the playback time difference between any two first video frames based on the playback time corresponding to each first video frame in the first video frame set.

[0291] The first target video frame determination unit is used to determine two first video frames whose playback time difference is less than or equal to a preset time threshold, thereby obtaining two first target video frames; the similarity between the two first target video frames is greater than or equal to the first threshold.

[0292] The first positive sample image pair determination unit is used to determine the two first target video frames as the positive sample image pair.

[0293] In some embodiments, the apparatus further includes:

[0294] The second video acquisition unit is used to acquire at least two second videos; the correlation parameter of any two of the at least two second videos is less than a preset parameter threshold; the second video is the preset sample image;

[0295] The second video frame determination unit is used to extract frames from the at least two second videos respectively to obtain the second video frame corresponding to each second video.

[0296] The negative sample image pair construction unit is used to construct the negative sample image pair based on the second video frame corresponding to each second video.

[0297] In some embodiments, the apparatus further includes:

[0298] The preset sample image set acquisition module is used to acquire the preset sample image set;

[0299] An image category determination module is used to determine the image category corresponding to each preset sample image in the preset sample image set;

[0300] The image category similarity determination module is used to determine the image category similarity between any two preset sample images in the preset sample image set based on the image category corresponding to each preset sample image.

[0301] The positive and negative sample image pair determination module is used to determine the positive sample image pair and the negative sample image pair based on the image category similarity of any two preset sample images in the preset sample image set.

[0302] In some embodiments, the positive and negative sample image pair determination module includes:

[0303] The first preset sample image determination submodule is used to determine two preset sample images whose image category similarity is greater than or equal to a third threshold, thereby obtaining two first preset sample images; the similarity between the two first preset sample images is greater than or equal to the first threshold.

[0304] The positive sample image pair determination submodule is used to determine the two first preset sample images as the positive sample image pair;

[0305] The second preset sample image determination submodule is used to determine two preset sample images whose image category similarity is less than the third threshold, thereby obtaining two second preset sample images; the similarity between the two second preset sample images is less than the first threshold.

[0306] The negative sample image pair determination submodule is used to determine the two second preset sample images as the negative sample image pair.

[0307] In some embodiments, the apparatus further includes:

[0308] The raw image acquisition unit is used to acquire the raw image.

[0309] An image processing unit is used to perform image enhancement processing on the original image to obtain at least one processed image;

[0310] An image pair construction unit is used to construct the positive sample image pair based on the original image and the at least one processed image.

[0311] The apparatus and method embodiments described herein are based on the same inventive concept.

[0312] This application also provides an image sharpness recognition system; please refer to [link to relevant documentation]. Figure 13The system includes: a server 1310, a content production terminal 1320, and a content consumption terminal 1330. The server 1310 includes: uplink and downlink content interface services, content metadata and storage, a resolution parameter recognition model, video frame extraction services, content storage services, the Internet, an image crawling and processing system, an image material and scoring sample library, a pre-trained model, image resolution scoring atomic services and video resolution scoring services, manual review services, a distribution content library, and a content distribution export service. The server 1310 is electrically connected to the content production terminal 1320 through the uplink and downlink content interface services, and the server 1310 is electrically connected to the content consumption terminal 1330 through the content distribution export service. The content production terminal 1320 includes Professionally Generated Content (PGC), User Generated Content (UGC), Multi-Channel Network (MCN), or Professional User Generated Content (PUGC) modes.

[0313] I. Content Production and Content Consumption

[0314] (1) PGC, UGC, MCN or PUGC content producers provide local or filmed video content or self-media articles or photo albums through mobile terminal or backend interface API system. Authors can choose to actively upload the cover image of the corresponding content. These are the main sources of content for distribution.

[0315] (2) By communicating with the upstream and downstream content interface services, first obtain the upload server interface address, and then upload the local file; during the shooting process, the local video content can be matched with music, filter templates and video beautification functions, etc.

[0316] (3) As a consumer, it communicates with the content distribution export server to obtain the index information of the corresponding content. For videos, it downloads the corresponding streaming media files through the video storage server and plays them through the local player. For images, it usually communicates through the content delivery network (CDN) service deployed at the edge.

[0317] (4) At the same time, the user's browsing behavior data, reading speed, completion rate, reading time, lag, loading time, playback clicks and other information during the upload and download process are reported to the server;

[0318] (5) Consumers typically browse consumption data through Feeds streams. For content with low clarity or blurriness, a direct feedback entry point is provided, which directly connects with the manual review system for confirmation and verification. The verified content is directly entered into the image sample library to provide low-segment training samples for subsequent algorithm models.

[0319] II. Uplink and Downlink Content Interface Server

[0320] (1) Communicate directly with the content production end. The content submitted from the front end is usually the title, publisher, summary, cover image, and publication time of the content, or the video being shot. It is directly entered into the server and stored in the video content storage service.

[0321] (2) Write the video content's metadata, such as video file size, cover image link, bitrate, file format, title, release time, author, etc., into the content database;

[0322] (3) Submit the uploaded files and content metadata to the content storage server for subsequent content processing and circulation.

[0323] III. Content Meta-information and Database

[0324] (1) The core database of video content. All the metadata of the content published by producers is stored in this business database. The focus is on the metadata of the video content and the text and image content itself, such as video file size, cover image link, bitrate, file format, title, publication time, author, whether it is original, possible categories and tags; it also includes the classification of content during the manual review process (including first, second and third level classification and tag information, such as a video about XX mobile phone with the first level classification as technology, the second level classification as mobile phone, the third level classification as domestic mobile phone, and the tag may be XX mobile phone, model YY, etc.);

[0325] (2) During the manual review process, information in the content database will be read. At the same time, the results and status of the manual review will also be sent back to the content database for storage. The results of the manual review are also an important basis for measuring the efficiency of the algorithm filtering model. The clarity assessment service mainly marks and distinguishes the clarity of the content. If it is divided into 10 levels, usually levels 1-3 will be filtered out directly, and the other levels will be marked. For content that the machine cannot judge, it will also go through the manual review system. The manual review system will send the review results back to this database.

[0326] IV. Content Ingestion Service

[0327] (1) Responsible for the entire scheduling process of video and text content flow, receiving video content through the uplink and downlink content interface server, and then obtaining the video metadata from the content metadata database;

[0328] (2) As the actual scheduler and controller of the image and video link operation, according to the content type, the corresponding clarity scoring service is scheduled to process the image and video content in the link;

[0329] (3) For the corresponding video content in the database, call the video frame extraction service to obtain the representative frame image of the video content, such as the 13-frame sampling mentioned above.

[0330] V. Manual review service and feedback on unclear content

[0331] (1) Usually it is a WEB system that receives the results of machine filtering on the link, performs manual confirmation and review of the results, writes the review results into the content information metadata database, and can evaluate the actual effect of the machine and filtering model online through the results of manual review.

[0332] (2) Report the source of the task, review results, review start and end times, and other detailed review process to the statistics server.

[0333] (3) Connect with the user's complaint and content feedback review system. For unclear text and images (including image sets) and video content, after confirmation, mark the score according to 10 levels and put it into the image material and scoring sample library as the basis for subsequent model iteration and optimization.

[0334] VI. Distributing Content Libraries

[0335] (1) Usually, it is a group of storage servers with a wide distribution range and the nearest access of client users; usually, there are also CDN acceleration servers in the periphery for distributed caching acceleration, and the video and image content uploaded by content producers is saved through the upstream and downstream content interface servers;

[0336] (2) After obtaining the content index information from the Feeds stream, end consumers can also directly access the content storage server to download the corresponding content;

[0337] (3) Regarding the issue of content distribution based on clarity, the fine-grained clarity score can be used as a factor for sorting or content filtering based on the application scenario and the richness of content supply in different categories. During the review process, high-definition videos can be reviewed first, and high-definition versions can be retained when deduplicating videos. This can reduce unnecessary review costs and improve user experience.

[0338] VII. Image Resources and Scoring Sample Library

[0339] (1) Obtain manually reviewed and marked image content from content metadata and the repository as a prototype library for establishing image clarity scoring tags;

[0340] (2) Collect large-scale image content from information flow links, public datasets and the Internet through the channels described above as the basic material data for self-supervised pre-training networks.

[0341] 8. Large-scale pre-trained models

[0342] (1) Following the method in the above embodiment, a self-supervised contrastive learning approach is adopted, and SwinTTransform is used as the Backbone to construct a pre-trained basic network model for large-scale information flow business image feature extraction.

[0343] 9. Clarity Scoring Model

[0344] (1) Based on deep learning, on the basis of large-scale image data in information flow, a basic model is constructed by transfer learning through a self-supervised learning pre-trained network, and then fine-tuned with a small sample set and the basic pre-trained model to obtain an image clarity scoring model.

[0345] (2) Based on unclear feedback from the client, for example, new labeled samples can be obtained periodically according to a week or day to update and fine-tune the training and upgrade the model;

[0346] 10. Image Clarity Scoring: Atomic Service and Video Clarity Scoring

[0347] (1) The image clarity content model constructed above is service-oriented, and a service that can be called on the link is built to realize the distinction of fine-grained clarity of content;

[0348] (2) For video clarity, there is a method of merging multiple video frames to obtain a video clarity score. This merging process is implemented by the image clarity scoring atomic service.

[0349] XI. Video Frame Extraction Service

[0350] (1) Download the video files stored in the database and extract the video files locally.

[0351] 12. Auditors

[0352] (1) Construct fine-grained distinguishing markers for image sharpness, which are called by the content storage scheduling server in the content processing chain to process content that the machine cannot recognize;

[0353] 13. Manual Review Service

[0354] (1) The main processing is unclear content feedback (images, image sets, videos) from the user side. After manual review and labeling, the content is added to the image library as the basis for subsequent training.

[0355] (2) As a supplement to machine review capabilities, it handles other quality issues related to content, such as legal issues;

[0356] XIV. Image Crawling and Processing System

[0357] (1) Using the query information constructed by the tags mined from the information flow content according to the method described above, crawl the corresponding image data from the Internet;

[0358] (2) Images with similar or identical labels can be considered similar, otherwise they are dissimilar, so as to construct weakly supervised sample data for comparative learning.

[0359] This application provides an image sharpness recognition device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the image sharpness recognition method provided in the above method embodiments.

[0360] Embodiments of this application also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing an image sharpness recognition method in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the image sharpness recognition method provided in the above method embodiment.

[0361] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image sharpness recognition method provided in the above-described method embodiments.

[0362] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0363] The memory described in this application embodiment can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0364] The image sharpness recognition method embodiments provided in this application can be executed on mobile terminals, computer terminals, servers, or similar computing devices. Taking running on a server as an example, Figure 14 This is a hardware structure block diagram of a server for an image sharpness recognition method provided in this application embodiment. As shown in Figure 14, the server 1400 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1410 (CPUs 1410 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 1430 for storing data, and one or more storage media 1420 (e.g., one or more mass storage devices) for storing application programs 1423 or data 1422. The memory 1430 and storage media 1420 may be temporary or persistent storage. The program stored in the storage media 1420 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 1410 may be configured to communicate with the storage media 1420 and execute the series of instruction operations in the storage media 1420 on the server 1400. Server 1400 may also include one or more power supplies 1460, one or more wired or wireless network interfaces 1450, one or more input / output interfaces 1440, and / or one or more operating systems 1421, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0365] The input / output interface 1440 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 1400. In one example, the input / output interface 1440 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one example, the input / output interface 1440 can be a radio frequency (RF) module for wireless communication with the Internet.

[0366] Those skilled in the art will understand that Figure 14 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 1400 may also include... Figure 14 The more or fewer components shown, or having the same Figure 14 The different configurations shown.

[0367] As can be seen from the embodiments of the image sharpness recognition method, apparatus, device, or storage medium provided in this application, this application acquires an image to be recognized; performs sharpness parameter recognition processing on the image to be recognized based on a sharpness parameter recognition model to obtain the target sharpness parameter of the image to be recognized; wherein, the training method of the sharpness parameter recognition model includes: acquiring a first sample image labeled with sharpness parameter tags; performing sharpness parameter recognition training on an image feature extraction model based on the first sample image to adjust the model parameters of the image feature extraction model until the sharpness parameter tags output by the image feature extraction model match the labeled sharpness parameter tags; the image feature extraction model is obtained by performing image feature extraction training on a preset model based on a second sample image pair, the second sample image pair including a positive sample image pair and a negative sample image pair, the similarity between two positive sample images in the positive sample image pair is greater than or equal to a first threshold, and the similarity between two negative sample images in the negative sample image pair is less than the first threshold; the image feature extraction model corresponding to the model parameters when the output sharpness parameter tags match the labeled sharpness parameter tags is used as the sharpness parameter recognition model. In the training process of the sharpness parameter recognition model, this application first obtains an image feature extraction model through unsupervised training. Based on this model, it then conducts supervised training using sample images labeled with sharpness parameters. This enables the modeling of combining low-level sharpness information with high-level semantic information, thereby allowing the obtained sharpness parameter recognition model to accurately identify the sharpness of images and improving the accuracy of image sharpness recognition.

[0368] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0369] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0370] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0371] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image sharpness recognition method, characterized in that, The method includes: Acquire the image to be recognized; The image to be identified is processed by a sharpness parameter recognition model to obtain the target sharpness parameter of the image to be identified. The training method for the sharpness parameter recognition model includes: Two preset sample images with a similarity greater than or equal to a first threshold are designated as positive sample image pairs, and two preset sample images with a similarity less than the first threshold are designated as negative sample image pairs; the positive sample image pairs and the negative sample image pairs are determined as the second sample image pairs; Based on the second sample image pair, a preset model is subjected to image feature extraction and self-supervised training through contrastive learning to adjust the model parameters of the preset model until the similarity loss value of the preset model is less than a second threshold. The similarity loss value is determined based on the first similarity and the second similarity corresponding to the second sample image pair. The first similarity is the similarity between two second sample images in the second sample image pair. The second similarity is the similarity between two features of two second sample images in the second sample image feature pair. The second sample image feature pair includes image features extracted from the two second sample images of the second sample image pair based on the preset model. The preset model corresponding to the model parameters when the similarity loss value is less than the second threshold is used as the image feature extraction model; Obtain the first sample image labeled with the sharpness parameter. Based on the first sample image, the image feature extraction model, regression classifier, and Softmax activation function are trained for sharpness parameter recognition to adjust the model parameters of the image feature extraction model until the sharpness parameter label output by the Softmax activation function matches the labeled sharpness parameter label. The image feature extraction model, regression classifier, and Softmax activation function corresponding to the model parameters when the output sharpness parameter label matches the labeled sharpness parameter label are used as the sharpness parameter recognition model.

2. The method according to claim 1, characterized in that, The acquisition of the image to be identified includes: Obtain the video to be recognized; The videos to be identified are divided into a first video to be identified and a second video to be identified according to their playback time; the first video to be identified is played earlier than the second video to be identified. Extract a first number of video frames from the first video to be identified, and use them as the first image to be identified; Extract a second number of video frames from the second video to be identified, and use them as the second image to be identified; The first image to be identified and the second image to be identified are determined as the images to be identified.

3. The method according to claim 2, characterized in that, Extracting a first number of video frames from the first video to be identified as the first image to be identified includes: The first video to be identified is parsed to obtain the first video frame set; From the first video frame set, extract the first video frame whose playback time is a first preset time, and use it as the first image to be identified; the first preset time includes the first number of time points.

4. The method according to claim 2, characterized in that, Extracting a first number of video frames from the first video to be identified as the first image to be identified includes: Determine the first playback duration of the first video to be identified; The first video to be identified is divided into the first number of first sub-videos according to the first playback duration; Extract one first sub-video frame from each first sub-video to obtain the first number of first sub-video frames, wherein the first number of first sub-video frames are video frames with equal time intervals. The first number of first sub-video frames are used as the first image to be identified.

5. The method according to claim 2, characterized in that, Extracting a second number of video frames from the second video to be identified, as the second image to be identified, includes: Determine the second playback duration of the second video to be identified; The second video to be identified is divided into the second number of second sub-videos according to the second playback duration; Extract one second sub-video frame from each second sub-video to obtain the second number of second sub-video frames; The second number of second sub-video frames are used as the second image to be identified.

6. The method according to claim 2, characterized in that, The process of performing sharpness parameter recognition on the image to be recognized based on the sharpness parameter recognition model to obtain the target sharpness parameters of the image to be recognized includes: Based on the sharpness parameter recognition model, sharpness parameter recognition processing is performed on the first image to be recognized and the second image to be recognized to obtain the target sharpness parameters corresponding to the first image to be recognized and the second image to be recognized respectively. The method further includes: The clarity parameters of the video to be identified are determined based on the target sharpness parameters corresponding to the first image to be identified and the second image to be identified, respectively.

7. The method according to claim 6, characterized in that, The step of determining the sharpness parameters of the video to be identified based on the target sharpness parameters corresponding to the first image to be identified and the second image to be identified includes: Calculate the sum of the first quantity and the second quantity to obtain the preset quantity; Calculate the product of the preset number of target sharpness parameters corresponding to the first image to be identified and the second image to be identified, and obtain the sharpness results corresponding to the first image to be identified and the second image to be identified. The initial clarity parameters of the video to be identified are obtained by summing the sharpness results corresponding to the first image to be identified and the second image to be identified. The resolution parameter of the video to be identified is obtained by calculating the preset number root of the initial resolution parameter.

8. The method according to claim 7, characterized in that, The method for constructing the positive sample image pairs includes: The first video is parsed to obtain a first video frame set; the first video in the first video frame set is the preset sample image. Calculate the playback time difference between any two first video frames based on the playback time corresponding to each first video frame in the first video frame set. Two first video frames are identified whose playback time difference is less than or equal to a preset time threshold, thus obtaining two first target video frames; the similarity between the two first target video frames is greater than or equal to the first threshold. The two first target video frames are identified as the positive sample image pair.

9. The method according to claim 1, characterized in that, The method for constructing the negative sample image pairs includes: At least two second videos are acquired; the correlation parameter between any two of the at least two second videos is less than a preset parameter threshold; the second video is the preset sample image. Frames are extracted from each of the at least two second videos to obtain a second video frame corresponding to each second video. The negative sample image pair is constructed based on the second video frame corresponding to each second video.

10. The method according to claim 1, characterized in that, The method further includes: Obtain a preset set of sample images; Determine the image category corresponding to each preset sample image in the preset sample image set; Based on the image category corresponding to each preset sample image, determine the image category similarity between any two preset sample images in the preset sample image set; Based on the image category similarity between any two preset sample images in the preset sample image set, the positive sample image pair and the negative sample image pair are determined.

11. The method according to claim 10, characterized in that, The step of determining the positive sample image pair and the negative sample image pair based on the image category similarity of any two preset sample images in the preset sample image set includes: Two preset sample images with an image category similarity greater than or equal to a third threshold are identified to obtain two first preset sample images; the similarity between the two first preset sample images is greater than or equal to the first threshold. The two first preset sample images are determined as the positive sample image pair; Two preset sample images with a category similarity less than the third threshold are identified to obtain two second preset sample images; the similarity between the two second preset sample images is less than the first threshold. The two second preset sample images are determined as the negative sample image pair.

12. The method according to claim 1, characterized in that, The method for constructing the positive sample image pairs includes: Obtain the original image; The original image is subjected to image enhancement processing to obtain at least one processed image; The positive sample image pair is constructed based on the original image and the at least one processed image.

13. An image sharpness recognition device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be recognized. The target sharpness parameter determination module is used to perform sharpness parameter recognition processing on the image to be recognized based on the sharpness parameter recognition model to obtain the target sharpness parameter of the image to be recognized. The model training module is used to train the sharpness parameter recognition model; The model training module includes: The positive and negative sample determination module is used to identify two preset sample images with a similarity greater than or equal to a first threshold as positive sample image pairs and two preset sample images with a similarity less than the first threshold as negative sample image pairs. The second sample image pair determination module is used to determine the positive sample image pair and the negative sample image pair as the second sample image pair; The parameter adjustment module is used to perform self-supervised training of a preset model based on the second sample image pair, extracting image features and conducting comparative learning, to adjust the model parameters of the preset model until the similarity loss value of the preset model is less than a second threshold. The similarity loss value is determined based on a first similarity and a second similarity corresponding to the second sample image pair. The first similarity is the similarity between two second sample images in the second sample image pair. The second similarity is the similarity between two features in a second sample image feature pair. The second sample image feature pair includes image features extracted from the two second sample images in the second sample image pair based on the preset model. The model determination module is used to select the preset model corresponding to the model parameters when the similarity loss value is less than the second threshold as the image feature extraction model; The first sample image acquisition submodule is used to acquire the first sample image labeled with the sharpness parameter. The model parameter adjustment submodule is used to train the image feature extraction model, regression classifier and softmax activation function for sharpness parameter recognition based on the first sample image, so as to adjust the model parameters of the image feature extraction model until the sharpness parameter label output by the softmax activation function matches the labeled sharpness parameter label. The sharpness parameter recognition model determination submodule is used to determine the image feature extraction model, regression classifier, and Softmax activation function corresponding to the model parameters when the output sharpness parameter label is matched with the labeled sharpness parameter label, and to use these as the sharpness parameter recognition model.

14. The apparatus according to claim 13, characterized in that, The image acquisition module includes: The video acquisition submodule is used to acquire the video to be recognized; The video decomposition submodule is used to divide the video to be identified into a first video to be identified and a second video to be identified according to the playback time; the playback time of the first video to be identified is earlier than the playback time of the second video to be identified. The first image to be identified determination submodule is used to extract a first number of video frames from the first video to be identified as the first image to be identified; The second image to be identified determination submodule is used to extract a second number of video frames from the second video to be identified as the second image to be identified. The image to be identified submodule is used to identify the first image to be identified and the second image to be identified as the image to be identified.

15. The apparatus according to claim 14, characterized in that, The first image determination submodule includes: The first video frame set determination unit is used to parse the first video to be identified to obtain the first video frame set; The first image to be identified unit is used to extract a first video frame with a playback time of a first preset time from the first video frame set as the first image to be identified; the first preset time includes the first number of time points.

16. The apparatus according to claim 14, characterized in that, The first image determination submodule includes: The first playback duration determination unit is used to determine the first playback duration of the first video to be identified. The first sub-video determining unit is used to divide the first video to be identified into the first number of first sub-videos according to the first playback duration. The first sub-video frame determination unit is used to extract a first sub-video frame from each first sub-video to obtain the first number of first sub-video frames, wherein the first number of first sub-video frames are video frames with equal time intervals. The first image acquisition unit is used to acquire the first number of first sub-video frames as the first image to be identified.

17. The apparatus according to claim 14, characterized in that, The second image to be identified determination submodule includes: The second playback duration determination unit is used to determine the second playback duration of the second video to be identified; The second sub-video determining unit is used to divide the second video to be identified into the second number of second sub-videos according to the second playback duration. The second sub-video frame determination unit is used to extract a second sub-video frame from each second sub-video to obtain the second number of second sub-video frames. The second image to be identified unit is used to determine the second number of second sub-video frames as the second image to be identified.

18. The apparatus according to claim 14, characterized in that, The target sharpness parameter determination module includes: The parameter recognition submodule is used to perform sharpness parameter recognition processing on the first image to be recognized and the second image to be recognized based on the sharpness parameter recognition model, so as to obtain the target sharpness parameters corresponding to the first image to be recognized and the second image to be recognized respectively. The device further includes: The sharpness parameter determination module is used to determine the sharpness parameter of the video to be identified based on the target sharpness parameters corresponding to the first image to be identified and the second image to be identified, respectively.

19. The apparatus according to claim 18, characterized in that, The sharpness parameter determination module includes: The preset quantity determination submodule is used to calculate the sum of the first quantity and the second quantity to obtain the preset quantity; The sharpness result determination submodule is used to calculate the product of the preset number of target sharpness parameters corresponding to the first image to be identified and the second image to be identified, so as to obtain the sharpness result corresponding to the first image to be identified and the second image to be identified. The initial sharpness parameter determination submodule is used to calculate the sum of the sharpness results corresponding to the first image to be identified and the second image to be identified, so as to obtain the initial sharpness parameter of the video to be identified; The sharpness parameter determination submodule is used to calculate the preset number root of the initial sharpness parameter to obtain the sharpness parameter of the video to be identified.

20. The apparatus according to claim 19, characterized in that, The device further includes: The first video parsing unit is used to parse the first video to obtain a first video frame set; the first video in the first video frame set is the preset sample image. The playback time difference determination unit is used to calculate the playback time difference between any two first video frames based on the playback time corresponding to each first video frame in the first video frame set. The first target video frame determination unit is used to determine two first video frames whose playback time difference is less than or equal to a preset time threshold, thereby obtaining two first target video frames; the similarity between the two first target video frames is greater than or equal to the first threshold. The first positive sample image pair determination unit is used to determine the two first target video frames as the positive sample image pair.

21. The apparatus according to claim 13, characterized in that, The device further includes: The second video acquisition unit is used to acquire at least two second videos; the correlation parameter of any two of the at least two second videos is less than a preset parameter threshold; the second video is the preset sample image; The second video frame determination unit is used to extract frames from the at least two second videos respectively to obtain the second video frame corresponding to each second video. The negative sample image pair construction unit is used to construct the negative sample image pair based on the second video frame corresponding to each second video.

22. The apparatus according to claim 13, characterized in that, The device further includes: The preset sample image set acquisition module is used to acquire the preset sample image set; An image category determination module is used to determine the image category corresponding to each preset sample image in the preset sample image set; The image category similarity determination module is used to determine the image category similarity between any two preset sample images in the preset sample image set based on the image category corresponding to each preset sample image. The positive and negative sample image pair determination module is used to determine the positive sample image pair and the negative sample image pair based on the image category similarity of any two preset sample images in the preset sample image set.

23. The apparatus according to claim 22, characterized in that, The positive and negative sample image pair determination module includes: The first preset sample image determination submodule is used to determine two preset sample images whose image category similarity is greater than or equal to a third threshold, thereby obtaining two first preset sample images; the similarity between the two first preset sample images is greater than or equal to the first threshold. The positive sample image pair determination submodule is used to determine the two first preset sample images as the positive sample image pair; The second preset sample image determination submodule is used to determine two preset sample images whose image category similarity is less than the third threshold, thereby obtaining two second preset sample images; the similarity between the two second preset sample images is less than the first threshold. The negative sample image pair determination submodule is used to determine the two second preset sample images as the negative sample image pair.

24. The apparatus according to claim 13, characterized in that, The device further includes: The raw image acquisition unit is used to acquire the raw image. An image processing unit is used to perform image enhancement processing on the original image to obtain at least one processed image; An image pair construction unit is used to construct the positive sample image pair based on the original image and the at least one processed image.

25. An image sharpness recognition device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image sharpness recognition method as described in any one of claims 1-12.

26. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the image sharpness recognition method as described in any one of claims 1-12.

27. A computer storage medium, characterized in that, The computer storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the image sharpness recognition method as described in any one of claims 1-12.

Images