Screening method and device of video polyphonic ringtone cover, equipment and storage medium

Abstract

The invention discloses a video polyphonic ringtone cover screening method and apparatus, a device and a storage medium. The method comprises the steps of obtaining an alternative picture set of a video polyphonic ringtone corresponding to a user and description information of the video polyphonic ringtone; determining a theme matching degree between each alternative cover picture and the description information; performing picture quality evaluation on each alternative cover picture to obtain a quality evaluation score corresponding to each alternative cover picture; performing weighted summation on the theme matching degree and the quality evaluation score corresponding to the alternative cover picture to obtain a comprehensive evaluation score corresponding to the alternative cover picture; and comparing the comprehensive evaluation scores corresponding to the alternative cover pictures, and determining the alternative cover picture corresponding to the maximum comprehensive evaluation score as a target cover picture corresponding to the video polyphonic ringtone. According to the method, the accuracy and efficiency of video polyphonic ringtone cover screening are improved, the implementation cost is low, and the video polyphonic ringtone use experience of a user is improved. The method can be widely applied to the technical field of video polyphonic ringtones.

Description

Technical Field

[0001] The present application relates to the technical field of video ringback tone, and in particular to a method, device, equipment and storage medium for screening video ringback tone covers. Background Art

[0002] With the advancement of information technology, more and more applications are being developed to provide services to users. For example, video ringback tone (CRBT) is a value-added service based on communication networks. When a user makes a voice call, it plays a short video to the called party, replacing the traditional ringback tone. When the caller calls the called party, while waiting on hold, they can view personalized video content pre-set by the called party, such as personal videos, corporate promotions, or holiday greetings.

[0003] With the widespread adoption of video ringback rings (RRTs), selecting appropriate video ringback ring covers is crucial for enhancing user experience and enhancing the user experience. However, existing RRT covers are often created manually by selecting from a library of pre-selected cover materials or through costly image generation technology. Manual selection is limited by the experience of the screener, resulting in inconsistent quality and low efficiency. Furthermore, existing image generation technology is immature and expensive. These factors contribute to the poor performance of current RRT covers, impacting the user experience.

[0004] In summary, the problems existing in related technologies need to be solved urgently. Summary of the Invention

[0005] The purpose of this application is to solve one of the technical problems existing in the related art to at least a certain extent.

[0006] To this end, one purpose of the embodiments of the present application is to provide a method, device, equipment and storage medium for screening video ringback tone covers.

[0007] In order to achieve the above technical objectives, the technical solutions adopted in the embodiments of the present application include:

[0008] In one aspect, an embodiment of the present application provides a method for screening video ringback tone covers, the method comprising:

[0009] Obtaining a candidate image set for a video ringback tone corresponding to the user and description information of the video ringback tone; wherein the candidate image set includes several candidate cover images;

[0010] Determining a degree of subject matching between each of the candidate cover images and the description information;

[0011] Performing image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image;

[0012] Performing a weighted summation of the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image;

[0013] Compare the comprehensive evaluation scores corresponding to the candidate cover pictures, and determine the candidate cover picture with the largest comprehensive evaluation score as the target cover picture corresponding to the video ringtone

[0014] In addition, the method for screening video ringback tone covers according to the above embodiment of the present application may also have the following additional technical features:

[0015] Furthermore, in one embodiment of the present application, determining the subject matching degree between each candidate cover image and the description information includes:

[0016] Inputting the candidate cover image and the description information into the CLIP model;

[0017] The image-text similarity between the candidate cover image and the description information is calculated using the CLIP model, and the image-text similarity is determined as the topic matching degree.

[0018] Furthermore, in one embodiment of the present application, performing image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image includes:

[0019] Performing a clarity evaluation on the candidate cover image to obtain a clarity evaluation score corresponding to the candidate cover image;

[0020] Performing composition evaluation on the candidate cover images to obtain composition evaluation scores corresponding to the candidate cover images;

[0021] Performing an aesthetic evaluation on the candidate cover image to obtain an aesthetic evaluation score corresponding to the candidate cover image;

[0022] Determine a quality evaluation score corresponding to the candidate cover image based on at least one of the clarity evaluation score, the composition evaluation score, and the aesthetic evaluation score.

[0023] Furthermore, in one embodiment of the present application, performing clarity evaluation on the candidate cover image to obtain a clarity evaluation score corresponding to the candidate cover image includes:

[0024] Convert the candidate cover image into a grayscale image and perform normalization processing to obtain a corresponding standard grayscale image;

[0025] Performing a two-dimensional discrete Fourier transform on the standard grayscale image to obtain a corresponding frequency domain representation;

[0026] Determine the center of the spectrum corresponding to the standard grayscale image, and determine a high-frequency region in the spectrum based on the center of the spectrum and a preset radius threshold; wherein the high-frequency region is a region where frequency domain points whose distance from the center of the spectrum is greater than the radius threshold are located;

[0027] determining high-frequency component energy according to the frequency domain representation corresponding to each of the frequency domain points in the high-frequency region, and determining total energy according to the frequency domain representation corresponding to all the frequency domain points in the spectrum;

[0028] The clarity evaluation score corresponding to the alternative cover image is determined based on a first ratio of the high-frequency component energy to the total energy; wherein the clarity evaluation score is positively correlated with the first ratio.

[0029] Furthermore, in one embodiment of the present application, performing composition evaluation on the candidate cover image to obtain a composition evaluation score corresponding to the candidate cover image includes:

[0030] Determining first coordinates of a plurality of golden section points in the candidate cover image;

[0031] Detecting the second coordinate of the subject center point in the candidate cover image using the Yolov8 model;

[0032] Calculating the Euclidean distances between the second coordinates and each of the first coordinates, and determining the minimum value of the Euclidean distances as a first distance value;

[0033] Determining the diagonal length of the candidate cover image as a second distance value, and determining a topic location score according to a second ratio of the first distance value to the second distance value; wherein the topic location score is negatively correlated with the second ratio;

[0034] Extracting the foreground and background regions of the candidate cover images using a semantic segmentation model;

[0035] Calculating a contrast difference between the foreground area and the background area, and determining a depth-of-field score based on the contrast difference; wherein the depth-of-field score is positively correlated with the contrast difference;

[0036] The subject position score and the depth of field score are weightedly summed to obtain the composition evaluation score corresponding to the alternative cover image.

[0037] Furthermore, in one embodiment of the present application, performing aesthetic evaluation on the candidate cover image to obtain an aesthetic evaluation score corresponding to the candidate cover image includes:

[0038] The candidate cover image is input into a pre-trained aesthetic scoring model, and the aesthetic evaluation score corresponding to the candidate cover image is predicted and output by the aesthetic scoring model.

[0039] Furthermore, in one embodiment of the present application, the method further includes:

[0040] Convert the candidate cover images into HSV space, determine multiple clusters corresponding to the candidate cover images through a clustering algorithm, and determine the HSV value corresponding to the cluster center of each cluster;

[0041] Determining a complementarity score based on the HSV value;

[0042] Determining a color score corresponding to the candidate cover image based on the complementarity score;

[0043] The aesthetic assessment score is modified based on the color score.

[0044] On the other hand, an embodiment of the present application provides a device for screening video ringback tone covers, the device comprising:

[0045] An acquiring unit, configured to acquire a candidate picture set of a video ringback tone corresponding to a user and description information of the video ringback tone; wherein the candidate picture set includes a plurality of candidate cover pictures;

[0046] A matching unit, configured to determine a degree of subject matching between each candidate cover image and the description information;

[0047] An evaluation unit, configured to perform image quality evaluation on each candidate cover image to obtain a quality evaluation score corresponding to each candidate cover image;

[0048] a summarizing unit, configured to perform a weighted summation of the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image;

[0049] The comparison unit is used to compare the comprehensive evaluation scores corresponding to the candidate cover pictures, and determine the candidate cover picture corresponding to the largest comprehensive evaluation score as the target cover picture corresponding to the video ringtone.

[0050] On the other hand, an embodiment of the present application provides an electronic device, including:

[0051] at least one processor;

[0052] at least one memory for storing at least one program;

[0053] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned method for screening video ringback tone covers.

[0054] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to implement the above-mentioned method for screening video ringtone covers.

[0055] The advantages and benefits of this application will be partially given in the following description, and partially become apparent from the following description, or learned through practice of this application:

[0056] The embodiments of the present application disclose a method, apparatus, device, and storage medium for screening video ringback tone covers, which obtains a set of candidate images for a user's corresponding video ringback tone and description information of the video ringback tone; wherein the set of candidate images includes several candidate cover images; determines the theme matching degree between each candidate cover image and the description information; performs image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image; performs weighted summation of the theme matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image; compares the comprehensive assessment scores corresponding to each candidate cover image, and determines the candidate cover image with the largest comprehensive assessment score as the target cover image corresponding to the video ringback tone. This method is based on a fusion scoring algorithm of theme matching degree and image quality, improves the accuracy and efficiency of video ringback tone cover screening, has a low implementation cost, and is conducive to improving the user's video ringback tone usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0058] Figure 1 A schematic diagram of an implementation environment for a method for screening video ringback tone covers provided in an embodiment of the present application;

[0059] Figure 2 A flowchart of a method for screening video ringback tone covers provided in an embodiment of the present application is shown;

[0060] Figure 3This is a schematic structural diagram of a device for screening video ringback tone covers provided in an embodiment of the present application;

[0061] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be considered as limiting the present application. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0065] 1) Video Ringback Tone: refers to a video played instead of the traditional audio ringback tone during a mobile phone call. It is usually a short video content selected by the ringback tone user.

[0066] 2) Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0067] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, pre-trained models, operating / interaction systems, and mechatronics. Pre-trained models, also known as large models or basic models, can be fine-tuned and widely applied to downstream tasks across various AI disciplines. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0068] 3) Machine Learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning. Pretrained models are the latest development in deep learning, integrating these techniques.

[0069] 4) Deep learning is a subfield of machine learning that simulates the way the human brain processes information by building and training multi-layer neural networks. Deep learning models can automatically extract complex features from large amounts of data and apply them to a variety of tasks, such as image recognition, natural language processing, speech recognition, and recommendation systems.

[0070] 5) CLIP (Contrastive Language–Image Pretraining) is a multimodal model proposed by OpenAI in 2021. It maps images and text into the same semantic space through contrastive learning to achieve cross-modal understanding.

[0071] 6) YOLOv8 is a real-time object detection model from the YOLO (You Only Look Once) family. It has become a popular choice in the computer vision field for its high speed, high accuracy, and ease of use.

[0072] With the advancement of information technology, more and more applications are being developed to provide services to users. For example, video ringback tone (CRBT) is a value-added service based on communication networks. When a user makes a voice call, it plays a short video to the called party, replacing the traditional ringback tone. When the caller calls the called party, while waiting on hold, they can view personalized video content pre-set by the called party, such as personal videos, corporate promotions, or holiday greetings.

[0073] With the widespread adoption of video ringback rings (RRTs), selecting appropriate video ringback ring covers is crucial for enhancing user experience and enhancing the user experience. However, existing RRT covers are often created manually by selecting from a library of pre-selected cover materials or through costly image generation technology. Manual selection is limited by the experience of the screener, resulting in inconsistent quality and low efficiency. Furthermore, existing image generation technology is immature and expensive. These factors contribute to the poor performance of current RRT covers, impacting the user experience.

[0074] In view of this, an embodiment of the present application provides a method for screening video ringback tone covers, which obtains a set of alternative pictures for the video ringback tone corresponding to the user and the description information of the video ringback tone; wherein the set of alternative pictures includes several alternative cover pictures; determines the subject matching degree between each of the alternative cover pictures and the description information; performs image quality assessment on each of the alternative cover pictures to obtain a quality assessment score corresponding to each of the alternative cover pictures; performs weighted summation on the subject matching degree and the quality assessment score corresponding to the alternative cover pictures to obtain a comprehensive assessment score corresponding to the alternative cover pictures; compares the comprehensive assessment scores corresponding to each of the alternative cover pictures, and determines the alternative cover picture with the largest comprehensive assessment score as the target cover picture corresponding to the video ringback tone. This method is based on a fusion scoring algorithm of subject matching degree and image quality, which improves the accuracy and efficiency of video ringback tone cover screening, has a low implementation cost, and is conducive to improving the user's video ringback tone usage experience.

[0075] Please refer to Figure 1 , Figure 1 The following is a schematic diagram showing an implementation environment of a method for selecting video ringback tone covers provided in an embodiment of the present application. In this implementation environment, the main hardware and software components involved include a terminal device 110 and a backend server 120. The terminal device 110 and the backend server 120 are in communication connection with each other.

[0076] Specifically, a method for screening video ringback tone covers provided in an embodiment of the present application can be executed solely on the terminal device 110 side, or solely on the background server 120 side, or based on data interaction between the terminal device 110 and the background server 120.

[0077] The terminal device 110 in the above embodiment may include, but is not limited to, a mobile phone, a computer, a smart wearable device, a PDA device, an intelligent voice interaction device, a smart home appliance, an in-vehicle terminal, etc. The backend server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0078] The terminal device 110 and the backend server 120 may establish a communication connection via a wireless network or a wired network. The wireless network or wired network uses standard communication technologies and / or protocols, and the network may be the Internet or any other network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network or any combination of a virtual private network.

[0079] Of course, it is understandable that Figure 1 The implementation environment in the embodiment of the present application is only a method for screening video ringback tone covers. Some optional application scenarios are not fixed in the actual application. Figure 1 The hardware and software environment shown.

[0080] Below, in combination with the introduction of the aforementioned implementation environment, a method for screening video ringback tone covers provided in an embodiment of the present application is introduced and explained.

[0081] Please refer to Figure 2 , Figure 2 : is a schematic diagram of a method for screening video ringback tone covers provided in an embodiment of the present application. The method for screening video ringback tone covers includes but is not limited to:

[0082] Step 210: Obtain a candidate image set for the video ringback tone corresponding to the user and description information of the video ringback tone; wherein the candidate image set includes several candidate cover images;

[0083] Step 220: Determine the subject matching degree between each candidate cover image and the description information;

[0084] Step 230: Perform image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image;

[0085] Step 240: Perform a weighted sum of the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image;

[0086] Step 250: Compare the comprehensive evaluation scores corresponding to the candidate cover pictures, and determine the candidate cover picture with the largest comprehensive evaluation score as the target cover picture corresponding to the video ringtone.

[0087] Video ringback ringtones (RRTs) are personalized call services provided by operators, allowing users to customize the video content played while on hold. As the first visual entry point for a video ringback ringtone, the RRT cover directly impacts the user experience. In related technologies, RRT covers are typically manually selected or generated using artificial intelligence (AI). Manual selection is time-consuming and subjective, resulting in inconsistent quality. AI-based generation is also expensive, impacting the user experience.

[0088] In an embodiment of the present application, a method for screening video ringtone covers is provided. The method realizes the automatic and intelligent selection of video ringtone cover images by combining theme matching analysis and image quality evaluation, which can improve the accuracy and efficiency of video ringtone cover screening and has a low implementation cost.

[0089] Specifically, in an embodiment of the present application, when selecting a video ringback tone cover, a candidate image set and description information of the video ringback tone corresponding to the user can be obtained. The candidate image set may include several candidate cover images. The method in the embodiment of the present application requires selecting one of these candidate cover images as the target cover image corresponding to the video ringback tone. This application does not impose a limit on the number of candidate cover images included in the candidate image set.

[0090] In the embodiments of the present application, the sources of the alternative cover images in the alternative image set can be diverse. For example, in some embodiments, the alternative cover images can be uploaded by the user; in some embodiments, the alternative cover images can be obtained from a third-party image library; in some embodiments, the alternative cover images can also be obtained from screenshots of video ringback rings. Of course, it should be noted that the alternative cover images themselves need to have a certain relevance to the video ringback ring. For example, if the video ringback ring content is related to "seaside vacation", the alternative cover images may include images of themes such as beaches, sunsets, and swimming pools. This application does not limit their specific sources.

[0091] In an embodiment of the present application, the description information of the video ringback tone is used to describe the core content of the video ringback tone, which can be manually edited, such as including text tags corresponding to the video ringback tone (such as "summer travel"), keywords (such as "beach, waves"), etc., or extracted based on images, audio, etc. of the video ringback tone.

[0092] In an embodiment of the present application, after obtaining the description information of the alternative image set and the video ringtone, the theme matching degree between the alternative cover image and the description information can be determined. In an embodiment of the present application, the theme matching degree between the alternative cover image and the description information refers to the degree of matching between the alternative cover image and the description information at the theme level. The higher the theme matching degree, the closer the theme reflected by the alternative cover image and the description information is; conversely, the higher the theme matching degree, the less similar the theme reflected by the alternative cover image and the description information is. For example, the description information corresponding to the video ringtone is "Romantic Seaside Wedding", and the theme matching degree with the alternative cover image of "Beach Wedding Photos" may be 0.92, and the theme matching degree with the alternative cover image of "City Night View" may be 0.15.

[0093] Then, the image quality of the alternative cover images can be evaluated. In an embodiment of the present application, when evaluating the image quality of the alternative cover images, the evaluation can be performed in multiple dimensions, such as clarity, composition, color distribution, aesthetics, noise level and other dimensions, and the present application does not impose any restrictions on this. For example, for the clarity dimension, the image sharpness can be calculated by an edge detection algorithm (such as the Sobel operator) to determine the corresponding evaluation score; for the color distribution dimension, the overexposed / underexposed areas can be detected (such as histogram equalization analysis) to determine the corresponding evaluation score; for the noise level dimension, the peak signal-to-noise ratio (PSNR) can be used to evaluate the compression distortion to determine the corresponding evaluation score. After obtaining the evaluation score for each dimension, they can be weighted and summed to obtain the quality evaluation score corresponding to the alternative cover image.

[0094] In the embodiment of the present application, after obtaining the theme matching degree and quality evaluation score corresponding to the candidate cover image, a weighted sum of the two can be performed to determine the comprehensive evaluation score corresponding to the candidate cover image. This process can be expressed as follows:

[0095]

[0096] Where S (i) Indicates the comprehensive evaluation score corresponding to the alternative cover images, Indicates the theme matching degree of the alternative cover images. It represents the quality assessment score corresponding to the candidate cover image, α and β are weighted parameters, and the sum of the two is 1.

[0097] After determining the comprehensive evaluation scores corresponding to the candidate cover images, they can be compared and sorted by size, and then the candidate cover image with the largest comprehensive evaluation score can be determined as the target cover image corresponding to the video ringtone.

[0098] In some embodiments, a score threshold can be set. If a candidate cover image exceeds the score threshold, the candidate cover image can be determined as the target cover image for the video ringtone. If all candidate cover images score below the score threshold, an image generation strategy can be adopted, using the staledifussion model as input, the image, background mask, and prompt word, and then the AdaIN algorithm is used to transfer the template style to the image to obtain the target cover image for the video ringtone.

[0099] It can be understood that an embodiment of the present application provides a method for screening video ringback tone covers, obtaining a set of candidate images for the video ringback tone corresponding to the user and the description information of the video ringback tone; wherein the set of candidate images includes several candidate cover images; determining the subject matching degree between each candidate cover image and the description information; performing image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image; performing weighted summation on the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image; comparing the comprehensive assessment scores corresponding to each candidate cover image, and determining the candidate cover image with the largest comprehensive assessment score as the target cover image corresponding to the video ringback tone. This method is based on a fusion scoring algorithm of subject matching degree and image quality, which improves the accuracy and efficiency of video ringback tone cover screening, has a low implementation cost, and is conducive to improving the user's video ringback tone usage experience.

[0100] Specifically, in some embodiments, determining the subject matching degree between each candidate cover image and the description information includes:

[0101] Inputting the candidate cover image and the description information into the CLIP model;

[0102] The image-text similarity between the candidate cover image and the description information is calculated using the CLIP model, and the image-text similarity is determined as the topic matching degree.

[0103] In an embodiment of the present application, when determining the subject matching between the alternative cover image and the description information, the CLIP model can be used to match the two. The CLIP model (Contrastive Language–Image Pre-training) is a multimodal model proposed by OpenAI in 2021. It maps images and text to the same vector space through contrastive learning (Contrastive Learning), thereby achieving a joint understanding of images and text. In an embodiment of the present application, the alternative cover image and description information are input into the CLIP model, and the image-text similarity between the alternative cover image and the description information is calculated by the CLIP model, and then the calculated image-text similarity can be determined as the subject matching degree.

[0104] Specifically, in some embodiments, performing image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image includes:

[0105] Performing a clarity evaluation on the candidate cover image to obtain a clarity evaluation score corresponding to the candidate cover image;

[0106] Performing composition evaluation on the candidate cover images to obtain composition evaluation scores corresponding to the candidate cover images;

[0107] Performing an aesthetic evaluation on the candidate cover image to obtain an aesthetic evaluation score corresponding to the candidate cover image;

[0108] Determine a quality evaluation score corresponding to the candidate cover image based on at least one of the clarity evaluation score, the composition evaluation score, and the aesthetic evaluation score.

[0109] In an embodiment of the present application, the alternative cover images can be evaluated in terms of clarity, composition, and at least one of aesthetics to obtain corresponding evaluation scores, such as a clarity evaluation score, a composition evaluation score, and an aesthetics evaluation score. These evaluation scores are then weighted to obtain a quality evaluation score corresponding to the alternative cover image.

[0110] The following introduces and explains the specific process of determining the clarity evaluation score, composition evaluation score, and aesthetic evaluation score.

[0111] Specifically, in some embodiments, performing clarity evaluation on the candidate cover image to obtain a clarity evaluation score corresponding to the candidate cover image includes:

[0112] Convert the candidate cover image into a grayscale image and perform normalization processing to obtain a corresponding standard grayscale image;

[0113] Performing a two-dimensional discrete Fourier transform on the standard grayscale image to obtain a corresponding frequency domain representation;

[0114] Determine the center of the spectrum corresponding to the standard grayscale image, and determine a high-frequency region in the spectrum based on the center of the spectrum and a preset radius threshold; wherein the high-frequency region is a region where frequency domain points whose distance from the center of the spectrum is greater than the radius threshold are located;

[0115] determining high-frequency component energy according to the frequency domain representation corresponding to each of the frequency domain points in the high-frequency region, and determining total energy according to the frequency domain representation corresponding to all the frequency domain points in the spectrum;

[0116] The clarity evaluation score corresponding to the alternative cover image is determined based on a first ratio of the high-frequency component energy to the total energy; wherein the clarity evaluation score is positively correlated with the first ratio.

[0117] In the embodiment of the present application, clarity can reflect the sharpness of image edges and details, which is quantified by gradient information and frequency domain analysis. Technically, a frequency domain analysis quantization method can be used, which can better focus on details and edges. Specifically, in the embodiment of the present application, the alternative cover image can be converted into a grayscale image, and then normalized to obtain the corresponding standard grayscale image, which is recorded as I(x,y). Then, a two-dimensional discrete Fourier transform can be used to obtain the corresponding frequency domain representation, which is expressed as follows:

[0118]

[0119] Where F(u,v) represents the frequency domain representation, M×N is the image resolution, and (u,v) represents the frequency domain coordinates.

[0120] After Fourier transform, the zero frequency component is further moved to the center of the spectrum. In the frequency domain, the high frequency area usually refers to the frequency component far from the center of the spectrum. In the embodiment of the present application, the center of the spectrum corresponding to the standard grayscale image can be determined, and then the high frequency area in the spectrum can be determined based on the center of the spectrum and the preset radius threshold. For example, for example, the radius threshold near the center is defined as r cut , all satisfied The points belong to the high-frequency area, where u c =M / 2,v c = N / 2. Generally speaking, the radius threshold can be set to 10%-20% of the short side size of the image, and this application does not impose any limitation on this.

[0121] After distinguishing the high-frequency area, the high-frequency component energy can be calculated. Specifically, for each frequency domain point (u, v), the square of its amplitude |F(u, v)| is calculated. 2 =Re(F(u,v))2 +Im(F(u,v)) 2 According to the frequency domain representation corresponding to each frequency domain point in the high frequency area, the high frequency component energy can be determined, and the formula is expressed as: E high =∑ (u,v)∈高频区域 |F(u,v)| 2 Similarly, the total energy can be determined based on the frequency domain representation corresponding to all frequency domain points in the spectrum, and the formula is expressed as: E total =∑ (u,v) |F(u,v)| 2 Then, based on the first ratio of high-frequency energy to total energy, the clarity evaluation score corresponding to the candidate cover image can be determined. In the embodiment of the present application, the clarity evaluation score and the first ratio are positively correlated, and there is no limitation on the specific functional relationship between the two. For example, the first ratio can be directly determined as the clarity evaluation score, and the formula is expressed as:

[0122] Specifically, in some embodiments, performing composition evaluation on the candidate cover image to obtain a composition evaluation score corresponding to the candidate cover image includes:

[0123] Determining first coordinates of a plurality of golden section points in the candidate cover image;

[0124] Detecting the second coordinate of the subject center point in the candidate cover image using the Yolov8 model;

[0125] Calculating the Euclidean distances between the second coordinates and each of the first coordinates, and determining the minimum value of the Euclidean distances as a first distance value;

[0126] Determining the diagonal length of the candidate cover image as a second distance value, and determining a topic location score according to a second ratio of the first distance value to the second distance value; wherein the topic location score is negatively correlated with the second ratio;

[0127] Extracting the foreground and background regions of the candidate cover images using a semantic segmentation model;

[0128] Calculating a contrast difference between the foreground area and the background area, and determining a depth-of-field score based on the contrast difference; wherein the depth-of-field score is positively correlated with the contrast difference;

[0129] The subject position score and the depth of field score are weightedly summed to obtain the composition evaluation score corresponding to the alternative cover image.

[0130] In the embodiment of the present application, in terms of composition, the composition evaluation score can be determined by the rule of thirds and the subject prominence index. Specifically, the coordinates of several golden section points in the candidate cover image can be determined first. For example, four golden section points can be determined, namely, the upper left intersection point is The upper right intersection is The lower left intersection is The lower right intersection is These coordinates are marked as the first coordinates. The Yolov8 model can be used to detect the coordinates of the subject center point in the candidate cover image and mark its coordinates as the second coordinates (x obj ,y obj ). Then, the Euclidean distance between the second coordinate and the four first coordinates can be calculated using the formula:

[0131]

[0132] In this embodiment of the present application, the minimum value of the Euclidean distance can be determined as the first distance value d1, and the diagonal length of the candidate cover image can be determined as the second distance value. Based on the second ratio d2 of the first distance value and the second distance value, the subject position score can be determined. The subject position score and the second ratio are negatively correlated, and the specific functional relationship between the two is not limited. For example, the subject position score can be calculated using the following formula:

[0133] In the embodiment of the present application, a semantic segmentation model (such as DeepLabv3+) can be used to extract the foreground and background areas of the candidate cover image, and the contrast difference between the two is calculated to obtain a depth of field score. The depth of field score can be positively correlated with the contrast difference, such as the formula: Where u represents the mean brightness and σ is the standard deviation.

[0134] Then, the subject position score and the depth of field score can be weighted and summed to obtain the composition evaluation score corresponding to the alternative cover image.

[0135] Specifically, in some embodiments, performing an aesthetic evaluation on the candidate cover image to obtain an aesthetic evaluation score corresponding to the candidate cover image includes:

[0136] The candidate cover image is input into a pre-trained aesthetic scoring model, and the aesthetic evaluation score corresponding to the candidate cover image is predicted and output by the aesthetic scoring model.

[0137] In an embodiment of the present application, when determining the aesthetic evaluation scores corresponding to candidate cover images, the candidate cover images can be input into a pre-trained aesthetic scoring model to obtain the corresponding aesthetic evaluation scores. Here, the aesthetic scoring model can be a model built and trained based on a convolutional neural network for predicting aesthetic evaluation scores. The specific training process can be implemented with reference to existing technologies and is not limited in this application.

[0138] In some embodiments, the method further comprises:

[0139] Convert the candidate cover images into HSV space, determine multiple clusters corresponding to the candidate cover images through a clustering algorithm, and determine the HSV value corresponding to the cluster center of each cluster;

[0140] Determining a complementarity score based on the HSV value;

[0141] Determining a color score corresponding to the candidate cover image based on the complementarity score;

[0142] The aesthetic assessment score is modified based on the color score.

[0143] In the embodiment of the present application, color harmony can also be considered in terms of aesthetics, that is, the aesthetic evaluation score can be corrected by color harmony. In terms of color harmony, the image is first converted from RGB space to HSV space, and then the K-meas clustering algorithm is used to obtain M color cluster centers C1, C2, C3, ... C M The cluster center is the HSV value of the main color of the image, which is more consistent with human perception of color. Next, the complementarity score is calculated. Specifically, the complementarity of the color pair is calculated based on the color wheel model (such as the CIELab color wheel), and the HSV hue Hi of the main color is mapped to the color wheel angle θi = Hi × 179 / 360. In the color definition, the hue difference is 180°, so the score formula can be calculated as:

[0144]

[0145] In the embodiment of the present application, the color score can be expressed as:

[0146]

[0147] The technical solution of this application is introduced and explained below in conjunction with specific application examples.

[0148] A user has uploaded 10 Spring Festival-themed images, and the description for the video ringtone is "New Year's Eve Fireworks Show." First, the theme matching score for each image is calculated. For example, if Image A contains "fireworks" elements, the theme matching score is 0.92, while Image B contains "family portrait" elements, the theme matching score is 0.65. Next, the image quality can be assessed. For example, if Image A has a skewed composition, the quality assessment score is 0.75, while Image B has insufficient clarity, the quality assessment score is 0.68. A comprehensive assessment score is then determined for the candidate cover images. For example, the comprehensive assessment score for Image A is 0.7 × 0.92 + 0.3 × 0.75 = 0.824. After determining the comprehensive assessment scores for each image, the cover image with the highest comprehensive assessment score can be selected as the target cover image for the video ringtone. In other words, a threshold value, such as 0.85, can be set. If there is a picture that exceeds the threshold value, it can be determined as the target cover picture corresponding to the video ringtone. If not, AI enhancement can be started, and the alternative cover picture with the highest comprehensive evaluation score and descriptive information can be used as input. The picture can be adjusted through AI technology to obtain the target cover picture corresponding to the video ringtone.

[0149] The technical solutions in the embodiments of the present application have at least the following advantages:

[0150] 1. A scoring algorithm that integrates theme matching and image quality improves the accuracy and efficiency of video ringtone cover screening;

[0151] 2. AI generation is triggered only when the quality of the pre-selected materials is insufficient. Compared with the full generation solution, it consumes less GPU resources, shortens the response time, and improves efficiency.

[0152] Reference Figure 3 In an embodiment of the present application, a device for screening video ringback tone covers is also provided, comprising:

[0153] The acquisition unit 310 is configured to acquire a candidate image set for a video ringback tone corresponding to the user and description information of the video ringback tone; wherein the candidate image set includes a plurality of candidate cover images;

[0154] A matching unit 320 is configured to determine a subject matching degree between each candidate cover image and the description information;

[0155] An evaluation unit 330 is configured to perform image quality evaluation on each candidate cover image to obtain a quality evaluation score corresponding to each candidate cover image;

[0156] A summarizing unit 340 is configured to perform a weighted summation of the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image;

[0157] The comparison unit 350 is used to compare the comprehensive evaluation scores corresponding to the candidate cover pictures, and determine the candidate cover picture corresponding to the largest comprehensive evaluation score as the target cover picture corresponding to the video ringtone.

[0158] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0159] Reference Figure 4 , an embodiment of the present application provides an electronic device, including:

[0160] at least one processor 410;

[0161] at least one memory 420, for storing at least one program;

[0162] When at least one program is executed by at least one processor 410, the at least one processor 410 implements the above-mentioned method for screening video ringback tone covers.

[0163] Similarly, the contents of the above method embodiments are applicable to the present electronic device embodiment. The functions specifically implemented by the present electronic device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0164] An embodiment of the present application also provides a computer-readable storage medium, which stores a program executable by the processor 410. When the program executable by the processor 410 is executed by the processor 410, it is used to perform the above-mentioned method for screening video ringtone covers.

[0165] Similarly, the contents of the above method embodiments are applicable to the computer-readable storage medium embodiments. The functions specifically implemented by the computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0166] In some optional embodiments, the functions / operations mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the functions / operations involved, the two boxes shown in succession may actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logic flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0167] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0168] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0169] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0170] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0171] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0172] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0173] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0174] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for screening video ringback tone covers, characterized in that: The method comprises: Obtaining a candidate image set for a video ringback tone corresponding to the user and description information of the video ringback tone; wherein the candidate image set includes several candidate cover images; Determining a degree of subject matching between each of the candidate cover images and the description information; Performing image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image; Performing a weighted summation of the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image; The comprehensive evaluation scores corresponding to the candidate cover pictures are compared, and the candidate cover picture corresponding to the largest comprehensive evaluation score is determined as the target cover picture corresponding to the video ringtone.

2. The method for screening video ringback tone covers according to claim 1, characterized in that: Determining the subject matching degree between each candidate cover image and the description information includes: Inputting the candidate cover image and the description information into the CLIP model; The image-text similarity between the candidate cover image and the description information is calculated using the CLIP model, and the image-text similarity is determined as the topic matching degree.

3. The method for screening video ringback tone covers according to claim 1, characterized in that: The performing of image quality assessment on each candidate cover image to obtain a quality assessment score corresponding to each candidate cover image includes: Performing a clarity evaluation on the candidate cover image to obtain a clarity evaluation score corresponding to the candidate cover image; Performing composition evaluation on the candidate cover images to obtain composition evaluation scores corresponding to the candidate cover images; Performing an aesthetic evaluation on the candidate cover image to obtain an aesthetic evaluation score corresponding to the candidate cover image; Determine a quality evaluation score corresponding to the candidate cover image based on at least one of the clarity evaluation score, the composition evaluation score, and the aesthetic evaluation score.

4. The method for screening video ringback tone covers according to claim 3, characterized in that: The performing clarity evaluation on the candidate cover image to obtain a clarity evaluation score corresponding to the candidate cover image includes: Convert the candidate cover image into a grayscale image and perform normalization processing to obtain a corresponding standard grayscale image; Performing a two-dimensional discrete Fourier transform on the standard grayscale image to obtain a corresponding frequency domain representation; Determine the center of the spectrum corresponding to the standard grayscale image, and determine a high-frequency region in the spectrum based on the center of the spectrum and a preset radius threshold; wherein the high-frequency region is a region where frequency domain points whose distance from the center of the spectrum is greater than the radius threshold are located; determining high-frequency component energy according to the frequency domain representation corresponding to each of the frequency domain points in the high-frequency region, and determining total energy according to the frequency domain representation corresponding to all the frequency domain points in the spectrum; The clarity evaluation score corresponding to the alternative cover image is determined based on a first ratio of the high-frequency component energy to the total energy; wherein the clarity evaluation score is positively correlated with the first ratio.

5. The method for screening video ringback tone covers according to claim 3, characterized in that: The performing composition evaluation on the candidate cover image to obtain a composition evaluation score corresponding to the candidate cover image includes: Determining first coordinates of a plurality of golden section points in the candidate cover image; Detecting the second coordinate of the subject center point in the candidate cover image using the Yolov8 model; Calculating the Euclidean distances between the second coordinates and each of the first coordinates, and determining the minimum value of the Euclidean distances as a first distance value; Determining the diagonal length of the candidate cover image as a second distance value, and determining a topic location score according to a second ratio of the first distance value to the second distance value; wherein the topic location score is negatively correlated with the second ratio; Extracting the foreground and background regions of the candidate cover images using a semantic segmentation model; Calculating a contrast difference between the foreground area and the background area, and determining a depth-of-field score based on the contrast difference; wherein the depth-of-field score is positively correlated with the contrast difference; The subject position score and the depth of field score are weightedly summed to obtain the composition evaluation score corresponding to the candidate cover image.

6. The method for screening video ringback tone covers according to claim 3, characterized in that: The performing aesthetic evaluation on the candidate cover images to obtain aesthetic evaluation scores corresponding to the candidate cover images includes: The candidate cover image is input into a pre-trained aesthetic scoring model, and the aesthetic evaluation score corresponding to the candidate cover image is predicted and output by the aesthetic scoring model.

7. The method for screening video ringback tone covers according to claim 6, characterized in that: The method further comprises: Convert the candidate cover images into HSV space, determine multiple clusters corresponding to the candidate cover images through a clustering algorithm, and determine the HSV value corresponding to the cluster center of each cluster; Determining a complementarity score based on the HSV value; Determining a color score corresponding to the candidate cover image based on the complementarity score; The aesthetic assessment score is modified based on the color score.

8. A device for screening video ringback tone covers, characterized in that: The device comprises: An acquiring unit, configured to acquire a candidate picture set of a video ringback tone corresponding to a user and description information of the video ringback tone; wherein the candidate picture set includes a plurality of candidate cover pictures; A matching unit, configured to determine a degree of subject matching between each candidate cover image and the description information; An evaluation unit, configured to perform image quality evaluation on each candidate cover image to obtain a quality evaluation score corresponding to each candidate cover image; a summarizing unit, configured to perform a weighted summation of the subject matching degree and the quality assessment score corresponding to the candidate cover image to obtain a comprehensive assessment score corresponding to the candidate cover image; The comparison unit is used to compare the comprehensive evaluation scores corresponding to the candidate cover pictures, and determine the candidate cover picture corresponding to the largest comprehensive evaluation score as the target cover picture corresponding to the video ringtone.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for screening video ringback tone covers as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to implement a method for screening video ringback tone covers as described in any one of claims 1-7 when executed by the processor.

Images