Video ringback tone multi-dimensional review and storage method and device, electronic equipment and medium

Abstract

The application discloses a video color ring multi-dimension auditing and warehousing method and device, electronic equipment and medium, comprising: preprocessing a target video color ring to obtain media information, a background sound file, a plurality of target I frames and target hash values corresponding to the target I frames; performing picture score auditing on the target video color ring according to the media information, the target I frames and the target hash values; performing picture duplication checking auditing on the target video color ring according to the target hash values; performing multi-level risk control auditing on the target video color ring according to the background sound file; and when the target video color ring passes the picture score auditing, the picture duplication checking auditing and the multi-level risk control auditing, performing transcoding storage on the target video color ring according to the media information. The application improves the efficiency and reliability of video color ring auditing and warehousing, and can be widely applied to the technical field of video processing.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a method, apparatus, electronic device, and medium for multi-dimensional review and database entry of video ringback tones. Background Technology

[0002] With the widespread adoption of 4G / 5G networks and the extensive use of smartphones and mobile internet applications, video ringback tones, as a new and personalized communication method, have rapidly attracted the attention and use of a large number of users. Video ringback tones not only provide users with a richer and more vivid call experience, but also offer businesses and individuals a platform to showcase their brand image and creative content.

[0003] To ensure smooth playback, visual appeal, and content compliance of video ringback tones, developing an efficient and intelligent video ringback tone review system is of paramount importance.

[0004] The existing video ringback tone review system only has risk control review and video transcoding processes. The clarity and richness of the video are subjectively judged by the editors. Furthermore, with a large number of online video ringback tones in the database, duplicate video materials are easily purchased, so deduplication is required. However, taking 200,000 video ringback tones as an example, each video ringback tone plays for about 25 seconds. If deduplication is done manually, it would take 200,000 * 25 / 60 / 60 / 8 ≈ 170 days just to review all these videos, which is a huge workload.

[0005] Furthermore, in existing video ringback tone review systems, the video preprocessing for each part is mostly performed independently. Ultra-high-definition video ringback tones have high resolution and require a large amount of memory; if each stage processes the video, it will consume a significant amount of time and storage space. Modifying the preprocessing workflow would require individual modifications to each stage, resulting in a substantial development workload.

[0006] In summary, the existing video ringback tone review system has the following drawbacks:

[0007] 1) The clarity and richness of the video footage are subjectively judged by the editors, which is prone to errors;

[0008] 2) The number of existing videos is large, and manual plagiarism checking is inefficient;

[0009] 3) Each stage preprocesses the video independently, and the same preprocessing process needs to be executed multiple times, which takes a long time;

[0010] 4) Each stage preprocesses the video independently, and the preprocessing results are stored independently for each stage, which occupies a large amount of space;

[0011] 5) Each stage of preprocessing is implemented independently, and the underlying implementation is not unified. If the review standards need to be modified, each system must be modified independently, resulting in a large amount of development work.

[0012] The above problems urgently need to be addressed.

[0013] Terminology Explanation:

[0014] Video ringback tones: Video ringback tones are a unique feature of VoLTE, evolving from "listened ringback tones" to "viewed ringback tones." When a user makes a voice or video call under the VoLTE network, the video content they see before the call is connected is called a "video ringback tone."

[0015] FFmpeg: An open-source computer program that can be used to record, convert, and stream digital audio and video.

[0016] Resolution: Also known as resolution or image resolution, it refers to the ability of a measurement or display system to distinguish details. It determines the fineness of details in a bitmap image and is usually used as a measure of the detail and sharpness displayed in images, videos, or display devices.

[0017] Bit rate (also called bit rate, variable R) in the telecommunications and computing fields refers to the number of bits transmitted or processed per unit of time, measured in bps (bits per second).

[0018] Frame rate: The frequency (rate) at which bitmap images, measured in frames, appear continuously on a display screen. This term also applies to film and cameras, computer graphics, and motion capture systems. Frame rate can also be called frame frequency and is expressed in Hertz (Hz).

[0019] H.264: Also known as H.264, or MPEG-4 Part 10, it is a highly compressed digital video codec standard proposed by the Joint Video Team (JVT), which is jointly formed by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG).

[0020] I-frame: Also known as an intra picture, an I-frame is usually the first frame of each GOP (a video compression technique used by MPEG). It is moderately compressed and serves as a reference point for random access, and can be treated as an image.

[0021] Grayscale image: also known as gray level image. It divides the range of white and black into several levels according to a logarithmic relationship, called grayscale. Grayscale is divided into 256 levels.

[0022] Discrete Cosine Transform: A transform related to the Fourier Transform, similar to the Discrete Fourier Transform (DFT), but using only real numbers.

[0023] Hamming distance is a concept used in data transmission error control coding. It represents the number of distinct characters at corresponding positions in two strings (of the same length). Let d(x,y) represent the Hamming distance between two characters x and y. The distance is calculated by performing an XOR operation on the two strings and counting the number of 1s in the result. Summary of the Invention

[0024] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0025] Therefore, one objective of this invention is to provide a multi-dimensional review and database entry method for video ringback tones, which improves the efficiency and reliability of video ringback tone review and database entry.

[0026] Another objective of this invention is to provide a multi-dimensional review and database entry device for video ringback tones.

[0027] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include:

[0028] On one hand, embodiments of the present invention provide a method for multi-dimensional review and database entry of video ringback tones, including the following steps:

[0029] The target video ringback tone is preprocessed to obtain media information, background sound file, multiple target I-frames, and target hash value corresponding to each target I-frame;

[0030] The target video ringback tone is scored and reviewed based on the media information, the target I-frame, and the target hash value.

[0031] The target video ringback tone is checked for duplicate content based on the target hash value.

[0032] The target video ringback tone is subject to multi-level risk control review based on the background audio file.

[0033] Once the target video ringback tone passes the image scoring review, image duplication check review, and multi-level risk control review, the target video ringback tone is transcoded and stored according to the media information.

[0034] Furthermore, in one embodiment of the present invention, the preprocessing of the target video ringback tone to obtain media information, background audio files, multiple target I-frames, and target hash values ​​corresponding to each target I-frame specifically includes:

[0035] Use FFmpeg to read the media information of the target video ringback tone;

[0036] Use FFmpeg to separate the background audio file of the target video;

[0037] Use FFmpeg to separate multiple target I-frames of the target video;

[0038] The target I-frame is converted into a grayscale image, and a pixel grayscale matrix of each target I-frame is generated based on the grayscale image.

[0039] A two-dimensional discrete cosine transform is performed on the pixel grayscale matrix to obtain the pixel transformation matrix of each target I-frame;

[0040] The pixel transformation matrix is ​​truncated according to a preset pixel window to obtain the pixel truncating matrix of each target I-frame;

[0041] Determine the average value of all elements in the cropping matrix for each pixel, and determine whether each element is greater than the average value.

[0042] The elements greater than the average value of the elements are assigned a value of 1, and the elements less than or equal to the average value of the elements are assigned a value of 0, thus obtaining the pixel assignment matrix of each target I-frame;

[0043] The elements of the pixel assignment matrix are read sequentially and concatenated in order to obtain the target hash value corresponding to each target I-frame.

[0044] Furthermore, in one embodiment of the present invention, the step of performing a picture scoring review of the target video ringback tone based on the media information, the target I-frame, and the target hash value specifically includes:

[0045] The resolution, bitrate, and frame rate of the target video ringback tone are determined based on the media information.

[0046] Determine the number of colors in each target I-frame, and determine the average color value of the target video ringback tone based on the number of colors;

[0047] The first Hamming distance between the target hash values ​​corresponding to two adjacent target I-frames is determined, and the number of adjacent target I-frame pairs with the first Hamming distance greater than or equal to a preset distance threshold is counted. Then, the frame change rate of the target video ringback tone is determined based on the ratio of the number of adjacent target I-frame pairs to the total number of adjacent target I-frame pairs.

[0048] The resolution, bitrate, frame rate, average color value, and change rate between consecutive frames are weighted and summed according to preset weighting coefficients to obtain the picture score value of the target video ringback tone.

[0049] When the image score is greater than or equal to the preset image score threshold, the image score is deemed to have passed the review.

[0050] If the image score is less than the image score threshold, the image score review is deemed unsuccessful.

[0051] Furthermore, in one embodiment of the present invention, the step of performing image deduplication verification on the target video ringback tone based on the target hash value specifically includes:

[0052] Determine the first hash value corresponding to multiple I frames of each first video ringback tone in the preset video ringback tone library;

[0053] The similarity between the target video ringback tone and each of the first video ringback tones is determined based on the target hash value and the first hash value;

[0054] If there is no first video ringback tone with a similarity greater than or equal to the preset similarity threshold that is the target video ringback tone, the screen deduplication review is deemed passed.

[0055] If there is a first video ringback tone with a similarity to the target video ringback tone that is greater than or equal to the similarity threshold, the video deduplication check is determined to be unsuccessful.

[0056] Furthermore, in one embodiment of the present invention, determining the similarity between the target video ringback tone and each of the first video ringback tones based on the target hash value and the first hash value specifically includes:

[0057] Determine the second Hamming distance between the target hash value and the first hash value;

[0058] When the second Hamming distance is less than a preset distance threshold, it is determined that the target video ringback tone and the corresponding first video ringback tone have a similar I-frame;

[0059] The number of similar I-frame pairs between the target video ringback tone and each of the first video ringback tones is counted, and the similarity between the target video ringback tone and each of the first video ringback tones is determined based on the ratio of the number of similar I-frame pairs to the total number of target I-frames.

[0060] Furthermore, in one embodiment of the present invention, the step of performing multi-level risk control review on the target video ringback tone based on the background audio file specifically includes:

[0061] The background audio file is subjected to speech recognition to obtain target text information, and the video source file of the target video ringtone is obtained;

[0062] The target text information is subject to text review;

[0063] The video source file was subject to content review.

[0064] When the target text information passes text review and the video source file passes content review, the multi-level risk control review is confirmed to be successful.

[0065] Furthermore, in one embodiment of the present invention, the step of transcoding and storing the target video ringback tone according to the media information specifically includes:

[0066] Based on the media information, the video source file of the target video ringback tone is transcoded into multiple video ringback tone files with preset format and preset resolution using FFmpeg;

[0067] The video ringback tone file is distributed to a preset video ringback tone library for storage.

[0068] On the other hand, embodiments of the present invention provide a multi-dimensional review and database entry device for video ringback tones, including:

[0069] The preprocessing module is used to preprocess the target video ringback tone to obtain media information, background sound file, multiple target I-frames, and target hash value corresponding to each target I-frame;

[0070] The image scoring and review module is used to score and review the target video ringback tone based on the media information, the target I-frame, and the target hash value.

[0071] The image deduplication verification module is used to perform image deduplication verification on the target video ringback tone based on the target hash value.

[0072] A multi-level risk control review module is used to perform multi-level risk control review on the target video ringback tone based on the background sound file;

[0073] The transcoding and storage module is used to transcode and store the target video ringback tone according to the media information when the target video ringback tone passes the image scoring review, image deduplication review, and multi-level risk control review.

[0074] On the other hand, embodiments of the present invention provide an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, it implements the video ringback tone multi-dimensional review and database entry method described above.

[0075] On the other hand, embodiments of the present invention also provide a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the video ringback tone multi-dimensional review and database entry method as described above.

[0076] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention:

[0077] This invention preprocesses the target video ringback tone to obtain media information, background audio files, multiple target I-frames, and target hash values ​​corresponding to each target I-frame. Based on the media information, target I-frames, and target hash values, the target video ringback tone undergoes image scoring and review. Based on the target hash values, image deduplication is checked. Based on the background audio files, multi-level risk control is implemented. Once the target video ringback tone passes the image scoring, deduplication, and multi-level risk control reviews, it is transcoded and stored based on the media information. This invention preprocesses and extracts the media information, background audio files, multiple target I-frames, and target hash values ​​corresponding to each target I-frame for the target video ringback tone. The extracted information is then used for subsequent image scoring, deduplication, and multi-level risk control reviews, reducing repetitive video processing and manual review workload, and improving the efficiency and reliability of video ringback tone review and storage. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0079] Figure 1 This is a flowchart illustrating one step of the multi-dimensional review and database entry method for video ringback tones provided in this embodiment of the invention.

[0080] Figure 2 This is a schematic diagram of the data flow of the video ringback tone multi-dimensional review and database entry method provided in this embodiment of the invention;

[0081] Figure 3 A flowchart of step S101 provided in an embodiment of the present invention;

[0082] Figure 4A flowchart of step S102 provided in an embodiment of the present invention;

[0083] Figure 5 A flowchart of step S103 provided in an embodiment of the present invention;

[0084] Figure 6 A flowchart of step S1032 provided in an embodiment of the present invention;

[0085] Figure 7 A flowchart of step S104 provided in an embodiment of the present invention;

[0086] Figure 8 A flowchart of step S105 provided in an embodiment of the present invention;

[0087] Figure 9 This is a schematic diagram of the structure of the video ringback tone multi-dimensional review and storage device provided in an embodiment of the present invention;

[0088] Figure 10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;

[0089] Figure 11 This is a schematic diagram of the structure of the storage medium provided in an embodiment of the present invention. Detailed Implementation

[0090] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. It should be noted that although functional modules are divided in the system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system schematic diagram or the order in the flowchart. The step numbers in the following embodiments are only set for ease of explanation and do not limit the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0091] In the description of this invention, "multiple" means two or more. The use of "first" and "second" is for distinguishing technical features only and should not be construed as indicating or implying relative importance, the number of indicated technical features, or the order of the indicated technical features. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0092] The video ringback tone multi-dimensional review and database entry method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, set-top box, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the video ringback tone multi-dimensional review and database entry method, but is not limited to the above forms.

[0093] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0094] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0095] like Figure 1 The diagram shows a flowchart of one step of the multi-dimensional review and database entry method for video ringback tones provided in an embodiment of the present invention. (Refer to...) Figure 1 This invention provides a method for multi-dimensional review and database entry of video ringback tones, specifically including the following steps:

[0096] S101. Preprocess the target video ringback tone to obtain media information, background sound file, multiple target I-frames, and target hash value corresponding to each target I-frame;

[0097] S102. Based on media information, target I-frame, and target hash value, perform image scoring and review of the target video ringback tone;

[0098] S103. Perform image deduplication verification on the target video ringback tone based on the target hash value;

[0099] S104. Conduct multi-level risk control review of the target video ringback tones based on the background sound file;

[0100] S105. When the target video ringback tone passes the image scoring review, image duplication check review, and multi-level risk control review, the target video ringback tone is transcoded and stored according to the media information.

[0101] like Figure 2The diagram illustrates the data flow of the multi-dimensional review and database entry method for video ringback tones provided in this embodiment of the invention. This embodiment independently develops a fully automated video ringback tone image scoring system and a video ringback tone image deduplication system, and performs unified preprocessing of video ringback tones, thereby unifying the entire process of video ringback tone review and database entry. The main steps include: preprocessing the video ringback tones using the video ringback tone preprocessing system; performing image scoring review of the video ringback tones using the video ringback tone image scoring system; performing image deduplication review of the video ringback tones using the video ringback tone image deduplication system; performing multi-level risk control review of the video ringback tones using the video ringback tone multi-level risk control system; converting the video ringback tones into video ringback tones files of different formats using the video ringback tones transcoding system; and storing the transcoded video ringback tones files using the video ringback tones distribution system. The method steps of this embodiment of the invention will be further explained below.

[0102] like Figure 3 The diagram shown is a flowchart of step S101 provided in an embodiment of the present invention. (Refer to...) Figure 3 As an optional implementation, the target video ringback tone is preprocessed to obtain media information, background audio files, multiple target I-frames, and target hash values ​​corresponding to each target I-frame, specifically including:

[0103] S1011. Use FFmpeg to read the media information of the target video ringback tone;

[0104] S1012. Use FFmpeg to separate the background audio file of the target video;

[0105] S1013. Use FFmpeg to separate multiple target I-frames of the target video;

[0106] S1014. Convert the target I-frame into a grayscale image, and generate the pixel grayscale matrix of each target I-frame based on the grayscale image.

[0107] S1015. Perform a two-dimensional discrete cosine transform on the pixel grayscale matrix to obtain the pixel transformation matrix of each target I-frame.

[0108] S1016. Extract the pixel transformation matrix according to the preset pixel window to obtain the pixel extraction matrix of each target I-frame.

[0109] S1017. Determine the average value of all elements in the cropping matrix for each pixel, and determine whether each element is greater than the average value.

[0110] S1018. Assign a value of 1 to elements that are greater than the average value of the elements, and assign a value of 0 to elements that are less than or equal to the average value of the elements, to obtain the pixel assignment matrix of each target I-frame.

[0111] S1019. Read each element of the pixel assignment matrix in sequence and concatenate them in order to obtain the target hash value corresponding to each target I-frame.

[0112] Specifically, the preprocessing process for video ringback tones using the video ringback tone preprocessing system is as follows:

[0113] 1) Use FFmpeg to read video ringback tone media information. The media information mainly includes video format, resolution (ppi), bitrate (br), frame rate (fps), etc.

[0114] 2) Use FFmpeg to separate and save the background audio from the video;

[0115] 3) Use FFmpeg to separate the video into N H264 I-frame images and save them;

[0116] 4) Convert N I-frame images into N 32*32 grayscale images;

[0117] 5) Save the gray values ​​of N grayscale images into N 32*32 matrices A;

[0118] 6) Calculate the two-dimensional discrete cosine transform of N matrices A to obtain N 32*32 matrices B; the formula for the two-dimensional discrete cosine transform is:

[0119]

[0120] Where F(μ,v) represents the elements in the transformed coefficient matrix, indicating the weights of different frequency components; f(x,y) is the gray value at pixel (x,y) in the original image; M and N represent the length and width of the matrix; C(μ) and C(v) are orthogonal normalization coefficients, where C(μ) and C(v) are the coefficients when μ,v = 0. Otherwise, C(μ) = C(v) = 1;

[0121] 7) Extract the values ​​from the top-left 8*8 region of N matrices B to obtain N 8*8 matrices C;

[0122] 8) Calculate the average of the 64 elements of each matrix C, denoted as average;

[0123] 9) Based on the order from left to right and from top to bottom, determine whether the 64 elements of matrix C are greater than the average. If they are, record them as 1; otherwise, record them as 0.

[0124] 10) Concatenate the 0s and 1s of the above records in sequence to form a 64-bit hash value in the form of “0011011011100010110110011001100000001010000001100000010000011”, denoted as hash, and save it to the database;

[0125] 11) After processing, a video ringback tone has one background sound, N key I-frame images and N hashes.

[0126] like Figure 4 The diagram shown is a flowchart of step S102 provided in an embodiment of the present invention. (Refer to...) Figure 4 As an optional implementation, the target video ringback tone is further evaluated based on media information, the target I-frame, and the target hash value, specifically including:

[0127] S1021. Determine the resolution, bitrate, and frame rate of the target video ringback tone based on the media information;

[0128] S1022. Determine the number of colors in each target I-frame, and determine the average color value of the target video ringback tone based on the number of colors;

[0129] S1023. Determine the first Hamming distance between the target hash values ​​corresponding to two adjacent target I-frames, and count the number of adjacent target I-frame pairs whose first Hamming distance is greater than or equal to a preset distance threshold. Then, determine the frame change rate of the target video ringback tone based on the ratio of the number of adjacent target I-frame pairs to the total number of adjacent target I-frame pairs.

[0130] S1024. The resolution, bit rate, frame rate, average color value and the change rate between consecutive frames are weighted and summed according to the preset weight coefficients to obtain the picture score of the target video ringback tone.

[0131] S1025. When the image score is greater than or equal to the preset image score threshold, the image score is deemed to have passed the review.

[0132] S1026. When the image score is less than the image score threshold, the image score review is deemed unsuccessful.

[0133] Specifically, the process for reviewing and approving video ringback tones through the video ringback tone scoring system is as follows:

[0134] 1) Read the video resolution (ppi), bitrate (br), and frame rate (fps) from the video ringback tone preprocessing system;

[0135] 2) Read N I-frame images from the video ringback tone preprocessing system;

[0136] 3) Count the number of colors in each I-frame image, denoted as num_color, and calculate the average color value of N I-frames as num_color_average. Then we have: This value reflects the richness of colors in the video;

[0137] 4) Read the hash corresponding to N I frames of the video from the video ringback tone preprocessing system, and calculate the Hamming distance between the hash values ​​of the two consecutive frames: hamming_distance = bit_count(hash^hash_next), where hash_next represents the hash value of the next adjacent frame, "^" represents the XOR operation, and bit_count(x) represents the number of 1s in the binary representation of a number.

[0138] 5) Count the number of values ​​where hamming_distance ≥ 12, and denote it as hamming_num;

[0139] 6) Calculate the change score between the previous and next frames: hamming_s = hamming_num / (N-1);

[0140] 7) Calculate the screen score s = ppi*w1 / ppi_max + br*w2 / br_max + fps*w3 / fps_max + num_color_average*w4 / num_color_max + hamming_s*w5;

[0141] Where w1, w2, w3, w4, and w5 are the weight values ​​of ppi, br, fps, num_color_average, and hamming_s, respectively, satisfying w1+w2+w3+w4+w5=1; ppi_max is the preset resolution value of the ultra-high-definition lossless video. If the actual ppi is greater than ppi_max, then let ppi*w1 / ppi_max=w1; br_max is the preset video bitrate value of the ultra-high-definition lossless video. If the actual br is greater than br_max, then let br*w2 / br_max=w2; fps_max is the preset frame rate value of the ultra-high-definition lossless video. If the actual fps is greater than fps_max, then let fps*w3 / fps_max=w3; num_color_max is the preset color quantity value of the ultra-high-definition lossless video. If the actual num_color_average is greater than num_color_max, then let num_color_average*w4 / num_color_max=w4;

[0142] 8) Determine whether the video score s is greater than or equal to the preset video score threshold. If it is, the video score will pass the review; otherwise, it will fail.

[0143] like Figure 5 The diagram shown is a flowchart of step S103 provided in an embodiment of the present invention. (Refer to...) Figure 5 As an optional implementation, the target video ringback tone is further subjected to image deduplication verification based on the target hash value, which specifically includes:

[0144] S1031. Determine the first hash value corresponding to multiple I frames of each first video ringback tone in the preset video ringback tone library;

[0145] S1032. Determine the similarity between the target video ringback tone and each first video ringback tone based on the target hash value and the first hash value;

[0146] S1033. If there is no first video ringback tone with a similarity greater than or equal to the preset similarity threshold with the target video ringback tone, the screen deduplication review is deemed passed.

[0147] S1034. If there is a first video ringback tone with a similarity greater than or equal to the similarity threshold with the target video ringback tone, the image plagiarism check is determined to be unsuccessful.

[0148] like Figure 6 The diagram shown is a flowchart of step S1032 provided in an embodiment of the present invention. (Refer to...) Figure 6 As an optional implementation, the similarity between the target video ringback tone and each of the first video ringback tones is determined based on the target hash value and the first hash value, specifically including:

[0149] S10321. Determine the second Hamming distance between the target hash value and the first hash value;

[0150] S10322. When the second Hamming distance is less than the preset distance threshold, it is determined that the target video ringback tone and the corresponding first video ringback tone have similar I-frames.

[0151] S10323. Count the number of similar I-frame pairs between the target video ringback tone and each first video ringback tone, and determine the similarity between the target video ringback tone and each first video ringback tone based on the ratio of the number of similar I-frame pairs to the total number of target I-frames.

[0152] Specifically, the process for reviewing and verifying video ringback tones using a video ringback image plagiarism detection system is as follows:

[0153] 1) Record the video to be checked as V1, and the video in the video library as V2;

[0154] 2) Read the hashes corresponding to the N I-frames of video V1 from the video ringback tone preprocessing system and compare them with the hash value hash_v2 of the I-frames of each video V2;

[0155] 3) Calculate the Hamming distance between the two frames' hash and hash_v2: hamming_distance = bit_count(hash^hash_v2), where "^" represents the XOR operation, and bit_count(x) represents counting the number of 1s in the binary representation of a number;

[0156] 4) If a hash of V1 can find at least one hash_v2 in a V2 that matches hamming_distance < 12, then this frame has similar frames.

[0157] 5) The number of similar frames in the target video is counted as similar_num. Then the similarity between V1 and a V2 is similar = similar_num / N.

[0158] 6) If similarity ≥ 50%, it means that more than 50% of the content in V1 is similar to V2, then we call V2 a similar video to V1;

[0159] 7) If V1 can find a similar video, the screen plagiarism check will fail; otherwise, it will pass.

[0160] like Figure 7 The diagram shown is a flowchart of step S104 provided in an embodiment of the present invention. (Refer to...) Figure 7 As an optional implementation, the target video ringback tone is subject to multi-level risk control review based on the background audio file, which specifically includes:

[0161] S1041. Perform speech recognition on the background audio file to obtain the target text information, and obtain the video source file of the target video ringback tone;

[0162] S1042. Conduct text review of the target text information;

[0163] S1043. Conduct content review of video source files;

[0164] S1044. When the target text information passes text review and the video source file passes content review, the multi-level risk control review is confirmed to be passed.

[0165] Specifically, the process for multi-level risk control review of video ringback tones through the multi-level risk control system is as follows:

[0166] 1) Read the background audio file and video source file containing the approved video from the video ringback tone preprocessing system;

[0167] 2) Convert background audio into text using an AI text-to-speech interface;

[0168] 3) Use the text risk control interface to determine whether the text is compliant; videos that do not comply with the rules will not pass the review.

[0169] 4) If the text risk control passes, the video source file will be sent to a third-party authoritative agency for risk control review (usually Xinhua News Agency). If it fails, the video review will also fail.

[0170] 5) If the third-party review is approved, it will be submitted to the internal manual review. If it fails, the video review will also fail. Otherwise, it will pass.

[0171] like Figure 8 The diagram shown is a flowchart of step S105 provided in an embodiment of the present invention. (Refer to...) Figure 8 As an optional implementation, the target video ringback tone is transcoded and stored based on media information, specifically including:

[0172] S1051. Based on the media information, use FFmpeg to transcode the target video ringback tone video source file into multiple video ringback tone files with preset format and preset resolution;

[0173] S1052. Distribute the video ringback tone file to the preset video ringback tone library for storage.

[0174] Specifically, the video source file and media information of the target video ringback tone are read from the video ringback tone preprocessing system. Based on the media information, the video ringback tone transcoding system uses FFmpeg to transcode the target video ringback tone into video ringback tone files of different formats and resolutions. The transcoded video ringback tone files are saved through the video ringback tone distribution system. After the video ringback tone distribution system notifies the storage system that the saving was successful, the storage system will change the status of the video to successful storage.

[0175] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention extract the media information, background sound file, multiple target I-frames, and target hash values ​​corresponding to each target I-frame of the target video ringback tone through preprocessing. The extracted information is then used for subsequent image scoring review, image deduplication review, and multi-level risk control review, which reduces repetitive work in video processing and the workload of manual review, and improves the efficiency and reliability of video ringback tone review and database entry.

[0176] Compared with the prior art, the embodiments of the present invention also have the following advantages:

[0177] 1) We independently developed a fully automatic video quality scoring system to avoid errors caused by subjective judgment during video editing;

[0178] 2) Independently developed fully automatic video plagiarism checker, greatly reducing the workload of editors;

[0179] 3) Unified video preprocessing: the same process only needs to be executed once, saving time;

[0180] 4) Unified video preprocessing; the preprocessing results are saved only once, saving space.

[0181] 5) Unified video preprocessing with a unified underlying implementation. If the review standards need to be modified, only the preprocessing system needs to be modified, saving development workload.

[0182] like Figure 9 The diagram shown is a structural schematic of the video ringback tone multi-dimensional review and database entry device provided in an embodiment of the present invention. (Refer to...) Figure 9 This invention provides a multi-dimensional review and database entry device for video ringback tones, comprising:

[0183] The preprocessing module is used to preprocess the target video ringback tone to obtain media information, background sound files, multiple target I-frames, and the target hash value corresponding to each target I-frame;

[0184] The image scoring and review module is used to score and review the target video ringback tones based on media information, target I-frames, and target hash values.

[0185] The image deduplication verification module is used to perform image deduplication verification on target video ringback tones based on the target hash value;

[0186] The multi-level risk control review module is used to conduct multi-level risk control review of the target video ringback tone based on the background sound file;

[0187] The transcoding and storage module is used to transcode and store the target video ringback tone based on media information when the target video ringback tone passes the image scoring review, image duplication review, and multi-level risk control review.

[0188] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0189] This invention also provides an electronic device, comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned multi-dimensional review and database entry method for video ringback tones. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0190] like Figure 10 The diagram shown is a hardware structure schematic of an electronic device provided in an embodiment of the present invention. (Refer to...) Figure 10 This invention provides an electronic device, comprising:

[0191] The processor 1001 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0192] The memory 1002 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1002 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1002 and is called and executed by the processor 1001 to execute the video ringback tone multi-dimensional review and database entry method of the embodiments of this invention.

[0193] Input / output interface 1003 is used to implement information input and output;

[0194] The communication interface 1004 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0195] Bus 1005 transmits information between various components of the device (e.g., processor 1001, memory 1002, input / output interface 1003, and communication interface 1004);

[0196] The processor 1001, memory 1002, input / output interface 1003 and communication interface 1004 are connected to each other within the device via bus 1005.

[0197] like Figure 11 The diagram shown is a structural schematic of the storage medium provided in an embodiment of the present invention. (Refer to...) Figure 11 The present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs 1101, which can be executed by one or more processors to implement the above-mentioned multi-dimensional review and storage method for video ringback tones.

[0198] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0199] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform... Figure 1 The method shown.

[0200] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0201] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0202] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0203] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0204] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or, if necessary, processing in other suitable ways, and then stored in computer memory.

[0205] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0206] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0207] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0208] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for multi-dimensional review and database entry of video ringback tones, characterized in that, Includes the following steps: The target video ringback tone is preprocessed to obtain media information, background sound file, multiple target I-frames, and target hash value corresponding to each target I-frame; The target video ringback tone is scored and reviewed based on the media information, the target I-frame, and the target hash value. The target video ringback tone is checked for duplicate content based on the target hash value. The target video ringback tone is subject to multi-level risk control review based on the background audio file. Once the target video ringback tone passes the image scoring review, image duplication check review, and multi-level risk control review, the target video ringback tone is transcoded and stored according to the media information; The step of scoring and reviewing the target video ringback tone based on the media information, the target I-frame, and the target hash value specifically includes: The resolution, bitrate, and frame rate of the target video ringback tone are determined based on the media information. Determine the number of colors in each target I-frame, and determine the average color value of the target video ringback tone based on the number of colors; The first Hamming distance between the target hash values ​​corresponding to two adjacent target I-frames is determined, and the number of adjacent target I-frame pairs with the first Hamming distance greater than or equal to a preset distance threshold is counted. Then, the frame change rate of the target video ringback tone is determined based on the ratio of the number of adjacent target I-frame pairs to the total number of adjacent target I-frame pairs. The resolution, bitrate, frame rate, average color value, and change rate between consecutive frames are weighted and summed according to preset weighting coefficients to obtain the picture score value of the target video ringback tone. When the image score is greater than or equal to the preset image score threshold, the image score is deemed to have passed the review. If the image score is less than the image score threshold, the image score review is deemed unsuccessful.

2. The method for multi-dimensional review and database entry of video ringback tones according to claim 1, characterized in that, The preprocessing of the target video ringback tone to obtain media information, background audio files, multiple target I-frames, and target hash values ​​corresponding to each target I-frame specifically includes: Use FFmpeg to read the media information of the target video ringback tone; Use FFmpeg to separate the background audio file of the target video; Use FFmpeg to separate multiple target I-frames of the target video; The target I-frame is converted into a grayscale image, and a pixel grayscale matrix of each target I-frame is generated based on the grayscale image. A two-dimensional discrete cosine transform is performed on the pixel grayscale matrix to obtain the pixel transformation matrix of each target I-frame; The pixel transformation matrix is ​​truncated according to a preset pixel window to obtain the pixel truncating matrix of each target I-frame; Determine the average value of all elements in the cropping matrix for each pixel, and determine whether each element is greater than the average value. The elements greater than the average value of the elements are assigned a value of 1, and the elements less than or equal to the average value of the elements are assigned a value of 0, thus obtaining the pixel assignment matrix of each target I-frame; The elements of the pixel assignment matrix are read sequentially and concatenated in order to obtain the target hash value corresponding to each target I-frame.

3. The method for multi-dimensional review and database entry of video ringback tones according to claim 1, characterized in that, The step of performing image deduplication verification on the target video ringback tone based on the target hash value specifically includes: Determine the first hash value corresponding to multiple I frames of each first video ringback tone in the preset video ringback tone library; The similarity between the target video ringback tone and each of the first video ringback tones is determined based on the target hash value and the first hash value; If there is no first video ringback tone with a similarity greater than or equal to the preset similarity threshold that is the target video ringback tone, the screen deduplication review is deemed passed. If there is a first video ringback tone with a similarity to the target video ringback tone that is greater than or equal to the similarity threshold, the video deduplication check is determined to be unsuccessful.

4. The method for multi-dimensional review and database entry of video ringback tones according to claim 3, characterized in that, The step of determining the similarity between the target video ringback tone and each of the first video ringback tones based on the target hash value and the first hash value specifically includes: Determine the second Hamming distance between the target hash value and the first hash value; When the second Hamming distance is less than a preset distance threshold, it is determined that the target video ringback tone and the corresponding first video ringback tone have a similar I-frame; The number of similar I-frame pairs between the target video ringback tone and each of the first video ringback tones is counted, and the similarity between the target video ringback tone and each of the first video ringback tones is determined based on the ratio of the number of similar I-frame pairs to the total number of target I-frames.

5. The method for multi-dimensional review and database entry of video ringback tones according to claim 1, characterized in that, The multi-level risk control review of the target video ringback tone based on the background audio file specifically includes: The background audio file is subjected to speech recognition to obtain target text information, and the video source file of the target video ringtone is obtained; The target text information is subject to text review; The video source file was subject to content review. When the target text information passes text review and the video source file passes content review, the multi-level risk control review is confirmed to be successful.

6. A method for multi-dimensional review and database entry of video ringback tones according to any one of claims 1 to 5, characterized in that, The step of transcoding and storing the target video ringback tone based on the media information specifically includes: Based on the media information, the video source file of the target video ringback tone is transcoded into multiple video ringback tone files with preset format and preset resolution using FFmpeg; The video ringback tone file is distributed to a preset video ringback tone library for storage.

7. A multi-dimensional review and database entry device for video ringback tones, characterized in that, The method for implementing the multi-dimensional review and database entry of video ringback tones as described in any one of claims 1 to 6 includes: The preprocessing module is used to preprocess the target video ringback tone to obtain media information, background sound file, multiple target I-frames, and target hash value corresponding to each target I-frame; The image scoring and review module is used to score and review the target video ringback tone based on the media information, the target I-frame, and the target hash value. The image deduplication verification module is used to perform image deduplication verification on the target video ringback tone based on the target hash value. A multi-level risk control review module is used to perform multi-level risk control review on the target video ringback tone based on the background sound file; The transcoding and storage module is used to transcode and store the target video ringback tone according to the media information when the target video ringback tone passes the image scoring review, image deduplication review, and multi-level risk control review.

8. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for establishing communication between the processor and the memory. When the program is executed by the processor, it implements the steps of the video ringback tone multi-dimensional review and database entry method as described in any one of claims 1 to 6.

9. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the video ringback tone multi-dimensional review and database entry method as described in any one of claims 1 to 6.

Images