Video polyphonic ringtone multi-dimensional auditing and warehousing method and device, electronic equipment and medium

Abstract

The invention discloses a video polyphonic ringtone multi-dimensional auditing and warehousing method and device, electronic equipment and a medium, and the method comprises the steps: carrying out the preprocessing of a target video polyphonic ringtone, and obtaining media information, a background sound file, a plurality of target I frames, and a target hash value corresponding to each target I frame; according to the media information, the target I frame and the target hash value, carrying out picture score auditing on the target video polyphonic ringtone; according to the target hash value, performing picture duplicate checking auditing on the target video polyphonic ringtone; performing multi-level risk control auditing on the target video polyphonic ringtone according to the background audio file; and transcoding and storing the target video polyphonic ringtone according to the media information when the target video polyphonic ringtone passes picture scoring auditing, picture duplicate checking auditing and multi-level risk control auditing. The method improves the efficiency and reliability of audit and storage of the video polyphonic ringtone, and can be widely applied to the technical field of video processing.

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and in particular to a method, device, electronic equipment and medium for multi-dimensional review and storage of video ringback tones. Background Art

[0002] With the popularization of 4G / 5G networks and the widespread use of smartphones and mobile Internet applications, video ringback tone, as an emerging personalized communication method, has rapidly attracted the attention and use of a large number of users. Video ringback tone can not only provide users with a richer and more vivid call experience, but also provide a platform for enterprises and individuals to showcase their brand image and creative content.

[0003] In order to ensure the smoothness of video ringback tone playback, the attractiveness of visual effects and the compliance of content, it is particularly important to develop an efficient and intelligent video ringback tone review system.

[0004] The existing video ringtone review system only has risk control review and video transcoding processes, and whether the video picture is clear and rich is subjectively judged by the editor. In addition, there are many online video ringtones in the library, and it is easy to purchase duplicate video materials when purchasing, so it is necessary to check for duplicates. However, taking 200,000 video ringtones as an example, the playback time of a video ringtone is about 25 seconds. If it relies on manual checking for duplicates, it will take 200,000*25 / 60 / 60 / 8≈170 days just to watch all these videos, which is a huge workload.

[0005] In addition, in the existing video ringback tone review system, the video preprocessing of each part is mostly carried out independently. Ultra-high-definition video ringback tone has high definition and occupies a large amount of memory. If the video needs to be processed in each link, it will take a lot of time and storage space. If the preprocessing process needs to be modified, each link must be modified separately, which requires a lot of development work.

[0006] In summary, the existing video ringback tone review system has the following shortcomings:

[0007] 1) Whether the video is clear and rich is determined by the editor's subjective judgment, which is prone to errors;

[0008] 2) There are a large number of existing videos, and manual duplication checking is inefficient;

[0009] 3) Each link pre-processes the video independently, and the same pre-processing process needs to be executed multiple times, which takes a long time;

[0010] 4) Each link pre-processes the video independently, and each link stores the pre-processing results independently, which takes up a lot of space;

[0011] 5) Each link is implemented through independent preprocessing, and the underlying implementation is not unified. If the audit standards need to be modified, each system must be modified independently, resulting in a large amount of development work.

[0012] The above problems need to be solved urgently.

[0013] Terminology explanation:

[0014] Video ringback tone: Video ringback tone is a special service of VoLTE, which evolves "audible ringback tone" into "visual ringback tone". When a user makes a voice call or video call on the VoLTE network, the video content that is displayed before the call is connected is called "video ringback tone".

[0015] FFmpeg: A set of open source computer programs that can be used to record, convert, and stream digital audio and video.

[0016] Resolution: Also known as resolution, it generally refers to the ability of a measurement or display system to resolve details. It determines the fineness of the details of a bitmap image and is usually used as a measure of the details and clarity displayed in an image, video or display device.

[0017] Bit rate: Also known as bit rate (bit rate, variable R) in the field of telecommunications and computing, it refers to the number of bits transmitted or processed per unit time (bit), with the unit of bps (bit per second).

[0018] Frame rate: The frequency (rate) at which bitmap images, called frames, appear continuously on a display. The term also applies to film and video cameras, computer graphics, and motion capture systems. Frame rate can also be called frame frequency and is expressed in Hertz (Hz).

[0019] H264: H.264, also known as the tenth part of MPEG-4, is a highly compressed digital video codec standard proposed by the Joint Video Team (JVT) jointly composed of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG).

[0020] I frame: Also known as intra picture, I frame is usually the first frame of each GOP (a video compression technology used by MPEG). After moderate compression, it can be used as a reference point for random access and can be regarded as an image.

[0021] Grayscale image: Also known as grayscale image. The relationship between white and black is divided into several levels according to the logarithmic relationship, which is called grayscale. Grayscale is divided into 256 levels.

[0022] Discrete Cosine Transform: A transform related to the Fourier transform, which is similar to the Discrete Fourier Transform (DFT) but uses only real numbers.

[0023] Hamming distance: It is a concept used in data transmission error control coding. It represents the number of different characters in corresponding positions of two strings (of the same length). D(x,y) represents the Hamming distance between two characters x and y. Perform an XOR operation on two strings and count the number of results that are 1. This number is the Hamming distance. Summary of the invention

[0024] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0025] To this end, an object of an embodiment of the present invention is to provide a method for multi-dimensional review and storage of video ringback tones, which improves the efficiency and reliability of review and storage of video ringback tones.

[0026] Another object of an embodiment of the present invention is to provide a multi-dimensional review and storage device for video ringback tones.

[0027] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0028] On the one hand, an embodiment of the present invention provides a method for multi-dimensional review and storage of video ringback tone, including the following steps:

[0029] Preprocessing the target video ringback tone to obtain media information, background sound files, multiple target I frames, and target hash values ​​corresponding to each of the target I frames;

[0030] Performing a picture scoring review on the target video ring back tone according to the media information, the target I frame and the target hash value;

[0031] Performing a duplicate check on the target video ringback tone according to the target hash value;

[0032] Performing a multi-level risk control review on the target video ringback tone according to the background sound file;

[0033] When the target video ringtone passes the screen scoring review, screen duplication review and multi-level risk control review, the target video ringtone is transcoded and stored according to the media information.

[0034] Further, in one embodiment of the present invention, the target video ringback tone is preprocessed to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each of the target I frames, which specifically includes:

[0035] Using FFmpeg to read the media information of the target video ringback tone;

[0036] Use FFmpeg to separate the background sound file of the target video;

[0037] Separate the target I frames of the target video using FFmpeg;

[0038] Converting the target I frame into a grayscale image, and generating a pixel grayscale matrix of each target I frame according to the grayscale image;

[0039] Performing a two-dimensional discrete cosine transform on the pixel grayscale matrix to obtain a pixel transformation matrix of each target I frame;

[0040] The pixel transformation matrix is ​​intercepted according to a preset pixel window to obtain a pixel interception matrix of each target I frame;

[0041] Determine an element average value of all elements in each pixel interception matrix, and judge whether each element is greater than the element average value;

[0042] Assigning a value of 1 to an element greater than the element average value, and assigning a value of 0 to an element less than or equal to the element average value, to obtain a pixel assignment matrix of each target I frame;

[0043] Each element of the pixel assignment matrix is ​​read in sequence and concatenated in order to obtain the target hash value corresponding to each target I frame.

[0044] Further, in an embodiment of the present invention, the performing picture scoring review on the target video ring back tone according to the media information, the target I frame and the target hash value specifically includes:

[0045] Determine the resolution, bit rate and frame rate of the target video ringback tone according to the media information;

[0046] Determine the number of colors of each of the target I frames, and determine the average color value of the target video ringtone according to the number of colors;

[0047] Determine a first Hamming distance of the target hash values ​​corresponding to two adjacent target I frames, and count the number of target adjacent target I frame pairs whose first Hamming distance is greater than or equal to a preset distance threshold, and then determine the before-after frame change rate of the target video ringback tone according to the ratio of the target adjacent target I frame pairs to the total number of adjacent target I frame pairs;

[0048] The resolution, the bit rate, the frame rate, the average color value and the rate of change of the previous and next frames are weighted and summed according to a preset weight coefficient to obtain a picture score value of the target video ring back tone;

[0049] When the screen score value is greater than or equal to a preset screen score threshold, it is determined that the screen score review is passed;

[0050] When the picture score value is less than the picture score threshold, it is determined that the picture score review fails.

[0051] Further, in one embodiment of the present invention, the performing of a screen duplication check on the target video ring back tone according to the target hash value specifically includes:

[0052] Determine a first hash value corresponding to a plurality of I frames of each first video ring back tone in a preset video ring back tone library;

[0053] Determine the similarity between the target video ringback tone and each of the first video ringback tone according to the target hash value and the first hash value;

[0054] When there is no first video ringback tone whose similarity with the target video ringback tone is greater than or equal to a preset similarity threshold, it is determined that the screen duplication check is passed;

[0055] When there is the first video ringback tone whose similarity with the target video ringback tone is greater than or equal to the similarity threshold, it is determined that the screen duplication check fails.

[0056] Further, in an embodiment of the present invention, determining the similarity between the target video ringback tone and each of the first video ringback tone according to the target hash value and the first hash value specifically includes:

[0057] Determine a second Hamming distance between the target hash value and the first hash value;

[0058] When the second Hamming distance is less than a preset distance threshold, it is determined that the target video ringback tone and the corresponding first video ringback tone have similar I frames;

[0059] The number of similar I frame pairs between the target video ring back tone and each of the first video ring back tone is counted, and the similarity between the target video ring back tone and each of the first video ring back tone is determined according to the ratio of the number of similar I frame pairs to the total number of target I frames.

[0060] Further, in an embodiment of the present invention, the multi-level risk control review of the target video ringback tone is performed according to the background sound file, which specifically includes:

[0061] Performing speech recognition on the background sound file to obtain target text information, and acquiring the video source file of the target video ringback tone;

[0062] Conducting text review on the target text information;

[0063] Conducting content review on the video source file;

[0064] When the target text information passes the text review and the video source file passes the content review, it is determined that the multi-level risk control review has passed.

[0065] Further, in an embodiment of the present invention, the step of transcoding and storing the target video ringback tone according to the media information specifically includes:

[0066] According to the media information, the video source file of the target video ringback tone is transcoded into a plurality of video ringback tone files of a preset format and a preset resolution by using FFmpeg;

[0067] The video ring back tone file is distributed to a preset video ring back tone library for storage.

[0068] On the other hand, an embodiment of the present invention provides a multi-dimensional review and storage device for video ringback tone, including:

[0069] A preprocessing module is used to preprocess the target video ringback tone to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each of the target I frames;

[0070] A picture scoring and reviewing module, used for performing a picture scoring and reviewing on the target video ring back tone according to the media information, the target I frame and the target hash value;

[0071] A screen duplication checking and reviewing module is used to check and review the screen duplication of the target video ringback tone according to the target hash value;

[0072] A multi-level risk control review module, used for performing a multi-level risk control review on the target video ringback tone according to the background sound file;

[0073] The transcoding storage module is used to transcode and store the target video ringtone according to the media information when the target video ringtone passes the picture scoring review, the picture duplication review and the multi-level risk control review.

[0074] On the other hand, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the multi-dimensional review and warehousing method for video ringtones as described above is realized.

[0075] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the multi-dimensional review and warehousing method for video ringtones as described above.

[0076] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0077] The embodiment of the present invention preprocesses the target video ringtone to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each target I frame, performs screen scoring review on the target video ringtone according to the media information, the target I frame and the target hash value, performs screen duplication review on the target video ringtone according to the target hash value, performs multi-level risk control review on the target video ringtone according to the background sound file, and when the target video ringtone passes the screen scoring review, screen duplication review and multi-level risk control review, the target video ringtone is transcoded and stored according to the media information. The embodiment of the present invention extracts the media information, background sound files, multiple target I frames and target hash values ​​corresponding to each target I frame of the target video ringtone through preprocessing, and uses the extracted information for subsequent screen scoring review, screen duplication review and multi-level risk control review, thereby reducing the duplication of video processing and the workload of manual review, and improving the efficiency and reliability of video ringtone review and storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0079] Figure 1 A flowchart of a method for multi-dimensional review and storage of video ringback tones provided by an embodiment of the present invention;

[0080] Figure 2 A schematic diagram of data flow of a multi-dimensional review and storage method for video ringback tone provided by an embodiment of the present invention;

[0081] Figure 3 A step flow chart of step S101 provided in an embodiment of the present invention;

[0082] Figure 4A step flow chart of step S102 provided in an embodiment of the present invention;

[0083] Figure 5 A step flow chart of step S103 provided in an embodiment of the present invention;

[0084] Figure 6 A step flow chart of step S1032 provided in an embodiment of the present invention;

[0085] Figure 7 A step flow chart of step S104 provided in an embodiment of the present invention;

[0086] Figure 8 A step flow chart of step S105 provided in an embodiment of the present invention;

[0087] Fig. 9 A schematic diagram of the structure of a multi-dimensional review and storage device for video ringback tone provided by an embodiment of the present invention;

[0088] Fig.10 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0089] Fig.11 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0090] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0091] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0092] The multi-dimensional review and storage method for video ringback tone provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the multi-dimensional review and storage method for video ringback tone, etc., but is not limited to the above forms.

[0093] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0094] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0095] like Figure 1 FIG. 1 is a flowchart of a method for multi-dimensional review and storage of video ringback tone provided by an embodiment of the present invention, referring to FIG. Figure 1 The embodiment of the present invention provides a method for multi-dimensional review and storage of video ringback tone, which specifically includes the following steps:

[0096] S101, pre-processing the target video ringback tone to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each target I frame;

[0097] S102, reviewing the target video ring back tone based on the media information, the target I frame, and the target hash value;

[0098] S103, checking and reviewing the target video ringback tone according to the target hash value;

[0099] S104, performing multi-level risk control review on the target video ringback tone according to the background sound file;

[0100] S105: When the target video ringtone passes the screen scoring review, screen duplication check review, and multi-level risk control review, the target video ringtone is transcoded and stored according to the media information.

[0101] like Figure 2The figure shows a data flow diagram of the multi-dimensional review and storage method of video ringtones provided by an embodiment of the present invention. The embodiment of the present invention independently develops a fully automatic video ringtone screen scoring system and a video ringtone screen duplication checking system, and uniformly preprocesses the video ringtones, thereby connecting the entire process of video ringtone review and storage. The main steps are: using the video ringtone preprocessing system to preprocess the video ringtone; using the video ringtone screen scoring system to review the screen of the video ringtone; using the video ringtone screen duplication checking system to review the screen of the video ringtone; using the video ringtone multi-level risk control system to review the video ringtone; using the video ringtone transcoding system to convert the video finance into video ringtone files of different formats; using the video ringtone distribution system to store the transcoded video ringtone files. The method steps of the embodiment of the present invention are further explained below.

[0102] like Figure 3 FIG. 1 is a flowchart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 3 As an optional implementation, the target video ringback tone is preprocessed to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each target I frame, which specifically includes:

[0103] S1011, using FFmpeg to read the media information of the target video ringback tone;

[0104] S1012, using FFmpeg to separate the background sound file of the target video;

[0105] S1013, using FFmpeg to separate multiple target I frames of the target video;

[0106] S1014, converting the target I frame into a grayscale image, and generating a pixel grayscale matrix of each target I frame according to the grayscale image;

[0107] S1015, performing a two-dimensional discrete cosine transform on the pixel grayscale matrix to obtain a pixel transformation matrix of each target I frame;

[0108] S1016, intercepting the pixel transformation matrix according to a preset pixel window to obtain a pixel interception matrix of each target I frame;

[0109] S1017, determining the element average value of all elements in each pixel interception matrix, and judging whether each element is greater than the element average value;

[0110] S1018, assigning a value of 1 to an element greater than an element average value, and assigning a value of 0 to an element less than or equal to the element average value, to obtain a pixel assignment matrix of each target I frame;

[0111] S1019, read each element of the pixel assignment matrix in sequence and splice them in order to obtain the corresponding target hash value of each target I frame.

[0112] Specifically, the specific process of preprocessing the video ring back tone by the video ring back tone preprocessing system is as follows:

[0113] 1) Use FFmpeg to read the video ringback tone media information, which mainly includes video format, resolution ppi, bit rate br, frame rate fps, etc.;

[0114] 2) Use FFmpeg to separate the background sound of the video and save it;

[0115] 3) Use FFmpeg to separate N I-frame images of H264 of the video and save them;

[0116] 4) Convert N I-frame images into N 32*32 grayscale images;

[0117] 5) Save the grayscale values ​​of the N grayscale images into N 32*32 matrices A;

[0118] 6) Calculate the two-dimensional discrete cosine transform of N matrices A to obtain N 32*32 matrices B; the two-dimensional discrete cosine transform formula is:

[0119]

[0120] Where F(μ,v) represents the elements in the transformed coefficient matrix, indicating the weights of different frequency components; f(x,y) is the grayscale value at the pixel point (x,y) in the original image; M and N represent the length and width of the matrix; C(μ), C(v) are orthogonal normalization coefficients. When μ,v=0, Otherwise C(μ)=C(v)=1;

[0121] 7) Cut the values ​​of the 8*8 area in the upper left corner of N matrices B to obtain N 8*8 matrices C;

[0122] 8) Calculate the average value of the 64 elements of each matrix C, denoted as average;

[0123] 9) From left to right and from top to bottom, determine whether the 64 elements of matrix C are greater than average. If so, record it as 1, otherwise record it as 0;

[0124] 10) Concatenate the 0s and 1s of the above records in order to form a 64-bit hash value such as “001101101110001011011011001100000001010000001100000010000011”, record it as hash, and save it to the database;

[0125] 11) After the processing is completed, a video ringback tone has a background sound, N key I-frame images and N hashes.

[0126] like Figure 4 FIG. 1 is a flow chart of step S102 provided in an embodiment of the present invention, referring to FIG. Figure 4 As an optional implementation, the target video ring back tone is reviewed based on the media information, the target I frame and the target hash value, which specifically includes:

[0127] S1021, determining the resolution, bit rate, and frame rate of the target video ringback tone according to the media information;

[0128] S1022, determining the number of colors of each target I frame, and determining the average color value of the target video ringtone according to the number of colors;

[0129] S1023, determining a first Hamming distance of target hash values ​​corresponding to two adjacent target I frames, and counting the number of target adjacent target I frame pairs whose first Hamming distance is greater than or equal to a preset distance threshold, and then determining the before-after frame change rate of the target video ringtone according to the ratio of the number of target adjacent target I frame pairs to the total number of adjacent target I frame pairs;

[0130] S1024, performing weighted summation of the resolution, bit rate, frame rate, average color value, and previous and next frame change rate according to a preset weight coefficient to obtain a picture score value of the target video ring back tone;

[0131] S1025: When the screen score value is greater than or equal to a preset screen score threshold, it is determined that the screen score review is passed;

[0132] S1026: When the screen score value is less than the screen score threshold, it is determined that the screen score review fails.

[0133] Specifically, the specific process of reviewing the screen rating of video ringback tone through the video ringback tone screen rating system is as follows:

[0134] 1) Read the video resolution ppi, bit rate br, and frame rate fps from the video ringback tone preprocessing system;

[0135] 2) Reading N I-frame images of the video from the video ringback tone preprocessing system;

[0136] 3) Count the number of colors in each I-frame image, recorded as num_color, and calculate the average color value of N I-frames as num_color_average, then: This value can reflect the richness of the video color;

[0137] 4) Read the hash values ​​corresponding to N I frames of the video from the video ring back tone preprocessing system, and calculate the Hamming distance between the hash values ​​of the previous and next frames: hamming_distance=bit_count(hash^hash_next), where hash_next represents the hash value of the next adjacent frame, "^" represents an XOR operation, and bit_count(x) represents the number of binary 1s of a number.

[0138] 5) Count the number of hamming_distance ≥ 12, recorded as hamming_num;

[0139] 6) Calculate the previous and next frame change score hamming_s = hamming_num / (N-1);

[0140] 7) Calculate the picture score s = ppi*w1 / ppi_max+br*w2 / br_max+fps*w3 / fps_max+num_color_average*w4 / num_color_max+hamming_s*w5;

[0141] Among them, w1, w2, w3, w4, and w5 are the weight values ​​of ppi, br, fps, num_color_average, and hamming_s, respectively, which meet w1+w2+w3+w4+w5=1; ppi_max is the preset resolution value of the ultra-high-definition lossless video. If the actual ppi is greater than ppi_max, ppi*w1 / ppi_max=w1 is set; br_max is the preset video bit rate value of the ultra-high-definition lossless video. If the actual br is greater than br_max, br*w2 / br_max=w2 is set; fps_max is the preset frame rate value of the ultra-high-definition lossless video. If the actual fps is greater than fps_max, fps*w3 / fps_max=w3 is set; num_color_max is the preset color number value of the ultra-high-definition lossless video. If the actual num_color_average is greater than num_color_max, num_color_average*w4 / num_color_max=w4 is set;

[0142] 8) Determine whether the picture score s is greater than or equal to a preset picture score threshold. If so, the video picture score review is passed, otherwise it is not passed.

[0143] like Figure 5 FIG. 1 is a flow chart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, the target video ring back tone is checked for duplicate content according to the target hash value, which specifically includes:

[0144] S1031. Determine first hash values ​​corresponding to multiple I frames of each first video ringback tone in a preset video ringback tone library;

[0145] S1032, determining the similarity between the target video ringback tone and each first video ringback tone according to the target hash value and the first hash value;

[0146] S1033: When there is no first video ringback tone whose similarity with the target video ringback tone is greater than or equal to a preset similarity threshold, it is determined that the screen duplication check is passed;

[0147] S1034. When there is a first video ringback tone whose similarity with the target video ringback tone is greater than or equal to the similarity threshold, it is determined that the screen duplication check fails.

[0148] like Figure 6 FIG. 1 is a flow chart of step S1032 provided in an embodiment of the present invention, referring to FIG. Figure 6 As an optional implementation, the similarity between the target video ringback tone and each first video ringback tone is determined according to the target hash value and the first hash value, which specifically includes:

[0149] S10321. Determine a second Hamming distance between the target hash value and the first hash value;

[0150] S10322: When the second Hamming distance is less than a preset distance threshold, it is determined that the target video ringback tone and the corresponding first video ringback tone have similar I frames;

[0151] S10323. Count the number of similar I-frame pairs between the target video ringback tone and each first video ringback tone, and determine the similarity between the target video ringback tone and each first video ringback tone according to the ratio of the number of similar I-frame pairs to the total number of target I frames.

[0152] Specifically, the process of checking the duplicate images of video ringback tone through the video ringback tone checking system is as follows:

[0153] 1) The video to be checked for duplicate content is recorded as V1, and the video in the video library is recorded as V2;

[0154] 2) Read the hash values ​​corresponding to the N I frames of video V1 from the video ringback tone preprocessing system, and compare them with the hash values ​​hash_v2 of each I frame of video V2;

[0155] 3) Calculate the Hamming distance between hash and hash_v2 of the two frames: hamming_distance = bit_count (hash^hash_v2), where “^” represents an XOR operation, and bit_count (x) represents the number of 1s in the binary value of a number.

[0156] 4) If a hash of V1 can find at least one hash_v2 in V2 that meets hamming_distance<12, it means that this frame has a similar frame;

[0157] 5) The number of frames with similar frames in the target video is counted as similar_num, then the similarity between V1 and V2 is similar=similar_num / N;

[0158] 6) If similar ≥ 50%, it means that more than 50% of the content in V1 is similar to that in V2, and we call V2 a similar video to V1;

[0159] 7) If V1 can find a similar video, the duplicate check will fail, otherwise it will pass.

[0160] like Figure 7 FIG. 1 is a flow chart of step S104 provided in an embodiment of the present invention, referring to FIG. Figure 7 As an optional implementation, a multi-level risk control review is performed on the target video ringback tone according to the background sound file, which specifically includes:

[0161] S1041, performing speech recognition on the background sound file to obtain target text information, and obtaining the video source file of the target video ringback tone;

[0162] S1042, conducting text review on the target text information;

[0163] S1043, review the content of the video source file;

[0164] S1044. When the target text information passes the text review and the video source file passes the content review, it is determined that the multi-level risk control review has passed.

[0165] Specifically, the specific process of multi-level risk control review of video ringback tone through the multi-level risk control system of video ringback tone is as follows:

[0166] 1) Read the background audio file and video source file with the audit video from the video ringback tone preprocessing system;

[0167] 2) Convert background sound into text through AI text-to-speech interface;

[0168] 3) Use the text risk control interface to determine whether the text is compliant. If it is not compliant, the video will not pass the review;

[0169] 4) If the text risk control passes, the video source file will be sent to a third-party authoritative agency for risk control review (generally, it will be reviewed by Xinhuanet). If it fails, the video review will fail;

[0170] 5) If the third-party review is passed, it will be submitted to internal manual review. If it fails, the video review will fail, otherwise it will pass.

[0171] like Figure 8 FIG. 1 is a flow chart of step S105 provided in an embodiment of the present invention, referring to FIG. Figure 8 As an optional implementation, the target video ringback tone is transcoded and stored according to the media information, which specifically includes:

[0172] S1051, according to the media information, transcoding the video source file of the target video ringback tone into multiple video ringback tone files of a preset format and a preset resolution through FFmpeg;

[0173] S1052: Distribute the video ringback tone file to a preset video ringback tone library for storage.

[0174] Specifically, the video source file and media information of the target video ringback tone are read from the video ringback tone preprocessing system, and the video ringback tone transcoding system transcodes the target video ringback tone into a video ringback tone file of different formats and resolutions through FFmpeg according to the media information; the transcoded video ringback tone file is saved through the video ringback tone distribution system; after the video ringback tone distribution system notifies the warehousing system of successful saving, the warehousing system will change the video status to successful warehousing.

[0175] The above is a description of the method steps of the embodiment of the present invention. It can be recognized that the embodiment of the present invention obtains the media information, background sound file, multiple target I frames and target hash values ​​corresponding to each target I frame of the target video ringback tone through pre-processing extraction, and uses the extracted information for subsequent picture scoring review, picture duplication review and multi-level risk control review, thereby reducing the duplication of video processing and the workload of manual review, and improving the efficiency and reliability of video ringback tone review and storage.

[0176] Compared with the prior art, the embodiments of the present invention also have the following advantages:

[0177] 1) Independently develop fully automatic video quality scoring to avoid errors caused by subjective judgment of video editors;

[0178] 2) Independently develop fully automatic video duplication check, greatly reducing the workload of editors;

[0179] 3) Unified video preprocessing, the same process only needs to be executed once, saving time;

[0180] 4) Unified video preprocessing, preprocessing results are only saved once, saving space;

[0181] 5) Unified video preprocessing and underlying implementation. If you want to modify the review standards, you only need to modify the preprocessing system, saving development workload.

[0182] like Fig. 9 FIG. 1 is a schematic diagram of the structure of a multi-dimensional video ringback tone review and storage device provided by an embodiment of the present invention, referring to FIG. Fig. 9 The embodiment of the present invention provides a multi-dimensional review and storage device for video ring back tone, including:

[0183] A preprocessing module is used to preprocess the target video ringback tone to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each target I frame;

[0184] A picture scoring and reviewing module is used to review the picture scoring of the target video ringtone based on the media information, the target I frame and the target hash value;

[0185] The image duplication checking and reviewing module is used to check the image duplication of the target video ringtone according to the target hash value;

[0186] Multi-level risk control review module, used to conduct multi-level risk control review on target video ringback tone according to background audio files;

[0187] The transcoding storage module is used to transcode and store the target video ringtone according to the media information when the target video ringtone passes the picture scoring review, picture duplication review and multi-level risk control review.

[0188] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0189] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the program is executed by the processor, the above-mentioned video ringtone multi-dimensional review and storage method is realized. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0190] like Fig.10 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Fig.10 , an embodiment of the present invention provides an electronic device, including:

[0191] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0192] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the multi-dimensional review and storage method of the video ringback tone in the embodiment of the present invention;

[0193] Input / output interface 1003, used to implement information input and output;

[0194] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0195] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0196] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0197] like Fig.11 FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.11 The embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1101, and the one or more programs 1101 can be executed by one or more processors to implement the above-mentioned video ringtone multi-dimensional review and warehousing method.

[0198] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0199] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0200] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0201] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0202] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0203] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0204] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0205] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0206] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0207] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0208] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A multi-dimensional review and storage method for video ringback tone, characterized in that: The following steps are involved: Preprocessing the target video ringback tone to obtain media information, background sound files, multiple target I frames, and target hash values ​​corresponding to each of the target I frames; Performing a picture scoring review on the target video ring back tone according to the media information, the target I frame and the target hash value; Performing a duplicate check on the target video ringback tone according to the target hash value; Performing a multi-level risk control review on the target video ringback tone according to the background sound file; When the target video ringtone passes the screen scoring review, screen duplication review and multi-level risk control review, the target video ringtone is transcoded and stored according to the media information.

2. A method for multi-dimensional review and storage of video ringback tone according to claim 1, characterized in that: The preprocessing of the target video ringback tone to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each of the target I frames specifically includes: Using FFmpeg to read the media information of the target video ringback tone; Use FFmpeg to separate the background sound file of the target video; Separate the target I frames of the target video using FFmpeg; Converting the target I frame into a grayscale image, and generating a pixel grayscale matrix of each target I frame according to the grayscale image; Performing a two-dimensional discrete cosine transform on the pixel grayscale matrix to obtain a pixel transformation matrix of each target I frame; The pixel transformation matrix is ​​intercepted according to a preset pixel window to obtain a pixel interception matrix of each target I frame; Determine an element average value of all elements in each pixel interception matrix, and judge whether each element is greater than the element average value; Assigning a value of 1 to an element greater than the element average value, and assigning a value of 0 to an element less than or equal to the element average value, to obtain a pixel assignment matrix of each target I frame; Each element of the pixel assignment matrix is ​​read in sequence and concatenated in order to obtain the target hash value corresponding to each target I frame.

3. The multi-dimensional review and storage method for video ringback tone according to claim 1, characterized in that: The performing picture scoring review on the target video ring back tone according to the media information, the target I frame and the target hash value specifically includes: Determine the resolution, bit rate and frame rate of the target video ringback tone according to the media information; Determine the number of colors of each of the target I frames, and determine the average color value of the target video ringtone according to the number of colors; Determine a first Hamming distance of the target hash values ​​corresponding to two adjacent target I frames, and count the number of target adjacent target I frame pairs whose first Hamming distance is greater than or equal to a preset distance threshold, and then determine the before-after frame change rate of the target video ringback tone according to the ratio of the target adjacent target I frame pairs to the total number of adjacent target I frame pairs; The resolution, the bit rate, the frame rate, the average color value and the rate of change of the previous and next frames are weighted and summed according to a preset weight coefficient to obtain a picture score value of the target video ring back tone; When the screen score value is greater than or equal to a preset screen score threshold, it is determined that the screen score review is passed; When the picture score value is less than the picture score threshold, it is determined that the picture score review fails.

4. The multi-dimensional review and storage method for video ringback tone according to claim 1 is characterized in that: The performing a duplicate checking on the target video ring back tone according to the target hash value specifically includes: Determine a first hash value corresponding to a plurality of I frames of each first video ring back tone in a preset video ring back tone library; Determine the similarity between the target video ringback tone and each of the first video ringback tone according to the target hash value and the first hash value; When there is no first video ringback tone whose similarity with the target video ringback tone is greater than or equal to a preset similarity threshold, it is determined that the screen duplication check is passed; When there is the first video ringback tone whose similarity with the target video ringback tone is greater than or equal to the similarity threshold, it is determined that the screen duplication check fails.

5. The method for multi-dimensional review and storage of video ringback tone according to claim 4, characterized in that: The determining the similarity between the target video ringback tone and each of the first video ringback tone according to the target hash value and the first hash value specifically includes: Determine a second Hamming distance between the target hash value and the first hash value; When the second Hamming distance is less than a preset distance threshold, it is determined that the target video ringback tone and the corresponding first video ringback tone have similar I frames; The number of similar I-frame pairs between the target video ring back tone and each of the first video ring back tone is counted, and the similarity between the target video ring back tone and each of the first video ring back tone is determined according to the ratio of the number of similar I-frame pairs to the total number of target I-frames.

6. The multi-dimensional review and storage method for video ringback tone according to claim 1, characterized in that: The multi-level risk control review of the target video ringback tone according to the background sound file specifically includes: Performing speech recognition on the background sound file to obtain target text information, and acquiring the video source file of the target video ringback tone; Conducting text review on the target text information; Conducting content review on the video source file; When the target text information passes the text review and the video source file passes the content review, it is determined that the multi-level risk control review has passed.

7. A method for multi-dimensional review and storage of video ringback tones according to any one of claims 1 to 6, characterized in that: The step of transcoding and storing the target video ringback tone according to the media information specifically includes: According to the media information, the video source file of the target video ringback tone is transcoded into a plurality of video ringback tone files of a preset format and a preset resolution by using FFmpeg; The video ring back tone file is distributed to a preset video ring back tone library for storage.

8. A multi-dimensional review and storage device for video ringback tone, characterized in that: include: A preprocessing module is used to preprocess the target video ringback tone to obtain media information, background sound files, multiple target I frames and target hash values ​​corresponding to each of the target I frames; A picture scoring and reviewing module, used for performing a picture scoring and reviewing on the target video ring back tone according to the media information, the target I frame and the target hash value; A screen duplication checking and reviewing module is used to check and review the screen duplication of the target video ringback tone according to the target hash value; A multi-level risk control review module, used for performing a multi-level risk control review on the target video ringback tone according to the background sound file; The transcoding storage module is used to transcode and store the target video ringtone according to the media information when the target video ringtone passes the picture scoring review, the picture duplication review and the multi-level risk control review.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the multi-dimensional review and warehousing method for video ringtones as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the multi-dimensional review and warehousing method for video ringback tones as described in any one of claims 1 to 7.

Images