Coloring back tone auditing method and device based on dual-mode watermark, electronic equipment and medium

Abstract

The invention discloses a coloring ring back tone auditing method and device based on a dual-mode watermark, electronic equipment and a medium. The method comprises the following steps: carrying out watermark extraction on a coloring ring back tone to be audited; when the to-be-audited polyphonic ringtone does not carry the robust watermark and the fragile watermark, determining that the to-be-audited polyphonic ringtone is a platform newly-added polyphonic ringtone, and embedding the robust watermark and the fragile watermark after the to-be-audited polyphonic ringtone passes content audit to obtain a warehousing polyphonic ringtone; when the to-be-audited polyphonic ringtone carries the robust watermark and the fragile watermark, determining that the to-be-audited polyphonic ringtone is an existing polyphonic ringtone of the platform, determining a use range of the to-be-audited polyphonic ringtone according to a watermark extraction result, and performing use audit on the to-be-audited polyphonic ringtone according to the use range; and when the to-be-audited polyphonic ringtone carries the robust watermark but does not carry the fragile watermark, determining that the to-be-audited polyphonic ringtone is a platform modified polyphonic ringtone, determining copyright information of the to-be-audited polyphonic ringtone according to a watermark extraction result, and performing copyright auditing on the to-be-audited polyphonic ringtone according to the copyright information. The polyphonic ringtone auditing efficiency and accuracy are improved, and the method can be widely applied to the technical field of polyphonic ringtones.

Description

Technical Field

[0001] The present invention relates to the technical field of color ring back tone, and in particular to a color ring back tone auditing method, device, electronic equipment and medium based on dual-mode watermark. Background Art

[0002] Audio digital watermarking is a mature and effective audio copyright protection technology. It embeds copyright information into the audio to be protected. People's auditory system is usually unable to distinguish the difference between watermarked audio and original audio. When necessary, matching extraction technology can be used to extract the original embedded information from the audio to determine the ownership of the audio copyright or to trace the source of the leak. Existing audio digital watermarking technology is aimed at improving the robustness when attacked and tampered, that is, robust watermarking technology. For example, when the watermarked audio is cropped, resampled, or even ripped, the correct watermark information can still be extracted.

[0003] Business ringback tone is a ringback tone service for government and enterprise customers. Customers create promotional audio or video and set this audio or video as the ringtone for the government and enterprise fixed-line phones or employee work mobile phones belonging to the customer. When a third party dials these government and enterprise fixed-line phones or employee numbers, the preset promotional audio or video will be played on the calling phone during the ringing period.

[0004] After the customer's audio and video ringtones are produced and uploaded to the business ringtone platform, manual and machine review are required to ensure the legality of the audio and video ringtones. In addition to checking whether there are any illegal texts, speeches, pictures, and videos, it is also necessary to manually check whether the audio and video content is consistent with the customer. It is necessary to avoid the situation where private enterprises steal the ringtones of government agencies or steal the carefully produced ringtones of other companies. This may introduce heavy manual review. At the same time, the ringtone business forms are ever-changing. In some business scenarios, it is hoped that a ringtone will be reused or used by multiple customers. For example, customers may use the original audio and video again (the customer thinks the original ringtone is better); the organization may hope that small customers use promotional ringtones to achieve the purpose of fission promotion (the Cultural and Tourism Bureau produced a ringtone to promote local tourism, and multiple cultural and tourism-related small and micro enterprise customers use this ringtone); customers from different government departments use exactly the same ringtone (multiple police stations in this city use the same anti-fraud promotional ringtone). This may introduce a large number of repeated manual and machine reviews.

[0005] Considering that digital watermark technology should be used to add copyright marks to business ringtones, it is more important to combine watermark technology with the review process of business ringtones. The existing digital watermark technology focuses on restoring watermark information without error, but ignores whether the audio and video carrier itself is attacked (such as cropping, modification), thus affecting the efficiency and accuracy of ringtone review.

[0006] Terminology explanation:

[0007] Digital Watermark: Embed specific information into a digital carrier (such as a picture, audio, or video). If the embedded information is not perceived by people and does not affect the presentation of the original digital carrier, such watermark information is a hidden digital watermark.

[0008] FFT (Fast Fourier Transform): It is an efficient algorithm for calculating discrete Fourier transform (DFT). It can convert time domain signals into frequency domain signals, thereby realizing the analysis and processing of the frequency components of the signals.

[0009] DCT (Discrete Cosine Transform): A signal processing method that can decompose a signal (such as audio) and extract different frequency components in the signal.

[0010] IDCT (Inverse Discrete Cosine Transform): The inverse processing of DCT, which is the process of restoring the coefficients after DCT processing to the original signal. Summary of the invention

[0011] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0012] To this end, one purpose of an embodiment of the present invention is to provide a color ring tone review method based on dual-mode watermark, which embeds robust watermarks and fragile watermarks into stored color ring tones, so that dual-mode watermarks can be used to complete rapid review of customers' newly added color ring tones, thereby improving the efficiency and accuracy of color ring tone review.

[0013] Another object of an embodiment of the present invention is to provide a color ring back tone auditing device based on dual-mode watermark.

[0014] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0015] On the one hand, an embodiment of the present invention provides a method for auditing a color ring back tone based on a dual-mode watermark, comprising the following steps:

[0016] Acquire the color ring tone to be reviewed, extract watermark from the color ring tone to be reviewed, and determine whether the color ring tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result;

[0017] When the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, it is determined that the CRBT to be reviewed is a newly added CRBT on the platform, and then after the CRBT to be reviewed passes the content review, a robust watermark and a fragile watermark are embedded to obtain a stored CRBT;

[0018] When the CRBT to be reviewed carries a robust watermark and a fragile watermark, it is determined that the CRBT to be reviewed is an existing CRBT on the platform, and then the usage scope of the CRBT to be reviewed is determined according to the watermark extraction result, and the usage of the CRBT to be reviewed is reviewed according to the usage scope;

[0019] When the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, it is determined that the ringtone to be reviewed is a platform-modified ringtone, and then the copyright information of the ringtone to be reviewed is determined according to the watermark extraction result, and the copyright review of the ringtone to be reviewed is performed according to the copyright information.

[0020] Further, in one embodiment of the present invention, the robust watermark is embedded in the embedded ring back tone through the following steps:

[0021] Segmenting the color ring tone to be embedded to obtain a plurality of first audio regions;

[0022] Performing a fast Fourier transform on each of the first audio regions to obtain first frequency spectrum information of each of the first audio regions;

[0023] determining a first high frequency component of each of the first audio regions according to the first frequency spectrum information, and selecting a plurality of the first audio regions as first embedding regions according to the first high frequency components;

[0024] Performing frequency domain transformation on the first embedding region using DCT to obtain first frequency domain information, and selecting a plurality of medium and low frequency bands of the first frequency domain information as first embedding frequency bands;

[0025] Generate a to-be-embedded robust watermark according to a preset robust watermark generation rule, and determine a plurality of bits of first binary embedding information according to the to-be-embedded robust watermark;

[0026] Modify the mid- and low-frequency coefficients of the first embedding frequency band of the first frequency domain information according to the first binary embedding information to obtain a first watermark frequency domain;

[0027] The first watermark frequency domain is subjected to inverse frequency domain transformation using IDCT to obtain a first watermark region, and then the first embedding region to be embedded in the ringback tone is replaced with the first watermark region to obtain a ringback tone carrying a robust watermark.

[0028] Furthermore, in one embodiment of the present invention, robust watermark extraction is performed on the color ring tone to be reviewed through the following steps:

[0029] Segmenting the color ring tone to be reviewed to obtain a plurality of second audio regions;

[0030] Performing a fast Fourier transform on each of the second audio regions to obtain second frequency spectrum information of each of the second audio regions;

[0031] determining a second high frequency component of each of the second audio regions according to the second frequency spectrum information, and selecting a plurality of the second audio regions as first extraction regions according to the second high frequency components;

[0032] Performing frequency domain transformation on the first extraction region using DCT to obtain second frequency domain information, and selecting a plurality of medium and low frequency bands of the second frequency domain information as first extraction frequency bands;

[0033] Determine a plurality of bits of first binary extraction information according to the mid- and low-frequency coefficients of the first extraction frequency band, and reconstruct the first binary extraction information to obtain a robust watermark to be verified;

[0034] Determining whether the robust watermark to be verified complies with a preset robust watermark generation rule;

[0035] If the robust watermark to be verified conforms to the robust watermark generation rule, it is determined that the color ring tone to be reviewed carries a robust watermark; if the robust watermark to be verified does not conform to the robust watermark generation rule, it is determined that the color ring tone to be reviewed does not carry a robust watermark.

[0036] Further, in one embodiment of the present invention, the fragile watermark is embedded into the embedded ring back tone through the following steps:

[0037] Segmenting the color ring tone to be embedded to obtain a plurality of third audio regions, and using the third audio regions as second embedding regions;

[0038] Performing frequency domain transformation on the second embedding region by using DCT to obtain third frequency domain information, and selecting a plurality of high frequency bands of the third frequency domain information as second embedding frequency bands;

[0039] Generating a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determining a plurality of bits of second binary embedding information according to the fragile watermark to be embedded;

[0040] Modify the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain;

[0041] The second watermark frequency domain is inversely transformed using IDCT to obtain a second watermark region, and then the second embedding region to be embedded in the ringback tone is replaced with the second watermark region to obtain a ringback tone carrying a fragile watermark.

[0042] Furthermore, in one embodiment of the present invention, the fragile watermark is extracted from the color ring tone to be reviewed through the following steps:

[0043] Segmenting the color ring tone to be reviewed to obtain a plurality of fourth audio regions, and using the fourth audio regions as second extraction regions;

[0044] Performing frequency domain transformation on the second extraction area using DCT to obtain fourth frequency domain information, and selecting multiple high-frequency frequency bands of the fourth frequency domain information as second extraction frequency bands;

[0045] Determining a plurality of bits of second binary extraction information according to the high frequency coefficient of the second extraction frequency band, and recombining the second binary extraction information to obtain a fragile watermark to be verified;

[0046] Determining whether the fragile watermark to be verified complies with a preset fragile watermark generation rule;

[0047] If the fragile watermark to be verified meets the fragile watermark generation rule, it is determined that the ringback tone to be reviewed carries a fragile watermark; if the fragile watermark to be verified does not meet the fragile watermark generation rule, it is determined that the ringback tone to be reviewed does not carry a fragile watermark.

[0048] Further, in an embodiment of the present invention, the fragile watermark carries the ringback tone ownership information and the use permission information, and the use scope of the ringback tone to be reviewed is determined according to the watermark extraction result, and the use review of the ringback tone to be reviewed is conducted according to the use scope, which specifically includes:

[0049] Determine the target fragile watermark of the color ring tone to be reviewed according to the watermark extraction result, and determine the target color ring tone attribution information and target use permission information of the color ring tone to be reviewed according to the target fragile watermark;

[0050] Determine the usage scope of the to-be-approved CRBT according to the target CRBT attribution information and the target usage permission information;

[0051] Acquire the uploaded customer information of the to-be-reviewed ringback tone, and determine whether the uploaded customer information complies with the usage scope;

[0052] If the uploaded customer information conforms to the usage scope, it is determined that the to-be-reviewed ringtone has passed the usage review; if the uploaded customer information does not conform to the usage scope, it is determined that the to-be-reviewed ringtone has not passed the usage review.

[0053] Further, in an embodiment of the present invention, determining the copyright information of the to-be-reviewed ringtone according to the watermark extraction result, and performing copyright review on the to-be-reviewed ringtone according to the copyright information specifically includes:

[0054] Determine the target robust watermark of the CRBT to be reviewed according to the watermark extraction result, and perform a matching search in a preset CRBT database according to the target robust watermark to obtain an initial platform CRBT corresponding to the CRBT to be reviewed;

[0055] Determining the copyright information of the to-be-reviewed ringback tone according to the initial platform ringback tone;

[0056] Acquire the uploaded customer information of the to-be-reviewed ringback tone, and determine whether the uploaded customer information is consistent with the copyright information;

[0057] If the uploaded customer information is consistent with the copyright information, it is determined that the pending ringtone has passed the copyright review; if the uploaded customer information is inconsistent with the copyright information, it is determined that the pending ringtone has not passed the copyright review.

[0058] On the other hand, an embodiment of the present invention provides a dual-mode watermark-based color ring back tone verification device, comprising:

[0059] A watermark extraction module is used to obtain the color ring tone to be reviewed, extract the watermark of the color ring tone to be reviewed, and determine whether the color ring tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result;

[0060] A watermark embedding module, for determining that the CRBT to be reviewed is a newly added CRBT on the platform when the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, and then embedding a robust watermark and a fragile watermark after the CRBT to be reviewed passes the content review to obtain a stored CRBT;

[0061] A usage audit module is used to determine that the CRBT to be reviewed is an existing CRBT on the platform when the CRBT to be reviewed carries a robust watermark and a fragile watermark, and then determine the usage scope of the CRBT to be reviewed according to the watermark extraction result, and perform usage audit on the CRBT to be reviewed according to the usage scope;

[0062] The copyright audit module is used to determine that the ringtone to be reviewed is a platform-modified ringtone when the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, and then determine the copyright information of the ringtone to be reviewed according to the watermark extraction result, and perform copyright audit on the ringtone to be reviewed according to the copyright information.

[0063] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the color ringback tone review method based on dual-mode watermark as described above is realized.

[0064] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the color ringback tone review method based on dual-mode watermark as described above.

[0065] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0066] The embodiment of the present invention obtains a ringtone to be reviewed, performs watermark extraction on the ringtone to be reviewed, and determines whether the ringtone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result; when the ringtone to be reviewed does not carry the robust watermark and the fragile watermark, it is determined that the ringtone to be reviewed is a newly added ringtone to the platform, and then the robust watermark and the fragile watermark are embedded in the ringtone to be reviewed after the ringtone to be reviewed passes the content review to obtain a stored ringtone; when the ringtone to be reviewed carries the robust watermark and the fragile watermark, it is determined that the ringtone to be reviewed is an existing ringtone on the platform, and then the use scope of the ringtone to be reviewed is determined according to the watermark extraction result, and the use review is performed on the ringtone to be reviewed according to the use scope; when the ringtone to be reviewed carries the robust watermark but does not carry the fragile watermark, it is determined that the ringtone to be reviewed is a modified ringtone on the platform, and then the copyright information of the ringtone to be reviewed is determined according to the watermark extraction result, and the copyright review is performed on the ringtone to be reviewed according to the copyright information. The embodiment of the present invention embeds a robust watermark and a fragile watermark into a new ringback tone. Subsequently, the dual-mode watermark can be extracted to determine whether the ringback tone to be reviewed is an existing ringback tone on the platform and whether it has been modified. At the same time, the usage scope and copyright information of the ringback tone to be reviewed can also be quickly obtained, thereby realizing a rapid review of the customer's newly added ringback tone, thereby improving the efficiency and accuracy of the ringback tone review. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 A step flow chart of a method for auditing a color ring tone based on a dual-mode watermark provided by an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of an application scenario of a dual-mode watermark-based color ring back tone audit method provided by an embodiment of the present invention;

[0070] Figure 3A specific implementation flow chart of the color ring back tone audit method based on dual-mode watermark provided by an embodiment of the present invention;

[0071] Figure 4 A flow chart of embedding a robust watermark for embedding a color ring back tone provided by an embodiment of the present invention;

[0072] Figure 5 A flow chart of robust watermark extraction for a color ring tone to be reviewed provided by an embodiment of the present invention;

[0073] Figure 6 A flow chart of embedding a fragile watermark into a ring back tone provided by an embodiment of the present invention;

[0074] Figure 7 A flow chart of extracting fragile watermarks from a ringback tone to be reviewed provided by an embodiment of the present invention;

[0075] Figure 8 A step flow chart of step S103 provided in an embodiment of the present invention;

[0076] Fig. 9 A step flow chart of step S104 provided in an embodiment of the present invention;

[0077] Fig.10 A schematic diagram of the structure of a dual-mode watermark-based color ring back tone verification device provided by an embodiment of the present invention;

[0078] Fig.11 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0079] Fig.12 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0080] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0081] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0082] The dual-mode watermark-based ringback tone audit method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the dual-mode watermark-based ringback tone audit method, etc., but is not limited to the above forms.

[0083] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0084] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0085] Existing watermark technology focuses on improving robustness and researching different implementation algorithms. In the business audio and video ringback tone service, ringback tone review is a key link. Ringback tone operators need to prevent illegal audio and video from being spread among the public, and need to prevent criminals from using stolen ringback tone to commit fraud or even fraud. In the current business process, only the model of machine review + manual review is used, which is inefficient, low-cost, and has a high possibility of error. It is necessary to use technical means to strongly support the review.

[0086] The present invention innovatively proposes a dual-mode watermark technology. The dual-mode watermark includes a robust watermark that can still read information when attacked and a fragile watermark that is immediately destroyed when attacked. The dual-mode watermark information is embedded in the carrier audio itself, just like DNA, accompanied by the copying, compression, modification and other behaviors of the carrier. By detecting the dual-mode watermark, the copyright ownership of the corresponding carrier (such as an audio ringtone) and whether it has been attacked and tampered with can be determined; at the same time, by extracting the information carried by the dual-mode watermark, business processing can be assisted and accelerated. Finally, by applying the dual-mode watermark technology, the present invention not only achieves the purpose of reducing costs and increasing efficiency, but also improves the level of information security.

[0087] The current business ring back tone platform includes two types: audio ring back tone and video ring back tone. Some business ring back tone customers have business ring back tone audio, some have business ring back tone video, and some have business ring back tone audio + business ring back tone video. The carrier of the dual-mode watermark technology described in the present invention can be an audio ring back tone or a video ring back tone, that is, the embodiment of the present invention can be applied to audio ring back tone and video ring back tone.

[0088] In the specific implementation, considering the efficiency and maturity of audio watermarking at the current stage, the present invention uses watermark embedding and extraction technology with audio as the carrier, but it should be noted that the present invention can be applied to pure audio ringback tone, and can also be applied to the audio part (audio track) of video ringback tone, that is, the present invention is applicable to business ringback tone of various customers. The following mainly takes audio as an example for explanation.

[0089] In addition, the carrier of the dual-mode watermark can also be a video (the video part of the video ringback tone). Compared with the audio watermark, the advantage of the video watermark is that the amount of information that can be embedded is larger and the concealment is better. The disadvantage is that the embedding and extraction operations are relatively complex and the efficiency is low. It should be recognized that the application of the dual-mode watermark technology of the present invention to the video part of the video ringback tone should still be regarded as within the scope of protection of the present invention.

[0090] like Figure 1 FIG. 1 is a flow chart showing a method for auditing a color ring tone based on a dual-mode watermark according to an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for auditing a ring back tone based on a dual-mode watermark, which specifically includes the following steps:

[0091] S101, obtaining a ringback tone to be reviewed, extracting a watermark from the ringback tone to be reviewed, and judging whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result;

[0092] S102: when the CRBT to be reviewed does not carry the robust watermark and the fragile watermark, it is determined that the CRBT to be reviewed is a newly added CRBT on the platform, and then the robust watermark and the fragile watermark are embedded in the CRBT to be reviewed after the CRBT to be reviewed passes the content review, so as to obtain the stored CRBT;

[0093] S103: When the ringback tone to be reviewed carries a robust watermark and a fragile watermark, it is determined that the ringback tone to be reviewed is an existing ringback tone on the platform, and then the usage scope of the ringback tone to be reviewed is determined according to the watermark extraction result, and the usage of the ringback tone to be reviewed is reviewed according to the usage scope;

[0094] S104: When the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, the ringtone to be reviewed is determined to be a platform-modified ringtone, and then the copyright information of the ringtone to be reviewed is determined according to the watermark extraction result, and a copyright review is performed on the ringtone to be reviewed according to the copyright information.

[0095] Specifically, Figure 2 The figure shows an application scenario diagram of the dual-mode watermark-based color ringback audit method provided by an embodiment of the present invention, wherein the commercial color ringback business and management unified background is the hub of the entire commercial color ringback business, which is responsible for connecting with the front-end portal and other business entrances, receiving the commercial color ringback tones uploaded by customers or providing color ringback generation capabilities, and is also connected to the front-end management portal, and has the ability to call professional functional module interfaces, these functional modules include machine audit capabilities and watermark processing algorithm capabilities, in addition, the commercial color ringback business and management unified background is also connected to the database and file storage warehouse. The devices used in the embodiment of the present invention include but are not limited to: application server, SQL database, Ceph file storage server, Redis cache server, message queue server, etc.

[0096] like Figure 3FIG. 1 is a flowchart of a specific implementation of a dual-mode watermark-based color ring back tone audit method according to an embodiment of the present invention. Figure 3 The dual-mode watermark of the embodiment of the present invention includes a robust watermark whose information can still be read after being attacked and a fragile watermark which is immediately destroyed after being attacked. When the robust watermark and the fragile watermark cannot be extracted from the color ring tone to be reviewed, it indicates that the ring tone is a new ring tone without a watermark embedded. At this time, it is necessary to conduct a content review on the color ring tone to be reviewed, and embed the robust watermark and the fragile watermark after passing the content review, and then put it into the warehouse; when the robust watermark and the fragile watermark can be extracted from the ring tone to be reviewed, it indicates that the ring tone has appeared on the system and has not been modified / transcribed. At this time, the information carried by the dual-mode watermark is used to speed up the review of the ring tone; when the robust watermark can be extracted from the color ring tone to be reviewed but the fragile watermark cannot be extracted, it indicates that the ring tone has appeared on the system but has been modified / transcribed. At this time, it is necessary to compare the original ring tone of the platform to determine the copyright information, so as to conduct a copyright review on the color ring tone to be reviewed and confirm whether the modification / transcription behavior is legal.

[0097] It can be recognized that the use of dual-mode watermark technology can provide strong copyright protection for the audio and video ringtones of ringtone operators; the use of dual-mode watermark technology can accurately determine whether the ringtone to be reviewed has been modified / transcribed, thereby assisting the ringtone review, which can not only quickly check out counterfeit / tampered ringtones, but also quickly confirm reused ringtones, reducing the difficulty and repetitive workload of manual review, reducing the number of calls to the paid machine review, and achieving the goal of reducing costs and increasing efficiency.

[0098] like Figure 4 FIG. 1 is a flowchart of a method for embedding a robust watermark for a ring back tone according to an embodiment of the present invention. Figure 4 As an optional implementation, the following steps are performed to embed the color ring back tone into a robust watermark:

[0099] S201, segmenting the embedded ringback tone to obtain a plurality of first audio regions;

[0100] S202, performing fast Fourier transform on each first audio region to obtain first frequency spectrum information of each first audio region;

[0101] S203, determining a first high frequency component of each first audio region according to the first frequency spectrum information, and selecting a plurality of first audio regions as first embedding regions according to the first high frequency components;

[0102] S204, performing frequency domain transformation on the first embedding region to obtain first frequency domain information, and selecting a plurality of medium and low frequency bands of the first frequency domain information as first embedding frequency bands;

[0103] S205, generating a robust watermark to be embedded according to a preset robust watermark generation rule, and determining a plurality of bits of first binary embedding information according to the robust watermark to be embedded;

[0104] S206, modifying the mid- and low-frequency coefficients of the first embedded frequency band of the first frequency domain information according to the first binary embedded information to obtain a first watermark frequency domain;

[0105] S207: Perform inverse frequency domain transformation on the first watermark frequency domain to obtain a first watermark region, and then replace the first embedding region to be embedded in the color ring back tone with the first watermark region to obtain a color ring back tone carrying a robust watermark.

[0106] Specifically, common audio modifications include cutting out the head and tail, resampling, ripping, etc. Robust watermarking technology has two major requirements: First, robustness, that is, the information embedded in it can still be extracted completely after a series of modifications, by reducing the BER (bit error rate), or by using encoding and decoding technology to reduce the information error rate. Second, hiddenness, that is, people cannot perceive any difference between the audio and video after the embedded information is different from the original, and it does not affect the quality of the audio and video.

[0107] Usually, robustness and concealment are contradictory requirements. Thanks to the small amount of information embedded in practical applications, the embodiments of the present invention use mature technology to perfectly meet the requirements of robustness and concealment at the same time. The specific process includes: 1) selecting a specific area in the carrier for embedding watermarks; 2) performing frequency domain transformation on each embedded area; 3) embedding watermark information in a specific frequency band in the frequency domain; 4) performing inverse domain transformation and replacing the original embedded area. The following are explained separately.

[0108] 1) Select a specific area in the carrier for embedding watermark

[0109] In order to meet the requirements of robustness and concealment, and to more effectively resist various attacks, some watermark technologies do not embed watermarks in the entire audio segment, but select specific areas to embed. The principle of selection is the area where the audio signal changes significantly, such as using discrete wavelet transform to obtain the carrier's detail signal as the area to be embedded; or by calculating the first-order difference and second-order difference of the carrier to determine the area to be embedded.

[0110] Taking advantage of the fact that Fast Fourier Transform (FFT) is mature, simple and fast in implementation and calculation, the present invention proposes a self-developed segmented FFT method for selecting the embedded area, which efficiently solves the task of selecting the embedded area. The solution is as follows:

[0111] First, divide the entire audio carrier C into n regions C with roughly the same energy. i :

[0112] C1,C2,…C i …C n ∈C, and Power(C i )≈Power(Cj )

[0113] Among them, when the volume is low at the beginning and end of the audio, the time corresponding to the area is longer, and when the volume is high, the time corresponding to the area is shorter.

[0114] Perform fast Fourier transform on each region to obtain the spectrum F of each region. i :

[0115] F i =FFT(C i )

[0116] According to the capacity of the information to be embedded, select the m with the largest high-frequency components, and their corresponding time domain area C k The embedded area.

[0117] 2) Perform frequency domain transformation on each embedded region

[0118] Mainstream watermarking algorithms all embed watermark information into the frequency domain and use DCT (discrete cosine transform) or DWT (discrete wavelet transform) for domain transformation. The embodiment of the present invention selects DCT to perform frequency domain transformation on each embedded area:

[0119] D k =DCT(C k )

[0120] D obtained by frequency domain transformation k That is, it is used for subsequent watermark embedding.

[0121] 3) Embed watermark information in a specific frequency band in the frequency domain

[0122] Considering that the human ear is more sensitive to the low-frequency part of the audio, if the watermark is embedded in it, the low-frequency part will be changed, which will affect the concealment requirement; and if the watermark is embedded in the high-frequency part, when the audio is resampled or ripped, the high-frequency part will be damaged, which will affect the robustness requirement. In combination with the embodiment of the present invention, the amount of information to be embedded is not large, so several medium and low frequency bands are selected to embed the watermark information. When the watermark information of some frequency bands is destroyed, the original information can still be restored based on the information of other frequency bands, which is conducive to improving robustness.

[0123] Embedded watermark is a p Is 0 or 1, modify the low-frequency coefficient u p For u' p The process can be expressed by the following formula:

[0124]

[0125] Among them, step is the quantization step size, and int() is the rounding process.

[0126] For example, if step is 2, when u p When the information to be embedded is 0, u' p is 36.5; when the information to be embedded is 1, u' p is 37.5; when u p When the information to be embedded is 0, u' p is 38.5; when the information to be embedded is 1, u' p is 39.5. Therefore, the DCT coefficient of the embedded area is k becomes D′ k .

[0127] 4) Perform inverse frequency domain transform and replace the original embedded area

[0128] D′ k Perform inverse discrete cosine transform (or inverse discrete cosine transform) to obtain C′ k :

[0129] C′ k = IDCT(D′ k )

[0130] Then use C′ k Replace the original audio area C k , so that the original audio carrier C becomes C′ with watermark information.

[0131] like Figure 5 FIG. 1 is a flowchart of robust watermark extraction for a ring back tone to be reviewed provided by an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, robust watermark extraction is performed on the ringback tone to be reviewed through the following steps:

[0132] S301, segmenting the ringback tone to be reviewed to obtain a plurality of second audio regions;

[0133] S302, performing fast Fourier transform on each second audio region to obtain second frequency spectrum information of each second audio region;

[0134] S303, determining a second high frequency component of each second audio region according to the second frequency spectrum information, and selecting a plurality of second audio regions as first extraction regions according to the second high frequency components;

[0135] S304, performing frequency domain transformation on the first extraction region to obtain second frequency domain information, and selecting a plurality of medium and low frequency bands of the second frequency domain information as first extraction frequency bands;

[0136] S305, determining a plurality of bits of first binary extraction information according to the mid- and low-frequency coefficients of the first extraction frequency band, and recombining the first binary extraction information to obtain a robust watermark to be verified;

[0137] S306, judging whether the robust watermark to be verified conforms to the preset robust watermark generation rule;

[0138] S307: if the robust watermark to be verified complies with the robust watermark generation rule, it is determined that the ringback tone to be reviewed carries the robust watermark; if the robust watermark to be verified does not comply with the robust watermark generation rule, it is determined that the ringback tone to be reviewed does not carry the robust watermark.

[0139] Specifically, extracting watermark information is a process very similar to embedding watermarks. ′ C can be used * Indicates. * Use the "segmented FFT to select embedded areas" method that is exactly the same as in the previous steps to confirm the watermark embedding area; then perform DCT transformation on these areas; and then extract the low- and medium-frequency coefficients u * p , according to the following formula to determine whether the originally embedded information is 0 or 1:

[0140]

[0141] Continuing from the previous example, if the extracted mid- and low-frequency coefficients u * p is 37.52, it can be judged that the previously embedded bit is 1; if the proposed low-frequency coefficient u * p is 38.44, so it can be determined that the previously embedded bit was 0.

[0142] According to the determined multi-bit binary extracted information, the robust watermark to be verified can be reconstructed and restored. In addition, the original watermark information needs to be preprocessed such as spreading, which is not the focus of the embodiment of the present invention and will not be described in detail here.

[0143] like Figure 6 FIG. 1 is a flow chart of embedding a fragile watermark into a ring back tone according to an embodiment of the present invention. Figure 6 As an optional implementation, the following steps are performed to embed a fragile watermark into the ringback tone:

[0144] S401, segmenting the color ring back tone to be embedded to obtain a plurality of third audio regions, and using the third audio region as a second embedding region;

[0145] S402, performing frequency domain transformation on the second embedding region to obtain third frequency domain information, and selecting a plurality of high frequency bands of the third frequency domain information as second embedding frequency bands;

[0146] S403, generating a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determining a plurality of bits of second binary embedding information according to the fragile watermark to be embedded;

[0147] S404, modifying the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain;

[0148] S405: Perform inverse frequency domain transformation on the second watermark frequency domain to obtain a second watermark region, and then replace the second embedding region to be embedded in the color ring back tone with the second watermark region to obtain a color ring back tone carrying a fragile watermark.

[0149] Specifically, different from the traditional robust watermark, the fragile watermark in the present invention is designed to meet two major requirements: first, the fragility requirement, that is, the information embedded in it cannot be extracted once it is attacked; second, the hidden requirement, that is, people cannot perceive that the audio and video after the embedded information is different from the original, and it does not affect the quality of the audio and video.

[0150] The technical solution of fragile watermark is similar to that of the aforementioned robust watermark, and is also divided into the same four main steps, but corresponding adjustments need to be made to meet the requirements of fragility.

[0151] 1) Selecting a specific area in the carrier for embedding watermarks: In order to be sensitive to possible attacks on audio, including common cropping attacks, the entire time domain needs to be selected as the embedding area. In the embodiment of the present invention, the entire audio carrier C is divided into m embedding areas of equal length.

[0152] 2) Perform frequency domain transformation on each embedded region: In the embodiment of the present invention, DCT is selected to transform each embedded region.

[0153] 3) Embed watermark information in a specific frequency band in the frequency domain: The embodiment of the present invention selects a high-frequency frequency band to embed watermark information, so as to detect resampling or ripping tampering. And select a very small quantization step, such as 0.05. A small quantization step means that the modification of the original audio during the embedding process is extremely small, so we can select multiple frequency bands to embed watermarks. When the watermark information of any frequency band is destroyed, it is considered that the audio has been attacked, which is conducive to improving fragility.

[0154] 4) Perform inverse domain transformation and replace the original embedded region.

[0155] like Figure 7 FIG. 1 is a flow chart of extracting fragile watermarks for a ringback tone to be reviewed provided by an embodiment of the present invention, referring to FIG. Figure 7 As an optional implementation, the fragile watermark is extracted from the ringback tone to be reviewed through the following steps:

[0156] S501, segmenting the color ring tone to be reviewed to obtain a plurality of fourth audio regions, and using the fourth audio region as a second extraction region;

[0157] S502, performing frequency domain transformation on the second extraction region to obtain fourth frequency domain information, and selecting a plurality of high frequency bands of the fourth frequency domain information as second extraction frequency bands;

[0158] S503, determining a plurality of bits of second binary extraction information according to the high frequency coefficient of the second extraction frequency band, and recombining the second binary extraction information to obtain a fragile watermark to be verified;

[0159] S504, determining whether the fragile watermark to be verified conforms to a preset fragile watermark generation rule;

[0160] S505: If the fragile watermark to be verified meets the fragile watermark generation rule, it is determined that the ringback tone to be reviewed carries the fragile watermark; if the fragile watermark to be verified does not meet the fragile watermark generation rule, it is determined that the ringback tone to be reviewed does not carry the fragile watermark.

[0161] Specifically, the process of extracting a fragile watermark is the inverse process of embedding a fragile watermark. For details, please refer to the specific process of extracting a robust watermark. According to the determined multi-bit binary embedded information, the fragile watermark to be verified can be reconstructed and restored. In addition, the original watermark information needs to be preprocessed such as spreading, which is not the focus of the embodiment of the present invention and will not be described in detail here.

[0162] In an embodiment of the present invention, at the stage of embedding a watermark in a new ring tone, the information carried by the robust watermark is generated according to the following rules: a fixed Chinese string "XXXXX (operator name)" + 16 random digits is used as the original information, and then encrypted using the National Cryptography Standard 4 (SM4.0).

[0163] I r ="XXXXX"+rand_number 16

[0164] W r =SM4_Encode(I r )

[0165] The relationship between the 16-bit random number and the new ring tone index is saved in the robust_watermark_table table in the database, as shown in Table 1 below.

[0166] Table 1

[0167]

[0168] A batch (e.g., 100) of 32-bit strings consisting of numbers and letters are preset. Different strings vary greatly, and they represent different categories (such as the industry affiliation of the ringtone, whether the ringtone is allowed to be used by other customers, etc.), and are written into the fragile_watermark_table table of the database. An example is shown in Table 2 below.

[0169] Table 2

[0170]

[0171] The information carried by the fragile watermark is generated according to the following rules: the prefix is ​​"XXXXX (operator name)", and then according to the classification of the ring tone, a matching string is selected as the suffix.

[0172] I f =W f ="XXXXX"+string 32

[0173] Using the robust watermark embedding technique and fragile watermark embedding technique described above, embed W r and W f .

[0174] In the embodiment of the present invention, in the watermark extraction stage, the robust watermark extraction technology and the fragile watermark extraction technology described above are used to calculate the robust watermark W′ of the ringtone to be processed. r and fragile watermark W′ f . And for W′ r Call the National Encryption 4.0 to decrypt, and you will get

[0175] I′ r =SM4_Decode(W′ r )

[0176] I′ f =W′ f

[0177] According to I' r and I′ f There are three situations to determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark:

[0178] 1) I′ r If it is not a pattern of "XXXXX" + 16 digits, it can be considered that the ringtone does not carry a robust watermark. In this case, it should not carry a fragile watermark either.

[0179] 2) I′ rIf the pattern is "XXXXX" + 16 digits, the ringtone is considered to have a robust watermark. And the obtained 16 digits should be searchable in the robust_watermark_table. f It is "XXXXX" + 32-bit string pattern, and the string can also be found in fragile_watermark_table, that is, it carries a fragile watermark.

[0180] 3) I′ r If the pattern is "XXXXX" + 16 digits, the ringtone is considered to have a robust watermark. And the obtained 16 digits should be searchable in the robust_watermark_table. f It is not a "XXXXX" + 32-bit string, or the string cannot be found in the fragile_watermark_table, that is, it does not carry a fragile watermark.

[0181] The last two cases of appeal can be used to verify the copyright of the ringtone. In addition, different business strategies are applied to the three cases of appeal to perform machine review and manual review.

[0182] like Figure 8 FIG. 1 is a flow chart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 8 As an optional implementation, the fragile watermark carries the ring tone ownership information and the use permission information. The use scope of the ring tone to be reviewed is determined according to the watermark extraction result. The use review of the ring tone to be reviewed is performed according to the use scope, which specifically includes:

[0183] S1031, determining a target fragile watermark of the color ring tone to be reviewed according to the watermark extraction result, and determining target color ring tone ownership information and target use permission information of the color ring tone to be reviewed according to the target fragile watermark;

[0184] S1032, determining the usage scope of the CRBT to be reviewed according to the target CRBT ownership information and the target usage permission information;

[0185] S1033, obtaining the uploaded customer information of the ringback tone to be reviewed, and determining whether the uploaded customer information complies with the usage scope;

[0186] S1034. If the uploaded customer information meets the usage scope, it is determined that the ringback tone to be reviewed has passed the usage review. If the uploaded customer information does not meet the usage scope, it is determined that the ringback tone to be reviewed has not passed the usage review.

[0187] like Fig. 9 FIG. 1 is a flowchart of step S104 provided in an embodiment of the present invention, referring to FIG. Fig. 9As an optional implementation, the copyright information of the ringback tone to be reviewed is determined according to the watermark extraction result, and the copyright review of the ringback tone to be reviewed is performed according to the copyright information, which specifically includes:

[0188] S1041, determining a target robust watermark of the CRBT to be reviewed according to the watermark extraction result, and performing a matching search in a preset CRBT database according to the target robust watermark to obtain an initial platform CRBT corresponding to the CRBT to be reviewed;

[0189] S1042, determining the copyright information of the ringback tone to be reviewed according to the initial platform ringback tone;

[0190] S1043, obtaining the uploaded customer information of the ringback tone to be reviewed, and determining whether the uploaded customer information is consistent with the copyright information;

[0191] S1044. If the uploaded customer information is consistent with the copyright information, it is determined that the ringback tone to be reviewed has passed the copyright review. If the uploaded customer information is inconsistent with the copyright information, it is determined that the ringback tone to be reviewed has not passed the copyright review.

[0192] Specifically, for ringtones that do not carry dual-mode watermarks, calculate the file's md5 checksum. If it cannot be found in the platform system, it can double-ensure that the ringtone to be processed is a brand new ringtone for the platform, and save the md5 checksum. According to the old process, the ringtone needs to be machine- and manually-reviewed to ensure the compliance and legality of the ringtone, and the certificate of the ringtone's approval (such as the serial number of the machine-reviewed pass) is also saved. For the approved ringtones, generate a robust watermark W according to the previous rules. r (and write the correspondence between random numbers and ringtones into the robust_watermark_table table) and the fragile watermark W f The robust watermark embedding technology and the fragile watermark embedding technology in the previous sections are called in sequence to form a ring tone with a dual-mode watermark, and the relevant information and files are imported into a database or file storage warehouse.

[0193] For ringtones with both robust watermarks and fragile watermarks, it can be determined that the original ringtone of the ringtone exists on the platform, and the original ringtone can be found by searching the robust_watermark_table with a 16-digit number, and the current ringtone is consistent with the original ringtone. The use scope of the ringtone to be reviewed can be determined based on the ringtone ownership information and usage permission information carried by the fragile watermark. For example, if the usage permission information is "can be promoted and reused", the corresponding use scope is "unrestricted to customers". If the usage permission information is "can be reused in various police stations and street offices", the corresponding use scope is "various police stations and street offices". If the usage permission information is "exclusive", the corresponding use scope can be determined as the original uploading customer based on the ringtone ownership information. Then, the uploaded customer information of the ringtone to be reviewed is obtained to determine whether the uploaded customer information meets the scope of use. For example, if the 32-bit suffix of the extracted fragile watermark indicates that the current ringtone can be disseminated and used within a certain range, and the current customer meets the relevant rules, it can be directly reviewed and approved; if the current ringtone is dedicated and reuse is not allowed, and the customer to whom the current ringtone belongs is different from the customer to whom the original ringtone belongs, it can be directly reviewed and rejected. This can avoid calling the payment machine review interface and reduce the workload of manual reviewers.

[0194] For ringtones with robust watermarks but without fragile watermarks, it can be determined that the original ringtone of the ringtone exists on the platform, and the original ringtone can be found by searching the robust_watermark_table with a 16-bit number. But now the original ringtone has been modified (it may be ripped, the head and tail may be removed, etc.). In this case, it needs to attract the attention of the auditor. The current ringtone can be compared with the original ringtone to strictly examine whether there is any doubt about the ownership of the current ringtone. For the behavior of counterfeiting the original ringtone, the ringtone will be marked as not passing the audit and the reason for failure.

[0195] The above is an explanation of the method flow of the embodiment of the present invention. It can be recognized that the embodiment of the present invention embeds a robust watermark and a fragile watermark into a new ringback tone, and can subsequently determine whether the ringback tone to be reviewed is an existing ringback tone on the platform and whether it has been modified by extracting the dual-mode watermark. At the same time, the use scope and copyright information of the ringback tone to be reviewed can also be quickly obtained, thereby realizing a rapid review of the new ringback tone added by the customer, and improving the efficiency and accuracy of the ringback tone review.

[0196] like Fig.10 FIG. 1 is a schematic diagram of the structure of a dual-mode watermark-based ring back tone verification device provided by an embodiment of the present invention, referring to FIG. Fig.10 The embodiment of the present invention provides a dual-mode watermark-based ring back tone auditing device, comprising:

[0197] The watermark extraction module is used to obtain the ringback tone to be reviewed, extract the watermark of the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result;

[0198] The watermark embedding module is used to determine that the to-be-reviewed ringtone is a new ringtone added to the platform when the to-be-reviewed ringtone does not carry a robust watermark and a fragile watermark, and then embed the robust watermark and the fragile watermark into the to-be-reviewed ringtone after the to-be-reviewed ringtone passes the content review to obtain the stored ringtone;

[0199] The use audit module is used to determine that the ringback tone to be reviewed is an existing ringback tone on the platform when the ringback tone to be reviewed carries a robust watermark and a fragile watermark, and then determine the use scope of the ringback tone to be reviewed based on the watermark extraction result, and perform a use audit on the ringback tone to be reviewed based on the use scope;

[0200] The copyright review module is used to determine that the ringtone to be reviewed is a platform-modified ringtone when the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, and then determine the copyright information of the ringtone to be reviewed based on the watermark extraction result, and perform copyright review on the ringtone to be reviewed based on the copyright information.

[0201] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0202] The embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and when the program is executed by the processor, the above-mentioned color ring tone review method based on dual-mode watermark is realized. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0203] like Fig.11 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Fig.11 , an embodiment of the present invention provides an electronic device, including:

[0204] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0205] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the dual-mode watermark-based color ringback tone audit method of the embodiment of the present invention;

[0206] Input / output interface 1103, used to implement information input and output;

[0207] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0208] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0209] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0210] like Fig.12 FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.12 The embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1201, and the one or more programs 1201 can be executed by one or more processors to implement the above-mentioned color ringback tone review method based on dual-mode watermark.

[0211] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0212] The embodiment of the present invention also discloses a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0213] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. The optional embodiment is expected, wherein the order of various operations is changed and the sub-operation described as a part of a larger operation is performed independently.

[0214] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0215] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0216] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0217] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0218] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0219] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0220] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0221] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A color ring back tone audit method based on dual-mode watermark, characterized in that: The following steps are involved: Acquire the color ring tone to be reviewed, extract watermark from the color ring tone to be reviewed, and determine whether the color ring tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result; When the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, it is determined that the CRBT to be reviewed is a newly added CRBT on the platform, and then after the CRBT to be reviewed passes the content review, a robust watermark and a fragile watermark are embedded to obtain a stored CRBT; When the CRBT to be reviewed carries a robust watermark and a fragile watermark, it is determined that the CRBT to be reviewed is an existing CRBT on the platform, and then the usage scope of the CRBT to be reviewed is determined according to the watermark extraction result, and the usage of the CRBT to be reviewed is reviewed according to the usage scope; When the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, it is determined that the ringtone to be reviewed is a platform-modified ringtone, and then the copyright information of the ringtone to be reviewed is determined according to the watermark extraction result, and the copyright review of the ringtone to be reviewed is performed according to the copyright information.

2. According to claim 1, a dual-mode watermark-based color ring back tone audit method is characterized in that: The following steps are used to embed the robust watermark into the ringback tone: Segmenting the color ring tone to be embedded to obtain a plurality of first audio regions; Performing a fast Fourier transform on each of the first audio regions to obtain first frequency spectrum information of each of the first audio regions; determining a first high frequency component of each of the first audio regions according to the first frequency spectrum information, and selecting a plurality of the first audio regions as first embedding regions according to the first high frequency components; Performing frequency domain transformation on the first embedding region to obtain first frequency domain information, and selecting a plurality of medium and low frequency bands of the first frequency domain information as first embedding frequency bands; Generate a to-be-embedded robust watermark according to a preset robust watermark generation rule, and determine a plurality of bits of first binary embedding information according to the to-be-embedded robust watermark; Modify the mid- and low-frequency coefficients of the first embedding frequency band of the first frequency domain information according to the first binary embedding information to obtain a first watermark frequency domain; The first watermark frequency domain is transformed in the inverse frequency domain to obtain a first watermark region, and then the first embedding region of the color ring tone to be embedded is replaced with the first watermark region to obtain a color ring tone carrying a robust watermark.

3. According to claim 1, a dual-mode watermark-based color ring back tone audit method is characterized in that: The robust watermark of the to-be-approved ring back tone is extracted by the following steps: Segmenting the color ring tone to be reviewed to obtain a plurality of second audio regions; Performing a fast Fourier transform on each of the second audio regions to obtain second frequency spectrum information of each of the second audio regions; determining a second high frequency component of each of the second audio regions according to the second frequency spectrum information, and selecting a plurality of the second audio regions as first extraction regions according to the second high frequency components; Performing frequency domain transformation on the first extraction area to obtain second frequency domain information, and selecting a plurality of medium and low frequency bands of the second frequency domain information as first extraction frequency bands; Determine a plurality of bits of first binary extraction information according to the mid- and low-frequency coefficients of the first extraction frequency band, and reconstruct the first binary extraction information to obtain a robust watermark to be verified; Determining whether the robust watermark to be verified complies with a preset robust watermark generation rule; If the robust watermark to be verified conforms to the robust watermark generation rule, it is determined that the color ring tone to be reviewed carries a robust watermark; if the robust watermark to be verified does not conform to the robust watermark generation rule, it is determined that the color ring tone to be reviewed does not carry a robust watermark.

4. The method for auditing color ring tone based on dual-mode watermark according to claim 1, characterized in that: Use the following steps to embed a fragile watermark into a ringback tone: Segmenting the color ring back tone to be embedded to obtain a plurality of third audio regions, and using the third audio regions as second embedding regions; Performing frequency domain transformation on the second embedding region to obtain third frequency domain information, and selecting a plurality of high frequency bands of the third frequency domain information as second embedding frequency bands; Generate a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determine a plurality of bits of second binary embedding information according to the fragile watermark to be embedded; Modify the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain; The second watermark frequency domain is transformed inversely to obtain a second watermark region, and then the second embedding region to be embedded in the ringback tone is replaced with the second watermark region to obtain a ringback tone carrying a fragile watermark.

5. The method for auditing color ring tone based on dual-mode watermark according to claim 1, characterized in that: The fragile watermark of the ring back tone to be reviewed is extracted by the following steps: Segmenting the color ring tone to be reviewed to obtain a plurality of fourth audio regions, and using the fourth audio regions as second extraction regions; Performing frequency domain transformation on the second extraction area to obtain fourth frequency domain information, and selecting a plurality of high frequency bands of the fourth frequency domain information as second extraction frequency bands; Determining a plurality of bits of second binary extraction information according to the high frequency coefficient of the second extraction frequency band, and recombining the second binary extraction information to obtain a fragile watermark to be verified; Determining whether the fragile watermark to be verified meets the preset fragile watermark generation rule; If the fragile watermark to be verified meets the fragile watermark generation rule, it is determined that the ringback tone to be reviewed carries a fragile watermark; if the fragile watermark to be verified does not meet the fragile watermark generation rule, it is determined that the ringback tone to be reviewed does not carry a fragile watermark.

6. A method for auditing color ring back tone based on dual-mode watermark according to any one of claims 1 to 5, characterized in that: The fragile watermark carries the ring tone ownership information and the use permission information. The method of determining the use scope of the ring tone to be reviewed according to the watermark extraction result and performing the use review on the ring tone to be reviewed according to the use scope specifically includes: Determine the target fragile watermark of the color ring tone to be reviewed according to the watermark extraction result, and determine the target color ring tone attribution information and target use permission information of the color ring tone to be reviewed according to the target fragile watermark; Determine the usage scope of the to-be-approved CRBT according to the target CRBT attribution information and the target usage permission information; Acquire the uploaded customer information of the to-be-reviewed ringback tone, and determine whether the uploaded customer information complies with the usage scope; If the uploaded customer information conforms to the usage scope, it is determined that the to-be-reviewed ringtone has passed the usage review; if the uploaded customer information does not conform to the usage scope, it is determined that the to-be-reviewed ringtone has not passed the usage review.

7. A method for auditing color ring back tone based on dual-mode watermark according to any one of claims 1 to 5, characterized in that: The step of determining the copyright information of the to-be-examined ringtone according to the watermark extraction result, and performing copyright examination on the to-be-examined ringtone according to the copyright information specifically includes: Determine the target robust watermark of the CRBT to be reviewed according to the watermark extraction result, and perform a matching search in a preset CRBT database according to the target robust watermark to obtain an initial platform CRBT corresponding to the CRBT to be reviewed; Determining the copyright information of the to-be-reviewed ringback tone according to the initial platform ringback tone; Acquire the uploaded customer information of the to-be-reviewed ringback tone, and determine whether the uploaded customer information is consistent with the copyright information; If the uploaded customer information is consistent with the copyright information, it is determined that the pending ringtone has passed the copyright review; if the uploaded customer information is inconsistent with the copyright information, it is determined that the pending ringtone has not passed the copyright review.

8. A dual-mode watermark-based ringback tone verification device, characterized in that: include: A watermark extraction module is used to obtain the color ring tone to be reviewed, extract the watermark of the color ring tone to be reviewed, and determine whether the color ring tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result; A watermark embedding module, for determining that the CRBT to be reviewed is a newly added CRBT on the platform when the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, and then embedding a robust watermark and a fragile watermark after the CRBT to be reviewed passes the content review to obtain a stored CRBT; A usage audit module is used to determine that the CRBT to be reviewed is an existing CRBT on the platform when the CRBT to be reviewed carries a robust watermark and a fragile watermark, and then determine the usage scope of the CRBT to be reviewed according to the watermark extraction result, and perform usage audit on the CRBT to be reviewed according to the usage scope; The copyright audit module is used to determine that the ringtone to be reviewed is a platform-modified ringtone when the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, and then determine the copyright information of the ringtone to be reviewed according to the watermark extraction result, and perform copyright audit on the ringtone to be reviewed according to the copyright information.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the color ringback tone review method based on dual-mode watermark are realized as described in any one of claims 1 to 7.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the color ring back tone audit method based on dual-mode watermark as described in any one of claims 1 to 7.

Images