Color ringback tone audit method, device, electronic device and medium based on dual-mode watermark

Abstract

The present invention discloses a dual-mode watermark-based color ringback tone (CRBT) auditing method, device, electronic device, and medium, including: extracting a watermark from a color ringback tone to be audited; when the color ringback tone to be audited does not carry a robust watermark and a fragile watermark, determining that the color ringback tone to be audited is a newly added color ringback tone to the platform, and embedding a robust watermark and a fragile watermark into the color ringback tone to obtain a stored color ringback tone after the color ringback tone to be audited passes content audit; when the color ringback tone to be audited carries a robust watermark and a fragile watermark, determining that the color ringback tone to be audited is an existing color ringback tone to the platform, determining the usage scope of the color ringback tone to be audited based on the watermark extraction result, and performing a usage audit on the color ringback tone to be audited based on the usage scope; when the color ringback tone to be audited carries a robust watermark but does not carry a fragile watermark, determining that the color ringback tone to be audited is a modified color ringback tone to the platform, determining the copyright information of the color ringback tone to be audited based on the watermark extraction result, and performing a copyright audit on the color ringback tone to be audited based on the copyright information. The present invention improves the efficiency and accuracy of color ringback tone auditing and can be widely applied to the field of color ringback tone technology.

Description

Technical Field

[0001] The present invention relates to the technical field of color ring back tone, and in particular to a color ring back tone auditing method, device, electronic equipment and medium based on dual-mode watermark. Background Art

[0002] Audio digital watermarking is a mature and effective audio copyright protection technology. It embeds copyright information into the audio being protected. The human auditory system is typically unable to distinguish watermarked audio from the original audio. When necessary, matching extraction techniques can be used to extract the original embedded information from the audio, thereby confirming audio copyright ownership or tracing leaks. Existing audio digital watermarking technologies aim to improve robustness against attacks and tampering, known as robust watermarking. For example, even if the watermarked audio is cropped, resampled, or even ripped, the correct watermark information can still be extracted.

[0003] Business ringback tone (CRBT) is a ringback tone service for government and enterprise customers. Customers create promotional audio or video and set it as a ringtone for their government or enterprise landline phones or employee work phones. When a third party calls these government or enterprise landline numbers or employee numbers, the pre-set promotional audio or video plays on the calling phone during the ringing period.

[0004] After a customer's audio or video ringtones are created and uploaded to the business ringtone platform, they undergo both manual and machine review to ensure their legality. In addition to checking for illegal text, speech, images, and videos, manual verification is also required to ensure the consistency of the audio and video content with the customer's. It is important to prevent private enterprises from misappropriating ringtones from government agencies or carefully crafted ringtones from other companies. This can lead to extensive manual review. Furthermore, ringtone services are ever-changing, and in some scenarios, it's desirable for a single ringtone to be reused or used by multiple customers. For example, a customer may reuse an existing audio or video (because they prefer the original ringtone). An organization may want small customers to use a promotional ringtone for synergistic marketing (e.g., the Cultural and Tourism Bureau produces a ringtone promoting local tourism, and multiple small and micro businesses in the cultural and tourism sector use the same ringtone). Clients from different government departments may use the same ringtone (e.g., multiple police stations in a city use the same anti-fraud ringtone). This can lead to extensive and repeated manual and machine review.

[0005] Considering that digital watermark technology should be used to add copyright identification to business ringtones, it is more important to combine watermark technology with the review process of business ringtones. However, the existing digital watermark technology focuses on restoring watermark information without error, but ignores whether the audio and video carrier itself is attacked (such as cropping, modification), thereby affecting the efficiency and accuracy of ringtone review.

[0006] Explanation of terms:

[0007] Digital Watermark: Embed specific information into a digital carrier (such as an image, audio, or video). If the embedded information is not perceived by humans and does not affect the presentation of the original digital carrier, such watermark information is called a hidden digital watermark.

[0008] FFT (Fast Fourier Transform): It is an efficient algorithm for calculating discrete Fourier transform (DFT). It can convert time domain signals into frequency domain signals, thereby realizing the analysis and processing of the frequency components of the signal.

[0009] DCT (Discrete Cosine Transform): A signal processing method that can decompose a signal (such as audio) and extract different frequency components in the signal.

[0010] IDCT (Inverse Discrete Cosine Transform): The inverse processing of DCT, which restores the coefficients after DCT processing to the original signal. Summary of the Invention

[0011] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0012] To this end, one purpose of an embodiment of the present invention is to provide a ringback tone review method based on dual-mode watermarks, which embeds robust watermarks and fragile watermarks into the stored ringback tones, so that the dual-mode watermarks can be used to quickly review the customer's newly added ringback tones, thereby improving the efficiency and accuracy of the ringback tone review.

[0013] Another object of an embodiment of the present invention is to provide a color ring back tone verification device based on dual-mode watermark.

[0014] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0015] In one aspect, an embodiment of the present invention provides a method for auditing a ring back tone based on a dual-mode watermark, comprising the following steps:

[0016] Acquire a ringback tone to be reviewed, extract a watermark from the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result;

[0017] When the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, the CRBT to be reviewed is determined to be a newly added CRBT to the platform, and then after the CRBT to be reviewed passes the content review, the robust watermark and the fragile watermark are embedded to obtain a stored CRBT;

[0018] When the CRBT to be reviewed carries a robust watermark and a fragile watermark, determining that the CRBT to be reviewed is an existing CRBT on the platform, further determining the usage scope of the CRBT to be reviewed based on the watermark extraction result, and performing a usage review on the CRBT to be reviewed based on the usage scope;

[0019] When the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, the ringtone to be reviewed is determined to be a platform-modified ringtone, and then the copyright information of the ringtone to be reviewed is determined according to the watermark extraction result, and the copyright review of the ringtone to be reviewed is performed according to the copyright information.

[0020] Furthermore, in one embodiment of the present invention, the following steps are performed to embed a robust watermark into the embedded ring back tone:

[0021] Segmenting the ringback tone to be embedded to obtain a plurality of first audio regions;

[0022] performing a fast Fourier transform on each of the first audio regions to obtain first spectrum information of each of the first audio regions;

[0023] determining a first high-frequency component of each of the first audio regions according to the first spectrum information, and selecting a plurality of the first audio regions as first embedding regions according to the first high-frequency components;

[0024] Performing frequency domain transformation on the first embedding region using DCT to obtain first frequency domain information, and selecting a plurality of mid- and low-frequency bands of the first frequency domain information as first embedding frequency bands;

[0025] generating a to-be-embedded robust watermark according to a preset robust watermark generation rule, and determining a plurality of bits of first binary embedding information according to the to-be-embedded robust watermark;

[0026] Modify the mid- and low-frequency coefficients of the first embedded frequency band of the first frequency domain information according to the first binary embedded information to obtain a first watermark frequency domain;

[0027] The first watermark frequency domain is subjected to inverse frequency domain transformation using IDCT to obtain a first watermark region, and the first embedding region of the ringback tone to be embedded is replaced with the first watermark region to obtain a ringback tone carrying a robust watermark.

[0028] Furthermore, in one embodiment of the present invention, robust watermark extraction is performed on the CRBT to be reviewed through the following steps:

[0029] Segmenting the ringback tone to be reviewed to obtain a plurality of second audio regions;

[0030] performing a fast Fourier transform on each of the second audio regions to obtain second spectrum information of each of the second audio regions;

[0031] determining a second high-frequency component of each second audio region according to the second frequency spectrum information, and selecting a plurality of second audio regions as first extraction regions according to the second high-frequency components;

[0032] Performing frequency domain transformation on the first extraction area using DCT to obtain second frequency domain information, and selecting multiple mid- and low-frequency bands of the second frequency domain information as first extraction frequency bands;

[0033] Determining a plurality of bits of first binary extraction information according to the mid- and low-frequency coefficients of the first extraction frequency band, and recombining the first binary extraction information to obtain a robust watermark to be verified;

[0034] Determining whether the robust watermark to be verified complies with a preset robust watermark generation rule;

[0035] If the robust watermark to be verified meets the robust watermark generation rule, it is determined that the CRBT to be reviewed carries a robust watermark; if the robust watermark to be verified does not meet the robust watermark generation rule, it is determined that the CRBT to be reviewed does not carry a robust watermark.

[0036] Furthermore, in one embodiment of the present invention, a fragile watermark is embedded in the embedded ring back tone through the following steps:

[0037] Segmenting the ringback tone to be embedded to obtain a plurality of third audio regions, and using the third audio regions as second embedding regions;

[0038] Performing frequency domain transformation on the second embedding region using DCT to obtain third frequency domain information, and selecting multiple high-frequency frequency bands of the third frequency domain information as second embedding frequency bands;

[0039] generating a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determining a plurality of bits of second binary embedding information according to the fragile watermark to be embedded;

[0040] Modify the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain;

[0041] The second watermark frequency domain is inversely transformed using IDCT to obtain a second watermark region, and the second embedding region to be embedded in the ringback tone is replaced with the second watermark region to obtain a ringback tone carrying a fragile watermark.

[0042] Furthermore, in one embodiment of the present invention, a fragile watermark is extracted from the ringback tone to be reviewed through the following steps:

[0043] Segmenting the CRBT to be reviewed to obtain a plurality of fourth audio regions, and using the fourth audio regions as second extraction regions;

[0044] Performing frequency domain transformation on the second extraction area using DCT to obtain fourth frequency domain information, and selecting multiple high-frequency bands of the fourth frequency domain information as second extraction frequency bands;

[0045] determining a plurality of bits of second binary extraction information according to the high frequency coefficients of the second extraction frequency band, and recombining the second binary extraction information to obtain a fragile watermark to be verified;

[0046] Determining whether the fragile watermark to be verified complies with a preset fragile watermark generation rule;

[0047] If the fragile watermark to be verified meets the fragile watermark generation rule, it is determined that the ringback tone to be reviewed carries a fragile watermark; if the fragile watermark to be verified does not meet the fragile watermark generation rule, it is determined that the ringback tone to be reviewed does not carry a fragile watermark.

[0048] Furthermore, in one embodiment of the present invention, the fragile watermark carries ring tone ownership information and usage permission information. Determining the usage scope of the ring tone to be reviewed based on the watermark extraction result and performing a usage review on the ring tone to be reviewed based on the usage scope specifically includes:

[0049] Determining a target fragile watermark of the CRBT to be reviewed based on the watermark extraction result, and determining target CRBT ownership information and target usage permission information of the CRBT to be reviewed based on the target fragile watermark;

[0050] determining the usage scope of the to-be-approved CRBT according to the target CRBT ownership information and the target usage permission information;

[0051] Obtaining the uploaded customer information of the ringback tone to be reviewed, and determining whether the uploaded customer information complies with the usage scope;

[0052] If the uploaded customer information meets the usage scope, it is determined that the pending ringtone has passed the usage review; if the uploaded customer information does not meet the usage scope, it is determined that the pending ringtone has not passed the usage review.

[0053] Furthermore, in one embodiment of the present invention, determining the copyright information of the CRBT to be reviewed based on the watermark extraction result, and performing copyright review on the CRBT to be reviewed based on the copyright information, specifically includes:

[0054] Determining a target robust watermark of the CRBT to be reviewed based on the watermark extraction result, and performing a matching search in a preset CRBT database based on the target robust watermark to obtain an initial platform CRBT corresponding to the CRBT to be reviewed;

[0055] Determining the copyright information of the to-be-reviewed ringback tone according to the initial platform ringback tone;

[0056] Obtaining the uploaded customer information of the ringback tone to be reviewed, and determining whether the uploaded customer information is consistent with the copyright information;

[0057] If the uploaded customer information is consistent with the copyright information, it is determined that the pending ringtone has passed the copyright review; if the uploaded customer information is inconsistent with the copyright information, it is determined that the pending ringtone has failed the copyright review.

[0058] On the other hand, an embodiment of the present invention provides a dual-mode watermark-based ringback tone verification device, comprising:

[0059] a watermark extraction module, configured to obtain a ringback tone to be reviewed, extract a watermark from the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result;

[0060] A watermark embedding module is configured to, when the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, determine that the CRBT to be reviewed is a newly added CRBT to the platform, and then embed a robust watermark and a fragile watermark into the CRBT to be reviewed after the CRBT to be reviewed passes content review, thereby obtaining a stored CRBT;

[0061] A usage audit module is configured to determine, when the CRBT to be reviewed carries a robust watermark and a fragile watermark, that the CRBT to be reviewed is an existing CRBT on the platform, further determine the usage scope of the CRBT to be reviewed based on the watermark extraction result, and perform a usage audit on the CRBT to be reviewed based on the usage scope;

[0062] The copyright audit module is used to determine that the ringtone to be reviewed is a platform-modified ringtone when the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, and then determine the copyright information of the ringtone to be reviewed based on the watermark extraction result, and perform copyright audit on the ringtone to be reviewed based on the copyright information.

[0063] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored on the memory and runnable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the color ringback tone review method based on dual-mode watermark as described above is realized.

[0064] On the other hand, an embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the color ringback tone review method based on dual-mode watermark as described above.

[0065] The advantages and benefits of the present invention will be described in part in the following description and will become apparent from the following description or learned through practice of the present invention:

[0066] An embodiment of the present invention obtains a ringtone to be reviewed, performs watermark extraction on the ringtone to be reviewed, and determines whether the ringtone to be reviewed carries a robust watermark and a fragile watermark based on the watermark extraction result; when the ringtone to be reviewed does not carry the robust watermark and the fragile watermark, it is determined that the ringtone to be reviewed is a newly added ringtone to the platform, and then the robust watermark and the fragile watermark are embedded after the ringtone to be reviewed passes the content review to obtain a stored ringtone; when the ringtone to be reviewed carries the robust watermark and the fragile watermark, it is determined that the ringtone to be reviewed is an existing ringtone on the platform, and then the scope of use of the ringtone to be reviewed is determined based on the watermark extraction result, and a usage review is performed on the ringtone to be reviewed based on the scope of use; when the ringtone to be reviewed carries the robust watermark but does not carry the fragile watermark, it is determined that the ringtone to be reviewed is a modified ringtone on the platform, and then the copyright information of the ringtone to be reviewed is determined based on the watermark extraction result, and a copyright review is performed on the ringtone to be reviewed based on the copyright information. The embodiment of the present invention embeds a robust watermark and a fragile watermark into a new ringback tone. Subsequently, the dual-mode watermark can be extracted to determine whether the ringback tone to be reviewed is an existing ringback tone on the platform and whether it has been modified. At the same time, the usage scope and copyright information of the ringback tone to be reviewed can also be quickly obtained, thereby realizing a rapid review of the customer's newly added ringback tone, thereby improving the efficiency and accuracy of the ringback tone review. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0068] Figure 1 A flowchart of a method for auditing a color ring tone based on a dual-mode watermark according to an embodiment of the present invention;

[0069] Figure 2 A schematic diagram of an application scenario of the dual-mode watermark-based ringback tone verification method provided by an embodiment of the present invention;

[0070] Figure 3A flowchart of a specific implementation of the dual-mode watermark-based ringback tone verification method provided by an embodiment of the present invention;

[0071] Figure 4 A flow chart of embedding a robust watermark for a ring back tone according to an embodiment of the present invention;

[0072] Figure 5 A flowchart of robust watermark extraction for a ring back tone to be reviewed provided by an embodiment of the present invention;

[0073] Figure 6 A flow chart of embedding a fragile watermark into a ring back tone according to an embodiment of the present invention;

[0074] Figure 7 A flow chart of extracting fragile watermarks from a ringback tone to be reviewed provided by an embodiment of the present invention;

[0075] Figure 8 A flowchart of step S103 provided in an embodiment of the present invention;

[0076] Figure 9 A flowchart of step S104 provided in an embodiment of the present invention;

[0077] Figure 10 A schematic diagram of the structure of a dual-mode watermark-based ringback tone verification device provided by an embodiment of the present invention;

[0078] Figure 11 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;

[0079] Figure 12 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0080] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limitations on the present application. It should be noted that, although the functional modules are divided in the system schematic and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system schematic or the order in the flow chart. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and no limitation is placed on the order between the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0081] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of "first" or "second", it is only used to distinguish technical features and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this document are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0082] The dual-mode watermark-based ringback tone audit method provided in the embodiments of the present application can be applied to a terminal, can be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the dual-mode watermark-based ringback tone audit method, etc., but is not limited to the above forms.

[0083] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0084] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0085] Existing watermarking technology focuses on improving robustness and researching different implementation algorithms. However, ringtone review is a critical step in the commercial audio and video ringtone (RBT) service. Ringtone operators must prevent the public dissemination of illegal audio and video content and prevent criminals from using stolen ringtones to commit fraud or even scams. The current business process relies solely on a combination of machine review and manual auditing, which is inefficient, costly, and prone to errors. Strong technical support is needed for this review process.

[0086] The present invention innovatively proposes a dual-mode watermark technology. The dual-mode watermark includes a robust watermark that can still read information even when attacked, and a fragile watermark that is immediately destroyed when attacked. The dual-mode watermark information is embedded in the carrier audio itself, just like DNA, and is accompanied by the carrier's copying, compression, modification, and other behaviors. By detecting the dual-mode watermark, the copyright ownership of the corresponding carrier (such as an audio ringtone) and whether it has been attacked and tampered with can be determined; at the same time, by extracting the information carried by the dual-mode watermark, business processing can be assisted and accelerated. Ultimately, by applying the dual-mode watermark technology, the present invention not only achieves the goal of reducing costs and increasing efficiency, but also improves the level of information security.

[0087] Current CRBT platforms offer both audio and video CRBTs. Some CRBT customers have audio, some have video, and some have both audio and video. The dual-mode watermarking technology described in this invention can be used as a carrier for either audio or video CRBTs, meaning that the embodiments of this invention are applicable to both audio and video CRBTs.

[0088] In practice, considering the current efficiency and maturity of audio watermarking, this invention uses audio as the carrier for watermark embedding and extraction. However, it should be noted that this invention can be applied to both audio-only ringback tone (CRBT) and the audio portion (audio track) of video CRBTs, meaning it is applicable to business CRBTs for a variety of customers. The following explanation primarily uses audio as an example.

[0089] Furthermore, the carrier of the dual-mode watermark can also be video (the video portion of a video ringback tone). Compared to audio watermarks, video watermarks have the advantage of being able to embed more information and providing better concealment. However, their disadvantage is that the embedding and extraction operations are relatively complex and inefficient. It should be understood that applying the dual-mode watermarking technology of the present invention to the video portion of a video ringback tone should still be considered within the scope of protection of the present invention.

[0090] like Figure 1 FIG2 is a flowchart of a method for auditing a ring back tone based on a dual-mode watermark according to an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a method for auditing ringback tone based on dual-mode watermark, which specifically includes the following steps:

[0091] S101: Obtain a ringback tone to be reviewed, extract a watermark from the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark based on the watermark extraction result;

[0092] S102: If the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, the CRBT to be reviewed is determined to be a newly added CRBT to the platform, and then the robust watermark and the fragile watermark are embedded in the CRBT to be reviewed after the CRBT to be reviewed passes the content review, thereby obtaining a stored CRBT;

[0093] S103: If the CRBT to be reviewed carries a robust watermark and a fragile watermark, the CRBT to be reviewed is determined to be an existing CRBT on the platform, and then the usage scope of the CRBT to be reviewed is determined based on the watermark extraction result, and the usage of the CRBT to be reviewed is reviewed based on the usage scope;

[0094] S104: When the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, the ringtone to be reviewed is determined to be a platform-modified ringtone, and the copyright information of the ringtone to be reviewed is determined based on the watermark extraction result. The ringtone to be reviewed is subjected to copyright review based on the copyright information.

[0095] Specifically, if Figure 2 The figure shows an application scenario for the dual-mode watermark-based ringback tone (CRBT) auditing method provided by an embodiment of the present invention. The unified backend for commercial lottery services and management serves as the core of the entire commercial lottery business. It connects to front-end portals and other service portals, receiving customer-uploaded CRBT tones and providing CRBT generation capabilities. It also connects to the front-end management portal and has the ability to call specialized functional module interfaces, including machine auditing capabilities and watermark processing algorithm capabilities. Furthermore, the commercial lottery services and management backend connect to databases and file storage repositories. Devices used in this embodiment of the present invention include, but are not limited to, application servers, SQL databases, Ceph file storage servers, Redis cache servers, and message queue servers.

[0096] like Figure 3FIG2 is a flowchart showing a specific implementation of the dual-mode watermark-based ring back tone verification method provided by an embodiment of the present invention, referring to FIG2. Figure 3 The dual-mode watermark of the embodiment of the present invention includes a robust watermark whose information can still be read even when attacked, and a fragile watermark that is immediately destroyed when attacked. When the robust watermark and the fragile watermark cannot be extracted from the ringtone to be reviewed, it indicates that the ringtone is a new ringtone without an embedded watermark. At this time, it is necessary to conduct a content review on the ringtone to be reviewed, and embed the robust watermark and the fragile watermark after passing the content review, and then put it into the warehouse; when the robust watermark and the fragile watermark can be extracted from the ringtone to be reviewed, it indicates that the ringtone has appeared on the system and has not been modified / transcribed. At this time, the information carried by the dual-mode watermark is used to speed up the review of the ringtone; when the robust watermark can be extracted from the ringtone to be reviewed but the fragile watermark cannot be extracted, it indicates that the ringtone has appeared on the system but has been modified / transcribed. At this time, it is necessary to compare the original ringtone on the platform to determine the copyright information, so as to conduct a copyright review on the ringtone to be reviewed and confirm whether the modification / transcription behavior is legal.

[0097] It can be recognized that the use of dual-mode watermark technology can provide strong copyright protection for the audio and video ringtones of ringtone operators; the use of dual-mode watermark technology can accurately determine whether the ringtone to be reviewed has been modified / transcribed, thereby assisting the ringtone review, and can quickly detect counterfeit / tampered ringtones, and can also quickly confirm reused ringtones, reducing the difficulty and repetitive workload of manual review, reducing the number of calls to the paid machine review, and achieving the goal of reducing costs and increasing efficiency.

[0098] like Figure 4 FIG2 is a flow chart of a method for embedding a robust watermark into a ring back tone according to an embodiment of the present invention. Figure 4 As an optional implementation, the following steps are performed to embed a robust watermark into the embedded ring back tone:

[0099] S201, segmenting the embedded ringback tone to obtain a plurality of first audio regions;

[0100] S202, performing a fast Fourier transform on each first audio region to obtain first spectrum information of each first audio region;

[0101] S203, determining a first high frequency component of each first audio region according to the first spectrum information, and selecting a plurality of first audio regions as first embedding regions according to the first high frequency components;

[0102] S204: Perform frequency domain transformation on the first embedding region to obtain first frequency domain information, and select multiple mid- and low-frequency bands of the first frequency domain information as first embedding frequency bands;

[0103] S205: generating a robust watermark to be embedded according to a preset robust watermark generation rule, and determining a plurality of bits of first binary embedding information according to the robust watermark to be embedded;

[0104] S206: Modify the mid- and low-frequency coefficients of the first embedded frequency band of the first frequency domain information according to the first binary embedded information to obtain a first watermark frequency domain;

[0105] S207: Perform inverse frequency domain transformation on the first watermark frequency domain to obtain a first watermark region, and then replace the first embedding region to be embedded in the CRBT with the first watermark region to obtain a CRBT carrying a robust watermark.

[0106] Specifically, common audio modifications include cropping, removing headers and tails, resampling, and ripping. Robust watermarking technology has two major requirements: First, robustness, which requires that the embedded information can still be fully extracted after a series of modifications. This can be achieved by reducing the bit error rate (BER) or by using codecs. Second, invisibility, which requires that the audio or video after the embedded information is not noticeably different from the original, without affecting the quality of the audio or video.

[0107] Robustness and concealment are often conflicting requirements. However, due to the limited amount of information embedded in practical applications, the present invention utilizes mature technology to achieve both robustness and concealment requirements. The specific process includes: 1) selecting specific regions within the carrier for watermark embedding; 2) performing frequency domain transformation on each embedded region; 3) embedding watermark information within a specific frequency band within the frequency domain; and 4) performing inverse domain transformation and replacing the original embedded region. Each of these steps is described below.

[0108] 1) Select a specific area in the carrier for embedding the watermark

[0109] To ensure robustness and concealment, and to more effectively resist various attacks, some watermarking technologies embed watermarks in specific regions rather than the entire audio segment. These regions are chosen based on significant changes in the audio signal. For example, discrete wavelet transforms can be used to obtain detailed signals from the carrier, or the first and second order differences of the carrier can be used to determine the embedding region.

[0110] Taking advantage of the maturity, simplicity, and speed of Fast Fourier Transform (FFT) in implementation and calculation, this paper proposes a self-developed segmented FFT method for selecting the embedding region, which efficiently solves the task of embedding region selection. The solution is as follows:

[0111] First, divide the entire audio carrier C into n segments C with roughly the same energy. i :

[0112] C1,C2,…C i …C n ∈C, and Power(C i )≈Power(Cj )

[0113] Among them, when the volume is low at the beginning and end of the audio, the time corresponding to the area is longer, and when the volume is high, the time corresponding to the area is shorter.

[0114] Perform fast Fourier transform on each region to obtain the spectrum F of each region i :

[0115] F i =FFT(C i )

[0116] According to the capacity of the information to be embedded, select the m with the largest high-frequency components, and their corresponding time domain area C k The embedded area.

[0117] 2) Perform frequency domain transformation on each embedded area

[0118] Mainstream watermarking algorithms all embed watermark information into the frequency domain and use DCT (discrete cosine transform) or DWT (discrete wavelet transform) for domain transformation. The embodiment of the present invention selects DCT to perform frequency domain transformation on each embedded area:

[0119] D k =DCT(C k )

[0120] D obtained by frequency domain transformation k That is, it is used for subsequent watermark embedding.

[0121] 3) Embed watermark information in a specific frequency band in the frequency domain

[0122] Given that the human ear is more sensitive to low-frequency audio, embedding a watermark there would alter this low-frequency portion, compromising stealth. Conversely, embedding the watermark in high-frequency audio could damage it when the audio is resampled or ripped, impacting robustness. In this embodiment of the present invention, the amount of information to be embedded is relatively small, so several mid- and low-frequency bands are selected for watermarking. Even if the watermark information in some bands is corrupted, the original information can still be restored based on information in other bands, improving robustness.

[0123] Embedded watermark is a p Is 0 or 1, modify the low and medium frequency coefficient u p for u' p The process can be expressed as follows:

[0124]

[0125] Among them, step is the quantization step size, and int() is the rounding process.

[0126] For example, if step is 2, when u p When the value is between 36 and 38, and the information to be embedded is 0, u' p is 36.5; when the information to be embedded is 1, u' p is 37.5; when u p When the value is between 38 and 40, when the information to be embedded is 0, u' p is 38.5; when the information to be embedded is 1, u' p is 39.5. Therefore, the DCT coefficient of the embedded area is k becomes D′ k .

[0127] 4) Perform inverse frequency domain transformation and replace the original embedded area

[0128] For D′ k Perform inverse discrete cosine transform (or inverse discrete cosine transform) to obtain C′ k :

[0129] C′ k =IDCT(D′ k )

[0130] Then use C′ k Replace the original audio area C k , so that the original audio carrier C becomes C′ with watermark information.

[0131] like Figure 5 FIG2 is a flowchart of robust watermark extraction for a ring back tone to be audited provided by an embodiment of the present invention, referring to FIG2. Figure 5 As an optional implementation, robust watermark extraction is performed on the ringback tone to be reviewed through the following steps:

[0132] S301, segmenting the ringback tone to be reviewed to obtain multiple second audio regions;

[0133] S302, performing a fast Fourier transform on each second audio region to obtain second spectrum information of each second audio region;

[0134] S303, determining a second high-frequency component of each second audio region according to the second spectrum information, and selecting a plurality of second audio regions as first extraction regions according to the second high-frequency components;

[0135] S304, performing frequency domain transformation on the first extraction region to obtain second frequency domain information, and selecting multiple mid- and low-frequency bands of the second frequency domain information as first extraction bands;

[0136] S305: determining a plurality of bits of first binary extraction information according to the mid- and low-frequency coefficients of the first extraction frequency band, and recombining the first binary extraction information to obtain a robust watermark to be verified;

[0137] S306: Determine whether the robust watermark to be verified complies with a preset robust watermark generation rule;

[0138] S307: If the robust watermark to be verified complies with the robust watermark generation rule, it is determined that the ringback tone to be reviewed carries the robust watermark; if the robust watermark to be verified does not comply with the robust watermark generation rule, it is determined that the ringback tone to be reviewed does not carry the robust watermark.

[0139] Specifically, extracting watermark information is a process very similar to embedding watermark. ′ You can use C * Indicates. * Use the "segmented FFT selection embedding area" method that is exactly the same as in the previous steps to confirm the watermark embedding area; then perform DCT transform on these areas; and then extract the low- and medium-frequency coefficients u * p , use the following formula to determine whether the originally embedded information is 0 or 1:

[0140]

[0141] Continuing from the previous example, if the extracted mid- and low-frequency coefficients u * p is 37.52, it can be judged that the previously embedded bit is 1; if the proposed low-frequency coefficient u * p The value is 38.44, indicating that the previously embedded bit was 0.

[0142] The determined multi-bit binary extracted information can be reconstructed and restored to obtain the robust watermark to be verified. In addition, the original watermark information needs to be pre-processed such as spread spectrum, which is not the focus of the embodiment of the present invention and will not be described in detail here.

[0143] like Figure 6 The flowchart of embedding fragile watermark into a ring back tone according to an embodiment of the present invention is shown. Figure 6 As an optional implementation, the following steps are performed to embed a fragile watermark into the embedded ringback tone:

[0144] S401: Segment the ringback tone to be embedded to obtain multiple third audio regions, and use the third audio regions as second embedding regions;

[0145] S402: Perform frequency domain transformation on the second embedding region to obtain third frequency domain information, and select multiple high-frequency bands of the third frequency domain information as second embedding frequency bands;

[0146] S403: generating a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determining a plurality of bits of second binary embedding information according to the fragile watermark to be embedded;

[0147] S404: Modify the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain;

[0148] S405: Perform inverse frequency domain transformation on the second watermark frequency domain to obtain a second watermark region, and then replace the second embedding region to be embedded in the ringback tone with the second watermark region to obtain a ringback tone carrying a fragile watermark.

[0149] Specifically, unlike traditional robust watermarks, the fragile watermark in the present invention is designed to meet two major requirements: first, the fragility requirement, that is, the information embedded in it cannot be extracted once it is attacked; second, the hidden requirement, that is, people cannot perceive that the audio and video after the embedded information is different from the original, and it does not affect the quality of the audio and video.

[0150] The technical solution of fragile watermark is similar to that of the aforementioned robust watermark, and is also divided into the same four main steps, but corresponding adjustments need to be made to meet the fragility requirements.

[0151] 1) Selecting a specific region in the carrier for watermark embedding: To be sensitive to possible attacks on the audio, including common cropping attacks, the entire time domain needs to be selected as the embedding region. In this embodiment of the present invention, the entire audio carrier C is divided into m embedding regions of equal duration.

[0152] 2) Performing frequency domain transformation on each embedded region: In the embodiment of the present invention, DCT is selected to perform transformation on each embedded region.

[0153] 3) Embedding watermark information in specific frequency bands in the frequency domain: This embodiment of the present invention selects high-frequency bands for watermarking to facilitate detection of resampling or ripping. A very small quantization step size, such as 0.05, is also selected. A small quantization step size means that the embedding process minimally modifies the original audio, allowing the watermark to be embedded in multiple frequency bands. If the watermark information in any frequency band is corrupted, the audio is considered to have been attacked, which helps improve fragility.

[0154] 4) Perform inverse domain transformation and replace the original embedded area.

[0155] like Figure 7 The flowchart of extracting fragile watermark of the ring back tone to be audited provided by the embodiment of the present invention is shown as follows. Figure 7 As an optional implementation, the fragile watermark of the ringback tone to be reviewed is extracted through the following steps:

[0156] S501: Segment the CRBT to be reviewed to obtain multiple fourth audio regions, and use the fourth audio regions as second extraction regions;

[0157] S502: Perform frequency domain transformation on the second extraction region to obtain fourth frequency domain information, and select multiple high-frequency bands of the fourth frequency domain information as second extraction frequency bands;

[0158] S503, determining a plurality of bits of second binary extraction information according to the high-frequency coefficients of the second extraction frequency band, and recombining the second binary extraction information to obtain a fragile watermark to be verified;

[0159] S504: Determine whether the fragile watermark to be verified complies with the preset fragile watermark generation rules;

[0160] S505: If the fragile watermark to be verified meets the fragile watermark generation rule, it is determined that the ringback tone to be reviewed carries the fragile watermark; if the fragile watermark to be verified does not meet the fragile watermark generation rule, it is determined that the ringback tone to be reviewed does not carry the fragile watermark.

[0161] Specifically, the process of extracting a fragile watermark is the inverse of embedding a fragile watermark. For details, refer to the detailed process of extracting a robust watermark. Based on the determined multi-bit binary embedded information, the fragile watermark to be verified can be reconstructed and restored. Furthermore, the original watermark information must undergo preprocessing, such as spread spectrum processing, but this is not the focus of the present invention and will not be discussed here.

[0162] In an embodiment of the present invention, during the watermark embedding stage of a new ringtone, the information carried by the robust watermark is generated according to the following rules: a fixed Chinese character string "XXXXX (operator name)" + a 16-bit random number is used as the original information, and then encrypted using the State Secret 4 (SM4.0).

[0163] I r ="XXXXX"+rand_number 16

[0164] W r =SM4_Encode(I r )

[0165] The relationship between the 16-bit random number and the new ringtone index is saved in the robust_watermark_table table in the database. An example is shown in Table 1 below.

[0166] Table 1

[0167]

[0168] A batch (e.g., 100) of 32-bit strings consisting of numbers and letters are preset. Different strings vary greatly, and they represent different categories (such as the industry affiliation of the ringtone, whether the ringtone is allowed to be used by other customers, etc.). They are written into the fragile_watermark_table table in the database. An example is shown in Table 2 below.

[0169] Table 2

[0170]

[0171] The information carried by the fragile watermark is generated according to the following rules: the prefix is ​​"XXXXX (operator name)", and then according to the classification of the ring tone, a matching string is selected as the suffix.

[0172] I f =W f ="XXXXX"+string 32

[0173] Using the robust watermark embedding technology and fragile watermark embedding technology mentioned above, embed W r and W f .

[0174] In the embodiment of the present invention, in the watermark extraction stage, the robust watermark extraction technology and the fragile watermark extraction technology described above are used to calculate the robust watermark W′ of the ringtone to be processed. r and fragile watermark W′ f . And for W′ r Call the National Cryptography 4.0 to decrypt, and you will get

[0175] I′ r =SM4_Decode(W′ r )

[0176] I′ f =W′ f

[0177] According to I′ r and I′ f There are three situations to determine whether the ringback tone to be reviewed carries a robust watermark or a fragile watermark:

[0178] 1) I′ r If the pattern is not "XXXXX" + 16 digits, the ringtone is considered not to carry a robust watermark. In this case, it should not carry a fragile watermark either.

[0179] 2) I′ rIf the pattern is "XXXXX" + 16 digits, the ringtone is considered to have a robust watermark. The resulting 16 digits should be searchable in the robust_watermark_table. f It is "XXXXX" + 32-bit string pattern, and the string can also be found in fragile_watermark_table, that is, it carries a fragile watermark.

[0180] 3) I′ r If the pattern is "XXXXX" + 16 digits, the ringtone is considered to have a robust watermark. And the obtained 16 digits should be searchable in the robust_watermark_table. f The string is not "XXXXX" + 32 bits, or the string cannot be found in the fragile_watermark_table. In other words, the string does not carry a fragile watermark.

[0181] The last two cases can be used to verify the copyright of the ringtone. In addition, different business strategies are applied to the three cases to perform machine review and manual review.

[0182] like Figure 8 FIG. 1 is a flowchart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 8 As an optional implementation, the fragile watermark carries ring tone ownership information and usage permission information. The usage scope of the ring tone to be reviewed is determined based on the watermark extraction result. The usage review of the ring tone to be reviewed is performed based on the usage scope, which specifically includes:

[0183] S1031. Determine a target fragile watermark of the CRBT to be reviewed based on the watermark extraction result, and determine target CRBT ownership information and target usage permission information of the CRBT to be reviewed based on the target fragile watermark.

[0184] S1032: Determine the usage scope of the CRBT to be reviewed based on the target CRBT ownership information and the target usage permission information;

[0185] S1033. Obtain the uploaded customer information of the ringback tone to be reviewed, and determine whether the uploaded customer information meets the usage scope;

[0186] S1034. If the uploaded customer information meets the usage scope, it is determined that the ringback tone to be reviewed has passed the usage review. If the uploaded customer information does not meet the usage scope, it is determined that the ringback tone to be reviewed has not passed the usage review.

[0187] like Figure 9 FIG. 1 is a flowchart of step S104 provided in an embodiment of the present invention, referring to FIG. Figure 9As an optional implementation, the copyright information of the ringback tone to be reviewed is determined based on the watermark extraction result, and copyright review is performed on the ringback tone to be reviewed based on the copyright information, which specifically includes:

[0188] S1041: Determine a target robust watermark for the CRBT to be reviewed based on the watermark extraction result, and perform a matching search in a preset CRBT database based on the target robust watermark to obtain an initial platform CRBT corresponding to the CRBT to be reviewed;

[0189] S1042. Determine the copyright information of the ringback tone to be reviewed based on the initial platform ringback tone;

[0190] S1043. Obtain the uploaded customer information of the ringback tone to be reviewed, and determine whether the uploaded customer information is consistent with the copyright information;

[0191] S1044. If the uploaded customer information is consistent with the copyright information, it is determined that the ringtone to be reviewed has passed the copyright review. If the uploaded customer information is inconsistent with the copyright information, it is determined that the ringtone to be reviewed has failed the copyright review.

[0192] Specifically, for ringtones that do not carry dual-mode watermarks, calculate the file's md5 checksum. If it cannot be found in the platform system, it can double-check that the ringtone to be processed is brand new to the platform, and save the md5 checksum. According to the old process, the ringtone must be machine- and manually reviewed to ensure its compliance and legality, and the certificate of the ringtone's approval (such as the serial number of the machine-approved approval form) must also be saved. For the ringtone that has passed the review, generate a robust watermark W according to the previous rules. r (And write the correspondence between random numbers and ringtones into the robust_watermark_table table) and the fragile watermark W f The robust watermark embedding technology and the fragile watermark embedding technology in the above sections are called in sequence to form a ring tone with a dual-mode watermark, and the relevant information and files are imported into a database or file storage warehouse.

[0193] For ringtones with both robust and fragile watermarks, it can be determined that the original ringtone exists on the platform, can be found by searching the robust_watermark_table using a 16-digit number, and that the current ringtone is identical to the original. The ringtone ownership information and usage permission information carried by the fragile watermark can be used to determine the scope of use of the pending ringtone. For example, if the usage permission information is "Can be reused for general use," the corresponding usage scope is "Unrestricted customers." If the usage permission information is "Can be reused in various police stations and sub-district offices," the corresponding usage scope is "Various police stations and sub-district offices." If the usage permission information is "Exclusive use," the ringtone ownership information can be used to determine that the corresponding usage scope is the original uploader. The system then retrieves the customer information that uploaded the ringtone to be reviewed and determines whether it complies with the usage scope. For example, if the extracted 32-bit suffix of the fragile watermark indicates that the ringtone can be distributed and used within a certain range and that the current customer meets the relevant regulations, the ringtone will be approved directly. If the ringtone is dedicated and reuse is not allowed, and the customer to whom the current ringtone belongs is different from the original customer, the ringtone will be rejected directly. This avoids calling the payment machine's review interface and reduces the workload of manual reviewers.

[0194] For ringtones with robust watermarks but no fragile watermarks, it can be determined that the original ringtone exists on the platform and can be found by searching the robust_watermark_table using a 16-digit number. However, the original ringtone has been modified (perhaps by ripping, removing the header and tail, etc.). In this case, the auditor should pay close attention. The current ringtone can be compared with the original ringtone to strictly examine whether there are any doubts about the current ringtone's ownership. In the case of counterfeiting the original ringtone, the ringtone will be marked as unapproved and the reason for the failure will be stated.

[0195] The above describes the method flow of an embodiment of the present invention. It can be appreciated that the embodiment of the present invention embeds both a robust watermark and a fragile watermark into a new ringback tone. Subsequently, by extracting the dual-mode watermark, it can determine whether the ringback tone to be reviewed is an existing ringback tone on the platform and whether it has been modified. It can also quickly obtain the usage scope and copyright information of the ringback tone to be reviewed, thereby enabling rapid review of the customer's newly added ringback tone, improving the efficiency and accuracy of ringback tone review.

[0196] like Figure 10 FIG2 is a schematic diagram showing the structure of a dual-mode watermark-based ring back tone verification device according to an embodiment of the present invention. Figure 10 The embodiment of the present invention provides a dual-mode watermark-based ringback tone verification device, comprising:

[0197] The watermark extraction module is used to obtain the ringback tone to be reviewed, extract the watermark of the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark or a fragile watermark based on the watermark extraction result;

[0198] The watermark embedding module is used to determine that the CRBT to be reviewed is a new CRBT on the platform when it does not carry the robust watermark and the fragile watermark. Then, after the CRBT to be reviewed passes the content review, the robust watermark and the fragile watermark are embedded in the CRBT to obtain the stored CRBT.

[0199] The usage audit module is used to determine whether the ringtone to be reviewed is an existing ringtone on the platform when it carries a robust watermark and a fragile watermark, and then determine the usage scope of the ringtone to be reviewed based on the watermark extraction result, and perform usage audit on the ringtone to be reviewed based on the usage scope;

[0200] The copyright review module is used to determine that the ringtone to be reviewed is a platform-modified ringtone when the ringtone to be reviewed carries a robust watermark but does not carry a fragile watermark, and then determine the copyright information of the ringtone to be reviewed based on the watermark extraction result, and perform copyright review on the ringtone to be reviewed based on the copyright information.

[0201] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0202] An embodiment of the present invention further provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned dual-mode watermark-based color ring tone verification method is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0203] like Figure 11 FIG2 is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention, referring to FIG2 Figure 11 , an embodiment of the present invention provides an electronic device, including:

[0204] The processor 1101 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0205] The memory 1102 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called by the processor 1101 to execute the dual-mode watermark-based color ring back tone verification method of the embodiment of the present invention.

[0206] Input / output interface 1103, used to implement information input and output;

[0207] Communication interface 1104, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0208] Bus 1105 , which transmits information between various components of the device (e.g., processor 1101 , memory 1102 , input / output interface 1103 , and communication interface 1104 );

[0209] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via a bus 1105 .

[0210] like Figure 12 FIG2 is a schematic diagram of the structure of the storage medium provided by the embodiment of the present invention, referring to FIG2 Figure 12 An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1201, and the one or more programs 1201 can be executed by one or more processors to implement the above-mentioned color ringback tone review method based on dual-mode watermark.

[0211] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0212] The embodiment of the present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0213] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0214] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present invention set forth in the claims using ordinary skills without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0215] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0216] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0217] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable media on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0218] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0219] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0220] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0221] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for auditing ringback tone based on dual-mode watermark, characterized in that: The following steps are involved: Acquire a ringback tone to be reviewed, extract a watermark from the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result; When the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, the CRBT to be reviewed is determined to be a newly added CRBT to the platform, and then after the CRBT to be reviewed passes the content review, the robust watermark and the fragile watermark are embedded to obtain a stored CRBT; When the CRBT to be reviewed carries a robust watermark and a fragile watermark, determining that the CRBT to be reviewed is an existing CRBT on the platform, further determining the usage scope of the CRBT to be reviewed based on the watermark extraction result, and performing a usage review on the CRBT to be reviewed based on the usage scope; When the CRBT to be reviewed carries a robust watermark but does not carry a fragile watermark, determining that the CRBT to be reviewed is a platform-modified CRBT, further determining the copyright information of the CRBT to be reviewed based on the watermark extraction result, and performing a copyright review on the CRBT to be reviewed based on the copyright information; The following steps are used to embed a robust watermark into the ring back tone: Segmenting the ringback tone to be embedded to obtain a plurality of first audio regions; performing a fast Fourier transform on each of the first audio regions to obtain first spectrum information of each of the first audio regions; determining a first high-frequency component of each of the first audio regions according to the first spectrum information, and selecting a plurality of the first audio regions as first embedding regions according to the first high-frequency components; Performing frequency domain transformation on the first embedding region to obtain first frequency domain information, and selecting a plurality of mid- and low-frequency bands of the first frequency domain information as first embedding frequency bands; generating a to-be-embedded robust watermark according to a preset robust watermark generation rule, and determining a plurality of bits of first binary embedding information according to the to-be-embedded robust watermark; Modify the mid- and low-frequency coefficients of the first embedded frequency band of the first frequency domain information according to the first binary embedded information to obtain a first watermark frequency domain; Performing an inverse frequency domain transform on the first watermark frequency domain to obtain a first watermark region, and then replacing the first embedding region of the ringback tone to be embedded with the first watermark region to obtain a ringback tone carrying a robust watermark; To embed a fragile watermark into a ringback tone, perform the following steps: Segmenting the ringback tone to be embedded to obtain a plurality of third audio regions, and using the third audio regions as second embedding regions; Performing frequency domain transformation on the second embedding region to obtain third frequency domain information, and selecting a plurality of high frequency bands of the third frequency domain information as second embedding frequency bands; generating a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determining a plurality of bits of second binary embedding information according to the fragile watermark to be embedded; Modify the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain; An inverse frequency domain transform is performed on the second watermark frequency domain to obtain a second watermark region, and then the second embedding region to be embedded in the ringback tone is replaced with the second watermark region to obtain a ringback tone carrying a fragile watermark.

2. The method for auditing color ring back tone based on dual-mode watermark according to claim 1, characterized in that: Robust watermark extraction is performed on the pending ring back tone through the following steps: Segmenting the ringback tone to be reviewed to obtain a plurality of second audio regions; performing a fast Fourier transform on each of the second audio regions to obtain second spectrum information of each of the second audio regions; determining a second high-frequency component of each second audio region according to the second frequency spectrum information, and selecting a plurality of second audio regions as first extraction regions according to the second high-frequency components; Performing frequency domain transformation on the first extraction area to obtain second frequency domain information, and selecting multiple mid- and low-frequency bands of the second frequency domain information as first extraction frequency bands; Determining a plurality of bits of first binary extraction information according to the mid- and low-frequency coefficients of the first extraction frequency band, and recombining the first binary extraction information to obtain a robust watermark to be verified; Determining whether the robust watermark to be verified complies with a preset robust watermark generation rule; If the robust watermark to be verified meets the robust watermark generation rule, it is determined that the CRBT to be reviewed carries a robust watermark; if the robust watermark to be verified does not meet the robust watermark generation rule, it is determined that the CRBT to be reviewed does not carry a robust watermark.

3. The method for auditing color ring back tone based on dual-mode watermark according to claim 1, characterized in that: Extract the fragile watermark from the ringback tone to be reviewed by the following steps: Segmenting the CRBT to be reviewed to obtain a plurality of fourth audio regions, and using the fourth audio regions as second extraction regions; Performing frequency domain transformation on the second extraction area to obtain fourth frequency domain information, and selecting multiple high-frequency bands of the fourth frequency domain information as second extraction frequency bands; determining a plurality of bits of second binary extraction information according to the high frequency coefficients of the second extraction frequency band, and recombining the second binary extraction information to obtain a fragile watermark to be verified; Determining whether the fragile watermark to be verified complies with a preset fragile watermark generation rule; If the fragile watermark to be verified meets the fragile watermark generation rule, it is determined that the ringback tone to be reviewed carries a fragile watermark; if the fragile watermark to be verified does not meet the fragile watermark generation rule, it is determined that the ringback tone to be reviewed does not carry a fragile watermark.

4. A method for auditing color ring back tone based on dual-mode watermark according to any one of claims 1 to 3, characterized in that: The fragile watermark carries ring tone ownership information and usage permission information. The method of determining the usage scope of the ring tone to be reviewed based on the watermark extraction result and performing usage review on the ring tone to be reviewed based on the usage scope specifically includes: Determining a target fragile watermark of the CRBT to be reviewed based on the watermark extraction result, and determining target CRBT ownership information and target usage permission information of the CRBT to be reviewed based on the target fragile watermark; determining the usage scope of the to-be-approved CRBT according to the target CRBT ownership information and the target usage permission information; Obtaining the uploaded customer information of the ringback tone to be reviewed, and determining whether the uploaded customer information complies with the usage scope; If the uploaded customer information meets the usage scope, it is determined that the pending ringtone has passed the usage review; if the uploaded customer information does not meet the usage scope, it is determined that the pending ringtone has not passed the usage review.

5. A method for auditing color ring back tone based on dual-mode watermark according to any one of claims 1 to 3, characterized in that: The step of determining the copyright information of the CRBT to be reviewed based on the watermark extraction result, and performing copyright review on the CRBT to be reviewed based on the copyright information, specifically includes: Determining a target robust watermark of the CRBT to be reviewed based on the watermark extraction result, and performing a matching search in a preset CRBT database based on the target robust watermark to obtain an initial platform CRBT corresponding to the CRBT to be reviewed; Determining the copyright information of the to-be-reviewed ringback tone according to the initial platform ringback tone; Obtaining the uploaded customer information of the ringback tone to be reviewed, and determining whether the uploaded customer information is consistent with the copyright information; If the uploaded customer information is consistent with the copyright information, it is determined that the pending ringtone has passed the copyright review; if the uploaded customer information is inconsistent with the copyright information, it is determined that the pending ringtone has failed the copyright review.

6. A dual-mode watermark-based ringback tone verification device, characterized in that: include: a watermark extraction module, configured to obtain a ringback tone to be reviewed, extract a watermark from the ringback tone to be reviewed, and determine whether the ringback tone to be reviewed carries a robust watermark and a fragile watermark according to the watermark extraction result; A watermark embedding module is configured to, when the CRBT to be reviewed does not carry a robust watermark and a fragile watermark, determine that the CRBT to be reviewed is a newly added CRBT to the platform, and then embed a robust watermark and a fragile watermark into the CRBT to be reviewed after the CRBT to be reviewed passes content review, thereby obtaining a stored CRBT; A usage audit module is configured to determine, when the CRBT to be reviewed carries a robust watermark and a fragile watermark, that the CRBT to be reviewed is an existing CRBT on the platform, further determine the usage scope of the CRBT to be reviewed based on the watermark extraction result, and perform a usage audit on the CRBT to be reviewed based on the usage scope; a copyright review module, configured to determine, when the CRBT to be reviewed carries a robust watermark but not a fragile watermark, that the CRBT to be reviewed is a platform-modified CRBT, further determine the copyright information of the CRBT to be reviewed based on the watermark extraction result, and perform a copyright review on the CRBT to be reviewed based on the copyright information; The following steps are used to embed a robust watermark into the ring back tone: Segmenting the ringback tone to be embedded to obtain a plurality of first audio regions; performing a fast Fourier transform on each of the first audio regions to obtain first spectrum information of each of the first audio regions; determining a first high-frequency component of each of the first audio regions according to the first spectrum information, and selecting a plurality of the first audio regions as first embedding regions according to the first high-frequency components; Performing frequency domain transformation on the first embedding region to obtain first frequency domain information, and selecting a plurality of mid- and low-frequency bands of the first frequency domain information as first embedding frequency bands; generating a to-be-embedded robust watermark according to a preset robust watermark generation rule, and determining a plurality of bits of first binary embedding information according to the to-be-embedded robust watermark; Modify the mid- and low-frequency coefficients of the first embedded frequency band of the first frequency domain information according to the first binary embedded information to obtain a first watermark frequency domain; Performing an inverse frequency domain transform on the first watermark frequency domain to obtain a first watermark region, and then replacing the first embedding region of the ringback tone to be embedded with the first watermark region to obtain a ringback tone carrying a robust watermark; To embed a fragile watermark into a ringback tone, perform the following steps: Segmenting the ringback tone to be embedded to obtain a plurality of third audio regions, and using the third audio regions as second embedding regions; Performing frequency domain transformation on the second embedding region to obtain third frequency domain information, and selecting a plurality of high frequency bands of the third frequency domain information as second embedding frequency bands; generating a fragile watermark to be embedded according to a preset fragile watermark generation rule, and determining a plurality of bits of second binary embedding information according to the fragile watermark to be embedded; Modify the high frequency coefficients of the second embedded frequency band of the third frequency domain information according to the second binary embedded information to obtain a second watermark frequency domain; An inverse frequency domain transform is performed on the second watermark frequency domain to obtain a second watermark region, and then the second embedding region to be embedded in the ringback tone is replaced with the second watermark region to obtain a ringback tone carrying a fragile watermark.

7. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the color ringback tone review method based on dual-mode watermarking as described in any one of claims 1 to 5 are realized.

8. A storage medium, which is a computer-readable storage medium and is used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the color ring back tone audit method based on dual-mode watermark as described in any one of claims 1 to 5.

Images