Hash-based sound source plagiarism detection method for copyright protection in distributed music sharing, and system using same
The hash-based sound source plagiarism detection method using perceptual hashing addresses the lack of copyright protection in decentralized music sharing by analyzing audio files with Chromaprint and blockchain storage, effectively preventing infringement and ensuring platform security and transparency.
Patent Information
- Application Number
- PCT/KR2023/019105
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-06-19
- Filing Date
- 2023-11-24
- Publication Date
- 2024-12-26
AI Technical Summary
Decentralized music sharing platforms lack effective mechanisms to prevent copyright infringement, allowing pirated audio files to circulate due to the absence of preventative measures to reject modified audio before it is uploaded online.
A hash-based sound source plagiarism detection method using perceptual hashing, where audio files are analyzed using Chromaprint to generate hash values, compared against existing hashes in a blockchain network, and if similar, the upload is rejected, while new audio files are stored with a Uniform Resource Identifier (URI) recorded on the blockchain through a smart contract.
The system effectively detects copyright infringement with an average match rate of 92.64% and ensures audio availability, immutability, and transparency, with an execution time of 616.3ms from hash to upload, leveraging the security of blockchain and distributed file storage.
Smart Images

Figure KR2023019105_26122024_PF_FP_ABST
Abstract
Description
A hash-based music plagiarism detection method and system using the same for copyright protection in decentralized music sharing. The present invention relates to a music plagiarism detection system, and more specifically, to a hash-based music plagiarism detection method for copyright protection in distributed music sharing and a system using the same. Preventative measures to prevent copyright infringement are currently not implemented on decentralized music sharing platforms. Because there is no mechanism to reject altered audio before it appears online, some decentralized music platforms have become hotbeds for pirated audio files. The present invention has been proposed to solve the above technical problems, and proposes a system for detecting manipulated audio for copyright protection in a distributed music sharing platform. According to one embodiment of the present invention for solving the above problem, a hash-based audio plagiarism detection method is provided, including the steps of extracting a perceptual hash value of an audio source to be uploaded to an Interoperable File Storage System (IPFS) using an audio fingerprint analysis tool, comparing the similarity between the extracted perceptual hash value and a hash value stored in a blockchain network to determine whether the audio source is a previously uploaded audio source, and returning a Uniform Resource Identifier (URI) from the Interoperable File Storage System (IPFS) if the audio source is determined to be a new audio source. In addition, the present invention further includes a step of recording a Uniform Resource Identifier (URI) returned from a distributed file storage system (IPFS) to a blockchain network through a smart contract when it is determined to be a new audio source. Additionally, the present invention further includes a step of detecting copyright infringement and rejecting the upload process of the audio source and preventing the blockchain network from being updated if it is determined to be a similar audio source. In addition, according to another embodiment of the present invention, a hash-based audio plagiarism detection system is provided, including a node terminal for uploading an audio source to an IPFS (Internet Protocol Storage System), a processing unit for extracting a perceptual hash value of the audio source to be uploaded to the IPFS (Internet Protocol Storage System) using an audio fingerprint analysis tool, a comparison unit for comparing the similarity between the extracted perceptual hash value and a hash value stored in a blockchain network to determine whether the audio source is a previously uploaded audio source, and the IPFS (Internet Protocol Storage System) for returning a Uniform Resource Identifier (URI) when the audio source is determined to be a new audio source. In addition, the present invention further includes a blockchain network that records a Uniform Resource Identifier (URI) returned from a distributed file storage system (IPFS) through a smart contract when it is determined to be a new audio source. In addition, in the present invention, if it is determined to be a similar audio source, it is detected as a copyright infringement, and the upload process of the audio source is rejected and the blockchain network is not updated. In this invention, a perceptual hash-based audio detection method and system for copyright protection in decentralized music sharing is proposed. Chromaprint, an open-source audio fingerprinting program, generates hash values to detect copyright infringement. To evaluate the robustness of the perceptual hashing technique, Chromaprint generates hash values from audio files that can be modified using various signal processing attacks. The detection system results show that Chromaprint is highly effective in detecting copyright infringement, with an average match rate of 92.64%. When implemented using the public Ethereum blockchain test network, the execution time from hashing to upload to the Interprovisioned File System (IPFS) distributed storage is only 616.3 ms. Figures 1a to 1e are drawings showing original and modified audio. Figures 2a and 2b are drawings showing modified audio. Figure 3 is a configuration diagram of the system of the present invention. Hereinafter, in order to explain in detail to a degree that a person having ordinary skill in the art to which the present invention pertains can easily practice the technical idea of the present invention, an embodiment of the present invention will be described with reference to the attached drawings. First, the abbreviations in the present invention are defined as follows. AES: Advanced Encryption Standard CNN: Convolutional Neural Network DAG: Directed Acyclic Graph DRM: Digital Rights Management IPFS: InterPlanetary File System LPF: Low-pass Filter SNR: Signal to Noise Ratio STFT: Short-time Fourier Transform SVD: Singular Value Decomposition URI: Uniform Resource Identifier Meanwhile, perceptual hash functions are a method for identifying multimedia data content. Unlike conventional hash functions, which are sensitive to changes, perceptual hash functions are designed to remain largely unchanged even when multimedia data undergoes minor modifications. Therefore, even slight changes or manipulations to the input result in slight differences. In other words, the perceptual hash function can be treated as a unique identifier for each audio. Perceptual hashing has also proven effective in detecting altered multimedia content, with Shazam being one of the most popular applications implementing perceptual hashing. This invention proposes a system for detecting manipulated audio for copyright protection purposes on a decentralized music sharing platform. Perceptual hashing is used to measure the similarity between audio and existing audio in the system. If the similarity of an audio file exceeds a threshold, the system rejects the audio and notifies the user that the audio already exists. If the audio file similarity is lower than a threshold, the audio is stored off-chain on the IPFS Distributed File Storage System. After obtaining the IPFS Uniform Resource Identifier (URI), it is stored on the Ethereum blockchain network via a smart contract. The robustness of the proposed system's audio perception hash, audio detection, and upload time are measured. The main contributions of the present invention are as follows. 1) We propose a model to detect manipulated audio in the distributed file storage system (IPFS) using perceptual hashing. 2) The proposed model can guarantee audio availability, immutability, transparency, and copyright protection in a decentralized music platform. 3) We implemented the first blockchain-based audio file detection system using perception hashing. - Proposal system FIGS. 1A to 1E are diagrams showing original and modified audio (3KHz low-pass filter (LPF), 0dB noise, 0.96x time scale correction, and 0.96x pitch shift), FIGS. 2A and 2B are diagrams showing modified audio (32KHz MP3 compression, 32KHz AAC compression), and FIG. 3 is a configuration diagram of the system of the present invention. This section details the detection of copyright infringement using the perception hash method (A) and the proposed system (B). The method used in the system of the present invention is illustrated in Figure 3, and the steps are as follows. 1) A node terminal attempts to upload an audio file to the system. Node users may be musicians or composers looking to share their creations, or they may be individuals who want to steal audio and pass it off as their own. 2) The perceptual hash of the audio file is calculated as a unique identifier. Chromaprint is used as the perceptual hash in the present invention. That is, the processing unit calculates the perceptual hash of the audio file. 3) The comparison unit compares the acquired perception hash with the existing hash recorded in the blockchain network. 4) If the hash value similarity is less than 50%, the audio file is added to the IPFS (Internet Protocol File System). After adding the audio, IPFS returns a Uniform Resource Identifier (URI), which is then recorded on the blockchain network using a smart contract. 5) If the hash value similarity exceeds 50%, copyright infringement is detected, the upload process audio is rejected, and the blockchain network is not updated. The threshold for similarity between perceived hash values is set to 50% as an example, as some studies have utilized this threshold for evaluation. A. Perceptual hashing method The perceptual hashing approach of the present invention is based on Chromaprint, an open-source audio fingerprinting tool (audio fingerprint analysis tool). Lukas Lalinsky designed Chromaprint as a client-side library that extracts fingerprints from any audio source. Below is a step-by-step explanation of how Chromaprint generates a perceptual hash from audio. 1) The input audio is converted to mono and downsampled to 11,025 Hz. 2) The audio signal is converted to the frequency domain using STFT with a frame size of 4096 samples and a 2 / 3 overlap. 3) The spectrum is divided into 12 bins, each representing a different chroma value. 4) Six filters are created to calculate the Chromaprint hash value. 5) 16 filters with different sizes are generated from 6 filters using the AdaBoost method. 6) A 12x16 sliding window slides across the chroma values of each sample to create sub-images. That is, the 12x16 sliding window moves across the chroma values to create sub-images. 7) Filter the window with 16 generated filters for each frame. 8) The filter adds the energy values of the white area and subtracts them from the black area to produce a single value. Additionally, the energy value is encoded as a 2-bit number (range 0 to 3) using Gray code. 9) Repeat steps 7 and 8 for all sub-images to obtain an audio fingerprint. B. Copyright infringement detection for audio files To detect audio similar to what exists in the system, we first need to generate a perceptual hash value of the audio using the Chromaprint method. Algorithm 1 The pseudocode for generating a perceptual hash from audio is described in Algorithm 1. Algorithm 1 shows how to obtain a perceptual hash from audio. First, select the audio file you want to add to the system. Chromaprint's getFingerprint function generates a perception hash of the selected audio, storing the value in the audioPHash variable. Algorithm 2 After obtaining the hash value, the audioPHash value is compared with the existing perception hash value in the blockchain network using the Hamming distance rule in Algorithm 2. The purpose of Algorithm 2 is to compute the similarity between the perceptual hash of the candidate audio file and the perceptual hash stored in the blockchain network. Before using the Hamming distance method, store the audioPHash value in h1, extract all the perception hashes stored in the blockchain network (hades.phashList) and store them in the variable audioPHashList, and create an empty array percentage to store the similarity values of h1 and each audioPHashList. The boolean value of the variable similarity, which indicates whether the candidate audio is similar to the audio file in the system, is set to FALSE. Then, check the similarity between h1 and audioPHashList one by one. In the first loop, the first value of audioPHashList is stored in h2 and the minimum length (minLen) of h1 and h2 is calculated. The Hamming distance is then used to obtain a similarity percentage by matching the perceptual hash values bit by bit until minLen is reached, which represents the bit positions where the two values have the same value. We assume that the similarity between the candidate audio and the system's audio file is greater than 50%. In this case, the system rejects adding the audio to the system. However, if the similarity is less than 50%, the loop continues until the audio list in audioPHashList is finished. Algorithm 3 After the similarity check is complete, the audio file and its perceptual hash are stored on the Distributed File Storage System (IPFS) using Algorithm 3, returning a URI (Iaudio). Finally, Iaudio is stored on the Ethereum blockchain network using a smart contract with h1. - Experimental setup and results Here, we detail the experimental setup (A), execution time (B), and robustness (C) of the perceptual hashing technique against signal processing attacks. A. Experimental setup The simulations of the present invention were performed on a Macbook Air powered by an M1 chip with an 8-core GPU, 8 GB of RAM, and 512 GB of storage. The audio files tested for the proposed approach had a 16-bit resolution, 47-second duration, and a 44,100 Hz sampling rate. Audio manipulation was performed using MATLAB 2021a. The blockchain was built using the Ethereum blockchain and tested on the Goerli public test network. Finally, the smart contracts were written in Solidity. B. Perceptual hash execution time Here, we discuss the execution time of the proposed system. The proposed system was tested by measuring the time required for the perceptual hashing process, audio detection, file upload to the IPFS (International Platform for File Storage), and data updates to the Ethereum blockchain network. Hash FunctionSimilarity CheckUpload to IPFSTotal Time156168.3292616.3 MethodAverageTimeRobustCPS
[0016] 2,776HADES156 Table 1 shows the average execution time of the proposed system, and Table 2 compares the average hashing time between the proposed system (HADES) and existing studies. According to Table 1, the hashing time using the Chromaprint program was approximately 156 ms. The detection time, which determines whether the input audio is similar to the system's existing audio, was approximately 168.3 ms. The average time required to upload an audio file and the total perceived hash value was 292 ms. However, the upload time depends on the user's internet bandwidth, so if there are internet connection issues, the file upload time may be longer. Therefore, the total execution time for audio detection is 616.3 ms. Table 2 also shows that the system of the present invention (Hash-based Audio Copy Detection System for Copyright Protection in Decentralized Music Sharing, HADES) has a much faster hashing process than RobustCPS, which has an average hashing time of 2,776 ms. However, this fast hashing process has its drawbacks. As explained below, accuracy can be reduced. C. Detecting Modified Audio Files To evaluate the robustness of Chromaprint perceptual hashes in detecting copyright infringement, various signal processing attacks are applied to audio files. The parameters for each operation are set as follows: 1) LPF: LPF was implemented using a second-order Butterworth filter with cutoff frequencies of 3, 6, and 9 kHz. 2) Adding noise: The amount of noise was increased until the SNR reached 0, 10, and 20 dB. 3) Time scale correction: Slowed down the tempo using the 99%, 98%, 97%, and 96% modifiers. 4) Pitch shift: Pitch was reduced from normal level to 99%, 98%, 97%, and 96%. 5) MP3 and AAC Compression: Audio bit rates were compressed to 32, 64 and 128 Kbps. After applying a signal processing attack to the original audio, we compared the perceptual hashes of the original audio and the attacked audio using the Hamming distance (see Algorithm 2). A higher similarity indicates better performance in identifying modified audio. AttacksParameterSimilarity(%)RobustCPSHADESLPF3 kHz6 kHz9 kHz96.8499.0599.3796.9499.6699.97Noise Addition0 dB10 dB20 dB76.3496.2199.6883.9597.2299.44TimeScaleModification99 %98 %97 %96 %99.0598.7498.7499.6898.4097.2997.0496.51Pitch-Shifting99 %98 %97 %96 %99.3799.6899.0599.0583.2678.9678.3177.93MP3Compression32 kps64 kps128 kps90.2090.8590.8592.5592.7592.67AACCompression32 kps64 kps128 kps93.6793.6793.6796.6196.6196.61Average Similarity95.6992.64 Table 3 shows the accuracy of the proposed system and the system cited in RobustPCS for detecting copyright infringement after applying signal processing attacks to the original audio. The performance of RobustCPS is slightly better than that of the present invention's system (HADES), with an average similarity 3.05% higher. There is little difference in performance when detecting audio filtered with a low-pass filter (LPF) and compressed with MP3 and AAC encoders. However, performance differences are evident when dealing with noise addition and pitch shift manipulation. Adding noise until the SNR reaches 0 dB significantly affects the audio signal, with the most noticeable difference occurring in the time domain (see Figure 1), making it difficult for both RobustCPS and the present system (HADES) to detect the audio. However, the present system (HADES) significantly outperforms RobustCPS in detecting audio with added noise, with similarities of 83.95% and 76.34%, respectively. In contrast, the present system (HADES) is inferior to RobustCPS in pitch shift manipulation, especially when the pitch of the audio changes by 96%. The system of the present invention (HADES) can only achieve a similarity level of 77.93%, unlike RobustCPS, which can achieve a similarity value of 99.05%. Since pitch shift manipulation affects the frequency of the audio stream (see Figure 1), it is difficult for the system of the present invention to perfectly identify the altered audio. Therefore, the system of the present invention (HADES) is better at detecting audio with added noise, but worse at detecting pitch-shifted audio than RobustCPS. D. Smart contract of the system of the present invention (HADES) As described in Experimental Setup (A), the smart contract of the proposed system was written in Solidity and deployed on the Goerli public test network. FunctionsTransactionFee(Gas)TransactionFee(ETH)Deploying Contract531,1600.01594register Audio()3,182,9470.09552getURI()00getpHash()00getAllInfo()00 "string _artistName": "Blue Bottles","string _album": "Here l,e Are","string _title": "Audiol","string _pHash": "-1061420370 -1066663002 1076625638 1076629590 2043910 1008646 67987463 67983621 202295557135186694 156162886 181861318 183862726 1827'51670 183275958 1256174006 1250796454 1250206694 -897277082 -897274075-1954235612 -1954104540 -1719223772 -1719563724 -1182758364 -1182692828 -1185838556 -1456109019 -14'5 6633242-1456633242 -1456485658 -145652252.2 -1423099146 -1423099450 -1427294010 -1431488314 -1431488266 -1163052825-1163970459 -1701893947 -1697699643 -1936839484 -863097212 -997312859 -1064435033 -1063517522 -1058274642-1066663450 1076625510 1077743622 2060294 68119559 67987717 202332421 202.295559 139380998 198105926 181853638183800214 183275958 184390070 1256174006 12502:06630 -89727695' 4 -897276057 -897274076 -1954170076 -1987659228-1719420364 -1719629260 -1182758364 -1178498'524 -1458206172 -1456108953 -14'56633242 -1456616858 -145' 6530842-1456'52252.2 -1456653354 -1456654138 -370329466 -374522730 -374525770 2042441894 2022518934 1485643906 143466690143073346 143093761 1216757044 1212578084 1208129812 -939357948 -922580220 -901608892 1251446340 12'528199241252819188 1'521258724 1521262821 1512890607 -63463810'5 -634452761 -651813657 -668582971 -131188156 2025806404202.2496836 1485630020 153594'5236 1519168036 -896751577 -896796634 -1979054042 -1978071002 -1975646170 -1892808362-1943135673 -2001847739 -2001921467 -2001987003 -2001986841 -200197865 0 -2001978650 -1997792282 -2006181466-198938812.2 -919734090 -919996282 -887490426 -1961907834 ","string_Iaudio": "ipfs: / bafybeibiebywsovirb62ue5oyzolijti4foaqjjzufjbbm4sjkystfs72q". Table 4 shows the transaction fees for each function of the smart contract of this system (HADES), and Table 41 shows the transaction contents of the smart contract captured in Remix IDE. Table 4 shows the highest cost of 3,183,947 Gas when executing registerAudio(), a function that registers an audio file after successfully passing the similarity check process. Compared to registerAudio(), contract deployment only has a transaction fee of 531,160 Gas. The reason registerAudio() is more expensive than deploying a smart contract to a blockchain network is because storing strings on a blockchain is more expensive than storing other types of variables. getURI() is a function that retrieves the IPFS Uniform Resource Identifier (URI) of the audio, while getpHash() returns the perceptual hash value of the audio stored on the blockchain. getAllInfo() is used to return all information about the audio, including the perception hash value and IPFS URI. The last three functions are just getter functions, so they don't cost anything to call. - conclusion The present invention presents a robust audio detection system for copyright protection in distributed music sharing, and the proposed system uses perceptual hashing to detect copyright infringement of audio files. The perceptual hash technology used in the present invention is Chromaprint, an open source audio fingerprinting library developed for the AcoustID project, and the proposed system is built on the Ethereum blockchain and the IPFS (Distributed File Storage System) to enable musicians to safely record music and use audio files thanks to the security, transparency, immutability of the blockchain and the advantages of the IPFS (Distributed File Storage System). The proposed system proved highly robust against signal processing attacks such as low-pass filtering (LPF), noise addition, time scaling modification, pitch shifting, MP3 compression, and AAC compression. In this study, the similarity threshold was set at 50%. The average similarity between the original audio and the modified version was over 90% in all scenarios, indicating that the proposed system can effectively detect copyright infringement. Furthermore, the average audio detection time of the proposed system was only 616.3 ms. As such, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering its technical spirit or essential characteristics. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the present invention.
Claims
1. A step of extracting a perceptual hash value of an audio source to be uploaded to the distributed file storage system (IPFS) using an audio fingerprint analysis tool; A step of comparing the similarity between the extracted perception hash value and the hash value stored in the blockchain network to determine whether it is a previously uploaded audio source; and If it is determined to be a new audio source, a step of returning a Uniform Resource Identifier (URI) from the Distributed File Storage System (IPFS); A hash-based music plagiarism detection method including .
2. In paragraph 1, A hash-based audio plagiarism detection method further comprising the step of recording a Uniform Resource Identifier (URI) returned from a distributed file storage system (IPFS) to a blockchain network via a smart contract when it is determined to be a new audio source.
3. In paragraph 1, A hash-based audio plagiarism detection method further comprising a step of detecting a copyright infringement and rejecting the upload process of the audio source and preventing the blockchain network from being updated, if the audio source is determined to be a similar audio source.
4. Node terminal for uploading audio sources to the distributed file storage system (IPFS); A processing unit for extracting a perceptual hash value of the audio source to be uploaded to the distributed file storage system (IPFS) using an audio fingerprint analysis tool; A comparison unit that compares the similarity between the extracted perception hash value and the hash value stored in the blockchain network to determine whether it is a previously uploaded audio source; and IPFS (Distributed File Storage System), which returns a Uniform Resource Identifier (URI) when it determines that the audio source is new; A hash-based music plagiarism detection system including .
5. In paragraph 4, A hash-based audio plagiarism detection system further comprising a blockchain network that records the Uniform Resource Identifier (URI) returned from the IPFS (Internet Protocol File Storage System) through a smart contract when it is determined to be a new audio source.
6. In paragraph 4, A hash-based audio plagiarism detection system characterized by detecting copyright infringement when a similar audio source is determined, thereby rejecting the upload process of the audio source and preventing the blockchain network from being updated.
Citation Information
Patent Citations
An original music composition management method based on a block chain and related equipment
CN109086577A
Digital asset processing method and system based on alliance chain, and storage medium
CN114756627A
System for managing document based on IPFS and method thereof
KR102447797B1
KR20230062817A
