System and method for authentication of rareness of a digital asset

Abstract

Systems and methods for asset fingerprinting and for authentication of rareness of a digital asset. The solution can be implemented in a peer-to-peer decentralized platform for securely registering, trading, and collecting non-fungible-tokens, which are associated with digital visual artwork and are recorded on a blockchain. The system comprises a Fingerprinting Engine (FE) and Relative Rareness Engine (RRE). The EE processes the digital asset (e.g., NET image) using trained neural network models, which generate a digital fingerprint vector representation of the image. The RRE compares the digital fingerprint vectors to a dataset of registered digital fingerprint vectors using multiple correlation measures. The correlation results are analyzed and combined to generate a Relative Rareness Score representing the rarity of the NET. Additionally, the RRE can evaluate rareness of the image relative to images on the Internet.

Description

W O 2 0 2 4 4 / 1 5 9 1 0 9 A V (12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property Organization International Bureau (43) International Publication Date 02 August 2024 (02.08.2024) (10) International Publication Number WO 2024 / 159109 A1 WIPO PCT (51) International Patent Classification: G06F 21 / 10 (2013.01) G06V 10 / 82 (2022.01) G06N 3 / 045 (2023.01) (72) Inventor: EMANUEL, Jeffrey; C / O Pastel Growth Fund LLC, 1 Deer Park Court, Greenwich, Connecticut 06830 (US). (21) International Application Number: PCT / US2024 / 013131 (74) Agent: GABATHULER, Henry et al.; Leason Ellis LLP, One Barker Avenue, Fifth Floor, White Plains, New York 10601 (US). (22) International Filing Date: 26 January 2024 (26.01.2024) (25) Filing Language: English (26) Publication Language: English (30) Priority Data: 18 / 160,515 27 January 2023 (27.01.2023) US (71) Applicant: PASTEL GROWTH FUND LLC [US / US]; 1 Deer Park Court, Greenwich, Connecticut 06830 (US). Designated States (unless otherwise indicated, for every kind of national protection available): AE, AG, AL, AM, AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, CA, CH, CL, CN, CO, CR, CU, CV, CZ, DE, DJ, DK, DM, DO, DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, HR, HU, ID, IL, IN, IQ, IR, IS, IT, JM, JO, JP, KE, KG, KH, KN, KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, MG, MK, MN, MU, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, PA, PE, PG, PH. PL, PT, QA, RO. RS, RU, RW, SA, SC, SD, SE, SG, SK, SL, ST, SV, SY, TH, (81) (54) Title: SYSTEM AND METHOD FOR AUTHENTICATION OF RARENESS OF A DIGITAL ASSET SuperNode 5 Storage 170 Server 10 Dupe Detection Input Files 108 Network SuperNode(s) Dupe. Detection Output Files 118 Client Devices 106 5 FIG. 1A (57) Abstract: Systems and methods for asset fingerprinting and for authentication of rareness of a digital asset. The solution can be implemented in a peer-to-peer decentralized platform for securely registering, trading, and collecting non-fungible-tokens, which are associated with digital visual artwork and are recorded on a blockchain. The system comprises a Fingerprinting Engine (FE) and Relative Rareness Engine (RRE). The EE processes the digital asset (e.g., NET image) using trained neural network models, which generate a digital fingerprint vector representation of the image. The RRE compares the digital fingerprint vectors to a dataset of registered digital fingerprint vectors using multiple correlation measures. The correlation results are analyzed and combined to generate a Relative Rareness Score representing the rarity of the NET. Additionally, the RRE can evaluate rareness of the image relative to images on the Internet. [Continued on next page] WO 2024 / 159109 A1 (84) TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, WS, ZA, ZM, ZW. Designated States (unless otherwise indicated, for every kind of regional protection available): ARIPO (BW, CV, GH, GM, KE, LR, LS, MW, MZ, NA, RW, SC, SD, SL, ST. SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, ME, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Published: - with international search report (Art. 21(3)) _ before the expiration of the time limit for amending the claims and to be republished in the event of receipt of amendments (Rule 48.2(h)) 摘要 一種用於數字資産指紋識別和稀有度認證的系統及方法。該方案可應用 於點對點去中心化平臺，用於安全地注册、交易和收集與數字視覺藝術作品關聯並記 錄在區塊鏈上的非同質化代幣（NFT）。該系統包含指紋識別引擎（FE）和相對稀有度 引擎（RRE）。FE 使用訓練好的神經網絡模型處理數字資産（例如，網絡圖像），生 成圖像的數字指紋向量表示。RRE 使用多種相關性度量方法，將數字指紋向量與已注 册的數字指紋向量數據集進行比較。分析並組合相關性結果，生成代表網絡圖像稀有 度的相對稀有度評分。此外，RRE 還可以評估圖像相對於互聯網上其他圖像的稀有 度。

Claims

What is claimed is:

1. An automated method for authentication of rareness of a digital asset, the method comprising: generating a first digital fingerprint of the digital asset by running one or more deep learning models on a first visual appearance of the digital asset, the generated first digital fingerprint being a first vector representing the first visual appearance, the one or more deep learning models being trained to process visual appearances of digital assets and generate corresponding first vectors suitable for determining visual appearance similarity; evaluating similarity of the first digital fingerprint to a registry of registered digital fingerprints by computing corresponding dot products of the first digital fingerprint with the registered digital fingerprints, the registry of registered digital fingerprints being obtained by running the one or more deep learning models on corresponding visual appearances of registered digital assets; determining corresponding first measures of statistical dependency for the registered digital fingerprints by normalizing the computed dot products to a range having a first end corresponding to no similarity with the first visual appearance and a second end corresponding to identical similarity with the first visual appearance; selecting a fixed number N of the registered digital fingerprints whose corresponding N normalized dot products are closest to the second end of the normalized range; determining a corresponding N second measures of statistical dependency by applying a population measurement test of deviation from independence to N combinations of the first digital fingerprint with the selected N digital fingerprints; and authenticating the rareness of the digital asset using the determined first and second measures of statistical dependency.

2. The method of claim 1, wherein running the one or more deep learning models on the first visual appearance comprises: running a first deep learning model on the first visual appearance to produce a second vector comprising a plurality of values that characterize the first visual appearance, the first deep learning model being trained on a plurality of visual appearances to produce corresponding second vectors that respectively characterize the visual appearances; running a second deep learning model on the produced second vector to generate a third vector optimized for determining visual appearance similarity, the second deep learning modelbeing trained on second vector pairs, each pair of second vectors being produced by running the first deep learning model on a corresponding original visual appearance and a transformed visual appearance obtained by applying one of a plurality of transformations to the original visual appearance, and forcing the pair of produced second vectors to be closer together in an embedding space in order to generate corresponding third vectors optimized for determining visual appearance similarity; and compressing the generated third vector to the first vector by eliminating redundant information in order to generate the first digital fingerprint.

3. The method of claim 2, wherein the first deep learning model is a convolutional neural network.

4. The method of claim 2, wherein for each original visual appearance and corresponding transformed visual appearance, applying one of the transformations comprises performing one of cropping the original visual appearance, flipping the original visual appearance, rotating the original visual appearance, adding one or more rectangular overlays to the original visual appearance, and deleting one or more parts from the original visual appearance.

5. The method of claim 2, wherein eliminating the redundant information comprises using principal component analysis (PC A).

6. The method of claim 1, wherein the population measurement test of deviation from independence is selected from the group consisting of Hoeff ding’s Dependence measure (Hoeffding’s D) and Hilbert-Schmidt Independence Criteria (HSIC) .

7. The method of claim 1, wherein determining the corresponding N second measures of statistical dependency comprises applying a first population measurement test of deviation from independence to the N digital fingerprint combinations, the method further comprises determining a corresponding N third measures of statistical dependency by applying a second population measurement test of deviation from independence to the N digital fingerprint combinations, and authenticating the rareness of the digital asset is performed using the determined third measures of statistical dependency.

8. The method of claim 7, further comprising: ranking the N digital fingerprints based on their corresponding first measures of statistical dependency; computing differentials of the first measures of statistical dependency between consecutively ranked digital fingerprints; computing differentials of the second measures of statistical dependency between the consecutively ranked digital fingerprints; and computing differentials of the third measures of statistical dependency between the consecutively ranked digital fingerprints, wherein authenticating the rareness of the digital asset is performed using the computed differentials of the first, second, and third measures of statistical dependency.

9. The method of claim 8, wherein authenticating the rareness of the digital asset comprises running a machine learning model that uses logistic regression to, for each digital fingerprint of the N digital fingerprints, process a plurality of inputs of the digital fingerprint and produce a single output representing a probability that the corresponding visual appearance of the digital fingerprint is a duplicate of the first visual appearance, the plurality of inputs comprising the first, second, and third measures of statistical dependency of the digital fingerprint and the differentials of the first, second, and third measures of statistical dependency of the digital fingerprint.

10. The method of claim 9, wherein the logistic regression model is trained on tens of thousands of visual appearances and their corresponding digital fingerprints and six-input and single output combinations, as well as on tens of thousands of complex transformations of the visual appearances and their corresponding digital fingerprints and six-input and single output combinations.

11. The method of claim 10, wherein the logistic regression model is further trained using a model performance metric for binary responses as a loss function to measure how well the logistic model is working.

12. The method of claim 9, further comprising: determining a second fingerprint of the N digital fingerprints that has a highest probability of being a duplicate of the first digital fingerprint from among the output probabilities of the logistic regression model; anddetermining a rareness score of the digital asset by subtracting the output highest probability from one, wherein authenticating the rareness of the digital asset is based on the rareness score.

13. An automated system for authentication of rareness of a digital asset, the system comprising: a processing circuit; a non-transitory storage medium storing a registry of registered digital fingerprints, machine learning models, and instructions that, when executed by the processing circuit, configure the processing circuit to: generate a first digital fingerprint of the digital asset by running one or more deep learning models of the stored machine learning models on a first visual appearance of the digital asset, the generated first digital fingerprint being a first vector representing the first visual appearance, the deep learning models being trained to process visual appearances of digital assets and generate corresponding first vectors suitable for determining visual appearance similarity; evaluate similarity of the first digital fingerprint to the registered digital fingerprints in the registry by computing corresponding dot products of the first digital fingerprint with the registered digital fingerprints, the registry of registered digital fingerprints being obtained by running the deep learning models on corresponding visual appearances of registered digital assets; determine corresponding first measures of statistical dependency for the registered digital fingerprints by normalizing the computed dot products to a range having a first end corresponding to no similarity with the first visual appearance and a second end corresponding to identical similarity with the first visual appearance; select a fixed number N of the registered digital fingerprints whose corresponding N normalized dot products are closest to the second end of the normalized range; determine a corresponding N second measures of statistical dependency by applying a population measurement test of deviation from independence to N combinations of the first digital fingerprint with the selected N digital fingerprints; and authenticate the rareness of the digital asset using the determined first and second measures of statistical dependency.

14. The system of claim 13, wherein the instructions, when executed by the processing circuit, further cause the processing circuit to run the deep learning models on the first visual appearance by: running a first deep learning model on the first visual appearance to produce a second vector of values that characterize the first visual appearance, the first deep learning model being trained on a plurality of visual appearances to produce corresponding second vectors that characterize the visual appearances; running a second deep learning model on the produced second vector to generate a third vector of similar size to the second vector and optimized for determining visual appearance similarity, the second deep learning model being trained second vector pairs, each pair of second vectors being produced by running the first deep learning model on a corresponding original visual appearance and a transformed visual appearance obtained by applying one of a plurality of transformations to the original visual appearance, and forcing the pair of produced second vectors to be closer together in an embedding space in order to generate corresponding third vectors optimized for determining visual appearance similarity; and compressing the generated third vector to the first vector by eliminating redundant information in order to generate the first digital fingerprint.

15. The system of claim 14, wherein for each original visual appearance and corresponding transformed visual appearance, applying one of the transformations comprises performing one of cropping the original visual appearance, flipping the original visual appearance, rotating the original visual appearance, adding one or more rectangular overlays to the original visual appearance, and deleting one or more parts from the original visual appearance.

16. The system of claim 13, wherein determining the corresponding N second measures of statistical dependency comprises applying a first population measurement test of deviation from independence to the N digital fingerprint combinations, the instructions, when executed by the processing circuit, further cause the processing circuit to determine a corresponding N third measures of statistical dependency by applying a second population measurement test of deviation from independence to the N digital fingerprint combinations, andthe rareness of the digital asset is authenticated using the determined third measures of statistical dependency.

17. The system of claim 16, wherein the instructions, when executed by the processing circuit, further cause the processing circuit to: rank the N digital fingerprints based on their corresponding first measures of statistical dependency; compute differentials of the first measures of statistical dependency between consecutively ranked digital fingerprints; compute differentials of the second measures of statistical dependency between the consecutively ranked digital fingerprints; and compute differentials of the third measures of statistical dependency between the consecutively ranked digital fingerprints, wherein the rareness of the digital asset is authenticated using the computed differentials of the first, second, and third measures of statistical dependency.

18. The system of claim 17, wherein authenticating the rareness of the digital asset comprises running an other machine learning model of the stored machine learning models, the other machine learning model using logistic regression to, for each digital fingerprint of the N digital fingerprints, process a plurality of inputs of the digital fingerprint and produce a single output representing a probability that the corresponding visual appearance of the digital fingerprint is a duplicate of the first visual appearance, the plurality of inputs comprising the first, second, and third measures of statistical dependency of the digital fingerprint and the differentials of the first, second, and third measures of statistical dependency of the digital fingerprint.

19. The system of claim 18 , wherein the logistic regression model is trained on tens of thousands of visual appearances and their corresponding digital fingerprints and six-input and single output combinations, as well as on tens of thousands of complex transformations of the visual appearances and their corresponding digital fingerprints and six-input and single output combinations.

20. The system of claim 18, wherein the instructions, when executed by the processing circuit, further cause the processing circuit to: determine a second fingerprint of the N digital fingerprints that has a highest probability of being a duplicate of the first digital fingerprint from among the output probabilities of the logistic regression model; and determine a rareness score of the digital asset by subtracting the output highest probability from one, wherein the rareness of the digital asset is authenticated using the rareness score.

21. A non- transitory computer readable medium (CRM) having computer instructions stored therein that, when executed by a processing circuit, cause the processing circuit to carry out an automated process for authentication of rareness of a digital asset, the process comprising: generating a first digital fingerprint of the digital asset by running one or more deep learning models on a first visual appearance of the digital asset, the generated first digital fingerprint being a first vector of values representing the first visual appearance, the one or more deep learning models being trained process visual appearances of digital assets and to generate corresponding first vectors suitable for determining visual appearance similarity; evaluating similarity of the first digital fingerprint to a registry of registered digital fingerprints by computing corresponding dot products of the first digital fingerprint with the registered digital fingerprints, the registry of registered digital fingerprints being obtained by running the deep learning models on corresponding visual appearances of the registered digital assets; determining corresponding first measures of statistical dependency for the registered digital fingerprints by normalizing the computed dot products to a range having a first end corresponding to no similarity with the first visual appearance and a second end corresponding to identical similarity with the first visual appearance; selecting a fixed number N of the registered digital fingerprints whose corresponding N normalized dot products are closest to the second end of the normalized range; determining a corresponding N second measures of statistical dependency by applying a population measurement test of deviation from independence to N combinations of the first digital fingerprint with the selected N digital fingerprints; and authenticating the rareness of the digital asset using the determined first and second measures of statistical dependency.

22. The CRM of claim 21, wherein running the deep learning models on the first visual appearance comprises: running a first deep learning model on the first visual appearance to produce a second vector comprising a plurality of values that characterize the first visual appearance, the first deep learning model being trained on a plurality of visual appearances to produce corresponding second vectors that characterize the visual appearances; running a second deep learning model on the produced second vector to generate a third vector optimized for determining visual appearance similarity, the second deep learning model being trained on second vector pairs, each pair of second vectors being produced by running the first deep learning model on a corresponding original visual appearance and a transformed visual appearance obtained by applying one of a plurality of transformations to the original visual appearance, and forcing the pair of produced second vectors to be closer together in an embedding space in order to generate corresponding third vectors optimized for determining visual appearance similarity; and compressing the generated third vector to the first vector by eliminating redundant information in order to generate the first digital fingerprint.

23. The CRM of claim 22, wherein for each original visual appearance and corresponding transformed visual appearance, applying one of the transformations comprises performing one of cropping the original visual appearance, flipping the original visual appearance, rotating the original visual appearance, adding one or more rectangular overlays to the original visual appearance, and deleting one or more parts from the original visual appearance.

24. The CRM of claim 21 , wherein determining the corresponding N second measures of statistical dependency comprises applying a first population measurement test of deviation from independence to the N digital fingerprint combinations, the process further comprises determining a corresponding N third measures of statistical dependency by applying a second population measurement test of deviation from independence to the N digital fingerprint combinations, and authenticating the rareness of the digital asset comprises further using the determined third measures of statistical dependency.

25. The CRM of claim 24, wherein the process further comprises: ranking the N digital fingerprints based on their corresponding first measures of statistical dependency; computing differentials of the first measures of statistical dependency between consecutively ranked digital fingerprints; computing differentials of the second measures of statistical dependency between the consecutively ranked digital fingerprints; and computing differentials of the third measures of statistical dependency between the consecutively ranked digital fingerprints, wherein authenticating the rareness of the digital asset comprises further using the computed differentials of the first, second, and third measures of statistical dependency.

26. The CRM of claim 25, wherein authenticating the rareness of the digital asset comprises running a machine learning model that uses logistic regression to, for each digital fingerprint of the N digital fingerprints, process a plurality of inputs of the digital fingerprint and produce a single output representing a probability that the corresponding visual appearance of the digital fingerprint is a duplicate of the first visual appearance, the plurality of inputs comprising the first, second, and third measures of statistical dependency of the digital fingerprint and the differentials of the first, second, and third measures of statistical dependency of the digital fingerprint.

27. The CRM of claim 26, wherein the logistic regression model is trained on tens of thousands of visual appearances and their corresponding digital fingerprints and six-input and single output combinations, as well as on tens of thousands of complex transformations of the visual appearances and their corresponding digital fingerprints and six-input and single output combinations.

28. The CRM of claim 26, wherein the process further comprises: determining a second fingerprint of the N digital fingerprints that has a highest probability of being a duplicate of the first digital fingerprint from among the output probabilities of the logistic regression model; and determining a rareness score of the digital asset by subtracting the output highest probability from one, wherein authenticating the rareness of the digital asset comprises further using the rareness score.

Images