Method and system, device and medium for processing and trading of multi-source heterogeneous marketing data

Abstract

The disclosure provides a multi-source heterogeneous marketing data processing and transaction method and system, device and medium, the method comprising: S1, using a marketing knowledge graph to automatically compile a data source connector, and establishing a connection between the data source connector and an external data interface. S2, automatically locate the change point of the abnormal interface and generate a repair patch. Update the data source connector corresponding to the abnormal interface. S3, get the sensitivity level of the access data. S4, differentially secure the access data, and cast the high-sensitive data and / or medium-sensitive data after security processing into non-homogeneous vouchers NFT. S5, complete the transaction of the corresponding NFT through the alliance chain. According to the embodiments of the disclosure, the comprehensive technical effects of exponential improvement of access efficiency, precise matching of security investment and data value, compliant release of data value, and optimization of storage and query performance can be realized.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to a method and system for processing and trading multi-source heterogeneous marketing data, electronic equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] As digital transformation deepens, businesses' marketing activities increasingly rely on data from multiple sources and in multiple formats. This data typically originates from various heterogeneous data sources, such as grid cloud, public operation platforms (e.g., WeChat official accounts, Weibo), digital sandboxes, business analysis systems, and cloud databases. These data sources differ significantly in terms of data structure (structured, semi-structured, unstructured), communication protocols, and data update frequency, thus constituting a "multi-source heterogeneous" data environment.

[0003] Existing technologies for processing this type of data include: 1. Customized ETL (Extract, Transform, Load) scripts: Independent data extraction, transformation, and loading scripts are written for each data source. However, this approach results in high coupling, a large development and maintenance workload, and difficulty in adapting to rapid changes in data sources.

[0004] 2. Single storage medium: There is a tendency to use a single relational database or a single file storage system (such as the Hadoop Distributed File System, HDFS) to store all types of data. However, relational databases are inefficient for storing unstructured data (such as images, videos, and documents), while file systems cannot meet the requirements for structured data that requires complex transactions and joins.

[0005] 3. Coarse-grained security processing: This approach typically involves uniform security processing (such as data masking) after data import or at the application layer. This method lacks a mechanism for differentiated and process-oriented security control based on data sensitivity levels during the data acquisition and access phase, posing a risk of sensitive data leakage.

[0006] The three solutions mentioned above suffer from problems such as low data collection efficiency, uneven utilization of storage resources, lagging data security management and control, and poor overall system coordination.

[0007] In addition to these, there are other problems, such as: Inefficient API adaptation: Marketing platforms (such as Xiaohongshu, Douyin, Taobao Union, etc.) frequently update their Application Programming Interfaces (APIs) to adapt to business upgrades. Traditional "hard-coded" connector development requires manual parsing of new interface documents, code reconstruction and deployment, with an average maintenance cycle of up to 10 person-days, which cannot meet the real-time access requirements of scenarios such as major promotions.

[0008] The sensitivity classification lacks commercial relevance: existing data sensitivity classifications mostly use regular expression matching to identify fixed format fields such as ID cards and mobile phone numbers, which can only achieve "format-level" classification and cannot quantify the actual impact of data leakage on revenue, resulting in a mismatch between security investment and risk (high sensitivity, low investment or low sensitivity, high investment).

[0009] The contradiction between compliance and revenue in data circulation: High-value marketing data (such as high-spending demographics and conversion attribution data) faces the dilemma of "locking up data with no revenue" and "selling data without authorization with high risk": Directly selling plaintext data does not comply with relevant regulations, while simply anonymizing the data results in the loss of commercial value, and there is a lack of a compliant transaction model that allows for the circulation of usage rights while retaining ownership. Summary of the Invention

[0010] This disclosure provides a method and system for processing and trading multi-source heterogeneous marketing data, an electronic device, a computer-readable storage medium, and a computer program product.

[0011] Firstly, this disclosure provides a method for processing and trading multi-source heterogeneous marketing data, including: S1, responding to receiving business concept terms input by a user, automatically compiling and generating a data source connector based on the user-input business concept terms using a marketing knowledge graph, and establishing a connection between the data source connector and an external data interface. S2, monitoring the operational status of the connected external data interfaces in real time, and automatically locating the change points of the abnormal interface and generating a repair patch to update the data source connector corresponding to the abnormal interface when an anomaly is detected. S3, acquiring access data through the data source connector. Performing dynamic sensitivity classification on the access data to obtain the sensitivity level of the access data. S4, performing differentiated security processing on the access data according to the sensitivity level of the access data, and casting the security-processed high-sensitivity data and / or medium-sensitivity data into non-fungible tokens (NFTs). S5, completing the transaction of the corresponding NFTs through a consortium blockchain.

[0012] Optionally, S1 includes: S101, constructing triples based on business entities, application programming interface (API) terms, and fields, and constructing a marketing knowledge graph based on the triples. S102, using a word vector model, generating term word vectors based on user-input business concept terms, and calculating the semantic similarity between the term word vectors and nodes in the marketing knowledge graph. S103, calling a graph matching algorithm to extract a relational subgraph from the marketing knowledge graph based on the semantic similarity between the term word vectors and nodes in the marketing knowledge graph. Generating an Open API differential specification based on the relational subgraph. The relational subgraph includes the API address, request method, and field mapping relationship of the target platform. The Open API differential specification marks the differences between the relational subgraph and the standard specification. S104, reading the Open API differential specification based on a template engine, adapting and filling a preset connector template according to the differences marked in the Open API differential specification, obtaining the filled code, and compiling the filled code into a container image. S105 deploys the container image to the runtime environment via rolling hot reloading, instantiates it as a data source connector, establishes a connection between the data source connector and the external data interface, and performs a connectivity test. If the data acquisition success rate is not lower than a preset threshold, the connection between the data source connector and the external data interface is successful.

[0013] Optionally, the marketing knowledge graph can be stored using the Neo4j graph database, and the marketing knowledge graph can be periodically updated through a crawler mechanism and a manual verification mechanism.

[0014] Optionally, S2 includes: S201, real-time collection of metrics returned by each interface in the connected external data interfaces, and calculation of the anomaly rate of each interface in the connected external data interfaces based on the metrics returned by each interface. The metrics returned by each interface in the connected external data interfaces include the HTTP response code, field missing rate, and data latency. S202, in response to the anomaly rate exceeding a preset threshold, triggering the syntax tree difference bot, comparing the data structure returned by the abnormal interface with the historically stored data structure of the abnormal interface, and locating the change point of the abnormal interface. S203, calling the reinforcement learning model to generate patch code adapted to the change point of the abnormal interface, the reward function of the reinforcement learning model consists of the decrease in the anomaly rate after repair and the repair time. S204, after deploying the patch code to the test environment for verification, arbitrarily selecting a portion of the online traffic of the data source connector corresponding to the abnormal interface for canary replacement. If the anomaly rate of the traffic after canary replacement is lower than a preset safety threshold, then all online traffic of the data source connector corresponding to the abnormal interface is updated; otherwise, automatic rollback is performed.

[0015] Optionally, S3 includes: S301, constructing a revenue loss ratio (ΔROI) prediction model using a federated learning framework; S302, performing streaming inference on the access data based on the ΔROI prediction model to obtain the predicted ΔROI value of the access data, and classifying the sensitivity level of the access data according to the ΔROI value of the access data and a preset sensitivity level threshold to obtain the sensitivity level of the access data.

[0016] Optionally, S301 includes: S311, defining sample requirements for federated learning, including sample features and prediction labels. Sample features include data type, usage scenario, and circulation frequency. Prediction labels include whether leakage has occurred and the change in revenue loss ratio ΔROI after leakage. S312, issuing sample requirements to each of the multiple data providers, so that each of the multiple data providers can select data from their local data that meets the sample requirements. S313, initializing the ΔROI prediction model and distributing the initialized ΔROI prediction model to each of the multiple data providers, so that each of the multiple data providers can use the gradient boosting tree algorithm to train the initialized ΔROI prediction model on the data from their local data that meets the sample requirements, obtaining local models for each of the multiple data providers, calculating the gradients of the local models for each of the multiple data providers, and encrypting and transmitting the gradients of the local models for each of the multiple data providers back using homomorphic encryption technology. S314: Receive the gradients of the local models of multiple data providers, encrypted and transmitted back by each of the data providers. Aggregate and decrypt the received gradients, and use the aggregated and decrypted gradients to iteratively update the initialized ΔROI prediction model. S315: Iterate and update until the model converges, obtaining the ΔROI prediction model.

[0017] Optionally, S3 also includes: S304, which encapsulates the sensitivity level classification results, inference basis and operator information of the accessed data into an audit block, writes it into the consortium blockchain through a consensus mechanism, and generates a classification certificate.

[0018] Optionally, in S4, differentiated security processing includes: S401, for highly sensitive data, encrypting it using the BFV homomorphic encryption algorithm to obtain ciphertext, passing the ciphertext into a TEE high-sensitivity computing pool built on Intel Software Protection Extensions (Intel SGX), and minting an NFT bound to the hash of the highly sensitive data, thus obtaining the NFT of the highly sensitive data, which is then stored in the InterPlanetary File System (IPFS) distributed storage system. Direct download of the ciphertext is prohibited. S402, for medium-sensitive data, format-preserving desensitization processing is performed on the mobile phone number and email fields in the medium-sensitive data, and NFTs labeled with the desensitization rules are minted, thus obtaining the NFT of the medium-sensitive data, which is then stored in the IPFS distributed storage system. The metadata of the NFTs of highly sensitive and medium-sensitive data includes data hash fingerprints, authorized usage scenarios, revenue sharing ratios, and data validity periods. S403, for low-sensitivity data, after data quality verification, storing it in plaintext in the Redis cache layer, and writing the hash fingerprint of the low-sensitivity data into the on-chain credential repository.

[0019] Optionally, S5 includes: S501, obtaining NFTs of high-sensitivity data and medium-sensitivity data from the IPFS distributed storage system, calculating the price of each NFT according to a formula through a smart contract on the consortium blockchain, and listing it on the trading market after confirmation by the data provider. Here, is the price of the NFT, is the predicted ΔROI value, is the historical cost per thousand impressions, and is the number of authorization days. S502, receiving payment and a deposit from the buyer according to the NFT price corresponding to the intended data, locking the funds and verifying the buyer's qualifications through a smart contract on the consortium blockchain, and allocating access to the TEE high-sensitivity computing pool to the buyer after successful verification. S503, granting the buyer permission to call the intended data for calculation within the TEE high-sensitivity computing pool, and writing the buyer's calculation process log to the audit log chain. During the calculation process, the intended data exists in an encrypted state, so that the buyer obtains the calculation result but not the intended data. S504: After the authorized period expires, the funds are distributed through the smart contract of the consortium blockchain according to the preset revenue sharing ratio. If no data leakage occurs, the deposit will be returned to the buyer, and the transaction of the NFT corresponding to the intended data will be completed.

[0020] Optionally, the processing and transaction method for multi-source heterogeneous marketing data also includes: S6, writing structured data from the differentiated and securely processed data into a Delta Lake-based integrated lakehouse architecture, writing unstructured data into object storage, extracting metadata from the unstructured data and writing it back to the integrated lakehouse architecture, and establishing a mapping between structured and unstructured data. S7, responding to a query request from a user, receiving the structured query language (SQL) statement input by the user, parsing the SQL statement through a cost optimizer, generating a cross-storage execution plan by combining metadata information, pulling data from the integrated lakehouse architecture and object storage through a parallel thread pool, aggregating it through an in-memory computing engine, and returning the query results to the user. S8, generating audit logs for all storage writes and query execution operations, the audit logs include the operator ID, data ID, operation type, and sensitivity level, and synchronously writing the audit logs to the audit log chain, which is linked with the consortium blockchain.

[0021] Secondly, this disclosure provides a system for processing and trading multi-source heterogeneous marketing data. This system includes: a connection establishment module, used to: respond to receiving business concept terms input by a user, automatically compile and generate a data source connector based on the user-input business concept terms using a marketing knowledge graph, and establish a connection between the data source connector and an external data interface; an anomaly monitoring module, used to: monitor the operational status of the connected external data interfaces in real time, and when an anomaly is detected in the external data interface, automatically locate the change point of the abnormal interface and generate a repair patch to update the data source connector corresponding to the abnormal interface; a sensitivity classification module, used to: obtain access data through the data source connector; dynamically classify the access data to obtain its sensitivity level; a certificate minting module, used to: perform differentiated security processing on the access data according to its sensitivity level, and mint the processed high-sensitivity data and / or medium-sensitivity data into data security certificates (NFTs); and a transaction module, used to: complete the transaction of the corresponding NFTs through a consortium blockchain.

[0022] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described method for processing and trading multi-source heterogeneous marketing data.

[0023] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for processing and trading multi-source heterogeneous marketing data.

[0024] Fifthly, this disclosure provides a computer program product that includes computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described method for processing and trading multi-source heterogeneous marketing data.

[0025] The embodiments provided in this disclosure form a closed-loop process from data access to value release through the organic coordination of multiple steps, including data access, anomaly self-healing, dynamic hierarchical classification, differentiated processing, and tokenized transactions. This solution systematically addresses technical issues such as low interface adaptation efficiency, lack of commercial relevance in sensitive classification, conflict between data circulation compliance and revenue, lagging security control, single storage structure, and incomplete operational auditing. It achieves comprehensive technical effects such as exponentially improved access efficiency, precise matching of security investment with data value, and compliant release of data value.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0027] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which: Figure 1 A flowchart illustrating a method for processing and trading multi-source heterogeneous marketing data, provided as an embodiment of this disclosure; Figure 2 A flowchart illustrating another method for processing and trading multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 3 A flowchart illustrating yet another method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 4 A flowchart illustrating yet another method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 5 A flowchart illustrating yet another method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 6 A flowchart illustrating yet another method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 7A flowchart illustrating yet another method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 8 A flowchart illustrating yet another method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 9 A system architecture diagram as an example of a method for processing and trading multi-source heterogeneous marketing data provided in this disclosure embodiment; Figure 10 A block diagram of a multi-source heterogeneous marketing data processing and transaction system provided in this disclosure embodiment; Figure 11 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0030] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0032] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0033] Figure 1 A flowchart illustrating a method for processing and transacting multi-source heterogeneous marketing data provided in this disclosure. (Refer to...) Figure 1 The method includes: Step S1: In response to receiving business concept terms input by the user, the marketing knowledge graph is used to automatically compile and generate a data source connector based on the business concept terms input by the user, and a connection is established between the data source connector and the external data interface.

[0034] Step S2: Monitor the running status of the connected external data interfaces in real time. When an abnormality is detected in the external data interface, automatically locate the change point of the abnormal interface and generate a repair patch to update the data source connector corresponding to the abnormal interface.

[0035] Step S3: Obtain access data through the data source connector. Perform dynamic sensitivity classification on the access data to obtain its sensitivity level.

[0036] Step S4: Perform differentiated security processing on the access data according to the sensitivity level of the access data, and mint the high-sensitivity data and / or medium-sensitivity data after security processing into non-fungible tokens (NFTs).

[0037] Step S5: Complete the transaction of the corresponding NFT through the consortium blockchain.

[0038] According to the embodiments of this disclosure, step S1 automatically compiles and generates a data source connector and completes deployment and access, realizing zero-code access to the new interface, reducing the access time from 10 person-days of traditional hard coding to minutes; step S2 monitors the interface running status in real time, automatically locates the change point and generates a repair patch for updating when an anomaly is detected, realizing minute-level self-healing of interface anomalies, solving the problem of low interface adaptation efficiency, and fully meeting the real-time access requirements of large-scale promotion scenarios.

[0039] Step S4 performs differentiated security processing operations on the data based on its sensitivity level. By adopting different levels of security policies based on the classification results during the data access phase, it achieves proactive and differentiated security control, rather than uniformly processing the data only after it has been stored in the traditional solution, thus solving the problem of delayed security control.

[0040] Steps S4 and S5 mint the securely processed high-sensitivity and / or medium-sensitivity data into data security credential NFTs and complete the transaction through the consortium blockchain. This mechanism transforms data usage rights into divisible and priced credentials. The buyer only obtains computing rights within a Trusted Execution Environment (TEE) and cannot access the original data, making the data usable but invisible. This satisfies relevant requirements and resolves the dilemma of data providers locking up their databases without profit versus selling them directly with high risk, thus addressing the conflict between data circulation compliance and profitability.

[0041] In some embodiments, such as Figure 2 As shown, step S1 includes: Step S101: Construct triples based on business entities, application programming interfaces (APIs), and fields, and construct a marketing knowledge graph based on the triples.

[0042] In some embodiments, step S101 further includes: storing the marketing knowledge graph using the Neo4j graph database, and periodically updating the marketing knowledge graph through a crawler mechanism and a manual verification mechanism.

[0043] For example, the data sources cover publicly available API documentation from mainstream marketing platforms, industry data access specifications, and historical successful access cases, and are dynamically updated every 24 hours through a dual mechanism of web crawling and manual verification.

[0044] Understandably, step S101 and the steps above establish a marketing knowledge graph, which is dynamically updated every 24 hours through a dual mechanism of web crawling and manual verification. This allows the system to continuously grasp the latest interface specifications and historical integration cases of mainstream marketing platforms. It can structure and store scattered and heterogeneous interface knowledge, providing a data foundation for subsequent automated integration and solving the problem of relying on manual review and memorization of interface information in traditional methods.

[0045] Step S102: Using a word vector model, generate term word vectors based on the business concept terms input by the user, and calculate the semantic similarity between the term word vectors and the nodes in the marketing knowledge graph.

[0046] For example, the word vector model can be the Word2Vec model (a word vector model, Word to Vector). After non-technical personnel (such as operations) input natural language business terms T (such as "Xiaohongshu note interaction data") into a visualization interface, the Word2Vec model generates word vectors for the terms and calculates the semantic similarity with the graph nodes.

[0047] Understandably, step S102 maps business concepts to technical specifications, enabling non-technical personnel (such as operations personnel) to trigger the access process by inputting business terms without needing to understand the underlying interface details. This breaks through the dependence of traditional "code-level" access on professional developers and achieves zero-code access.

[0048] Step S103: Invoke the graph matching algorithm to extract related subgraphs from the marketing knowledge graph based on the semantic similarity between term vectors and nodes in the marketing knowledge graph. Generate an Open Application Programming Interface (Open API) differential specification based on the related subgraphs.

[0049] In step S103, the association subgraph includes the target platform's API address, request method, and field mapping relationships. The OpenAPI Differential Specification marks the differences between the association subgraph and the standard specification.

[0050] For example, the graph matching algorithm can be an optimized graph matching algorithm for the graph sampling and aggregation (GraphSAGE) algorithm. The implementation method of step S103 can be: extracting related subgraphs (including target platform API address, request method, and field mapping relationship) from the marketing knowledge graph based on semantic similarity, automatically generating OpenAPI differential specifications, and marking the differences with the standard specifications.

[0051] Understandably, step S103 clearly identifies the personalized differences of the interface (such as the addition of new fields or changes in type) in a machine-readable manner, providing a precise basis for the subsequent customized generation of connectors and avoiding the inefficient work of manually reading documents and comparing differences line by line.

[0052] Step S104: Read the Open API differential specification based on the template engine, adapt and fill the preset connector template according to the differences marked in the Open API differential specification, obtain the filled code, and compile the filled code into a container image.

[0053] For example, the template engine adopts a template engine developed based on Freemarker. The implementation method of step S104 can be: the template engine developed based on Freemarker reads the associated subgraph information, automatically fills the connector template and compiles it into a Docker container image, and has built-in data cleaning and format conversion scripts.

[0054] Understandably, step S104 automatically converts the differentiated interface specifications into executable linker code, realizing automated compilation from "specification" to "code", eliminating the workload of manually writing linker code, and ensuring the consistency and correctness of the code.

[0055] Step S105: Deploy the container image to the runtime environment using a rolling hot reload method, instantiate it as a data source connector, establish a connection between the data source connector and the external data interface, and perform a connectivity test. If the data acquisition success rate is not lower than a preset threshold, the connection between the data source connector and the external data interface is successful.

[0056] For example, after receiving the image, the Kubernetes container orchestration system (Kubernetes, K8s) performs rolling hot reloading, which can complete the deployment without stopping existing services; after deployment, connectivity tests are automatically performed, and the connection is determined to be complete if the data collection success rate is ≥99.5%, and the whole process takes ≤5 minutes.

[0057] Understandably, step S105 enables uninterrupted deployment and automated verification of the connector, which can be put into use immediately after deployment. The entire connection process takes no more than 5 minutes, achieving an exponential improvement in efficiency compared to the traditional hard-coded 10 person-days (80 working hours).

[0058] Steps S101 to S105 automate the entire process from "business concept input" to "connector deployment": the user inputs business terms, the system automatically matches interface specifications from the knowledge graph, generates differential specifications, compiles them into container images, and rolls them out to the runtime environment for verification. The entire process requires no manual coding, significantly shortens the time for new interface integration, completely solves the technical problem of low interface adaptation efficiency, and meets the business needs of frequent interface upgrades and real-time integration during major promotional events for marketing platforms.

[0059] In some embodiments, such as Figure 3 As shown, step S2 includes: Step S201: Collect the metrics returned by each interface in the connected external data interfaces in real time, and calculate the anomaly rate of each interface in the connected external data interfaces based on the metrics returned by each interface.

[0060] In step S201, the metrics returned by each interface among the connected external data interfaces include the Hypertext Transfer Protocol (HTTP) response code, field missing rate, and data latency.

[0061] For example, step S201 can be implemented by: collecting interface return metrics in real time, including HTTP response codes (4xx / 5xx percentage), field missing rate, data latency, and calculating the anomaly rate η, with a monitoring frequency of once per second.

[0062] Understandably, this achieves second-level awareness of the interface's operational status, enabling the immediate detection of interface anomalies or changes, and providing triggering conditions for subsequent repairs. Compared to traditional methods that rely on manual inspections or passive responses after user reports, this step achieves proactive anomaly detection, significantly shortening the time required to discover problems.

[0063] Step S202: In response to the exception rate exceeding the preset threshold, the syntax tree difference Bot is triggered to compare the data structure returned by the exception interface with the historical data structure of the exception interface to locate the change point of the exception interface.

[0064] For example, step S201 can be implemented as follows: when η>5% (the preset threshold is 5%), the syntax tree difference Bot is automatically triggered to pull the JavaScript Object Notation (JSON) mode currently returned by the exception interface and perform a structured comparison with the historical mode stored in the knowledge graph, and locate the change point (such as adding a field or changing the field type) within 3 minutes.

[0065] Understandably, step S202 transforms the process of locating interface changes from manual field-by-field comparison to automated structured comparison, which is not only faster but also more accurate, avoiding omissions or errors that may occur when manually parsing new interface documents.

[0066] Step S203: Invoke the reinforcement learning model to generate patch code that adapts to the abnormal interface change points.

[0067] In step S203, the reward function of the reinforcement learning model consists of the decrease in the abnormality rate after repair and the repair time.

[0068] For example, step S203 can be implemented by calling a Deep Deterministic Policy Gradient (DDPG) reinforcement learning model, using "the decrease in the anomaly rate after repair + the repair time" as the composite reward function, to generate patch code that adapts to the change; the model is pre-trained through historical repair cases, and the patch generation accuracy is ≥98%.

[0069] Understandably, step S203 enables the automatic generation of patch code without the need for manual writing and debugging; the model is pre-trained through historical repair cases, resulting in high patch generation accuracy, ensuring the correctness of the repair scheme, and avoiding new problems that may be introduced by manual coding.

[0070] Step S204: After deploying the patch code to the test environment for verification, select a portion of the online traffic of the data source connector corresponding to the abnormal interface for canary replacement. If the abnormality rate of the traffic after canary replacement is lower than the preset safety threshold, then update all online traffic of the data source connector corresponding to the abnormal interface; otherwise, automatically roll back.

[0071] For example, step S204 can be implemented as follows: the patch code is first deployed to the test environment for verification. After verification, 10% of the online traffic is replaced in a gray-scale manner. The gray-scale traffic anomaly rate is continuously monitored. If the anomaly rate is ≤1% (the preset safety threshold is 1%), a full update is completed within 5 minutes. If the anomaly rate is ≥1%, an automatic rollback to the historical version is triggered.

[0072] Understandably, in step S204, the grayscale verification mechanism ensures that the repair will not cause a wider range of failures, and the automatic rollback mechanism ensures the stability of the system. The entire repair process, from the time of the exception trigger to the full update, can achieve minute-level self-healing of interface exceptions.

[0073] Understandably, steps S201 to S204 can solve the core pain point of "slow upgrade response" in the low efficiency of interface adaptation, enabling the system to cope with the frequent upgrades of marketing platform interfaces and the real-time access requirements in the context of major promotional events.

[0074] In some embodiments, such as Figure 4 As shown, step S3 includes: S301 uses a federated learning framework to build a revenue loss ratio (the change in return on investment, Delta Return on Investment, ΔROI) prediction model.

[0075] Federated learning is, understandably, a distributed machine learning framework whose core idea is "the model moves while the data remains stationary." Its standard process is as follows: a central server initializes the global model and distributes it to all participants; each participant trains its model using local data and calculates gradients; the gradients are encrypted using techniques such as homomorphic encryption and then uploaded to the central server; the central server aggregates the gradients from all participants and updates the global model; this iterative process is repeated until the model converges. This framework resolves the contradiction of needing to collaboratively model data when centralized training is impossible, achieving multi-party collaborative modeling while protecting data privacy.

[0076] For example, this embodiment uses the FedProx algorithm, which is an improved version of federated learning that adds a proximal regularization term to handle the heterogeneity problem of inconsistent data distribution among parties.

[0077] Step S301 employs a federated learning framework to construct a revenue loss ratio prediction model, resolving the conflict between data privacy and model training—data from each party remains locally, with only encrypted gradients uploaded, protecting data privacy while addressing the issue of insufficient sample size from a single data provider. Simultaneously, the model uses "ROI change after leakage" as its prediction target, elevating the sensitive classification criteria from the traditional "format-level" (e.g., ID number) to the "commercial-level" (revenue loss), achieving a deep binding between classification and commercial value, resulting in high model prediction accuracy.

[0078] Step S302: Perform streaming inference on the access data according to the ΔROI prediction model to obtain the ΔROI prediction value of the access data. Divide the access sensitivity level according to the ΔROI value of the access data and the preset sensitivity level threshold to obtain the sensitivity level of the access data.

[0079] For example, after new data is accessed, the model stream inferences and outputs ΔROI values, and classifies the sensitivity level according to preset rules: ΔROI ≥ 5% is high-sensitivity data, ΔROI in the range of 1%-5% is medium-sensitivity data, and ΔROI < 1% is low-sensitivity data; the classification results are synchronized to the classification rule engine in real time.

[0080] Understandably, the sensitivity level of accessed data is determined by dynamically classifying it based on revenue loss predictions. This classification method, which relies on the actual revenue loss that a data breach may cause, replaces the traditional static classification that depends solely on field format. This allows security investments to accurately match risk levels, avoiding the problem of insufficient investment in high-sensitivity data or excessive protection for low-sensitivity data.

[0081] In some embodiments, such as Figure 5 As shown, step S301 includes: Step S311: Define the sample requirements for federated learning, which include sample features and predicted labels.

[0082] In step S311, sample characteristics include data type, usage scenario, and circulation frequency. Prediction labels include whether a leak occurred and the change in revenue loss ratio (ΔROI) after the leak.

[0083] For example, the data provider can be an advertising platform, a brand, or other parties, and each party can provide ≥100,000 samples. The original data is stored on the local nodes of each party.

[0084] Step S312: Send sample requirements to each of the multiple data providers so that each of the multiple data providers can select data from their local data that meets the sample requirements.

[0085] Understandably, steps S311 and S312 provide a suitable dataset for subsequent training.

[0086] Step S313: Initialize the ΔROI prediction model and distribute the initialized ΔROI prediction model to each of the multiple data providers.

[0087] In step S313, after receiving the initialized ΔROI prediction model, each of the multiple data providers uses the gradient boosting tree algorithm to train the initialized ΔROI prediction model based on the data in the local data that meets the sample requirements, thereby obtaining the local models of each of the multiple data providers, calculating the gradients of the local models of each of the multiple data providers, and encrypting and sending back the gradients of the local models of each of the multiple data providers through homomorphic encryption technology.

[0088] Understandably, each party uses its own local sample pairs for local training, calculates gradients, and encrypts the gradients using homomorphic encryption before sending them back. This ensures that the original data remains locally on each party's end, with only the encrypted gradients being uploaded. This protects data privacy and solves the problem of insufficient sample size from a single data provider to train a high-quality model. Homomorphic encryption further guarantees the security of the gradient transmission process.

[0089] Step S314: Receive the gradients of the local models of the multiple data providers, which are encrypted and transmitted back by each of the multiple data providers. After aggregating and decrypting the received gradients, use the aggregated and decrypted gradients to iteratively update the initialized ΔROI prediction model.

[0090] For example, using the FedProx federated learning framework, each node trains a Light Gradient Boosting Machine (LightGBM) based on local samples. Gradients are encrypted using Paillier homomorphic encryption and then uploaded to the aggregation node. After aggregation, the global model is updated, with incremental updates performed every 7 days. The loss function is: The model's prediction accuracy is ≥92%.

[0091] Understandably, using the aggregated and decrypted gradient to iteratively update the initialized ΔROI prediction model can integrate the training results from various parties, enabling the model to absorb features from multiple sources, improve the model's generalization ability and prediction accuracy, and at the same time, the entire aggregation process does not touch the original data, thus ensuring data privacy.

[0092] Step S315: Iterate and update until the model converges to obtain the ΔROI prediction model.

[0093] Understandably, by continuously optimizing the model parameters through multiple rounds of iteration, the model can achieve the best predictive performance. The converged model can accurately predict the revenue loss ratio (ΔROI value) that may be caused by new data leakage, providing a reliable basis for subsequent sensitivity level classification.

[0094] Understandably, steps S311 to S315 implement model construction under the federated learning framework: defining a business-oriented training task (sample requirements), initializing and distributing the model, encrypting and uploading gradients after local training by each party, aggregating and updating the global model, and iterating until convergence. Elevating the sensitivity classification criterion from "fixed-format fields" to "revenue loss prediction" enables security investment to accurately match the actual business losses that may be caused by data breaches, thus solving the technical problems of sensitivity classification lacking business relevance and security investment not matching risk.

[0095] In some embodiments, such as Figure 6 As shown, step S3 further includes: Step S304: The sensitivity level classification result of the accessed data, the inference basis, and the operator information are encapsulated into an audit block, written into the consortium blockchain through the consensus mechanism, and a classification certificate is generated.

[0096] For example, the classification results, inference basis, and operator information are encapsulated into an audit block and written into the Data Security Credential Chain (DSC-Chain) through the PoA consensus mechanism to generate an immutable classification credential for subsequent security processing and audit traceability.

[0097] In some embodiments, such as Figure 7 As shown, in step S4, the differentiated security processing includes: Step S401: For highly sensitive data, the BFV homomorphic encryption algorithm (Brakerski-Fan-Vercauteren) is used to encrypt the data to obtain ciphertext. The ciphertext is then passed to a TEE highly sensitive computing pool built on Intel Software Guard Extensions (Intel SGX) and an NFT bound to the hash of the highly sensitive data is minted to obtain the NFT of the highly sensitive data. The NFT of the highly sensitive data is then stored in the InterPlanetary File System (IPFS) distributed storage system.

[0098] In step S401, the encrypted text is prohibited from being downloaded directly.

[0099] For example, step S401 can be implemented by: encrypting the original data using the BFV homomorphic encryption algorithm, and then directly transmitting the encrypted data to the TEE high-sensitivity computing pool built by Intel SGX technology; at the same time, triggering the credential casting process to generate a data security credential (DSC-NFT) bound to the unique hash of the data. The original ciphertext data is not allowed to be downloaded directly and only trusted computing within the TEE is supported.

[0100] Understandably, step S401 achieves the highest level of protection for highly sensitive data: after encryption, calculations can only be performed within the TEE, the original data always exists in ciphertext and cannot be downloaded or obtained, eliminating the risk of data leakage, and at the same time, through NFT tokenization, it lays the foundation for subsequent compliant transactions.

[0101] Step S402: For medium-sensitive data, perform format-preserving desensitization processing on the mobile phone number and email fields in the medium-sensitive data, and cast an NFT labeled with the desensitization rules to obtain the NFT of the medium-sensitive data. Store the NFT of the medium-sensitive data in the IPFS distributed storage system.

[0102] In step 402, the metadata of the NFTs of high-sensitivity data and medium-sensitivity data includes data hash fingerprints, authorized usage scenarios, revenue sharing ratios, and data validity periods.

[0103] For example, step S402 can be implemented as follows: Perform format-preserving de-identification processing, reversibly de-identifying privacy fields such as phone numbers and email addresses (preserving the original format), and store the processed data in a lake-warehouse integrated architecture built on Delta Lake, utilizing its ACID transaction properties (Atomicity, Consistency, Isolation, Durability) to ensure data consistency. Simultaneously, DSC-NFTs are minted, and de-identification rule information is labeled for traceability during use.

[0104] Understandably, Ethereum Request for Comments 721 (ERC-721) standard DSC-NFTs are generated for high / medium sensitive data. The metadata includes information such as data hash fingerprint, authorized use case (e.g., "advertising for baby formula"), revenue sharing ratio, and validity period. The metadata is stored in the IPFS distributed storage system to ensure immutability.

[0105] Understandably, step S402 preserves data availability while protecting privacy—format preservation and desensitization allow the data to still be used for business scenarios such as analysis and querying, while the recording of desensitization rules provides traceability for data use, achieving a balance between security and utility.

[0106] Step S403: For low-sensitivity data, after data quality verification, it is stored in plaintext in the Redis cache layer, and the hash fingerprint of the low-sensitivity data is written into the on-chain credential repository.

[0107] For example, step S403 can be implemented as follows: after data quality verification, it is stored in the Redis acceleration layer in plaintext to improve the query efficiency of high-frequency access scenarios. Only the data hash fingerprint is written to the on-chain credential library, without the need to mint DSC-NFT. The hash comparison ensures that the data has not been tampered with.

[0108] Understandably, a lightweight protection strategy is adopted for low-risk data, which stores it in plaintext and uses Redis to accelerate high-frequency access, ensuring query efficiency. At the same time, data integrity is ensured by uploading hash fingerprints to the blockchain, achieving a precise match between security costs and data value.

[0109] Understandably, steps S401 to S403 enable differentiated security processing based on the sensitivity level of the data during the data access phase, thereby achieving a precise match between security investment and risk level. This avoids the problem of "insufficient protection for high-sensitivity data and excessive protection for low-sensitivity data" caused by using a unified security strategy for all data in the traditional approach. At the same time, by bringing security control forward to the data access phase, the technical pain point of lagging security control is completely resolved.

[0110] In some embodiments, such as Figure 7 As shown, step S5 includes: Step S501: Obtain NFTs of high-sensitivity data and NFTs of medium-sensitivity data from the IPFS distributed storage system, calculate the price of each NFT according to formula (1) through the smart contract of the consortium blockchain, and place the order on the trading market after confirmation by the data provider.

[0111] (1)

[0112] In equation (1), For the price of NFT, This is the predicted value of ΔROI. Cost of displaying history a thousand times For the number of authorized days.

[0113] For example, the data provider can manually fine-tune the price of the NFT before confirming the listing.

[0114] Understandably, in step S501, the data usage right is transformed into a divisible and priced voucher, and the pricing formula incorporates risk ( ),value( ) and duration ( It achieves scientific valuation of data assets through three dimensions, and automatically prices data through smart contracts, avoiding the inefficiency and subjectivity of manual negotiation.

[0115] Step S502: Receive payment and deposit from the buyer based on the NFT price corresponding to the intended data. Lock the funds through the smart contract of the consortium blockchain and verify the buyer's qualifications. After verification, grant the buyer access to the TEE high-sensitivity computing pool.

[0116] For example, step S502 can be implemented as follows: The buyer browses the trading market and selects the target NFT, pays the corresponding amount, and deposits 50% of the transaction amount as a security deposit, which is locked by a smart contract. The system automatically verifies the buyer's qualifications and, upon approval, grants them access to the TEE computing pool.

[0117] Understandably, the margin mechanism constrains buyers to use data in compliance with regulations, reducing the risk of data misuse. For example, the margin amount is 50% of the NFT's price. The allocation of TEE access rights lays the foundation for subsequent data use. All transaction steps, including fund locking, qualification verification, and permission allocation, are executed automatically through smart contracts, ensuring the fairness and non-repudiation of the transaction.

[0118] Step S503: Grant the purchaser permission to call the intended data for calculation within the TEE high-sensitivity computing pool, and write the purchaser's calculation process log into the audit log chain.

[0119] In step S503, during the process of the buyer calling the intended data for calculation in the TEE high-sensitivity computing pool, the intended data exists in an encrypted state so that the buyer can obtain the calculation result but not the intended data.

[0120] For example, step S503 can be implemented as follows: the purchaser calls data to perform calculations (such as crowd package analysis) within the TEE trusted environment. During the calculation process, the original data always exists in an encrypted state, and only the calculation result report is output; the calculation process log is written to the audit chain in real time.

[0121] Understandably, in step S503, the buyer can only obtain the calculation results and not the original data, which satisfies the data usage needs and protects the rights and interests of the data owner. The entire calculation process is recorded on the blockchain, providing a basis for subsequent auditing and dispute tracing.

[0122] Step S504: After the authorized period expires, the funds are distributed through the smart contract of the consortium blockchain according to the preset revenue sharing ratio. If no data leakage occurs, the deposit will be returned to the buyer, and the transaction of the NFT corresponding to the intended data will be completed.

[0123] For example, step S504 can be implemented as follows: After the NFT expires, the smart contract automatically completes the fund distribution according to a preset revenue sharing ratio (e.g., 70% for the data provider and 30% for the platform). If no data breach or other violations occur, the deposit is fully refunded to the buyer, and the entire transaction takes ≤10 minutes.

[0124] Understandably, step S504 achieves automated settlement of the transaction loop, with the data provider receiving a share of the revenue, the platform receiving a service commission, and the buyer recovering their deposit after compliant use. The smart contract's revenue-sharing mechanism ensures a fair distribution of benefits among all parties without the need for manual intervention.

[0125] Understandably, steps S501 to S504 completely resolve the contradiction between "no profit from locking the database" and "high risk from selling naked data" in data circulation. This not only protects the profit of the data owner but also meets the usage needs of the buyer. At the same time, through TEE and blockchain technology, "data is usable but not visible" and full-process compliance auditing are achieved, which meets the relevant requirements for personal information protection.

[0126] In some embodiments, such as Figure 8 As shown in the embodiments of this disclosure, the method for processing and trading multi-source heterogeneous marketing data further includes: Step S6: Write the structured data from the data after differential security processing into the lake-warehouse integrated architecture based on Delta Lake, write the unstructured data into object storage, extract the metadata of the unstructured data and write it back to the lake-warehouse integrated architecture, and establish the association mapping between structured data and unstructured data.

[0127] For example, step S6 can be implemented as follows: After the differentiated security processing in step S4, structured data (such as user profiles, transaction records, etc.) is directly written to the Delta Lake integrated storage architecture, relying on its ACID transaction characteristics to ensure data consistency. Unstructured data (such as user behavior logs, note content, etc.) is stored in Alibaba Cloud Object Storage Service (OSS) object storage, and metadata such as filename, storage path, creation time, and data sensitivity level are extracted and written back to Delta Lake to establish a mapping between structured and unstructured data.

[0128] Understandably, the data storage method in step S6 solves the drawback that a single storage medium cannot accommodate multiple data types. It enables structured data to rely on the ACID transaction characteristics of Delta Lake to ensure data consistency and transaction processing capabilities, while storing unstructured data in object storage reduces storage costs (50% lower than traditional HDFS). The association between the two types of data is established through metadata mapping, laying the foundation for subsequent cross-source queries.

[0129] Step S7: In response to receiving a query request from a query user, the system receives the Structured Query Language (SQL) statement input by the query user, parses the SQL statement through the cost optimizer, generates a cross-storage execution plan by combining metadata information, pulls data from the lake warehouse architecture and object storage through a parallel thread pool, aggregates the data through the in-memory computing engine, and returns the query results to the query user.

[0130] For example, step S7 can be implemented as follows: the user inputs a standard SQL statement through a unified query gateway, the cost-based optimizer (CBO) integrated in the gateway parses the statement and combines it with metadata information to generate the optimal cross-storage execution plan; the optimizer automatically determines the distribution location of the target data (lake warehouse or OSS), pulls multi-source data through a parallel thread pool, and returns the results after real-time aggregation processing by the in-memory computing engine, with a total latency of ≤300ms and support for 500 concurrent queries per second.

[0131] Understandably, the query method in step S7 shields the heterogeneity of the underlying storage, allowing users to access multi-source data through standard SQL without worrying about the actual storage location of the data. The cost optimizer intelligently selects the optimal execution path, and parallel retrieval and memory aggregation ensure query performance with an average latency of no more than 300 milliseconds, supporting 500 concurrent queries per second, thus meeting the needs of real-time marketing decisions.

[0132] Step S8 generates audit logs for all storage write and query operations. The audit logs include the operator ID, data ID, operation type, and sensitivity level. The audit logs are then synchronously written to the audit log chain, which is linked with the consortium blockchain.

[0133] For example, step S8 can be implemented as follows: all storage write and query execution operations generate audit logs in real time, including information such as operator ID, data ID, operation type, execution duration, and hierarchical results. The logs are synchronously written to the audit log chain and linked with the DSC-Chain to achieve tamper-proof traceability of the entire operation process and meet the relevant requirements for personal information protection.

[0134] Understandably, step S8 addresses the issues of easily tampered and difficult-to-trace audit logs in traditional solutions; the audit logs are linked with the consortium blockchain to ensure that every operation has an immutable record, the association between the operator ID and the data ID makes responsibility traceable to the individual, and the sensitive level records provide a basis for tracing security incidents, meeting the relevant requirements for personal information protection.

[0135] The following example illustrates a method for processing and trading multi-source heterogeneous marketing data according to embodiments of this disclosure.

[0136] like Figure 9 As shown, this example adopts a four-layer architecture of "engine-computation-storage-transaction". Each module works together to realize the entire process of data access, classification, processing and transaction. The modules communicate with each other through an encrypted gateway. The data flow is as follows: smart adapter accesses data → hybrid storage layer is deployed → ROI classification module processes data → secure credential transaction module circulates data. The specific structure is as follows: - Smart Adapter Self-Evolution Engine Module (Knowledge Graph-based Data Access Engine, KDE): Core components include a marketing knowledge graph engine, a graph matching algorithm engine, a container template engine, and a K8s scheduling unit; the knowledge graph is stored using the Neo4j graph database, and the template engine is developed based on Freemarker, responsible for transforming business concepts into executable connectors and achieving anomaly self-healing.

[0137] - ROI Sensitivity Classification and Federated Learning Module (Dynamic ROI-based SensitivityClassification, DRS): It consists of federated learning nodes (FedProx framework), ΔROI prediction model (LightGBM gradient boosting tree), and a classification rule engine; the federated nodes are deployed locally on the data provider's site, and gradients are transmitted through homomorphic encryption to prevent the original data from leaving the country.

[0138] - Data Security Certificate Consortium Chain Module (Dynamic ROI-based Sensitivity Classification, DSC-Chain): Built on the Hyperledger Fabric consortium chain, it includes consensus nodes (5, using the PoA mechanism), a smart contract engine (supporting the Solidity language), and NFT minting units; the consensus nodes are jointly maintained by the data provider, the platform provider, and the regulatory agency to ensure transaction traceability.

[0139] -TEE High-Sensitivity Computing Pool: Composed of 4 Xeon E3 servers equipped with Intel SGX technology, it supports trusted computing of homomorphically encrypted data; the computing pool distributes tasks through a load balancer, and the computing process is completely isolated, without outputting the original ciphertext.

[0140] - Hybrid Storage Layer: Adopts an architecture of "lake warehouse + object storage + on-chain credential repository"; the lake warehouse is implemented based on Delta Lake (supporting ACID transactions) and stores structured hierarchical data; the object storage uses Alibaba Cloud OSS to store unstructured raw data (such as user behavior logs); the on-chain credential repository stores DSC-NFT metadata and hash fingerprints.

[0141] - Unified Query Gateway & Audit Log Chain: The gateway supports standard SQL parsing and integrates a CBO cost optimizer to generate cross-storage execution plans; the audit log chain is linked with the DSC-Chain, generating an immutable log for each operation (access, query, transaction), containing information such as operator ID, data ID, and hierarchical results.

[0142] 2. Key Methods and Steps

[0143] S1, Business Concept → Connector Auto-Compilation (Zero-Code Integration)

[0144] This step automates the conversion of business terminology into connectors using a marketing knowledge graph, eliminating the need for manual coding. The specific process is as follows: This step leverages a marketing knowledge graph to automate the entire compilation of business concepts into connectors, requiring no professional coding personnel. 1. Knowledge Graph Construction and Update: Construct a marketing knowledge graph G consisting of "business entity-API-field" triples, stored in the Neo4j graph database; the data sources cover publicly available API documents of mainstream marketing platforms, industry data access specifications, and historical successful access cases, and are dynamically updated every 24 hours through a dual mechanism of web crawling and manual verification.

[0145] 2. Intelligent parsing of business terms: Non-technical personnel (such as operations personnel) can input natural language business terms T (such as "Xiaohongshu note interaction data") into the visual interface. The system generates term word vectors through the Word2Vec model and calculates the semantic similarity with the graph nodes.

[0146] 3. Subgraph Matching and Specification Generation: The GraphSAGE optimized graph matching algorithm is called to extract the associated subgraph Gt (including target platform API address, request method, and field mapping relationship) from the marketing knowledge graph G based on semantic similarity, automatically generate the OpenAPI differential specification, and mark the differences with the standard specification.

[0147] 4. Container image compilation: Based on the template engine developed by Freemarker, it reads the associated subgraph Gt information, automatically populates the connector template and compiles it into a Docker container image, with built-in data cleaning and format conversion scripts.

[0148] 5. Rapid deployment and access: After receiving the image, the K8s scheduling unit performs rolling hot reloading, which can complete the deployment without stopping existing services; after deployment, connectivity tests are automatically performed, and the access is judged to be completed if the data collection success rate is ≥99.5%, and the whole process takes ≤5 minutes.

[0149] S2, interface anomaly self-healing (minute-level repair).

[0150] Automated interface upgrade repair is achieved through anomaly monitoring, syntax tree differencing, and reinforcement learning. The specific process is as follows: This step utilizes real-time monitoring and intelligent repair mechanisms to achieve minute-level self-healing of interface anomalies, ensuring continuous data collection. 1. Multi-dimensional anomaly monitoring: The system collects interface return metrics in real time, including HTTP response codes (4xx / 5xx ratio), field missing rate, and data latency, and calculates the anomaly rate η. The monitoring frequency is once per second.

[0151] 2. Anomaly Repair Trigger: When η>5% (configurable threshold), the syntax tree difference bot is automatically triggered to pull the JSON pattern returned by the abnormal interface and compare it with the historical pattern stored in the knowledge graph in a structured manner, and locate the change point (such as adding a field or changing the field type) within 3 minutes.

[0152] 3. Intelligent Patch Generation: The DDPG reinforcement learning model is invoked, and a patch code adapted to the changes is generated using the composite reward function of "the decrease in the anomaly rate after the fix + the fix time". The model is pre-trained through historical fix cases, and the patch generation accuracy is ≥98%.

[0153] 4. Gray-scale verification and full update: The patch code is first deployed to the test environment for verification. After passing the verification, a gray-scale replacement is performed on 10% of the online traffic. The gray-scale traffic anomaly rate is continuously monitored. If it is ≤1%, a full update is completed within 5 minutes. If it is ≥1%, an automatic rollback to the historical version is triggered.

[0154] S3, ROI Sensitive Tier (Dynamic Risk Pricing)

[0155] By training a ΔROI prediction model using federated learning, dynamic grading based on revenue loss is achieved. The specific process is as follows: This step builds a ΔROI prediction model based on federated learning to achieve dynamic data sensitivity classification. 1. Federated Sample Set Co-construction: A federated sample set is jointly constructed by advertising platforms, brands, and other parties. Each party can provide ≥100,000 samples. Sample characteristics include data type, usage scenario, and circulation frequency. The tags are "whether leakage has occurred" and "ROI change value 30 days after leakage". The original data is stored on the local nodes of each party.

[0156] 2. ΔROI Model Training: The FedProx federated learning framework is used. Each node trains a LightGBM gradient boosting tree model based on local samples. The gradients are encrypted using Paillier homomorphic encryption technology and then uploaded to the aggregation node. After aggregation, the global model is updated. Incremental updates are performed every 7 days. The loss function is L = |True ΔROI - Predicted ΔROI|. The model prediction accuracy is ≥92%.

[0157] 3. Streaming Inference Classification: After new data is input, the model outputs ΔROI value through streaming inference and classifies sensitivity levels according to preset rules: ΔROI ≥ 5% is high-sensitivity data, 1%-5% is medium-sensitivity data, and < 1% is low-sensitivity data; the classification results are synchronized to the classification rule engine in real time.

[0158] 4. On-chain grading results: The grading results, inference basis, and operator information are encapsulated into an audit block and written into the DSC-Chain consortium blockchain through the PoA consensus mechanism to generate an immutable grading certificate for subsequent security processing and audit traceability.

[0159] S4, Differentiated Security Processing (Precise Protection)

[0160] Based on the graded results, differentiated encryption / de-identification strategies are implemented to balance security and availability: Based on the sensitivity level results output by S3, differentiated security processing strategies are implemented to balance data security and access efficiency. The specific process is as follows: 1. High-sensitivity data processing: The original data is encrypted using the BFV homomorphic encryption algorithm. The encrypted data is then directly transmitted to the TEE high-sensitivity computing pool built with Intel SGX technology. At the same time, the voucher casting process is triggered to generate a DSC-NFT bound to the unique hash of the data. The original ciphertext data is not allowed to be downloaded directly and only trusted computing within the TEE is supported.

[0161] 2. Data Sensitivity Processing: Perform format-preserving desensitization processing, reversibly desensitizing privacy fields such as mobile phone numbers and email addresses (preserving the original format), and store the processed data in a lake-warehouse integrated architecture built on Delta Lake, utilizing its ACID transaction characteristics to ensure data consistency; simultaneously cast DSC-NFTs, annotating desensitization rule information for traceability when used.

[0162] 3. Low-sensitivity data processing: After data quality verification, it is stored in plaintext in the Redis acceleration layer to improve query efficiency in high-frequency access scenarios; only the data hash fingerprint is written to the on-chain credential repository, without the need to mint DSC-NFT, and the hash comparison ensures that the data has not been tampered with.

[0163] S5, Data Security Certificate Minting and Trading (Compliant Circulation)

[0164] On-chain transactions of data usage rights are achieved through DSC-NFT, ensuring the original data remains completely confidential. The specific process involves: using S4 processing results to realize the credentialed transaction of high / medium sensitivity data usage rights, guaranteeing that the original data never leaves the country. The detailed process is as follows: 1. DSC-NFT Minting: The credential minting unit generates ERC-721 standard DSC-NFTs from high / medium sensitive data. The metadata includes information such as data hash fingerprint, authorized use scenario (e.g., "advertising of baby formula"), revenue sharing ratio, and validity period. The metadata is stored in the IPFS distributed storage system to ensure immutability.

[0165] 2. Smart pricing and order placement: The smart contract automatically calculates the NFT price based on the formula "P=ΔROI×historical CPM×authorization days" (e.g., ΔROI=7.2%, historical CPM=100 yuan, authorization 30 days, then the price is 216 yuan). After the data provider confirms, the order is placed on the trading market, and the price can be manually adjusted.

[0166] 3. On-chain transaction execution: The buyer browses the trading market and selects the target NFT, pays the corresponding amount and pays 50% of the transaction amount as a deposit, which is locked by a smart contract; the system automatically verifies the buyer's qualifications, and after passing the verification, allocates access to the TEE computing pool to the buyer.

[0167] 4. Trusted Computation and Result Return: The buyer calls data to perform calculations (such as audience segmentation) within the TEE trusted environment. During the calculation process, the original data always exists in an encrypted state, and only the calculation result report is output; the calculation process log is written to the audit chain in real time.

[0168] 5. Automatic revenue sharing and deposit refund: After the NFT expires, the smart contract will automatically distribute the funds according to the preset revenue sharing ratio (e.g., 70% for the data provider and 30% for the platform); if no data leakage or other violations occur, the deposit will be fully refunded to the buyer, and the entire transaction will take ≤10 minutes.

[0169] S6, Hybrid Storage and Unified Queries

[0170] This step achieves efficient access and compliant control of multi-source data through "categorized storage - intelligent query - full-process auditing". The specific process is as follows: 1. Data Classification and Storage: After S4 differentiation processing, structured data (such as user profiles, transaction records, etc.) is directly written to the lake warehouse integrated architecture (Delta Lake), relying on its ACID transaction characteristics to ensure data consistency; unstructured data (such as user behavior logs, note content, etc.) is stored in Alibaba Cloud OSS object storage, and metadata such as file name, storage path, creation time, and data sensitivity level are extracted and written back to Delta Lake to establish a correlation mapping between structured and unstructured data.

[0171] 2. Cross-source intelligent query: Users input standard SQL statements through a unified query gateway. The CBO cost optimizer integrated into the gateway parses the statement and combines it with metadata information to generate the optimal cross-storage execution plan. The optimizer automatically determines the distribution location of the target data (lake warehouse or OSS), pulls multi-source data through a parallel thread pool, and returns the results after real-time aggregation and processing by the in-memory computing engine. The entire process has a latency of ≤300ms and supports 500 concurrent queries per second.

[0172] 3. Operation auditing and traceability: All storage write and query execution operations generate audit logs in real time, including information such as operator ID, data ID, operation type, execution duration, and hierarchical results. The logs are synchronously written to the audit log chain and linked with DSC-Chain to achieve tamper-proof traceability of the entire operation process, meeting the "auditability" requirement in the relevant requirements for personal information protection.

[0173] This example addresses the core pain points in traditional marketing data access, tiering, and distribution through technological innovation, achieving the following beneficial effects: 1. Exponentially improved access efficiency: The zero-code access method reduces the access time of new interfaces from 10 person-days (80 working hours) of traditional hard coding to 5 minutes, improving efficiency by 1152 times; the self-healing time of interface anomalies is ≤15 minutes, and the repair success rate is ≥98%, meeting the real-time access requirements of major promotional scenarios such as 618 and Double 11.

[0174] 2. Precise optimization of security investment: Dynamic tiering based on ΔROI enables security resources to be concentrated on highly sensitive data. 3. Compliant release of data value: The DSC-NFT transaction model achieves "data usable but not visible", and the whole process audit meets the compliance audit requirements and passes the compliance certification of regulatory agencies.

[0175] 4. Excellent query performance: Hybrid storage and caching optimization reduce the average latency of cross-source queries to ≤300ms, supporting 500 concurrent queries per second to meet the needs of real-time marketing decisions; the cost of unstructured data storage is reduced by 50% (compared to traditional HDFS storage).

[0176] Specific Example 1: Data Integration of Xiaohongshu Notes Interactions During the 618 Shopping Festival

[0177] Application scenario: An e-commerce company needs to access Xiaohongshu's note interaction data (likes, comments, reposts, and collections) before the 618 promotion (May 20th) to adjust its marketing strategy in real time; traditional access requires 10 person-days of development, and Xiaohongshu plans to upgrade its API interface on May 25th (adding a "collection" field), which poses a risk of interruption.

[0178] Implementation process: On May 20th, the operations staff entered the business term "Xiaohongshu Notes Interaction Data" into the system. The system used the GraphSAGE algorithm to match the knowledge graph subgraph and generate an OpenAPI specification that includes the note list API and interaction fields.

[0179] 2. The template engine compiles and generates a Docker image within 5 minutes. After rolling deployment in Kubernetes, the connection is completed. Connectivity tests show that the data collection success rate is 100%.

[0180] 3. At 9:00 AM on May 25th, the system monitored an interface anomaly rate η=8% (threshold 5%), triggering the syntax tree difference bot, which located the new addition to the "collection count" field within 3 minutes.

[0181] 4. The reinforcement learning model generated patch code within 10 minutes, and after gray-scale verification, it was fully updated. By 12:00, the anomaly rate dropped to 0.3%, without affecting the data collection for the promotion.

[0182] Implementation results: Access time was reduced from 10 person-days to 5 minutes, interface upgrade self-healing time was 12 minutes, data collection interruption time during major promotions was 0, and the response speed for marketing campaign strategy adjustments was increased by 3 times.

[0183] Specific Example 2: DSC-NFT Transactions for High-Spending Stay-at-Home Mothers

[0184] Application scenario: A data service provider has a data package of high-spending mothers (including spending amount, purchasing preferences, parenting stage, etc.) and needs to sell the right to use it for 30 days to a maternal and infant brand, requiring that the data not be disclosed in plaintext and that revenue sharing be guaranteed.

[0185] Implementation process: 1. The data service provider uploads the audience data to the system. The DRS module calls the federated training ΔROI model and outputs a predicted loss value of ΔROI=7.2%, which is determined to be high-sensitivity data.

[0186] 2. The system uses the BFV algorithm to encrypt data, which is then fed into the TEE computing pool to mint DSC-NFTs (metadata: hash value = 8a7f..., authorized scenario = "baby formula advertising", revenue sharing ratio = 7:3, validity period 30 days), priced at P = 7.2% × 100 yuan × 30 = 216 yuan.

[0187] 3. On May 1, a maternal and infant brand purchased the NFT on the trading market, paying 216 yuan and depositing 108 yuan as a security deposit, with the funds locked by a smart contract.

[0188] 4. From May 1st to May 30th, maternal and infant brands used data within TEE to complete advertising campaign analysis and generate a report on "Preferred Formula Types for High-Spending Mothers". The original encrypted data was not downloaded.

[0189] On June 1st, the smart contract automatically distributed the revenue (151.2 yuan for the data service provider and 64.8 yuan for the platform), refunded the deposit, and wrote the transaction log to the audit chain.

[0190] The method for processing and trading multi-source heterogeneous marketing data according to embodiments of this disclosure utilizes technologies such as marketing knowledge graphs, semantic parsing, and reinforcement learning to achieve automatic compilation from business concepts to connectors and minute-level self-healing of interface anomalies. This reduces the time for new interface access from the traditional 10 person-days to 5 minutes, solving the problem of low interface adaptation efficiency. By constructing a revenue loss ratio prediction model using the post-leakage ROI change value as the prediction label through a federated learning framework with multiple parties, the method performs streaming inference on the accessed data and automatically classifies sensitivity levels based on the ΔROI value, elevating the classification criteria from "format-level" to "commercial-level," thus addressing the lack of commercial relevance in sensitivity classification. Differentiated security processing is performed based on sensitivity levels—high-sensitivity data is homomorphically encrypted before being transmitted to the TEE, medium-sensitivity data retains its format and is de-identified, and low-sensitivity data is cached in plaintext, resolving issues of lagging security control and mismatch between security investment and risk. High / medium-sensitivity data is cast into data security certificates (NFTs), and automatic pricing, trading, and revenue sharing are achieved through consortium blockchain smart contracts. Buyers only obtain computing rights within the TEE and cannot access the original data, achieving "data usable but invisible," thus resolving the conflict between data circulation compliance and revenue. By employing a hybrid architecture combining lakeware and object storage, this system achieves categorized storage and relational mapping of structured and unstructured data. It provides cross-source intelligent querying (latency ≤300ms, concurrency 500 QPS) and on-chain linkage of audit logs, resolving issues such as single storage structure, highly heterogeneous data sources, and incomplete operational auditing. In summary, this disclosure forms a closed-loop process from access, hierarchical classification, processing, transaction to storage auditing, achieving a comprehensive technical effect of exponentially improved access efficiency, precise matching of security investment with data value, compliant release of data value, and a 50% reduction in storage costs.

[0191] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0192] In addition, this disclosure also provides a system for processing and trading multi-source heterogeneous marketing data, electronic devices, and computer-readable storage media. All of the above can be used to implement any of the methods for processing and trading multi-source heterogeneous marketing data provided in this disclosure. The corresponding technical solutions and descriptions are described in the relevant section of the method and will not be repeated here.

[0193] Figure 10This is a block diagram of a multi-source heterogeneous marketing data processing and transaction system provided in an embodiment of the present disclosure.

[0194] Reference Figure 10 This disclosure provides a system for processing and trading multi-source heterogeneous marketing data. The system 1000 includes a connection establishment module 1001, an anomaly monitoring module 1002, a sensitivity classification module 1003, a voucher casting module 1004, and a transaction module 1005.

[0195] The connection establishment module 1001 is used to: respond to receiving business concept terms input by the user, automatically compile and generate a data source connector based on the business concept terms input by the user using a marketing knowledge graph, and establish a connection between the data source connector and the external data interface.

[0196] The anomaly monitoring module 1002 is used to: monitor the running status of the connected external data interfaces in real time; when an anomaly is detected in the external data interface, automatically locate the change point of the abnormal interface and generate a repair patch to update the data source connector corresponding to the abnormal interface.

[0197] The sensitivity classification module 1003 is used to: acquire access data through the data source connector; dynamically classify the access data to obtain its sensitivity level.

[0198] The credential casting module 1004 is used to: perform differentiated security processing on the access data according to the sensitivity level of the access data, and cast the high-sensitivity data and / or medium-sensitivity data after security processing into data security credentials (NFTs).

[0199] The transaction module 1005 is used to complete the transaction of the corresponding NFT through the consortium blockchain.

[0200] Figure 11 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0201] Reference Figure 11 This disclosure provides an electronic device comprising: at least one processor 1101; at least one memory 1102; and one or more I / O interfaces 1103 connected between the processor 1101 and the memory 1102; wherein the memory 1102 stores one or more computer programs executable by the at least one processor 1101, the one or more computer programs being executed by the at least one processor 1101 to enable the at least one processor 1101 to perform the above-described method for processing and trading multi-source heterogeneous marketing data.

[0202] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for processing and trading multi-source heterogeneous marketing data. The computer-readable storage medium may be volatile or non-volatile.

[0203] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described method for processing and trading multi-source heterogeneous marketing data.

[0204] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0205] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0206] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0207] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0208] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0209] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0210] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0211] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0212] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0213] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for processing and trading multi-source heterogeneous marketing data, characterized in that, include: S1, in response to receiving business concept terms input by the user, automatically compile and generate a data source connector based on the business concept terms input by the user using a marketing knowledge graph, and establish a connection between the data source connector and the external data interface; S2, monitor the running status of the connected external data interfaces in real time, and when an abnormality is detected in the external data interface, automatically locate the change point of the abnormal interface and generate a repair patch to update the data source connector corresponding to the abnormal interface. S3, obtain access data through the data source connector; perform dynamic sensitivity classification on the access data to obtain the sensitivity level of the access data; S4, perform differentiated security processing on the access data according to the sensitivity level of the access data, and cast the high-sensitivity data and / or medium-sensitivity data after security processing into non-fungible credential NFTs; S5 enables the completion of corresponding NFT transactions through a consortium blockchain.

2. The method for processing and trading multi-source heterogeneous marketing data according to claim 1, characterized in that, S1 includes: S101, construct triples based on business entities, application programming interfaces (APIs), and fields, and construct a marketing knowledge graph based on the triples; S102, using a word vector model, generate term word vectors based on the business concept terms input by the user, and calculate the semantic similarity between the term word vectors and the nodes in the marketing knowledge graph; S103, invoke the graph matching algorithm to extract a related subgraph from the marketing knowledge graph based on the semantic similarity between the term vectors and the nodes in the marketing knowledge graph; generate an Open Application Programming Interface (Open API) differential specification based on the related subgraph; the related subgraph includes the API address, request method, and field mapping relationship of the target platform; the Open API differential specification marks the differences between the related subgraph and the standard specification; S104, based on the template engine, read the Open API differential specification, adapt and fill the preset connector template according to the difference points marked in the Open API differential specification, obtain the filled code, and compile the filled code into a container image; S105, the container image is deployed to the runtime environment by rolling hot loading, instantiated as the data source connector, the connection between the data source connector and the external data interface is established, and a connectivity test is performed. If the data acquisition success rate is not lower than a preset threshold, the connection between the data source connector and the external data interface is successful.

3. The method for processing and trading multi-source heterogeneous marketing data according to claim 2, characterized in that, S1 also includes: storing the marketing knowledge graph using the Neo4j graph database, and periodically updating the marketing knowledge graph through a crawler mechanism and a manual verification mechanism.

4. The method for processing and trading multi-source heterogeneous marketing data according to claim 1, characterized in that, S2 include: S201, Real-time collection of metrics returned by each interface in the connected external data interfaces, and calculation of the anomaly rate of each interface in the connected external data interfaces based on the metrics returned by each interface in the connected external data interfaces; the metrics returned by each interface in the connected external data interfaces include Hypertext Transfer Protocol (HTTP) response code, field missing rate, and data latency. S202, in response to the anomaly rate exceeding a preset threshold, trigger the syntax tree difference robot program Bot to compare the data structure returned by the anomaly interface with the historical data structure stored by the anomaly interface, and locate the change point of the anomaly interface; S203, Invoke the reinforcement learning model to generate patch code adapted to the abnormal interface change point, wherein the reward function of the reinforcement learning model consists of the decrease in the abnormality rate after repair and the repair time; S204. After deploying the patch code to the test environment for verification, arbitrarily select a portion of the online traffic of the data source connector corresponding to the abnormal interface for canary replacement. If the abnormality rate of the traffic after canary replacement is lower than a preset security threshold, then update all online traffic of the data source connector corresponding to the abnormal interface; otherwise, automatically roll back.

5. The method for processing and trading multi-source heterogeneous marketing data according to claim 1, characterized in that, S3 include: S301, uses a federated learning framework to build a revenue loss ratio ΔROI prediction model; S302, Perform streaming inference on the access data according to the ΔROI prediction model to obtain the ΔROI prediction value of the access data, and classify the sensitivity level of the access data according to the ΔROI value of the access data and the preset sensitivity level threshold to obtain the sensitivity level of the access data.

6. The method for processing and trading multi-source heterogeneous marketing data according to claim 5, characterized in that, S301 includes: S311 defines the sample requirements for federated learning, which include sample features and prediction labels; the sample features include data type, use case, and circulation frequency; the prediction labels include whether leakage has occurred and the change in revenue loss ratio ΔROI after leakage. S312, Send sample requirements to each of the multiple data providers so that each of the multiple data providers can select data from their local data that meets the sample requirements; S313, initialize the ΔROI prediction model and distribute the initialized ΔROI prediction model to each of the multiple data providers, so that each of the multiple data providers can use the gradient boosting tree algorithm to train the initialized ΔROI prediction model based on the data in the local data that meets the sample requirements, to obtain the local model of each of the multiple data providers, calculate the gradient of the local model of each of the multiple data providers, and encrypt and send back the gradient of the local model of each of the multiple data providers through homomorphic encryption technology; S314, Receive the gradients of the local models of each of the multiple data providers, which are encrypted and transmitted back by each of the multiple data providers; Aggregate and decrypt the received gradients; Use the aggregated and decrypted gradients to iteratively update the initialized ΔROI prediction model. S315, iterate and update until the model converges to obtain the ΔROI prediction model.

7. The method for processing and trading multi-source heterogeneous marketing data according to claim 5, characterized in that, S3 also includes: S304, the sensitivity level classification result, inference basis and operator information of the access data are encapsulated into an audit block, written into the consortium blockchain through a consensus mechanism, and a classification certificate is generated.

8. The method for processing and trading multi-source heterogeneous marketing data according to claim 1, characterized in that, In S4, the differentiated security processing includes: S401, for highly sensitive data, the BFV homomorphic encryption algorithm is used to encrypt the data to obtain ciphertext. The ciphertext is then fed into a TEE highly sensitive computing pool built on Intel Software Protection Extensions (Intel SGX), and an NFT bound to the hash of the highly sensitive data is minted to obtain the NFT of the highly sensitive data. The NFT of the highly sensitive data is then stored in the InterPlanetary File System (IPFS) distributed storage system. Direct download of the ciphertext is prohibited. S402, for medium-sensitive data, the mobile phone number and email fields in the medium-sensitive data are processed to retain the format and desensitize them, and an NFT labeled with the desensitization rules is cast to obtain the medium-sensitive data NFT, and the medium-sensitive data NFT is stored in the IPFS distributed storage system; the metadata of the high-sensitive data NFT and the medium-sensitive data NFT includes data hash fingerprint, authorized use scenario, revenue sharing ratio and data validity period; S403: For low-sensitivity data, after data quality verification, it is stored in plaintext in the Redis cache layer, and the hash fingerprint of the low-sensitivity data is written into the on-chain credential repository.

9. The method for processing and trading multi-source heterogeneous marketing data according to claim 8, characterized in that, S5 include: S501, Obtain the NFT of the high-sensitivity data and the NFT of the medium-sensitivity data from the IPFS distributed storage system, and then, through the smart contract of the consortium blockchain, execute the NFT according to the formula... Calculate the price of each NFT, and after confirmation by the data provider, list it on the trading market; among which, The price of the NFT, This is the predicted value of ΔROI. Cost of displaying history a thousand times For the number of authorized days; S502 receives the payment and deposit from the buyer based on the NFT price corresponding to the intended data, locks the funds through the smart contract of the consortium blockchain and verifies the buyer's qualifications. After the verification is passed, it allocates access to the TEE high-sensitivity computing pool to the buyer. S503, grant the buyer permission to call the intended data for calculation within the TEE high-sensitivity computing pool, and write the buyer's calculation process log into the audit log chain; during the calculation process, the intended data exists in an encrypted state, so that the buyer can obtain the calculation result but not the intended data. S504 After the authorized period expires, the funds are distributed through the smart contract of the consortium blockchain according to the preset revenue sharing ratio. If no data leakage occurs, the deposit will be returned to the buyer, and the transaction of the NFT corresponding to the intended data will be completed.

10. The method for processing and trading multi-source heterogeneous marketing data according to claim 1, characterized in that, Also includes: S6. Write the structured data in the data after differential security processing into the lake-warehouse integrated architecture based on Delta Lake, write the unstructured data into object storage, extract the metadata of the unstructured data and write it back to the lake-warehouse integrated architecture, and establish an association mapping between structured data and unstructured data. S7. In response to receiving a query request from a query user, the system receives the structured query language (SQL) statement input by the query user, parses the SQL statement through a cost optimizer, generates a cross-storage execution plan by combining metadata information, pulls data from the lake warehouse architecture and the object storage through a parallel thread pool, aggregates the data through an in-memory computing engine, and returns the query results to the query user. S8 generates audit logs for all storage write and query operations. The audit logs include the operator ID, data ID, operation type, and sensitivity level. The audit logs are then synchronously written to the audit log chain, which is linked with the consortium blockchain.

11. A system for processing and trading multi-source heterogeneous marketing data, characterized in that, include: The connection establishment module is used to: respond to receiving business concept terms input by the user, automatically compile and generate a data source connector based on the business concept terms input by the user using a marketing knowledge graph, and establish a connection between the data source connector and an external data interface; The anomaly monitoring module is used to: monitor the running status of the connected external data interfaces in real time; when an anomaly is detected in the external data interface, automatically locate the change point of the abnormal interface and generate a repair patch to update the data source connector corresponding to the abnormal interface. The sensitivity classification module is used to: acquire access data through the data source connector; perform dynamic sensitivity classification on the access data to obtain the sensitivity level of the access data; The credential casting module is used to: perform differentiated security processing on the access data according to the sensitivity level of the access data, and cast the high-sensitivity data and / or medium-sensitivity data after security processing into data security credentials (NFTs); The transaction module is used to complete the transaction of corresponding NFTs through the consortium blockchain.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more of the computer programs being executed by the at least one processor to enable the at least one processor to perform the method for processing and trading multi-source heterogeneous marketing data as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for processing and trading multi-source heterogeneous marketing data as described in any one of claims 1-10.

14. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs a method for processing and trading multi-source heterogeneous marketing data as described in any one of claims 1-10.

Images