Target information tracing method for internet big data

Abstract

The invention discloses a target information tracing method for internet big data, which comprises the following steps: capturing data from a plurality of internet data sources by using a distributed data capturing tool, collecting the data to a data processing platform, and preprocessing the captured data to generate a feature vector about the data; carrying out normalization processing and quantum state mapping on the data, and encoding the data into quantum state representation through a parameterized quantum circuit; constructing a quantum data index structure based on a quantum variation algorithm, and generating a quantum data index tree; adding a federal learning mechanism in the quantum data index tree, performing cooperative training on quantum state representations from different data sources, and updating global model parameters to obtain optimized quantum state representations; and inputting the optimized quantum state representation into the quantum neural network, and calculating and outputting a traceability result through forward propagation. According to the invention, efficient and accurate traceability of target information in an internet big data environment is realized, and the reliability of information traceability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of information processing and quantum computing, and in particular to a target information tracing method for Internet big data. Background Art

[0002] With the rapid development of the Internet, the amount of global data has shown an exponential growth and entered the era of big data. The massive amount of Internet data provides rich resources for information mining and analysis, but also brings huge challenges. Traditional data processing technology is often difficult to meet the requirements of efficiency and accuracy when faced with multi-source, heterogeneous, and real-time data. The emergence of distributed data crawling technology makes it possible to obtain data from multiple Internet data sources in real time. At the same time, the development of data preprocessing technology, such as data deduplication, cleaning, and formatting, has laid a solid foundation for subsequent data analysis.

[0003] In recent years, quantum computing, as an emerging computing paradigm, has gradually attracted the attention of academia and industry due to its potential advantages in parallel computing and solving complex problems. Research on quantum computing technologies such as quantum state mapping, quantum variational algorithms, and quantum neural networks has provided new ideas for big data processing and information traceability. In addition, federated learning, as a distributed machine learning method that protects data privacy, can achieve multi-party collaborative training without sharing original data, thereby improving the generalization ability of the model.

[0004] However, existing target information tracing methods mainly rely on traditional machine learning and statistical analysis techniques, such as keyword matching, association rule mining, and graph-based propagation path analysis. These methods face the problems of high computational complexity and poor real-time performance when processing massive amounts of Internet data, making it difficult to complete information tracing in a timely and accurate manner.

[0005] In addition, with the increasing awareness of data privacy and security, how to conduct effective data collaboration and model training while protecting the privacy of each data source has become an urgent problem to be solved; the shortcomings of existing technologies in data privacy protection and collaborative computing have limited the application and effectiveness of information traceability methods; at the same time, the application of quantum computing technology in big data processing is still in its infancy, lacking mature solutions, and unable to give full play to its advantages in computing efficiency and processing complexity. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract of the specification and the title of the invention of this application to avoid blurring the purpose of this section, the abstract of the specification and the title of the invention, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the above existing problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by the present invention is: the problems of low traceability efficiency and insufficient data privacy protection existing in the prior art.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: using a distributed data crawling tool to crawl data from multiple Internet data sources, collecting the data to a data processing platform, and performing deduplication, cleaning and formatting processing on the crawled data to generate a feature vector about the data;

[0010] Normalizing and quantum-state mapping the eigenvectors, and encoding them into quantum-state representations through parameterized quantum circuits;

[0011] Construct a quantum data index structure based on the quantum variational algorithm and generate a quantum data index tree;

[0012] Adding a federated learning mechanism in the quantum data index tree to collaboratively train the quantum state representations from different data sources, update global model parameters, and obtain an optimized quantum state representation;

[0013] The optimized quantum state representation is input into the quantum neural network, and the tracing result is output through forward propagation calculation:

[0014] If the output value of the traceability result is ≥ the set threshold, the traceability is successful and the credibility is high. A traceability report is generated to record the source information and the transmission path;

[0015] If the output value of the traceability result is less than the set threshold, the traceability fails and the credibility is low. The system prompts that more data is needed or the model parameters are adjusted.

[0016] As a preferred solution of the target information tracing method for Internet big data described in the present invention, the collecting of captured data to a data processing platform includes:

[0017] According to the tracing target, set keywords, URL list, and crawling frequency parameters;

[0018] The crawling tasks are distributed to each crawler node through the task scheduler, and the nodes do not interfere with each other;

[0019] Each node captures data according to the configured capture task;

[0020] Upload the captured data to the distributed data processing platform in real time.

[0021] As a preferred solution of the target information tracing method for Internet big data described in the present invention, generating a feature vector about the data includes:

[0022] Remove duplicate data by calculating unique identifiers for the data;

[0023] Remove HTML tags, special characters, and advertising content to complete noise filtering;

[0024] Remove or fill in missing data fields;

[0025] Convert text data to UTF-8 encoding;

[0026] Convert timestamps to a unified format;

[0027] Use Jieba word segmentation to segment Chinese text;

[0028] The text is then tagged with parts of speech to extract the parts of speech of nouns and verbs, and a text feature representation of the data, namely the feature vector, is generated.

[0029] As a preferred solution of the target information tracing method for Internet big data described in the present invention, the unique identifier includes a URL list and a message ID.

[0030] As a preferred solution of the target information tracing method for Internet big data described in the present invention, the timestamp is converted into a unified format of ISO 8601.

[0031] As a preferred solution of the target information tracing method for Internet big data described in the present invention, obtaining the quantum state representation includes:

[0032] Using quantum amplitude coding technology, classical data is loaded into quantum registers through quantum circuits;

[0033] The initial quantum state is transformed into the desired quantum state representation through a parameterized quantum circuit.

[0034] As a preferred solution of the target information tracing method for Internet big data described in the present invention, the construction of the quantum data index structure includes:

[0035] Quantum KD tree: for nearest neighbor search of high-dimensional data;

[0036] Quantum hash table: for fast retrieval;

[0037] Build index: Organize the optimized quantum state according to the index structure and store it in the quantum memory;

[0038] Search path optimization: Combined with Grover search algorithm to improve search efficiency;

[0039] Use an adaptive quantum parallel optimization algorithm to dynamically adjust the path based on the search results.

[0040] As a preferred solution of the target information tracing method for Internet big data described in the present invention, a quantum neural network is constructed using a parameterized quantum circuit, which includes an input layer for receiving quantum states, a hidden layer composed of a series of parameterized quantum gates, and an output layer for measuring specific quantum bits to output results;

[0041] Wherein, the parameterized quantum gate includes a rotating gate and a controlled gate.

[0042] As a preferred solution of the target information tracing method for Internet big data described in the present invention, a quantum state is input into a quantum neural network to obtain an output quantum state;

[0043] Measure the output quantum state, obtain the probability distribution P(y), and calculate the output value y. The mathematical expression formula is as follows:

[0044]

[0045] Among them, y is the output value of the traceability result, and i is the number of quantum states;

[0046] Use the optimization algorithm to adjust the parameter φ to minimize the cross entropy loss function;

[0047] Repeat forward propagation and parameter update until the model converges and outputs the quantum state.

[0048] The beneficial effects of the invention are as follows: The invention realizes efficient and accurate tracing of target information in the Internet big data environment, and utilizes quantum computing and federated learning advanced technologies to solve the problems of low computing efficiency, poor real-time performance, and insufficient data privacy protection in the prior art, thereby significantly improving the performance and reliability of information tracing. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0050] Figure 1 It is a flow chart of the target information tracing method for Internet big data shown in the present invention;

[0051] Figure 2 It is a schematic diagram of ROC curve analysis shown in the present invention. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0053] Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without making any creative work should fall within the scope of protection of the present invention.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0055] According to an embodiment of the present invention, Figure 1 The flowchart shown is a method for tracing the source of target information for Internet big data, which specifically includes the following steps:

[0056] S1. Use distributed data crawling tools to crawl data from multiple Internet data sources, collect them to the data processing platform, and perform deduplication, cleaning and formatting on the crawled data to generate feature vectors about the data. It should be noted that this step collects the crawled data to the data processing platform, including:

[0057] According to the tracing target, set keywords, URL list, and crawling frequency parameters;

[0058] The task scheduler distributes crawling tasks to each crawler node, so that the nodes do not interfere with each other, thus improving efficiency;

[0059] Each node crawls data according to the configuration and processes anti-crawling mechanisms (such as verification codes and IP bans) to ensure data integrity;

[0060] Upload the captured data to a distributed data processing platform such as Hadoop HDFS or NoSQL database in real time.

[0061] Furthermore, feature vectors about the data are generated, including:

[0062] Remove duplicate data by calculating the hash value of the data or using unique identifiers (such as URLs, message IDs);

[0063] Remove HTML tags, special characters, and advertising content to complete noise filtering;

[0064] Remove or fill in missing data fields;

[0065] Convert text data to UTF-8 encoding;

[0066] Convert timestamps to a uniform format (such as ISO 8601);

[0067] Use Jieba word segmentation to segment Chinese text;

[0068] The text is then tagged with parts of speech to extract the parts of speech of nouns and verbs, generating a text feature representation of the data, namely a feature vector.

[0069] Useful information extracted from the cleaned data is called features. For example, in text data, features may be the frequency of occurrence of keywords, sentiment tendency (positive or negative), and these features are arranged in the form of numerical values ​​to form a vector (list), called a feature vector:

[0070] V=[v1,v2,…,v n ]

[0071] Among them, v i represents the value of the i-th feature, and n is the total number of features.

[0072] Specifically, in the embodiment of the present invention, the word frequency feature v is obtained by calculating the TF-IDF value of the words in the text. tfidf , and then perform sentiment analysis on the text to get the sentiment score v sentiment , the value range is [-1,1], indicating negative to positive sentiment, and using the LDA model, we get the probability distribution v of the text on each topic topic , and finally merge all features to form a feature vector, whose mathematical expression formula is as follows:

[0073] v=[v tfidf ,v sentiment ,v topic ]

[0074] As an example, the distributed data crawling tool is implemented through a distributed crawler system, which can adopt ScrapyCluster, Apache Nutch or self-developed distributed crawler framework to achieve multi-node collaborative crawling.

[0075] In an optional implementation, a task scheduler (such as Apache Kafka, RabbitMQ) and a load balancer evenly distribute the crawling tasks to each crawler node.

[0076] As an example, the data source includes public Internet data sources such as social media APIs (such as Weibo API, Twitter API), news websites, forums, blogs, etc.

[0077] In an optional embodiment, the data cleaning tool is implemented using Python's pandas, NumPy, or a big data processing framework such as Apache Spark, and natural language processing (NLP) can be performed on text through Jieba, NLTK, or Stanford CoreNLP.

[0078] Preferably, this step ensures the quality and consistency of the data, reduces the impact of redundancy and noise on subsequent analysis, and thus improves the accuracy and efficiency of information traceability.

[0079] S2, normalize and map the feature vector to a quantum state, and encode it into a quantum state representation through a parameterized quantum circuit. Among them, what needs to be explained in this step is:

[0080] In order to ensure that the eigenvector satisfies the normalization condition of the quantum state, that is, the total probability is 1, the second norm of the eigenvector is calculated first:

[0081]

[0082] Then normalize it:

[0083]

[0084] The normalized eigenvector V′=[v′1,v′2,,v′ n ] satisfies the following conditions:

[0085]

[0086] Among them, v′ i is the normalized feature value.

[0087] Furthermore, the normalized eigenvector is mapped to a quantum state, and the data is represented using the quantum superposition principle. The mathematical expression formula for constructing the quantum state is as follows:

[0088]

[0089] Among them, |φ> is the quantum state, |i> is the standard computational basis vector, representing the i-th quantum basis state.

[0090] Using quantum amplitude coding technology, classical data is loaded into quantum registers through quantum circuits, such as using a quantum random access memory (QRAM) structure.

[0091] The initial quantum state is transformed into the desired quantum state representation through a parameterized quantum circuit.

[0092] As an example, we define a parameterized quantum circuit as U(θ), ​​which consists of multiple layers of quantum gates, such as the rotation gate RY(θ i) and controlled NOT gate (CNOT), parameter θ=[θ1,θ2,,θ m ]Control the rotation angle of the quantum gate.

[0093] As an example, the quantum state evolution process is:

[0094] The initial state is (All quantum bits are initialized to 0);

[0095] The coded state is obtained by evolving U(θ), ​​as follows:

[0096]

[0097] It should be noted that by normalizing and quantum-state mapping the eigenvectors and encoding them into quantum state representations through parameterized quantum circuits, efficient conversion of classical data vectors to quantum states is achieved. This allows data to be processed in a quantum computing environment, and by utilizing quantum superposition and parallel computing characteristics, the speed and computational efficiency of large-scale data processing are significantly improved.

[0098] S3. Construct a quantum data index structure based on a quantum variational algorithm to generate a quantum data index tree. The following are some instructions for this step:

[0099] According to the task requirements, the loss function of the quantum variational algorithm is defined as:

[0100] L(θ)=1-|<φ|ψ(θ)>| 2

[0101] Among them, 〈φ|ψ(θ)〉| 2 It represents the overlap between two quantum states and measures the similarity, |ψ(θ)> is the quantum state, and θ is the circuit parameter;

[0102] Use classic optimization algorithms such as gradient descent, Adam, and SPSA (stochastic perturbation gradient estimation) to calculate the derivative of the loss function with respect to the parameters

[0103] Update parameters:

[0104]

[0105] Where η is the learning rate;

[0106] Repeat the above steps until the loss function converges to the preset threshold.

[0107] Furthermore, the optimized quantum state is used to establish a quantum index structure to achieve efficient search, wherein the index structure is selected, including:

[0108] Quantum KD tree: for nearest neighbor search of high-dimensional data;

[0109] Quantum hash table: for fast retrieval;

[0110] Build index: Organize the optimized quantum state according to the index structure and store it in the quantum memory;

[0111] Search path optimization: Combined with Grover search algorithm to improve search efficiency;

[0112] Use an adaptive quantum parallel optimization algorithm to dynamically adjust the path based on the search results.

[0113] Preferably, this step optimizes the circuit parameter θ so that the quantum state |ψ(θ)> satisfies specific conditions, thereby improving the indexing efficiency.

[0114] Preferably, this step constructs a quantum data index structure based on a quantum variational algorithm and generates a quantum data index tree, thereby achieving efficient indexing and retrieval of data in a quantum computing environment. Through the optimization of the quantum variational algorithm, the query performance and accuracy of the index structure are improved, so that when processing high-dimensional and large-scale data, the target information can be quickly located, thereby enhancing the real-time nature of traceability.

[0115] S4. Add a federated learning mechanism to the quantum data index tree to collaboratively train quantum state representations from different data sources, update global model parameters, and obtain optimized quantum state representations. It should be noted that the construction of the federated learning framework includes:

[0116] Determine participating nodes: Set K participants, each with its own data and computing processing center;

[0117] Communication protocol: Adopt a secure federated learning protocol using homomorphic encryption or secure multi-party computing (SMPC) technology.

[0118] Furthermore, collaborative training is performed to update global model parameters, including:

[0119] Each node (participant) processes its own data according to steps S1 to S3 to generate a local quantum state;

[0120] Run VQA (Variational Quantum Algorithm) independently to obtain the local optimal parameter θ k ;

[0121] Each node will encrypt θ k Upload to the central server;

[0122] The server aggregates the parameters as follows:

[0123]

[0124] θglobal Distribute to each node and update the local model;

[0125] Repeat the above process and the model is gradually optimized.

[0126] Specifically, a quantum neural network (QNN) is constructed using a parameterized quantum circuit, which includes a circuit for receiving a quantum state |ψ(θ global )>, a hidden layer consisting of a series of parameterized quantum gates (such as rotating gates and controlled gates), and an output layer that measures specific quantum bits to output results, where:

[0127] Input the quantum state obtained above into the QNN circuit to obtain the output quantum state;

[0128] Measure the output quantum state, obtain the probability distribution P(y), and calculate the output value y. The mathematical expression formula is as follows:

[0129]

[0130] Use the optimization algorithm to adjust the parameter φ to minimize the cross entropy loss function:

[0131]

[0132] Repeat the forward propagation and parameter update until the model converges.

[0133] In an optional implementation, the adjustment of the search path includes the following steps:

[0134] Analyze model output: judge the validity of the current traceability path based on the output y of QNN;

[0135] Adjust search parameters: Dynamically adjust the parameters of the quantum search algorithm, such as the number of iterations t and the amplification factor, based on feedback;

[0136] Optimize search path: Re-execute quantum search and select the optimal path.

[0137] It should be noted that the federated learning mechanism improves the generalization ability of the model and the accuracy of traceability results while protecting the privacy of each data source, while avoiding the security and privacy risks brought about by data centralization.

[0138] Preferably, the optimized quantum state representation is input into the quantum neural network, and the tracing result is output through forward propagation calculation, thereby realizing deep learning and pattern recognition of quantum state data. The application of quantum neural network makes full use of the nonlinearity and parallel processing capabilities of quantum computing, and improves the accuracy and reliability of tracing analysis.

[0139] S5, input the optimized quantum state representation into the quantum neural network, and output the tracing result through forward propagation calculation. Among them, it is also necessary to explain that:

[0140] If the traceability result output value y ≥ the set threshold τ, the traceability is successful and the credibility is high. A traceability report is generated to record the source information and propagation path;

[0141] If the output value y of the traceability result is less than the set threshold τ, the traceability fails and the credibility is low. The system prompts that more data is needed or the model parameters need to be adjusted.

[0142] As an example, the traceability result output value y∈[0,1] represents the credibility of the traceability result. The larger the value, the higher the credibility. The threshold τ∈[0,1] is set.

[0143] As an example, the report content generated by the traceability report includes:

[0144] Source information: the original publisher or source node that can be traced back to;

[0145] Propagation path: the propagation path and key nodes of information in the network;

[0146] Credibility: Output result y and threshold τ, as well as credibility percentage;

[0147] Analysis and opinions: professional analysis and suggestions based on the results;

[0148] Report format: presented in a graphic and textual form, including flow charts, statistical charts and text descriptions.

[0149] As an example, the output of the traceability report includes:

[0150] Real-time display: display traceability results and analysis reports on the system interface;

[0151] Export file: support export to PDF and Word formats;

[0152] Notification mechanism: notify relevant personnel via email or SMS.

[0153] It is not difficult to understand that this embodiment selects a threshold value τ that balances the false alarm rate and the missed alarm rate as a credibility evaluation standard for the traceability result by statistically analyzing a large amount of historical data. For example, when τ=0.85 is set, the traceability is considered successful when the credibility exceeds 85%.

[0154] The traceability result is determined according to the output result y and the threshold τ, including the following steps:

[0155] (1) Credibility calculation:

[0156] Multiply the output y by 100% to get the confidence percentage:

[0157] Credibility = y × 100%

[0158] (2) Decision logic:

[0159] Successful tracing: The condition is y ≥ τ;

[0160] Operation: Output the traceability report and provide detailed information;

[0161] Tracing failure: the condition is y<τ;

[0162] Action: The prompt indicates that the credibility is insufficient. It is recommended to obtain more data or adjust the model parameters.

[0163] As an example, if the output result y=0.88, the threshold τ=0.85, and the credibility: 0.88×100%=88%; then the judgment: because 0.88≥0.85, it is judged that the traceability is successful, and the credibility is 88%; action: generate and output the traceability report.

[0164] Reference Figure 2 , the embodiment of the present invention draws the ROC curve and selects the threshold τ at the inflection point of the curve to balance the true positive rate and the false positive rate, such as Figure 2 As shown in the figure, the horizontal axis (X axis) represents FPR (false positive rate), ranging from 0 to 1, without unit; the vertical axis (Y axis) represents TPR (true positive rate), ranging from 0 to 1, without unit. The implementation steps include:

[0165] (1) Determine the value range of the model output result y, which is 0 to 1;

[0166] (2) Select a series of thresholds τ, for example, from 0 to 1, with a step size of 0.1;

[0167] (3) For each threshold τ, classification is performed according to the following rules:

[0168] If y ≥ τ, the prediction is positive (tracing is successful);

[0169] If y<τ, the prediction is negative (tracing failure);

[0170] (4) Based on the actual true labels and the model’s prediction results, calculate the corresponding TPR and FPR;

[0171] (6) Plot the TPR and FPR at each threshold in a coordinate system and connect the points to form a ROC curve.

[0172] In an optional implementation, based on business needs, such as high-security scenarios (financial fraud tracing) requiring high credibility, τ ≥ 0.9 can be set; such as general scenarios (market research) requiring general credibility, τ ≥ 0.85 can be set (in the test data set, it was found that when the threshold was 0.85, the F1-score of the model reached the highest); improve coverage.

[0173] It should be noted that in actual applications, users can automatically adjust the threshold according to real-time data characteristics and model performance, and users are allowed to manually set the threshold according to their own needs. The embodiments of the present invention do not make a unique limitation on this.

[0174] Preferably, the embodiment of the present invention determines the traceability results by setting thresholds, thereby realizing an automated decision-making and feedback mechanism for the traceability process, which enables the system to automatically take corresponding measures based on the credibility of the results, thereby improving the intelligence level of the system and user experience.

[0175] In order to verify the effectiveness of the target information tracing method for Internet big data described in the present invention, a variety of Internet data sources such as social media platforms, news websites and forums were selected to simulate the tracing process of an actual online public opinion event.

[0176] This experiment aims to verify the effectiveness and advantages of the proposed target information tracing method for Internet big data. By capturing, preprocessing, quantum state mapping, building quantum data index structure, applying federated learning mechanism and training quantum neural network of multi-source heterogeneous data, the performance of this method in tracing accuracy, efficiency and credibility is evaluated.

[0177] Experimental environment conditions:

[0178] Hardware environment:

[0179] Distributed server cluster: consists of 10 node servers, each with the following configuration:

[0180] CPU: Intel Xeon E5-2680 v4;

[0181] Memory: 128GB DDR4;

[0182] Storage: 2TB SSD;

[0183] Network: 10 Gigabit Ethernet;

[0184] Quantum computing simulator: IBM Qiskit Aer Simulator, supports simulation of 32 quantum bits.

[0185] Software environment:

[0186] Operating system: Ubuntu 20.04LTS;

[0187] Programming language: Python 3.8;

[0188] Data scraping tool: Scrapy 2.5.0;

[0189] Task scheduler: Apache Kafka 2.7.0;

[0190] Data processing platform: Hadoop HDFS 3.2.1;

[0191] Data cleaning and processing libraries: pandas, NumPy;

[0192] Natural language processing tools: Jieba 0.42.1, NLTK 3.5;

[0193] Quantum computing framework: IBM Qiskit 0.23.6;

[0194] Machine learning libraries: TensorFlow 2.4.1, TensorFlow Quantum 0.5.0;

[0195] Federated learning framework: TensorFlow Federated 0.19.0.

[0196] Data Source:

[0197] Social media APIs: Weibo API, Twitter API;

[0198] News websites: Xinhuanet, BBC News, CNN;

[0199] Forums and blogs: Zhihu, CSDN, Medium.

[0200] Experimental process:

[0201] According to the traceability goal, a keyword list was set, including "new product release", "market reaction", and "user evaluation", and a related URL list and crawling frequency parameters were set to ensure the real-time and comprehensiveness of the data; the task scheduler Apache Kafka was used to evenly distribute the crawling tasks to each crawler node to achieve distributed data crawling. Each crawler node uses the Scrapy framework and is configured with a proxy IP pool and a verification code recognition module to cope with the anti-crawling mechanism of each website to ensure the integrity and legality of the data.

[0202] During the crawling process, each crawler node uploads the data to the distributed data processing platform Hadoop HDFS in real time. In order to ensure data quality, the collected data is preprocessed. The Jieba word segmentation tool is used to segment the text data and perform part-of-speech tagging to extract key words such as nouns and verbs. The TF-IDF algorithm is used to calculate the keyword weights in the text to obtain the word frequency features. At the same time, a sentiment analysis model (such as SnowNLP) is used to calculate the sentiment score of the text. The LDA topic model is used to obtain the probability distribution of the text on each topic, and all features are merged to form a feature vector.

[0203] The feature vector is normalized and mapped to a quantum state to construct a quantum state. The initial quantum state is evolved into a coded state using a parameterized quantum circuit through multiple layers of rotation gates and controlled NOT gates. Based on the quantum variational algorithm (VQA), a loss function is defined and the Adam optimization algorithm is used to iteratively update the circuit parameters θ\thetaθ until the loss function converges. The optimized quantum state is used to construct a quantum KD tree and a quantum hash table, and the quantum state is organized according to an index structure and stored in a quantum memory. The Grover search algorithm and the adaptive quantum parallel optimization algorithm are combined to dynamically adjust the path according to the search results to improve the search efficiency.

[0204] In the quantum data index tree, a federated learning mechanism is added. In this experiment, three participants are set up. Each participant has its own data and computing processing center. A secure federated learning protocol and homomorphic encryption technology are used. Each node processes its own data according to the above steps, generates a local quantum state, and independently runs VQA to obtain the local optimal parameter θ k , each node will encrypt θ k Upload to the central server, the server aggregates the parameters and distributes them to each node, repeating the above process to gradually optimize the model.

[0205] Finally, the optimized quantum state is input into the quantum neural network (QNN), the output quantum state is measured, the probability distribution is obtained, the output value y is calculated, the cross entropy loss function and the gradient descent method are used to optimize the QNN parameters, and the forward propagation and parameter update are repeated until the model converges.

[0206] According to the output value y of the traceability result, the threshold τ=0.85 is set. If y≥τ, the traceability is judged to be successful and a traceability report is generated; if y<τ, the traceability is judged to be unsuccessful, and the system prompts that more data is needed or the model parameters are adjusted. The statistical results of the test data are shown in the following table:

[0207] Table 1. Experimental data results statistics

[0208]

[0209] Referring to Table 1, it can be seen that the method of the present invention can effectively extract features and make accurate traceability judgments when processing different samples. First, the construction of the feature vector covers a variety of information such as TF-IDF feature values, sentiment scores, and topic distribution, which fully reflects the content and sentiment characteristics of the text. The normalized feature vector length of all samples is 1, which meets the normalization requirements of the quantum state and lays the foundation for subsequent quantum computing.

[0210] In terms of the traceability result output value y, the y values ​​of samples 1, 2, 4, and 7 are 0.90, 0.88, 0.93, and 0.91, respectively, which are all higher than the set threshold τ=0.85. Therefore, they are judged to be successfully traced. For these samples, traceability reports are generated, recording the source information and propagation path, which shows that the method of the present invention can accurately trace the target information with high credibility.

[0211] The y values ​​of samples 3, 5, and 6 are 0.75, 0.80, and 0.65, respectively, which are lower than the threshold τ and are judged as traceability failures. The system prompts that more data is needed or the model parameters need to be adjusted. This shows that when the sentiment scores are low or negative (such as the sentiment scores of samples 3 and 6 are -0.20 and -0.50), the traceability credibility of the model will be reduced and further optimization is needed.

[0212] Compared with the existing technology, the method of the present invention significantly improves the accuracy and efficiency of traceability by introducing quantum computing and federated learning mechanisms. The application of quantum state mapping and parameterized quantum circuits enables high-dimensional feature data to be efficiently processed in a quantum computing environment. The combination of quantum variational algorithms and quantum neural networks enhances the learning and generalization capabilities of the model.

[0213] In addition, the introduction of the federated learning mechanism ensures the collaborative training of multiple data sources while protecting the data privacy of all participants, which is difficult to achieve in the traditional centralized model; experimental data show that the optimized global model parameters can improve the overall traceability effect.

[0214] The aforementioned preprocessing method for crawled data and the TF-IDF value calculation, sentiment analysis, and LDA model calculation method for text data can be performed using methods and means in the prior art and will not be repeated in this example.

[0215] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A target information tracing method for Internet big data, characterized in that: include: Using a distributed data crawling tool to crawl data from multiple Internet data sources, collecting the data to a data processing platform, and performing deduplication, cleaning and formatting on the crawled data to generate a feature vector for the data; Normalizing and quantum-state mapping the eigenvectors, and encoding them into quantum-state representations through parameterized quantum circuits; Construct a quantum data index structure based on the quantum variational algorithm and generate a quantum data index tree; Adding a federated learning mechanism in the quantum data index tree to collaboratively train the quantum state representations from different data sources, update global model parameters, and obtain an optimized quantum state representation; The optimized quantum state representation is input into the quantum neural network, and the tracing result is output through forward propagation calculation: If the output value of the traceability result is ≥ the set threshold, the traceability is successful and the credibility is high. A traceability report is generated to record the source information and the transmission path; If the output value of the traceability result is less than the set threshold, the traceability fails and the credibility is low. The system prompts that more data is needed or the model parameters are adjusted.

2. The target information tracing method for Internet big data according to claim 1 is characterized in that: The collecting of the captured data to the data processing platform includes: According to the tracing target, set keywords, URL list, and crawling frequency parameters; The crawling tasks are distributed to each crawler node through the task scheduler, and the nodes do not interfere with each other; Each node captures data according to the configured capture task; Upload the captured data to the distributed data processing platform in real time.

3. The target information tracing method for Internet big data according to claim 1 is characterized in that: The step of generating a feature vector about the data comprises: Remove duplicate data by calculating unique identifiers for the data; Remove HTML tags, special characters, and advertising content to complete noise filtering; Remove or fill in missing data fields; Convert text data to UTF-8 encoding; Convert timestamps to a unified format; Use Jieba word segmentation to segment Chinese text; The text is then tagged with parts of speech to extract the parts of speech of nouns and verbs, and a text feature representation of the data, namely the feature vector, is generated.

4. The target information tracing method for Internet big data according to claim 3 is characterized in that: The unique identifier includes a URL list and a message ID.

5. The target information tracing method for Internet big data according to claim 3 is characterized in that: Convert the timestamp to ISO 8601 format.

6. The target information tracing method for Internet big data according to claim 1 is characterized in that: Obtaining the quantum state representation comprises: Using quantum amplitude coding technology, classical data is loaded into quantum registers through quantum circuits; The initial quantum state is transformed into the desired quantum state representation through a parameterized quantum circuit.

7. The target information tracing method for Internet big data according to claim 1 is characterized in that: The step of constructing a quantum data index structure includes: Quantum KD tree: for nearest neighbor search of high-dimensional data; Quantum hash table: for fast retrieval; Build index: Organize the optimized quantum state according to the index structure and store it in the quantum memory; Search path optimization: Combined with Grover search algorithm to improve search efficiency; Use an adaptive quantum parallel optimization algorithm to dynamically adjust the path based on the search results.

8. The target information tracing method for Internet big data according to claim 1 is characterized in that: Using parameterized quantum circuits to construct a quantum neural network, which includes an input layer for receiving quantum states, a hidden layer consisting of a series of parameterized quantum gates, and an output layer for measuring specific quantum bits to output results; Wherein, the parameterized quantum gate includes a rotating gate and a controlled gate.

9. The target information tracing method for Internet big data according to claim 8 is characterized in that: Input the quantum state into the quantum neural network to obtain the output quantum state; Measure the output quantum state, obtain the probability distribution P(y), and calculate the output value y. The mathematical expression formula is as follows: Among them, y is the output value of the traceability result, and i is the number of quantum states; Use the optimization algorithm to adjust the parameter φ to minimize the cross entropy loss function; Repeat forward propagation and parameter update until the model converges and outputs the quantum state.

Images