Target information tracing method for internet big data

Abstract

The application discloses a target information tracing method for internet big data, comprising the following steps: using a distributed data crawling tool to crawl data from multiple internet data sources, collecting the data to a data processing platform, and preprocessing the crawled data to generate a feature vector about the data; normalizing and quantum state mapping are performed on the feature vector, and the feature vector is encoded into a quantum state representation through a parameterized quantum circuit; a quantum data index structure is constructed based on a quantum variation algorithm to generate a quantum data index tree; a federal learning mechanism is added to the quantum data index tree, quantum state representations from different data sources are collaboratively trained, global model parameters are updated, and optimized quantum state representations are obtained; and the optimized quantum state representations are input into a quantum neural network to calculate output tracing results through forward propagation. The application realizes efficient and accurate tracing of target information in an internet big data environment, and improves the reliability of information tracing.

Description

Technical Field

[0001] This invention relates to the fields of information processing and quantum computing, and in particular to a method for tracing the source of target information in Internet big data. Background Technology

[0002] With the rapid development of the Internet, the global data volume has grown exponentially, ushering in the era of big data. The massive amount of Internet data provides abundant resources for information mining and analysis, but it also brings enormous challenges. Traditional data processing technologies often struggle to meet the requirements of efficiency and accuracy when faced with multi-source, heterogeneous, and real-time data. The emergence of distributed data crawling technology has made it possible to obtain data from multiple Internet data sources in real time. At the same time, the development of data preprocessing technologies, such as data deduplication, cleaning, and formatting, has laid a solid foundation for subsequent data analysis.

[0003] In recent years, quantum computing, as an emerging computing paradigm, has gradually attracted the attention of academia and industry due to its potential advantages in parallel computing and solving complex problems. Research on quantum computing technologies such as quantum state mapping, quantum variational algorithms, and quantum neural networks has provided new ideas for big data processing and information traceability. In addition, federated learning, as a distributed machine learning method that protects data privacy, can achieve multi-party collaborative training without sharing the original data, thereby improving the generalization ability of the model.

[0004] However, existing methods for tracing the source of target information mainly rely on traditional machine learning and statistical analysis techniques, such as keyword matching, association rule mining, and graph-based propagation path analysis. When dealing with massive amounts of Internet data, these methods face problems of high computational complexity and poor real-time performance, making it difficult to complete information tracing in a timely and accurate manner.

[0005] Furthermore, with increasing awareness of data privacy and security, how to conduct effective data collaboration and model training while protecting the privacy of various data sources has become an urgent problem to be solved. The shortcomings of existing technologies in data privacy protection and collaborative computing limit the application and effectiveness of information tracing methods. At the same time, the application of quantum computing technology in big data processing is still in its early stages, lacking mature solutions and failing to fully leverage its advantages in computational efficiency and processing complexity. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract and title of the invention. Such simplifications or omissions shall not be used to limit the scope of the present invention.

[0007] In view of the aforementioned existing problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by this invention is the low traceability efficiency and insufficient data privacy protection in the prior art.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: using a distributed data crawling tool to crawl data from multiple Internet data sources, collecting it to a data processing platform, and performing deduplication, cleaning and formatting processing on the crawled data to generate feature vectors about the data;

[0010] The feature vector is normalized and mapped to a quantum state, and then encoded into a quantum state representation using a parameterized quantum circuit.

[0011] A quantum data index structure is constructed based on the quantum variational algorithm, and a quantum data index tree is generated.

[0012] A federated learning mechanism is added to the quantum data index tree to collaboratively train the quantum state representations from different data sources, update the global model parameters, and obtain the optimized quantum state representations.

[0013] The optimized quantum state representation is input into the quantum neural network, and the source tracing result is output through forward propagation calculation:

[0014] If the output value of the source tracing result is greater than or equal to the set threshold, the source tracing is successful, the credibility is high, a source tracing report is generated, and the source information and propagation path are recorded.

[0015] If the output value of the tracing result is less than the set threshold, the tracing fails, the credibility is low, and the system prompts that more data is needed or the model parameters need to be adjusted.

[0016] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, the step of collecting the captured data to a data processing platform includes:

[0017] Based on the source tracing target, set the keywords, URL list, and crawling frequency parameters;

[0018] The task scheduler distributes the crawling tasks to each crawler node, and the nodes do not interfere with each other.

[0019] Each node retrieves data according to the configured retrieval task;

[0020] The captured data is uploaded to the distributed data processing platform in real time.

[0021] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, the generation of feature vectors about the data includes:

[0022] Remove duplicate data by calculating the unique identifier of the data;

[0023] Remove HTML tags, special characters, and advertising content to filter noise.

[0024] Delete or populate missing data fields;

[0025] Convert text data to UTF-8 encoding;

[0026] Convert the timestamp to a uniform format;

[0027] Use Jieba word segmentation to process Chinese text into words;

[0028] The text is then tagged with parts of speech to extract the parts of speech of nouns and verbs, generating a text feature representation of the data, namely the feature vector.

[0029] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, the unique identifier includes a URL list and a message ID.

[0030] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, the timestamp is converted into the unified format of ISO 8601.

[0031] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, obtaining the quantum state representation includes:

[0032] Using quantum amplitude encoding technology, classical data is loaded into a quantum register through quantum circuits;

[0033] The initial quantum state is transformed into the desired quantum state representation by parameterizing the quantum circuit.

[0034] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, the construction of the quantum data index structure includes:

[0035] Quantum KD-tree: Nearest neighbor search for high-dimensional data;

[0036] Quantum hash tables: used for fast retrieval;

[0037] Index building: The optimized quantum states are organized according to an index structure and stored in the quantum memory;

[0038] Search path optimization: Improve search efficiency by combining the Grover search algorithm;

[0039] An adaptive quantum parallel optimization algorithm is used to dynamically adjust the path based on the search results.

[0040] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, a quantum neural network is constructed using parameterized quantum circuits, which includes an input layer for receiving quantum states, a hidden layer composed of a series of parameterized quantum gates, and an output layer for measuring specific qubits to output results.

[0041] The parameterized quantum gates include rotation gates and controlled gates.

[0042] As a preferred embodiment of the target information tracing method for Internet big data described in this invention, a quantum state is input into a quantum neural network to obtain an output quantum state.

[0043] The output quantum state is measured to obtain the probability distribution P(y), and the output value y is calculated. The mathematical expression of this value is as follows:

[0044]

[0045] Where y is the output value of the tracing result, and i is the number of quantum states;

[0046] The parameter φ is adjusted using an optimization algorithm to minimize the cross-entropy loss function;

[0047] Repeat the forward propagation and parameter updates until the model converges and outputs the quantum state.

[0048] The beneficial effects of this invention are as follows: This invention achieves efficient and accurate tracing of target information in the context of Internet big data. By utilizing advanced technologies such as quantum computing and federated learning, it solves the problems of low computational efficiency, poor real-time performance, and insufficient data privacy protection in existing technologies, and significantly improves the performance and reliability of information tracing. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0050] Figure 1 This is a flowchart illustrating the target information tracing method for Internet big data as shown in this invention;

[0051] Figure 2 This is a schematic diagram of ROC curve analysis as shown in this invention. Detailed Implementation

[0052] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0053] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the scope of protection of this invention.

[0054] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0055] According to an embodiment of the present invention, in combination Figure 1 The flowchart shown illustrates a method for tracing the source of target information using internet big data, which specifically includes the following steps:

[0056] S1. Use distributed data crawling tools to crawl data from multiple internet data sources, collect it to a data processing platform, and perform deduplication, cleaning, and formatting on the crawled data to generate feature vectors. Note that this step involves collecting the crawled data to the data processing platform, including:

[0057] Based on the source tracing target, set the keywords, URL list, and crawling frequency parameters;

[0058] The task scheduler distributes crawling tasks to each crawler node, ensuring that the nodes do not interfere with each other and improving efficiency.

[0059] Each node crawls data according to its configuration and handles anti-crawling mechanisms (such as CAPTCHA and IP blocking) to ensure data integrity;

[0060] The captured data is uploaded in real time to a distributed data processing platform, such as Hadoop HDFS or a NoSQL database.

[0061] Furthermore, feature vectors about the data are generated, including:

[0062] Remove duplicate data by calculating the hash value of the data or by using unique identifiers (such as URLs or message IDs);

[0063] Remove HTML tags, special characters, and advertising content to filter noise.

[0064] Delete or populate missing data fields;

[0065] Convert text data to UTF-8 encoding;

[0066] Convert the timestamp to a uniform format (such as ISO 8601);

[0067] Use Jieba word segmentation to process Chinese text into words;

[0068] Next, the text is tagged with parts of speech to extract the parts of speech of nouns and verbs, generating a text feature representation of the data, namely a feature vector.

[0069] Extracting useful information from cleaned data is called creating features. For example, in text data, features might be the frequency of keyword occurrences or sentiment (positive or negative). Arranging these features numerically to form a vector (list) is called a feature vector.

[0070] V = [v1, v2, ..., v n ]

[0071] Among them, v i This represents the value of the i-th feature, where n is the total number of features.

[0072] Specifically, in this embodiment of the invention, word frequency features v are obtained by calculating TF-IDF values ​​for words in the text. tfidf Then, sentiment analysis is performed on the text to obtain a sentiment score v. sentiment The value range is [-1, 1], representing negative to positive sentiment. The probability distribution v of the text across various topics is obtained using the LDA model. topic Finally, all features are merged to form a feature vector, the mathematical expression of which is as follows:

[0073] v = [v tfidf ,v sentiment ,v topic ]

[0074] As an example, distributed data crawling tools are implemented through distributed crawling systems, such as ScrapyCluster, Apache Nutch, or self-developed distributed crawling frameworks, to achieve collaborative crawling across multiple nodes.

[0075] In alternative implementations, a task scheduler (such as Apache Kafka or RabbitMQ) and a load balancer distribute crawling tasks evenly across the crawler nodes.

[0076] As an example, data sources include publicly available internet data sources such as social media APIs (e.g., Weibo API, Twitter API), news websites, forums, and blogs.

[0077] In alternative implementations, data cleaning tools are implemented using Python's pandas, NumPy, or big data processing frameworks such as Apache Spark, while natural language processing (NLP) can be performed using Jieba, NLTK, or Stanford CoreNLP.

[0078] Preferably, this step ensures data quality and consistency, reduces the impact of redundancy and noise on subsequent analysis, thereby improving the accuracy and efficiency of information tracing.

[0079] S2. Normalize the eigenvectors and map them to quantum states, then encode them into quantum state representations using parameterized quantum circuits. Note that the following points should be noted in this step:

[0080] To ensure that the eigenvectors satisfy the quantum state normalization condition, i.e., the total probability is 1, we first calculate the L2 norm of the eigenvectors:

[0081]

[0082] Then perform normalization:

[0083]

[0084] The normalized eigenvector V′=[v′1,v′2,,v′ n The following condition must be met:

[0085]

[0086] Where, v′ i These are the normalized feature values.

[0087] Furthermore, the normalized eigenvectors are mapped to quantum states, and the data is represented using the principle of quantum superposition. The mathematical expression formula for the quantum state is constructed as follows:

[0088]

[0089] Where |φ> is a quantum state, and |i> is a standard computational basis vector, representing the i-th quantum ground state.

[0090] Using quantum amplitude encoding technology, classical data can be loaded into quantum registers through quantum circuits, such as using a quantum random access memory (QRAM) structure.

[0091] The initial quantum state is transformed into the desired quantum state representation by parameterizing the quantum circuit.

[0092] As an example, a parameterized quantum circuit is defined as U(θ), ​​which consists of multiple layers of quantum gates, such as the rotation gate RY(θ). i) and controlled NOT gate (CNOT), with parameters θ = [θ1, θ2, ..., θ m Controlling the rotation angle of the quantum gate.

[0093] As an example, the quantum state evolution process is as follows:

[0094] The initial state is (All qubits are initialized to 0);

[0095] The encoded state is obtained through the evolution of U(θ), ​​as follows:

[0096]

[0097] It should be noted that by normalizing the feature vectors and mapping them to quantum states, and then encoding them into quantum state representations through parameterized quantum circuits, an efficient conversion from classical data vectors to quantum states is achieved. This enables data to be processed in a quantum computing environment, and by utilizing the characteristics of quantum superposition and parallel computing, the speed and computational efficiency of large-scale data processing are significantly improved.

[0098] S3. Construct a quantum data index structure based on the quantum variational algorithm to generate a quantum data index tree. Note the following in this step:

[0099] Based on the task requirements, the loss function of the quantum variational algorithm is defined as follows:

[0100] L(θ)=1-|<φ|ψ(θ)>| 2

[0101] Where, <φ|ψ(θ)>| 2 It represents the degree of overlap between two quantum states, measuring similarity, where |ψ(θ)> represents the quantum state and θ is a circuit parameter;

[0102] Classical optimization algorithms, such as gradient descent, Adam, and SPSA (stochastic perturbation gradient estimation), are used to calculate the derivative of the loss function with respect to the parameters.

[0103] Update parameters:

[0104]

[0105] Where η is the learning rate;

[0106] Repeat the above steps until the loss function converges to the preset threshold.

[0107] Furthermore, by utilizing the optimized quantum states, a quantum index structure is established to achieve efficient searching. The selection of the index structure includes:

[0108] Quantum KD-tree: Nearest neighbor search for high-dimensional data;

[0109] Quantum hash tables: used for fast retrieval;

[0110] Index building: The optimized quantum states are organized according to an index structure and stored in the quantum memory;

[0111] Search path optimization: Improve search efficiency by combining the Grover search algorithm;

[0112] An adaptive quantum parallel optimization algorithm is used to dynamically adjust the path based on the search results.

[0113] Preferably, this step improves indexing efficiency by optimizing the circuit parameters θ to make the quantum state |ψ(θ)> satisfy specific conditions.

[0114] Preferably, this step constructs a quantum data index structure based on the quantum variational algorithm and generates a quantum data index tree, realizing efficient indexing and retrieval of data in a quantum computing environment. Through the optimization of the quantum variational algorithm, the query performance and accuracy of the index structure are improved, enabling the rapid location of target information when processing high-dimensional and large-scale data, thus enhancing the real-time nature of traceability.

[0115] S4. Add a federated learning mechanism to the quantum data index tree to collaboratively train quantum state representations from different data sources, update global model parameters, and obtain optimized quantum state representations. Note that the federated learning framework setup in this step includes:

[0116] Determine participating nodes: Define K participants, each with its own data and computing processing center;

[0117] Communication protocol: Employs a secure federated learning protocol, using homomorphic encryption or secure multi-party computation (SMPC) technology.

[0118] Furthermore, collaborative training is performed to update the global model parameters, including:

[0119] Each node (each participant) processes its own data according to steps S1 to S3 to generate a local quantum state;

[0120] Run VQA (Variable Quantum Algorithm) independently to obtain the locally optimal parameters θ. k ;

[0121] Each node will encrypt θ k Uploaded to the central server;

[0122] The server aggregates the parameters as follows:

[0123]

[0124] θglobal Distribute to each node to update the local model;

[0125] Repeat the above process to gradually optimize the model.

[0126] Specifically, a quantum neural network (QNN) is constructed using parameterized quantum circuits, which includes a circuit for receiving quantum states |ψ(θ)|. global The input layer consists of a series of parameterized quantum gates (such as rotation gates and controlled gates), and the output layer measures specific qubits to output the results.

[0127] The quantum state obtained above is input into the QNN circuit to obtain the output quantum state;

[0128] The output quantum state is measured to obtain the probability distribution P(y), and the output value y is calculated. The mathematical expression of this value is as follows:

[0129]

[0130] Use an optimization algorithm to adjust the parameter φ and minimize the cross-entropy loss function:

[0131]

[0132] Repeat the forward propagation and parameter updates until the model converges.

[0133] In an optional implementation, adjusting the search path includes the following steps:

[0134] Analysis of model output: Determine the validity of the current source tracing path based on the output y of the QNN;

[0135] Adjust search parameters: Based on feedback, dynamically adjust the parameters of the quantum search algorithm, such as the number of iterations t and the amplification factor;

[0136] Optimize the search path: Re-execute the quantum search and select the optimal path.

[0137] It should be noted that the federated learning mechanism improves the generalization ability of the model and the accuracy of the source tracing results while protecting the privacy of each data source, and avoids the security and privacy risks brought about by data centralization.

[0138] Preferably, the optimized quantum state representation is input into the quantum neural network, and the source tracing result is output through forward propagation calculation, realizing deep learning and pattern recognition of quantum state data. The application of quantum neural network makes full use of the nonlinear and parallel processing capabilities of quantum computing, improving the accuracy and reliability of source tracing analysis.

[0139] S5. Input the optimized quantum state representation into the quantum neural network, and calculate the source tracing result through forward propagation. It should also be noted that:

[0140] If the output value y of the tracing result is greater than or equal to the set threshold τ, then the tracing is successful, the credibility is high, and a tracing report is generated, recording the source information and the propagation path;

[0141] If the output value y of the tracing result is less than the set threshold τ, the tracing fails, the credibility is low, and the system prompts that more data or model parameters need to be adjusted.

[0142] As an example, the source tracing result output value y∈[0,1] represents the credibility of the source tracing result. The larger the value, the higher the credibility. The threshold τ∈[0,1] is set.

[0143] As an example, the content generated by the source tracing report includes:

[0144] Source information: The original publisher or source node traced back to;

[0145] Propagation path: The path and key nodes through which information propagates in a network;

[0146] Credibility: Output result y and threshold τ, as well as the credibility percentage;

[0147] Analysis and Recommendations: Professional analysis and suggestions based on the results;

[0148] Report format: Presented in a graphic and textual format, including flowcharts, statistical charts and text descriptions.

[0149] As an example, the output methods of the source tracing report include:

[0150] Real-time display: The system interface displays the tracing results and analysis reports.

[0151] Export files: Supports exporting to PDF and Word formats;

[0152] Notification mechanism: Relevant personnel will be notified via email and SMS.

[0153] It is easy to understand that this embodiment selects a threshold τ that balances the false alarm rate and the false negative rate as the credibility evaluation standard for the source tracing results by statistically analyzing a large amount of historical data. For example, if τ = 0.85 is set, the source tracing is considered successful when the credibility exceeds 85%.

[0154] The source tracing result is determined based on the output result y and the threshold τ, including the following steps:

[0155] (1) Credibility calculation:

[0156] Multiply the output y by 100% to get the confidence percentage:

[0157] Credibility = y × 100%

[0158] (2) Decision logic:

[0159] Successful tracing: condition is y≥τ;

[0160] Operation: Output a traceability report, providing detailed information;

[0161] Source tracing failed: condition y < τ;

[0162] Action taken: The system indicates insufficient reliability. It is recommended to obtain more data or adjust the model parameters.

[0163] As an example, if the output result y = 0.88, the threshold τ = 0.85, and the confidence level is 0.88 × 100% = 88%, then the judgment is: because 0.88 ≥ 0.85, the tracing is successful, and the confidence level is 88%; action: generate and output the tracing report.

[0164] Reference Figure 2 In this embodiment of the invention, by plotting an ROC curve and selecting a threshold τ at the inflection point of the curve, the true positive rate and the false positive rate are balanced. Figure 2 As shown in the figure, the horizontal axis (X-axis) represents FPR (false positive rate), with a value range of 0 to 1 and no unit; the vertical axis (Y-axis) represents TPR (true positive rate), with a value range of 0 to 1 and no unit. The implementation steps include:

[0165] (1) Determine the range of values ​​for the model output y, which is from 0 to 1;

[0166] (2) Select a series of thresholds τ, for example from 0 to 1, with a step size of 0.1;

[0167] (3) For each threshold τ, classify according to the following rules:

[0168] If y≥τ, the prediction is positive (source tracing successful);

[0169] If y < τ, the prediction is negative (source tracing failed);

[0170] (4) Calculate the corresponding TPR and FPR based on the actual real labels and the prediction results of the model;

[0171] (6) Plot the TPR and FPR at each threshold on the coordinate system, connect the points to form the ROC curve.

[0172] In optional implementations, based on business needs, such as high-security scenarios (financial fraud tracing) requiring high credibility, τ ≥ 0.9 can be set; while general scenarios (market research) requiring moderate credibility can be set τ ≥ 0.85 (on the test dataset, it was found that the model's F1-score reached its highest when the threshold was 0.85); thus improving coverage.

[0173] It should be noted that in practical applications, users can automatically adjust the threshold based on real-time data characteristics and model performance, and users can also manually set the threshold according to their own needs. This embodiment of the invention does not impose a unique limitation on this.

[0174] Preferably, the embodiments of the present invention determine the traceability results by setting a threshold, thereby realizing an automated decision-making and feedback mechanism for the traceability process. This enables the system to automatically take corresponding measures based on the credibility of the results, thereby improving the system's intelligence level and user experience.

[0175] To verify the effectiveness of the target information tracing method for Internet big data described in this invention, multiple Internet data sources, such as social media platforms, news websites, and forums, were selected to simulate the tracing process of an actual online public opinion event.

[0176] This experiment aims to verify the effectiveness and advantages of the proposed method for tracing the source of target information for Internet big data. By capturing, preprocessing, mapping quantum states, constructing quantum data index structures, applying federated learning mechanisms, and training quantum neural networks for multi-source heterogeneous data, the performance of this method in terms of tracing accuracy, efficiency, and reliability is evaluated.

[0177] Experimental environment conditions:

[0178] Hardware environment:

[0179] Distributed server cluster: Consists of 10 node servers, each configured as follows:

[0180] CPU: Intel Xeon E5-2680 v4;

[0181] Memory: 128GB DDR4;

[0182] Storage: 2TB SSD;

[0183] Network: 10 Gigabit Ethernet;

[0184] Quantum computing simulator: IBM Qiskit Aer Simulator, supporting simulation of 32 qubits.

[0185] Software environment:

[0186] Operating system: Ubuntu 20.04LTS;

[0187] Programming language: Python 3.8;

[0188] Data scraping tool: Scrapy 2.5.0;

[0189] Task scheduler: Apache Kafka 2.7.0;

[0190] Data processing platform: Hadoop HDFS 3.2.1;

[0191] Data cleaning and processing libraries: pandas, NumPy;

[0192] Natural Language Processing Tools: Jieba 0.42.1, NLTK 3.5;

[0193] Quantum computing framework: IBM Qiskit 0.23.6;

[0194] Machine learning libraries: TensorFlow 2.4.1, TensorFlow Quantum 0.5.0;

[0195] Federated learning framework: TensorFlow Federated 0.19.0.

[0196] Data source:

[0197] Social media APIs: Weibo API, Twitter API;

[0198] News websites: Xinhua News Agency, BBC News, CNN;

[0199] Forums and blogs: Zhihu, CSDN, Medium.

[0200] Experimental procedure:

[0201] Based on the tracing objectives, a keyword list was set, including "new product launch," "market response," and "user reviews." A related URL list and crawling frequency parameters were also set to ensure the real-time nature and comprehensiveness of the data. The task scheduler Apache Kafka was used to evenly distribute crawling tasks to various crawler nodes to achieve distributed data crawling. Each crawler node adopted the Scrapy framework and was configured with a proxy IP pool and CAPTCHA recognition module to deal with the anti-crawling mechanisms of various websites and ensure the integrity and legality of the data.

[0202] During the crawling process, each crawler node uploads data to the distributed data processing platform Hadoop HDFS in real time. To ensure data quality, the collected data is preprocessed. The Jieba word segmentation tool is used to segment the text data and perform part-of-speech tagging to extract keywords such as nouns and verbs. The TF-IDF algorithm is used to calculate the keyword weights in the text to obtain word frequency features. At the same time, a sentiment analysis model (such as SnowNLP) is used to calculate the sentiment score of the text. The LDA topic model is used to obtain the probability distribution of the text on each topic. All features are merged to form a feature vector.

[0203] The feature vectors are normalized and mapped to quantum states. A parameterized quantum circuit is used to evolve the initial quantum state into an encoded state through multiple rotating gates and controlled NOT gates. Based on the quantum variational algorithm (VQA), a loss function is defined, and the Adam optimization algorithm is employed to iteratively update the circuit parameters θ until the loss function converges. Using the optimized quantum states, a quantum KD tree and a quantum hash table are constructed, organizing the quantum states according to an index structure and storing them in a quantum memory. The Grover search algorithm and an adaptive quantum parallel optimization algorithm are combined to dynamically adjust the path based on the search results, improving search efficiency.

[0204] In this experiment, a federated learning mechanism is added to the quantum data index tree. Three participants are involved, each with its own data and computational processing center. A secure federated learning protocol and homomorphic encryption are used. Each node processes its own data according to the steps described above, generates a local quantum state, and independently runs VQA to obtain the locally optimal parameter θ. k Each node will encrypt θ k The data is uploaded to a central server, which aggregates the parameters and distributes them to each node. This process is repeated to gradually optimize the model.

[0205] Finally, the optimized quantum state is input into the quantum neural network (QNN), the output quantum state is measured to obtain the probability distribution, the output value y is calculated, the cross-entropy loss function is used and the gradient descent method is employed to optimize the QNN parameters, and forward propagation and parameter updates are repeated until the model converges.

[0206] Based on the traceability result output value y, a threshold τ = 0.85 is set. If y ≥ τ, the traceability is considered successful, and a traceability report is generated; if y < τ, the traceability is considered unsuccessful, and the system prompts that more data is needed or model parameters need to be adjusted. The statistical results of the test data are shown in the table below:

[0207] Table 1. Statistical Table of Experimental Data Results

[0208]

[0209] Referring to Table 1, it can be seen that the method of the present invention can effectively extract features and make accurate source determination when processing different samples. First, the construction of the feature vector covers a variety of information such as TF-IDF feature values, sentiment scores and topic distribution, which fully reflects the content and sentiment characteristics of the text. The normalized feature vector of all samples has a length of 1, which meets the normalization requirements of quantum state and lays the foundation for subsequent quantum computing.

[0210] Regarding the output value y of the source tracing result, the y values ​​of samples 1, 2, 4, and 7 are 0.90, 0.88, 0.93, and 0.91, respectively, all of which are higher than the set threshold τ = 0.85. Therefore, they are judged to be successfully traced. For these samples, a source tracing report is generated, which records the source information and propagation path. This shows that the method of the present invention can accurately trace the target information with high reliability.

[0211] The y values ​​for samples 3, 5, and 6 are 0.75, 0.80, and 0.65, respectively, which are below the threshold τ, and are therefore judged as source tracing failures. The system prompts that more data or model parameters need to be adjusted. This indicates that when the sentiment score is low or negative (such as the sentiment scores of samples 3 and 6 being -0.20 and -0.50, respectively), the credibility of the model's source tracing will decrease, and further optimization is needed.

[0212] Compared with existing technologies, the method of this invention significantly improves the accuracy and efficiency of source tracing by introducing quantum computing and federated learning mechanisms. The application of quantum state mapping and parameterized quantum circuits enables high-dimensional feature data to be processed efficiently in a quantum computing environment. The combination of quantum variational algorithms and quantum neural networks enhances the model's learning and generalization capabilities.

[0213] Furthermore, the introduction of the federated learning mechanism ensures collaborative training from multiple data sources while protecting the data privacy of all participants, which is difficult to achieve in traditional centralized models. Experimental data shows that the optimized global model parameters can improve the overall source tracing effect.

[0214] The aforementioned methods for preprocessing the crawled data and for calculating the TF-IDF value, sentiment analysis, and LDA model of the text data can be carried out using existing technologies and methods, and will not be elaborated further in this example.

[0215] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for tracing the source of target information in Internet big data, characterized in that, include: S1. Use distributed data crawling tools to crawl data from multiple Internet data sources, collect the data to the data processing platform, and perform deduplication, cleaning and formatting on the crawled data to generate feature vectors about the data; S2. Normalize and map the feature vectors to quantum states, and encode them into quantum state representations using parameterized quantum circuits; S3. Construct a quantum data index structure based on the quantum variational algorithm to generate a quantum data index tree; The constructed quantum data index structure includes a quantum KD tree, a quantum hash table, and an index; S4. Add a federated learning mechanism to the quantum data index tree to collaboratively train the quantum state representations from different data sources, update the global model parameters, and obtain the optimized quantum state representations. The process of building the federated learning mechanism includes: Determine participating nodes: Define K participants, each with its own data and computing processing center; Communication protocol: Employs a secure federated learning protocol, using homomorphic encryption or secure multi-party computation techniques; The process of performing collaborative training and updating global model parameters includes: Each node processes its own data according to steps S1 to S3 to generate a local quantum state; Run the variable quantum algorithm independently to obtain the local optimal parameters; Each node uploads its encrypted local optimal parameters to the central server; The server aggregates the parameters as follows: Will Distribute to each node to update the local model; Repeat the above process to gradually optimize the model; A quantum neural network is constructed using parameterized quantum circuits, which includes an input layer for receiving quantum states, a hidden layer consisting of a series of parameterized quantum gates, and an output layer for measuring specific qubits to output the results. The parameterized quantum gates include rotation gates and controlled gates; The local quantum state is input into the quantum neural network circuit, and the optimized quantum state representation is output. The optimized quantum state representation is measured to obtain the probability distribution P(y), and the source tracing result output value y is calculated. Its mathematical expression formula is as follows: Adjust parameters using optimization algorithms. Minimize the cross-entropy loss function: Repeat the forward propagation and parameter updates until the model converges; S5. Input the optimized quantum state representation into the quantum neural network, and output the source tracing result through forward propagation: If the output value of the source tracing result is greater than or equal to the set threshold, the source tracing is successful, the credibility is high, a source tracing report is generated, and the source information and propagation path are recorded. If the output value of the tracing result is less than the set threshold, the tracing fails, the credibility is low, and the system prompts that more data or model parameters need to be adjusted.

2. The method for tracing target information in Internet big data according to claim 1, characterized in that, The captured data is collected and sent to a data processing platform, including: Based on the source tracing target, set the keywords, URL list, and crawling frequency parameters; The task scheduler distributes the crawling tasks to each crawler node, and the nodes do not interfere with each other. Each node retrieves data according to the configured retrieval task; The captured data is uploaded to the distributed data processing platform in real time.

3. The method for tracing target information in Internet big data according to claim 1, characterized in that, Generating a feature vector about the data includes: Remove duplicate data by calculating the unique identifier of the data; Remove HTML tags, special characters, and advertising content to filter noise. Delete or populate missing data fields; Convert text data to UTF-8 encoding; Convert the timestamp to a uniform format; Use Jieba word segmentation to process Chinese text into words; The text is then tagged with parts of speech to extract the parts of speech of nouns and verbs, generating a text feature representation of the data, namely the feature vector.

4. The method for tracing target information in Internet big data according to claim 3, characterized in that, The unique identifier includes a list of URLs and a message ID.

5. The method for tracing target information in Internet big data according to claim 3, characterized in that, Convert the timestamp to the ISO 8601 unified format.

6. The method for tracing target information in Internet big data according to claim 1, characterized in that, Obtaining the quantum state representation includes: Using quantum amplitude encoding technology, classical data is loaded into a quantum register through quantum circuits; The initial quantum state is transformed into the desired quantum state representation by parameterizing the quantum circuit.

7. The method for tracing target information in Internet big data according to claim 1, characterized in that, The construction of the quantum data index structure includes: Quantum KD-tree: Nearest neighbor search for high-dimensional data; Quantum hash tables: used for fast retrieval; Index building: The optimized quantum states are organized according to an index structure and stored in the quantum memory; Search path optimization: Improve search efficiency by combining the Grover search algorithm; An adaptive quantum parallel optimization algorithm is used to dynamically adjust the path based on the search results.

Images