Internet mass data accurate search method and system based on AI technology

Abstract

The invention discloses an internet mass data accurate search method and system based on an AI technology, and relates to the technical field of internet, the method comprises the following steps: receiving a query text input by a user, using a natural language processing technology to identify the input query text, and generating a structured query vector; based on the structured query vector, constructing a knowledge graph to obtain a rich semantic context graph; a knowledge transfer learning technology is introduced, historical search data and behavior characteristics of a user are analyzed, and a personalized search model is constructed; monitoring the real-time interaction behavior of the user, judging the cognitive load of the user, and generating an adaptive search interface; initiating a plurality of query requests by using the personalized search model, and outputting a personalized search result set through filtering and sorting; according to the method, the knowledge graph is constructed and the rich semantic context graph is generated, so that the system can identify entities and multiple relationships thereof, more relevant search results are provided, and a user is promoted to discover potential requirements.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to an AI-based method and system for accurately searching massive amounts of Internet data. Background Art

[0002] With the rapid development of information technology, the Internet has become a huge data resource library. The generation and storage of massive data have put forward higher requirements for information retrieval technology. Traditional search engines mainly rely on keyword matching and static algorithms to process user queries. This approach often seems to be inadequate when faced with complex queries and diverse information. In recent years, the rise of emerging technologies such as natural language processing (NLP), machine learning, and knowledge graphs has provided new ideas for precise search.

[0003] Although existing technologies have made certain progress in the field of information retrieval, there are still many shortcomings. First, traditional search engines have limited understanding of query semantics, especially when dealing with fuzzy or long-tail queries, and often find it difficult to provide highly relevant results. This results in users often having to make multiple queries in the process of obtaining information, which wastes time and energy. Secondly, existing personalized search technologies mostly rely on static models and lack the ability to monitor and dynamically adjust users' real-time behavior, making it difficult to adapt to changes in user needs. In addition, users' cognitive load is often ignored in the process of information retrieval, causing users to feel confused when faced with massive amounts of information, further affecting search efficiency and satisfaction. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an AI-based method and system for accurate search of massive Internet data to solve the problems of insufficient query understanding, lack of personalized recommendations and excessive cognitive load faced by users when obtaining accurate information from massive data.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for accurate search of Internet massive data based on AI technology, which includes receiving a query text input by a user, and using natural language processing technology to recognize the input query text to generate a structured query vector;

[0008] Based on the structured query vector, a knowledge graph is constructed to obtain a rich semantic context graph;

[0009] Introducing knowledge transfer learning technology to analyze users’ historical search data and behavior characteristics and build a personalized search model;

[0010] Monitor users’ real-time interactive behaviors, determine users’ cognitive load, and generate adaptive search interfaces;

[0011] Use the personalized search model to initiate multiple query requests, and after filtering and sorting, output a personalized search result set.

[0012] As a preferred solution of the method for accurate search of Internet massive data based on AI technology described in the present invention, wherein: receiving a query text input by a user, and using natural language processing technology to recognize the input query text, and generating a structured query vector includes the following steps:

[0013] The user enters query text in the search interface;

[0014] Use dependency parsing tools to perform basic grammatical analysis on the input text;

[0015] Build a prefix dictionary of common words and perform word segmentation on the analyzed text;

[0016] Perform part-of-speech tagging on the text after word segmentation;

[0017] After completing part-of-speech tagging, perform named entity recognition on the text;

[0018] Combine the results of word segmentation, part-of-speech tagging, and named entity recognition to perform keyword extraction and topic identification;

[0019] The extracted keywords, user intent and their related attributes are combined into a structured query vector and stored in the knowledge base.

[0020] As a preferred solution of the method for accurate search of Internet massive data based on AI technology described in the present invention, the basic grammatical analysis refers to parsing the input text using a dependency syntactic analysis tool, identifying each word in the sentence and its corresponding part of speech, automatically analyzing the relationship between words, and generating a dependency graph; the word segmentation processing refers to using a maximum word length word segmentation algorithm based on a prefix dictionary to cut the input string into independent words to obtain a word segmentation result;

[0021] The specific steps of part-of-speech tagging are as follows:

[0022] Select conditional random fields as the model for part-of-speech tagging;

[0023] Extract current word features, context features and part-of-speech features from the word segmentation results;

[0024] Pass each word segment and its features to the conditional random field model to calculate the part-of-speech probability of each word. The expression is:

[0025]

[0026] Among them, P(y i |w i ) means that in a given word w i In the case of i The probability of w i represents the i-th participle, y i Represents the i-th participle w i The associated part-of-speech tag, i represents the word index, θ represents the parameter vector of the model, f represents the set of features, T represents the transposition operation, Y represents the set of all possible part-of-speech tags, and y' represents the tag in the part-of-speech tag set Y;

[0027] For each word segment, select the one with the highest probability from all possible part-of-speech tags as the part-of-speech tag of the word segment;

[0028] The process of named entity recognition is as follows:

[0029] Select a model based on a combination of bidirectional long short-term memory network and conditional random field for named entity recognition;

[0030] Input the part-of-speech tagged words into the model for training;

[0031] Pass the word segmentation results and part-of-speech tags to the trained model, perform named entity recognition on each word, and output the corresponding entity category;

[0032] The specific steps of keyword extraction and topic identification are as follows:

[0033] Using the TF-IDF algorithm, the TF-IDF value of each word is calculated. The expression is:

[0034] TF-IDF(t,d)=TF(t,d)×IDF(t);

[0035] Among them, TF-IDF(t,d) represents the TF-IDF value of word t in document d, t represents a specific word, d represents a specific document, TF(t,d) represents the word frequency of word t in document d, and IDF(t) represents the inverse document frequency of word t;

[0036] Sort all words by TF-IDF value and select the top N words with the highest TF-IDF value as keywords;

[0037] By analyzing the relationships and context between key words, the main themes of the text are identified.

[0038] As a preferred solution of the method for accurate search of Internet massive data based on AI technology described in the present invention, wherein: based on the structured query vector, constructing a knowledge graph to obtain a rich semantic context graph includes the following steps:

[0039] Connect to the selected knowledge base via the database connection tool;

[0040] Extract relevant metadata based on the user's query conditions;

[0041] Based on the extracted keywords and topics, keyword matching is performed in the knowledge base to identify multiple entities related to the user query;

[0042] Retrieve relationships between the identified entities based on the related entities;

[0043] Classify the acquired relations and extract relevant attributes to determine the interactions between entities;

[0044] A node is created for each identified entity and the relationship between the identified entities is created;

[0045] Build the identified entities and their relationships into a knowledge graph;

[0046] For each node in the knowledge graph, initialize its feature vector;

[0047] The message passing mechanism of graph neural network is introduced, and nodes pass their feature vectors to adjacent nodes;

[0048] Set the number of inference layers K, and gradually update the feature vector of each node by combining the feature vectors of its neighboring nodes through multi-layer graph neural network inference to obtain the final feature vector of each node;

[0049] The final feature vector of each node is analyzed to identify other topics and concepts similar to the user's query intent and build a rich semantic context graph.

[0050] As a preferred solution of the method for accurate search of Internet massive data based on AI technology described in the present invention, the following steps are included: introducing knowledge transfer learning technology, analyzing the user's historical search data and behavior characteristics, and building a personalized search model:

[0051] Extract feature vectors of each node and edge from the semantic context graph;

[0052] Integrate the extracted node feature vectors and edge feature vectors into an input matrix;

[0053] Collect users’ historical search data and behavior characteristics and perform feature extraction to form a user feature matrix;

[0054] The input feature matrix and the user feature matrix are concatenated together to form a comprehensive feature matrix;

[0055] Construct an adjacency matrix to represent the connection relationship between nodes in the graph;

[0056] Select a transfer learning model suitable for processing graph structure data, input the comprehensive feature matrix and adjacency matrix into the transfer learning model to fine-tune the model, and build a personalized search model.

[0057] As a preferred solution of the method for accurate search of Internet massive data based on AI technology described in the present invention, wherein: monitoring the real-time interactive behavior of users, judging the cognitive load of users, and generating an adaptive search interface include the following steps:

[0058] Collect user interaction behavior data on personalized search results pages in real time;

[0059] Evaluate the user's cognitive load based on the user's interactive behavior data;

[0060] Develop dynamic interface adjustment strategies based on the user's cognitive load assessment results;

[0061] According to different cognitive load levels, the layout of the interface is adjusted to generate an adaptive search interface.

[0062] As a preferred solution of the method for accurate search of Internet massive data based on AI technology described in the present invention, the personalized search model is used to initiate multiple query requests, and after filtering and sorting, a personalized search result set is output, which includes the following steps:

[0063] Use personalized search models to analyze user query intent and generate multiple query requests;

[0064] Send the generated query request to the knowledge base and related databases to capture information related to the query request;

[0065] Based on the user's personalized characteristics and cognitive load assessment results, the crawled results are initially filtered to remove irrelevant information;

[0066] Sort the filtered results, evaluate the diversity of the results, and generate the final search result set.

[0067] In a second aspect, the present invention provides an AI-based Internet massive data precision search system, including a query input module, which receives a query text input by a user, and uses natural language processing technology to recognize the input query text and generate a structured query vector;

[0068] A knowledge graph construction module constructs a knowledge graph based on the structured query vector to obtain a rich semantic context graph;

[0069] The knowledge transfer learning module introduces knowledge transfer learning technology to analyze users' historical search data and behavior characteristics and build a personalized search model;

[0070] The cognitive load assessment module monitors the user's real-time interactive behavior, determines the user's cognitive load, and generates an adaptive search interface;

[0071] The query request generation module uses the personalized search model to initiate multiple query requests, and outputs a personalized search result set after filtering and sorting.

[0072] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the method for precise search of massive Internet data based on AI technology as described in the first aspect of the present invention.

[0073] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for precise search of massive Internet data based on AI technology as described in the first aspect of the present invention.

[0074] The beneficial effects of the present invention are as follows: by receiving the query text input by the user and generating a structured query vector using natural language processing technology, a deep understanding of the user's query intention is achieved; a knowledge graph is constructed and a rich semantic context graph is generated, so that the system can identify entities and their multiple relationships, thereby providing more relevant search results and promoting users to discover potential needs; knowledge transfer learning technology is introduced, and a personalized search model is constructed by analyzing the user's historical search data, so that accurate recommendations for individual users are achieved, the relevance of the search is improved, and the user's cognitive load is reduced; by real-time monitoring of user interaction behavior, an adaptive search interface is generated, and the user experience is dynamically optimized, making information retrieval more efficient and friendly; a personalized search model is used to initiate multiple query requests, and after filtering and sorting, a relevant and diverse search result set is output to ensure that users can efficiently obtain information. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0076] Figure 1 This is a flow chart of the method for accurately searching massive amounts of Internet data based on AI technology in Example 1.

[0077] Figure 2 This is a flowchart of the construction of the personalized search model in Example 1. DETAILED DESCRIPTION

[0078] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0079] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0080] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0081] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a method for accurately searching massive Internet data based on AI technology, comprising the following steps:

[0082] S1. Receiving a query text input by a user and using natural language processing technology to recognize the input query text, and generating a structured query vector includes the following steps:

[0083] S1.1. The user enters the query text in the search interface; the dependency parsing tool is used to perform basic grammatical analysis on the input text.

[0084] Specifically, basic grammatical analysis refers to using dependency syntactic analysis tools to parse the input text, identify each word in the sentence and its corresponding part of speech, automatically analyze the relationship between words, and generate a dependency graph.

[0085] It should be noted that through dependency syntactic analysis, the grammatical relationship between words in the text can be clearly identified, laying the foundation for subsequent semantic understanding. The text analysis methods mentioned in the prior art mainly rely on basic spelling checking and simple word segmentation, and fail to deeply analyze the sentence structure, which may lead to information loss and understanding deviation. Dependency syntactic analysis can provide a more detailed understanding of the sentence structure, thereby improving the accuracy of subsequent processing.

[0086] S1.2. Build a prefix dictionary of common words and perform word segmentation on the analyzed text.

[0087] Specifically, word segmentation processing refers to using a maximum word length word segmentation algorithm based on a prefix dictionary to segment an input string into independent words to obtain a word segmentation result.

[0088] It should be noted that the word segmentation algorithm based on the prefix dictionary can process complex Chinese texts more effectively and reduce errors in segmenting polysemous words and phrases. This word segmentation method can significantly improve the quality of word segmentation, thereby providing a more accurate basis for subsequent processing. However, the text processing method in the existing technology is not flexible enough and is prone to errors when processing polysemous words and phrases, affecting the overall text understanding.

[0089] S1.3, perform part-of-speech tagging on the text after word segmentation.

[0090] Specifically, the specific steps of part-of-speech tagging are as follows:

[0091] Select conditional random fields as the model for part-of-speech tagging;

[0092] Extract current word features, context features and part-of-speech features from the word segmentation results;

[0093] Pass each word segment and its features to the conditional random field model to calculate the part-of-speech probability of each word. The expression is:

[0094]

[0095] Among them, P(y i |w i ) means that in a given word w i In the case of i The probability of w i represents the i-th participle, y i Represents the i-th participle w i The associated part-of-speech tag, i represents the word index, θ represents the parameter vector of the model, f represents the set of features, T represents the transposition operation, Y represents the set of all possible part-of-speech tags, and y' represents the tag in the part-of-speech tag set Y;

[0096] For each word segmentation, select the one with the highest probability from all possible part-of-speech tags as the part-of-speech tag of the word segmentation.

[0097] It should be noted that the conditional random field model can combine contextual information to provide more accurate part-of-speech tagging and improve the overall accuracy of text analysis. However, the prior art does not elaborate on the part-of-speech tagging process, which may lead to inaccurate grammatical analysis and affect the effectiveness of information extraction.

[0098] S1.4. After completing part-of-speech tagging, perform named entity recognition on the text.

[0099] Specifically, a model based on a combination of bidirectional long short-term memory network and conditional random field is selected for named entity recognition. The bidirectional long short-term memory network can effectively capture contextual information. The reverse and forward information flow makes the model's understanding of the context more comprehensive, while the conditional random field is used to optimize the label sequence to ensure the consistency and accuracy of the recognition results. The specific steps are as follows:

[0100] Extract word features, part-of-speech features, and context features from the results of part-of-speech tagging; use the labeled data set in the training phase to train the model based on the combination of bidirectional long short-term memory network and conditional random field to learn how to identify various named entities. During the training process, the model continuously adjusts the weights through the back-propagation algorithm to improve recognition accuracy. In the inference phase, the input text data (including word segmentation results and part-of-speech tags) is passed to the trained model. For each word in the text, the model outputs a label indicating whether the word is a named entity and its category (such as a person's name, place name, organization name, etc.). After the entity recognition is completed, the identified entities and their categories are summarized.

[0101] Furthermore, word features refer to the text content of the current word, and part-of-speech features refer to the part-of-speech tag of each word, which helps the model understand the grammatical role of the word, and the lexical information within a certain range before and after the current word to capture the impact of context on entity recognition. For example, the word before and after the current word may affect whether the word is an entity.

[0102] It should be noted that combining the bidirectional long short-term memory network and the conditional random field model for named entity recognition can more comprehensively understand the context, thereby improving the accuracy of entity recognition, ensuring the consistency of the recognition results, and reducing inconsistencies in recognition. The lack of contextual understanding in the existing technology leads to a decrease in the accuracy of entity recognition and affects the quality of information retrieval.

[0103] S1.5. Combine the results of word segmentation, part-of-speech tagging and named entity recognition to perform keyword extraction and topic identification; combine the extracted keywords, user intent and their related attributes into a structured query vector and store it in the knowledge base.

[0104] Specifically, the specific steps of keyword extraction and topic identification are as follows:

[0105] The TF-IDF (term frequency - inverse document frequency) algorithm is used to calculate the TF-IDF value of each word. The expression is:

[0106] TF-IDF(t,d)=TF(t,d)×IDF(t);

[0107] Among them, TF-IDF(t,d) represents the TF-IDF value of word t in document d, t represents a specific word, d represents a specific document, TF(t,d) represents the word frequency of word t in document d, and IDF(t) represents the inverse document frequency of word t;

[0108] Sort all words by TF-IDF value and select the top N words with the highest TF-IDF value as keywords; identify the main topic of the text by analyzing the relationship and context between keywords.

[0109] It should be noted that the use of the TF-IDF algorithm for keyword extraction can quantify the importance of each word, thereby effectively identifying the text topic and providing a structured query vector, while the keyword extraction process of the existing technology is too simple and fails to fully utilize the important information in the text.

[0110] S2. Based on the structured query vector, construct a knowledge graph to obtain a rich semantic context graph, including the following steps:

[0111] Connect to the selected knowledge base through the database connection tool; extract relevant metadata according to the user's query conditions; perform keyword matching in the knowledge base based on the extracted keywords and topics, and identify multiple entities related to the user's query; retrieve the relationships between these entities based on the identified multiple related entities; classify the acquired relationships and extract relevant attributes to determine the interactions between entities; create a node for each identified entity, and create edges between the identified entities; construct the identified entities and their relationships into a knowledge graph; for each node in the knowledge graph, initialize its feature vector; introduce the message passing mechanism of the graph neural network, and the node passes its feature vector to the adjacent nodes; set the number of reasoning layers K, and through multi-layer graph neural network reasoning, gradually update the feature vector of each node in combination with the feature vectors of its neighboring nodes to obtain the final feature vector of each node; analyze the final feature vector of each node, identify other topics and concepts similar to the user's query intention, and construct a rich semantic context graph.

[0112] Specifically, metadata includes basic information of an entity (such as name, type, description, etc.) and related attributes and relationships.

[0113] Specifically, in the case where the same keyword may correspond to multiple entities, disambiguation processing is required.

[0114] Furthermore, disambiguation processing means that in the knowledge base, the same keyword may correspond to multiple entities. The system needs to identify these potential ambiguities to ensure the accuracy of subsequent operations. The system will analyze the attributes and related information of the identified entities. By comparing these attributes, the system can determine the entity that is most relevant to the query intent. In addition to attribute analysis, the system will also consider contextual information. Based on the complete sentence entered by the user or the query background, the system can more accurately determine the most appropriate entity. After attribute and context analysis, the system will select the most relevant entity as the final result.

[0115] It should be noted that keyword matching is performed based on the extracted keywords and topics to identify multiple entities related to the user query. Through dynamic matching, the system can quickly identify the entities most relevant to the user's intention and improve the accuracy of information retrieval, while the existing technology mainly focuses on static extraction; through relationship classification and attribute extraction, the interaction between entities is clarified, providing a deep semantic background for the construction of knowledge graphs, while the existing technology does not design in-depth analysis of entity relationships and lacks understanding of information relevance; a node is created for each identified entity, and edges are created based on the relationships between entities to form a complete knowledge graph, which can intuitively display entities and their relationships and help users Users can better understand the relationship between information, but the construction of knowledge graphs is not mentioned in the prior art, and it mainly relies on simple information extraction; through the reasoning ability of graph neural networks, it can achieve a deeper semantic understanding and identify other topics and concepts similar to user query intentions, but the application of graph neural networks is not mentioned in the prior art, and there is a lack of in-depth analysis of complex relationships; for the situation where the same keyword may correspond to multiple entities, disambiguation processing is performed, and the most relevant entity is determined through attribute and context analysis to ensure the accuracy of the final result, improve user experience, and avoid confusion caused by ambiguity, but the disambiguation processing mechanism is not mentioned in the prior art, which leads to ambiguity in information retrieval.

[0116] S3, introduce knowledge transfer learning technology, analyze the user's historical search data and behavior characteristics, and build a personalized search model including the following steps:

[0117] S3.1. Extract the feature vector of each node and the feature vector of each edge from the semantic context graph.

[0118] Specifically, the features of each node include the node's name, type, description, relevance score and other information. These feature vectors are generated using word embedding or graph embedding technology to ensure that they can effectively capture the semantic information of the nodes; the feature vectors of the edges are extracted to reflect the relationship between the nodes. The edge features include relationship type, weight (indicating the strength of the relationship) and contextual information.

[0119] S3.2. Integrate the extracted node feature vectors and edge feature vectors into an input matrix.

[0120] Specifically, the structure of the input matrix is ​​that the rows represent each node or edge and the columns represent the extracted features.

[0121] S3.3. Collect the user's historical search data and behavioral characteristics and perform feature extraction to form a user feature matrix; concatenate the input feature matrix and the user feature matrix together to form a comprehensive feature matrix.

[0122] Specifically, the user's historical search data includes query terms, click results, dwell time, feedback information, etc. These data help understand the user's preferences and needs.

[0123] Extract features from user behavior, such as:

[0124] Query frequency: How often users search for a particular topic.

[0125] Click-through rate: The rate at which users click on a particular search result.

[0126] Dwell time: How long a user stays on a specific results page.

[0127] User feedback: User satisfaction rating for search results (such as likes or dislikes).

[0128] S3.4. Construct an adjacency matrix to represent the connection relationship between nodes in the graph; select a transfer learning model suitable for processing graph structure data, input the comprehensive feature matrix and adjacency matrix into the transfer learning model to fine-tune the model, and construct a personalized search model.

[0129] Specifically, create an adjacency matrix A to represent the connection relationship between nodes in the graph. The element A of the adjacency matrix mn It can represent the strength of the relationship between node m and node n. If two nodes are connected, then A mn =1, otherwise 0. Through the adjacency matrix, the model can understand the structural information between nodes.

[0130] It should be noted that an adjacency matrix is ​​created to represent the connection relationship between nodes in the graph, and the comprehensive feature matrix and the adjacency matrix are input into the transfer learning model for fine-tuning, which helps the model understand the structural information between nodes and improves the model's ability to handle complex relationships. At the same time, the fine-tuned model can better adapt to the user's personalized needs. However, the prior art does not mention the construction process of the adjacency matrix, resulting in insufficient understanding of node relationships.

[0131] S4, monitoring the user's real-time interactive behavior, determining the user's cognitive load, and generating an adaptive search interface include the following steps:

[0132] Collect user interaction behavior data on personalized search results pages in real time; evaluate user cognitive load based on user interaction behavior data; formulate dynamic interface adjustment strategies based on user cognitive load evaluation results; adjust interface layout according to different cognitive load levels to generate an adaptive search interface.

[0133] Further, specific strategies include:

[0134] Simplify options: When cognitive load is too high, the system will hide or merge some unimportant options to reduce the amount of information users need to process.

[0135] Highlight key information: When cognitive load is moderate, the system will help users quickly obtain key information by highlighting or amplifying important information.

[0136] Provide recommendations: When cognitive load is low, the system can provide relevant recommended options to encourage users to explore further.

[0137] Going a step further, adjust the layout of the interface for different cognitive load levels. For example:

[0138] High load: Use a simple card-style layout to highlight key information and avoid visual clutter.

[0139] Medium load: Use groups and tags to organize information and ensure users can find what they need quickly.

[0140] Low load: Show more information and options to encourage users to explore further.

[0141] It should be noted that by using the collected interactive behavior data and evaluating the user's cognitive load level through analysis algorithms, it is possible to understand the user's psychological state at a specific moment and make corresponding adjustments. However, the existing technology lacks dynamic evaluation of user cognitive load and fails to consider the user's psychological changes during use; according to the cognitive load evaluation results, corresponding interface adjustment strategies are formulated to meet user needs, and adjustment strategies can be formulated in advance for different cognitive load levels to improve the interface's response speed and user experience. However, the existing technology does not mention the formulation of dynamic interface adjustment strategies, which may lead to rigid interface design; according to different cognitive load levels, the interface layout is adjusted to generate an adaptive search interface. Through reasonable interface layout adjustment, the system can effectively improve the user's ability to process information, reduce cognitive load, and improve search efficiency. However, the existing technology does not consider the impact of cognitive load on interface design, which leads to difficulties for users in information processing.

[0142] S5. Initiating multiple query requests using the personalized search model, and outputting personalized search result sets after filtering and sorting, including the following steps:

[0143] S5.1. Use the personalized search model to analyze the user's query intention and generate multiple query requests.

[0144] Specific operations include:

[0145] Intent recognition: The model generates relevant keywords and phrases based on the user’s current query and historical behavior.

[0146] Query expansion: Expand the identified intent to generate multiple variant queries. For example, if a user queries artificial intelligence, related queries such as AI applications, machine learning technology, and deep learning can be generated.

[0147] It should be noted that the model generates relevant keywords and phrases based on the user's current query and historical behavior, accurately identifies user intent, makes subsequent queries more targeted, enhances the relevance of search results, expands the identified intent, and generates multiple variant queries, which can cover a wider search range, thereby improving the comprehensiveness of information retrieval. The existing technology only relies on a single query input by the user and lacks in-depth analysis and expansion of user intent.

[0148] S5.2. Send the generated query request to the knowledge base and related databases to capture information related to the query request; perform preliminary filtering on the captured results based on the user's personalized characteristics and cognitive load assessment results to remove irrelevant information; sort the filtered results, evaluate the diversity of the results, and generate the final search result set.

[0149] Specifically, diversity evaluation refers to evaluating the diversity of the results in the sorted result set to ensure that the different needs of users are met. For example, the following diversity indicators can be used:

[0150] Coverage: Calculates the proportion of results for different topics or types.

[0151] Convergence: Evaluate the similarity between results and ensure diversity among different results.

[0152] It should be noted that sending the generated query request to the knowledge base and related databases to capture information related to the query request can obtain a large number of potential results, providing a rich basis for subsequent filtering and sorting. According to the user's personalized characteristics and cognitive load evaluation results, irrelevant information is removed to ensure the relevance of the search results, avoid users being disturbed by irrelevant information, and improve the effectiveness of the information. The filtered results are sorted, the diversity of the results is evaluated, and the final search result set is generated to ensure that the search results are not only relevant but also meet the different needs of users and enhance the richness of information. However, the prior art does not mention the evaluation of the diversity of results, which may result in the search results being too single and unable to meet the comprehensive needs of users.

[0153] This embodiment also provides an AI-based Internet massive data precision search system, including:

[0154] A query input module receives a query text input by a user, and uses natural language processing technology to recognize the input query text and generate a structured query vector;

[0155] A knowledge graph construction module constructs a knowledge graph based on the structured query vector to obtain a rich semantic context graph;

[0156] The knowledge transfer learning module introduces knowledge transfer learning technology to analyze users' historical search data and behavior characteristics and build a personalized search model;

[0157] The cognitive load assessment module monitors the user's real-time interactive behavior, determines the user's cognitive load, and generates an adaptive search interface;

[0158] The query request generation module uses the personalized search model to initiate multiple query requests, and outputs a personalized search result set after filtering and sorting.

[0159] This embodiment also provides a computer device, which is suitable for the case of a precise search method for Internet massive data based on AI technology, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the precise search method for Internet massive data based on AI technology as proposed in the above embodiment.

[0160] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0161] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for accurately searching massive Internet data based on AI technology as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0162] In summary, the present invention achieves a deep understanding of the user's query intention by: receiving the query text input by the user and generating a structured query vector using natural language processing technology; constructing a knowledge graph and generating a rich semantic context graph, so that the system can identify entities and their multiple relationships, thereby providing more relevant search results and promoting users to discover potential needs; introducing knowledge transfer learning technology, analyzing user historical search data, and building a personalized search model, thereby achieving accurate recommendations for individual users, improving the relevance of the search, and reducing the user's cognitive load; through real-time monitoring of user interaction behavior, generating an adaptive search interface, dynamically optimizing the user experience, and making information retrieval more efficient and friendly; using a personalized search model to initiate multiple query requests, and after filtering and sorting, outputting a relevant and diverse search result set to ensure that users can efficiently obtain information.

[0163] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of a precise search method for Internet massive data based on AI technology is provided.

[0164] First, prepare a dataset containing a variety of user query texts, covering different fields such as technology, health, and tourism. Each text will go through the following processing steps:

[0165] Simulate the query text entered by the user in the search interface, use the dependency syntax analysis tool to analyze the input text, identify the words and their parts of speech in the sentence, generate a dependency graph, build a prefix dictionary and use the maximum word length word segmentation algorithm to segment the analyzed text; select the conditional random field model for part-of-speech tagging, extract the features of each word segmentation, and calculate its part-of-speech probability to determine the part-of-speech tag of each word; use a model that combines a bidirectional long short-term memory network and a conditional random field to perform named entity recognition on the part-of-speech tagged words; use the TF-IDF algorithm to calculate the TF-IDF value of each word, extract keywords, and determine the main theme of the text by analyzing the relationship between keywords.

[0166] The details are shown in Table 1 below:

[0167] Table 1 Comparison data record table in query processing

[0168]

[0169]

[0170] Through the analysis of the above experimental data, it can be clearly seen that the present invention has significant advantages in processing user queries. The specific analysis is as follows:

[0171] For the query of how to improve immunity, the method of the present invention takes only 0.45 seconds, while the prior art takes 1.20 seconds; when processing the best tourist destination recommendation, the processing accuracy of the present invention reaches 92%, while the prior art is only 65%; for the query of healthy diet advice, 5 keywords are extracted, while the prior art only extracts 2 keywords. This shows that the present invention has higher performance in text analysis and information extraction capabilities.

[0172] In terms of named entity recognition, the present invention successfully recognizes more entities and further enhances the understanding of user intent, which is crucial for building accurate search results.

[0173] The accuracy of the present invention in identifying user intentions reaches 90%, while the accuracy of the prior art is 60%, which shows that the present invention can better meet user needs and provide a personalized search experience.

[0174] In summary, the present invention performs well in all aspects of processing user queries, showing its technical innovation and great potential in practical applications. Through further optimization and application, this method is expected to provide a more accurate and efficient solution in the field of Internet data search.

[0175] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for accurate search of Internet massive data based on AI technology, characterized by: include, Receive the query text input by the user, and use natural language processing technology to recognize the input query text to generate a structured query vector; Based on the structured query vector, a knowledge graph is constructed to obtain a rich semantic context graph; Introducing knowledge transfer learning technology to analyze users’ historical search data and behavior characteristics and build a personalized search model; Monitor users’ real-time interactive behaviors, determine users’ cognitive load, and generate adaptive search interfaces; Use the personalized search model to initiate multiple query requests, and after filtering and sorting, output a personalized search result set.

2. The method for accurately searching Internet massive data based on AI technology according to claim 1, characterized in that: Receiving the query text input by the user and using natural language processing technology to recognize the input query text, generating a structured query vector includes the following steps: The user enters query text in the search interface; Use dependency parsing tools to perform basic grammatical analysis on the input text; Build a prefix dictionary of common words and perform word segmentation on the analyzed text; Perform part-of-speech tagging on the text after word segmentation; After completing part-of-speech tagging, perform named entity recognition on the text; Combine the results of word segmentation, part-of-speech tagging, and named entity recognition to perform keyword extraction and topic identification; The extracted keywords, user intent and their related attributes are combined into a structured query vector and stored in the knowledge base.

3. The method for accurately searching Internet massive data based on AI technology as claimed in claim 2, characterized in that: The basic grammatical analysis refers to the use of dependency syntactic analysis tools to parse the input text, identify each word in the sentence and its corresponding part of speech, automatically analyze the relationship between words, and generate a dependency graph; the word segmentation processing refers to the use of a maximum word length word segmentation algorithm based on a prefix dictionary to segment the input string into independent words to obtain a word segmentation result; The specific steps of part-of-speech tagging are as follows: Select conditional random fields as the model for part-of-speech tagging; Extract current word features, context features and part-of-speech features from the word segmentation results; Pass each word segment and its features to the conditional random field model to calculate the part-of-speech probability of each word. The expression is: Among them, P(yi|w i ) means that given a word w i In the case of i The probability of w i represents the i-th participle, y i Represents the i-th participle w i The associated part-of-speech tag, i represents the word index, θ represents the parameter vector of the model, f represents the set of features, T represents the transposition operation, Y represents the set of all possible part-of-speech tags, and y′ represents the tag in the part-of-speech tag set Y; For each word segment, select the one with the highest probability from all possible part-of-speech tags as the part-of-speech tag of the word; The process of named entity recognition is as follows: Select a model based on a combination of bidirectional long short-term memory network and conditional random field for named entity recognition; Input the part-of-speech tagged words into the model for training; Pass the word segmentation results and part-of-speech tags to the trained model, perform named entity recognition on each word, and output the corresponding entity category; The specific steps of keyword extraction and topic identification are as follows: Using the TF-IDF algorithm, the TF-IDF value of each word is calculated. The expression is: TF-IDF(t,d)=TF(t,d)×IDF(t); Among them, TF-IDF(t, d) represents the TF-IDF value of word t in document d, t represents a specific word, d represents a specific document, TF(t, d) represents the word frequency of word t in document d, and IDF(t) represents the inverse document frequency of word t; Sort all words by TF-IDF value and select the top N words with the highest TF-IDF value as keywords; By analyzing the relationships and context between key words, the main themes of the text are identified.

4. The method for accurately searching Internet massive data based on AI technology as claimed in claim 3, characterized in that: Based on the structured query vector, constructing a knowledge graph to obtain a rich semantic context graph includes the following steps: Connect to the selected knowledge base via the database connection tool; Extract relevant metadata based on the user's query conditions; Based on the extracted keywords and topics, keyword matching is performed in the knowledge base to identify multiple entities related to the user query; Retrieve relationships between the identified entities based on the related entities; Classify the acquired relations and extract relevant attributes to determine the interactions between entities; A node is created for each identified entity and the relationship between the identified entities is created; Build the identified entities and their relationships into a knowledge graph; For each node in the knowledge graph, initialize its feature vector; The message passing mechanism of graph neural network is introduced, and nodes pass their feature vectors to adjacent nodes; Set the number of inference layers K, and gradually update the feature vector of each node by combining the feature vectors of its neighboring nodes through multi-layer graph neural network inference to obtain the final feature vector of each node; The final feature vector of each node is analyzed to identify other topics and concepts similar to the user's query intent and build a rich semantic context graph.

5. The method for accurately searching Internet massive data based on AI technology as claimed in claim 4, characterized in that: Introducing knowledge transfer learning technology, analyzing users' historical search data and behavior characteristics, and building a personalized search model includes the following steps: Extract feature vectors of each node and edge from the semantic context graph; Integrate the extracted node feature vectors and edge feature vectors into an input matrix; Collect users’ historical search data and behavior characteristics and perform feature extraction to form a user feature matrix; The input feature matrix and the user feature matrix are concatenated together to form a comprehensive feature matrix; Construct an adjacency matrix to represent the connection relationship between nodes in the graph; Select a transfer learning model suitable for processing graph structure data, input the comprehensive feature matrix and adjacency matrix into the transfer learning model to fine-tune the model, and build a personalized search model.

6. The method for accurately searching Internet massive data based on AI technology according to claim 5, characterized in that: Monitoring the user's real-time interactive behavior, determining the user's cognitive load, and generating an adaptive search interface include the following steps: Collect user interaction behavior data on personalized search results pages in real time; Evaluate the user's cognitive load based on the user's interactive behavior data; Develop dynamic interface adjustment strategies based on the user's cognitive load assessment results; According to different cognitive load levels, the layout of the interface is adjusted to generate an adaptive search interface.

7. The method for accurately searching Internet massive data based on AI technology according to claim 6, characterized in that: Using the personalized search model to initiate multiple query requests, and after filtering and sorting, outputting a personalized search result set includes the following steps: Use personalized search models to analyze user query intent and generate multiple query requests; Send the generated query request to the knowledge base and related databases to capture information related to the query request; Perform preliminary filtering on the crawled results based on the user's personalized characteristics and cognitive load assessment results; Sort the filtered results, evaluate the diversity of the results, and generate the final search result set.

8. An AI-based Internet mass data precision search system, based on the AI-based Internet mass data precision search method according to any one of claims 1 to 7, characterized in that: include, A query input module receives a query text input by a user, and uses natural language processing technology to recognize the input query text and generate a structured query vector; A knowledge graph construction module constructs a knowledge graph based on the structured query vector to obtain a rich semantic context graph; The knowledge transfer learning module introduces knowledge transfer learning technology to analyze users' historical search data and behavior characteristics and build a personalized search model; The cognitive load assessment module monitors the user's real-time interactive behavior, determines the user's cognitive load, and generates an adaptive search interface; The query request generation module uses the personalized search model to initiate multiple query requests, and outputs a personalized search result set after filtering and sorting.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for accurate search of Internet massive data based on AI technology described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for accurate search of Internet massive data based on AI technology described in any one of claims 1 to 7 are implemented.

Images