Large-model thinking-enhanced networking search question and answer method, system and device and medium

Abstract

The invention relates to the field of artificial intelligence, in particular to a large-model thinking-enhanced networking search question and answer method, system and device and a medium, and the method comprises the following steps: S1, obtaining a natural language question input by a user; s2, judging whether the problem needs to be networked to search and acquire information, and if so, executing a step S3; if networking is not needed, executing the step S5; s3, generating a search keyword according to the question, and calling a search engine to obtain a related webpage; s4, performing content analysis and related content extraction on the obtained webpage to form a complete prompt word; s5, if the complete cue word exists, inputting the complete cue word and the question into a large language model to generate a final answer; and if the complete cue word does not exist, directly inputting the question into a large language model to generate a final answer. Network resources and computing resources are saved, and question and answer efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, specifically to a networked search question-answering method, system, device, and medium for large-scale model-enhanced thinking. Background Technology

[0002] In recent years, with the development of deep learning technology, Language Large Models (LLMs) have made significant progress. LLMs fit human written symbols to a massive set of neural networks with huge parameters, thereby possessing human-like linguistic intelligence, enabling them to engage in natural language dialogue, think and reason based on text. These models perform exceptionally well in tasks such as language understanding and text generation, but their performance largely depends on the pre-training data and the number of parameters in the model.

[0003] Web search is a software system that returns relevant images, videos, web pages, and other media information based on keyword descriptions, possessing the ability to obtain real-time information. Traditional web search question-answering systems primarily rely on results returned by search engines, generating answers through keyword matching and simple text processing. However, these systems have limitations when handling complex problems, especially when it is necessary to integrate information from multiple sources or perform deep reasoning.

[0004] Existing question-answering systems typically face the following technical challenges when processing user questions: When processing data requiring real-time updates (such as weather and flight status), dynamic events (such as news and technological advancements), or personalized knowledge (such as the latest research papers, local information, and travel guides), online searches are often necessary to obtain the latest information. However, existing systems lack intelligent judgment in determining whether an online search is needed, leading to unnecessary searches or the omission of crucial information. When an online search is required, the search and parsing processes of existing systems are inefficient, resulting in long overall question-and-answer response times. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a large-scale model-enhanced thinking-based online search question-answering method, system, device, and medium.

[0006] In a first aspect, the technical solution of the present invention provides a networked search and question-answering method for large-scale model-enhanced thinking, including: S1. Obtaining natural language input from the user; S2. Determine whether the problem requires an online search for information. If an online search is required, proceed to step S3; if an online search is not required, proceed to step S5. S3. Generate search keywords based on the question, call the search engine to obtain relevant web pages, each web page containing title, link and summary information; S4. Perform content parsing and relevant content extraction on the acquired web pages, including: S41. Perform content analysis on each webpage, extract the content most relevant to the question, and obtain a content set; S42. Concatenate the acquired content sets sequentially to form a complete prompt word; S5. If the complete prompt word exists, input the complete prompt word and the question into the language model to generate the final answer; if the complete prompt word does not exist, directly input the question into the language model to generate the final answer.

[0007] By using a large-scale model to quickly determine whether a user's question requires an online search, unnecessary search operations are avoided, significantly improving the system's response speed. This method can accurately distinguish between situations requiring an online search (such as real-time data, dynamic events, and personalized knowledge) and situations that do not (such as scientific laws, historical common sense, and public formulas), ensuring the rational use of system resources. Deep semantic analysis of webpage content using a large-scale model extracts the most relevant content to the user's question, ensuring highly relevant information and improving the accuracy and completeness of the answer. The extracted content set is then sequentially concatenated to form complete prompts, providing richer contextual information for subsequent answer generation and further improving the quality of the answer.

[0008] As a further limitation of the technical solution of the present invention, the judgment criterion in step S2 is: Situations requiring internet connectivity include real-time data, dynamic events, and personalized knowledge; Situations that do not require an internet connection include scientific laws, historical common sense, and publicly available formulas.

[0009] Models with fewer parameters consume relatively fewer computational resources and can quickly determine whether a question requires an internet connection, while maintaining a certain level of accuracy, thus improving the overall response speed of the question-answering system. Furthermore, they clearly distinguish between situations requiring and not requiring an internet connection. For example, real-time data, dynamic events, and personalized knowledge require an internet connection, while scientific laws, historical facts, and publicly available formulas do not. This provides clear standards for judgment, reducing ambiguity and uncertainty, and enabling the system to more accurately decide whether to perform an internet search. This further optimizes the information retrieval process and improves the performance and efficiency of the question-answering system.

[0010] As a further limitation of the technical solution of the present invention, step S3 specifically includes: S31. Based on the user's input question, perform semantic analysis and automatically generate search keywords that match the intent of the question; S32. Call the search engine API, using the search keywords as input, to initiate a search request; S33. Receive the response results returned by the search engine and extract the N related web pages contained therein; S34. Extract structured data for each webpage, including title, webpage link, and summary information; S35. Based on the semantic matching degree between the title and summary of the webpage and the question, perform a preliminary relevance assessment on the N obtained webpages, and select the set of webpages with the highest relevance to the question.

[0011] By automatically generating search keywords that match the user's question intent through semantic analysis using a large-scale model, the core intent of the user's question can be captured more accurately, resulting in higher search precision compared to keywords generated manually or using simple rules. Secondly, by invoking the search engine API to initiate search requests and receive response results, the powerful information retrieval capabilities of the search engine are leveraged to quickly obtain a large number of relevant web pages. Then, structured data is extracted from each web page, including the title, page link, and summary information, and the obtained web pages are initially relevance evaluated to select the set of web pages with the highest relevance to the question. This series of operations not only improves search efficiency but also provides a high-quality information foundation for subsequent content parsing and answer generation, ensuring that the information in the large-scale language model is highly relevant to the question, thus contributing to the generation of more accurate answers.

[0012] As a further limitation of the technical solution of the present invention, the specific steps for extracting the abstract information in step S34 include: The summary is obtained directly from the metadata returned by the search engine; or if there is no summary in the metadata, the first preset characters of the webpage text are extracted as the summary.

[0013] Extracting summaries directly from the metadata returned by the search engine is a simple and quick method that provides a rapid overview of the core content of a webpage. If no summary is available in the metadata, the first few characters of the webpage text are extracted as a summary, ensuring that summary information is always available and avoiding the problem of missing key content due to a lack of summary. This flexible summary extraction method provides necessary information support for subsequent analysis of webpage content and relevance assessment, helping to more accurately determine the relevance of the webpage to the problem.

[0014] As a further limitation of the technical solution of the present invention, the preliminary correlation assessment step in step S35 includes: Calculate the semantic similarity between the webpage title and the question. Analyze the matching density of question keywords in the summary information of web pages; The retrieved web pages are sorted based on a weighted score of semantic similarity and matching density. Remove web pages whose weighted scores are below the threshold.

[0015] Calculating the semantic similarity between webpage titles and questions allows for a semantic assessment of the relevance between the two, providing a more accurate reflection of the content's relevance compared to simple keyword matching. Statistical analysis of keyword matching density within webpage summaries, considering keyword frequency, further quantifies the connection between the webpage and the question. Ranking webpages based on a weighted score of semantic similarity and matching density, and removing those with scores below a threshold, effectively selects the set of webpages most relevant to the question. This reduces the amount of data required for subsequent processing, improves system efficiency, and ensures the quality of information input to the large language model, contributing to the generation of more accurate and higher-quality answers.

[0016] As a further limitation of the technical solution of the present invention, the step of calculating the matching density of problem keywords in the statistical abstract includes: Extract the core keywords and their weights from the problem; Statistically analyze the frequency of occurrence of the core keywords in the webpage's summary information; Calculate matching density The formula is as follows:

[0017] In the formula, n is the total number of keywords extracted from the problem. For the first i The number of times each keyword appears in the webpage summary information. For the first i The semantic importance weight of each keyword. This represents the total number of words in the abstract.

[0018] By extracting core keywords and their weights from the question, and then calculating the frequency of these keywords in the webpage summary, this method is more scientific and accurate. It considers the semantic importance weight of different keywords within the question, making the matching density calculation more reflective of the keywords' importance to the question, avoiding the problem of simply counting keyword occurrences while ignoring their importance. This precise matching density calculation method provides a more accurate basis for the initial relevance assessment of webpages, helping to filter out webpages truly relevant to the question and improving the accuracy and reliability of the question-answering system.

[0019] As a further limitation of the technical solution of the present invention, step S41 specifically includes: The parsed text of each webpage is divided into blocks to generate several content blocks, each containing consecutive sentences or paragraphs; The semantic relevance score between each content block and the question is calculated using a large model. Content blocks with relevance scores exceeding a preset threshold are selected to form a preliminary set of relevant content. Redundancy removal is performed on the initial set of related content, and semantically overlapping content blocks are merged. Output the final set of content, where each element in the set is a text fragment that is highly relevant to the question.

[0020] The parsed text of each webpage is segmented into several content blocks, enabling more detailed and targeted analysis of the content. A large-scale model calculates the semantic relevance score of each content block to the question, accurately determining the degree of association between the content block and the question at the semantic level. Content blocks with relevance scores exceeding a preset threshold are selected to form a preliminary set of relevant content, removing a large amount of irrelevant content and reducing the amount of data for subsequent processing. Redundancy removal is then performed on the preliminary set of relevant content, merging semantically overlapping content blocks to further optimize the set and avoid information duplication and redundancy. Each element in the final output content set is a text fragment highly relevant to the question, providing high-quality information input for the language model to generate accurate answers, effectively improving the answer quality and performance of the question-answering system.

[0021] Secondly, the technical solution of the present invention provides a large-scale model-enhanced thinking networked search and question-answering system, including: The question retrieval module is used to retrieve natural language questions input by the user. The network connectivity judgment module is used to determine whether the problem requires network search to obtain information; The search module is used to generate search keywords based on the question when the network connection judgment module determines that a network connection is required, and call the search engine to obtain relevant web pages. Each web page contains a title, link and summary information. The content parsing module is used to parse the content of the acquired web pages and extract relevant content, including: The content analysis unit is used to perform content analysis on each webpage, extract the content most relevant to the question, and obtain a content set; The prompt word generation unit is used to concatenate the acquired content set sequentially to form a complete prompt word; The answer generation module is used to generate a final answer based on the complete hint words and the question if the complete hint words exist; otherwise, it directly generates a final answer based on the question.

[0022] Thirdly, the present invention also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing computer program instructions executable by the at least one processor, the computer program instructions being executed by the at least one processor to enable the at least one processor to execute the large-scale model-enhanced thinking networked search question-answering method as described in the first aspect.

[0023] Fourthly, the present invention also provides a non-transitory computer-readable storage medium that stores computer instructions that cause the computer to execute the large-model augmented thinking networked search and question-answering method as described in the first aspect.

[0024] As can be seen from the above technical solution, this application has the following advantages: By intelligently determining whether the natural language question input by the user requires an online search for information, it achieves flexible selection of information acquisition methods. For questions that can be answered without an internet connection, the answer is directly generated using a large language model, avoiding unnecessary online operations, saving network and computing resources, and improving question-and-answer efficiency. For questions that require an internet connection to obtain the latest information, relevant information is acquired and processed through a series of steps before being input into the large language model to generate the answer, ensuring the accuracy and timeliness of the answer. This method of dynamically adjusting the information acquisition strategy according to the characteristics of the question effectively solves the shortcomings of traditional question-and-answer systems in handling real-time data, dynamic events, and personalized knowledge, providing users with a more comprehensive, accurate, and timely question-and-answer service. Attached Figure Description

[0025] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the method provided in an embodiment of the present invention.

[0027] Figure 2 This is a system block diagram provided for an embodiment of the present invention. Detailed Implementation

[0028] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0030] like Figure 1 As shown, this embodiment of the invention provides a networked search and question-answering method for large-scale model-enhanced thinking, including the following steps: S1. Obtaining natural language input from the user; S2. Determine whether the problem requires an online search for information. If an online search is required, proceed to step S3; if an online search is not required, proceed to step S5. It should be noted that the specific steps in step S2 to determine whether the problem requires online searching to obtain information include: Construct preset prompt words containing judgment rules. The prompt words specify the data types that need to be obtained online (real-time data, dynamic events, personalized knowledge) and the situations that do not require online access (scientific laws, historical common sense, public formulas, personal feelings, hypothetical questions), and specify the output format requirements. The system replaces the user's query with the {question} placeholder in the preset prompt words to form the complete model input text. The assembled input text is then fed into the pre-trained large model to start the model inference process. The model analyzes the properties of the query based on the prompt word rules. The system receives the model output and parses it according to a preset format (such as JSON) to obtain a boolean value judgment result, where true means that an online search is required and false means that an online search is not required.

[0031] Training large-scale models is a current technology, generally including a pre-training phase and a fine-tuning phase. Pre-training aims to allow the model to learn the statistical regularities, grammatical structures, and semantic relationships of language from massive amounts of unsupervised text data, laying the foundation for subsequent downstream tasks. Pre-training typically employs self-supervised learning methods, using the prediction of certain parts of the text (such as the next token or the occluded portion) as the training objective. In terms of data preparation, large-scale unlabeled text datasets are used, with a wide range of data sources. Data processing involves word segmentation and embedding operations. Modern pre-trained models often use word-level segmentation methods, such as Byte Pair Encoding (BPE) or SentencePiece, to better handle out-of-vocabulary words. Each token is mapped to a high-dimensional vector representation through an embedding layer before being input into the model. Common pre-training tasks include autoregressive language modeling and autoencoder language modeling. Autoregressive language modeling is used for generative pre-training, such as GPT, where the model predicts the next token based on preceding context information. Autoencoder language modeling is used for models similar to BERT, training by predicting occluded tokens to capture bidirectional contextual information. During model training, the forward propagation algorithm is used to predict the target value, and the difference between the predicted value and the true value is calculated using a loss function (such as cross-entropy loss). Then, the model parameters are optimized using the gradient descent algorithm and the backpropagation algorithm. Adam is a commonly used optimization algorithm.

[0032] While the pre-trained model has learned some language knowledge, its performance is poor when handling specific tasks, thus requiring fine-tuning. The first stage of fine-tuning is instruction fine-tuning (supervised fine-tuning), which uses manually labeled QA question-answer pairs for supervised training. The second stage of fine-tuning is alignment / reinforcement training, which uses alignment data (including questions, accepted responses, and unaccepted responses) collected from the runtime logs to train the model, enabling it to understand the acceptance level of responses, thereby obtaining the final usable model.

[0033] S3. Generate search keywords based on the question, call the search engine to obtain relevant web pages, each web page containing title, link and summary information; S4. Perform content parsing and relevant content extraction on the acquired web pages, including: S41. Perform content analysis on each webpage, extract the content most relevant to the question, and obtain a content set; S42. Concatenate the acquired content sets sequentially to form a complete prompt word; S5. If the complete prompt word exists, input the complete prompt word and the question into the language model to generate the final answer; if the complete prompt word does not exist, directly input the question into the language model to generate the final answer.

[0034] In some embodiments, step S3 specifically includes: S31. Based on the user's input question, perform semantic analysis and automatically generate search keywords that match the intent of the question; The steps to generate search keywords include: Perform semantic analysis on the question to identify core entities and intents; Multiple sets of expanded keywords are generated by combining core entities and intents, and redundant words are filtered out.

[0035] S32. Call the search engine API, using the search keywords as input, to initiate a search request; When calling the search engine API, it also includes: Set parameters to limit the search area, language, or time range to narrow down the search results.

[0036] S33. Receive the response results returned by the search engine and extract the N related web pages contained therein; Explicitly use third-party search engine APIs or build your own web crawler engine.

[0037] Structured data fields should include a title, a webpage URL, and a snippet to support subsequent processing.

[0038] S34. Extract structured data for each webpage, including title, webpage link, and summary information; S35. Based on the semantic matching degree between the title and summary of the webpage and the question, perform a preliminary relevance assessment on the N obtained webpages, and select the set of webpages with the highest relevance to the question.

[0039] In some embodiments, step S34, the specific steps for extracting the summary information include: The summary is obtained directly from the metadata returned by the search engine; or if there is no summary in the metadata, the first preset characters of the webpage text are extracted as the summary.

[0040] In some embodiments, the preliminary correlation assessment step in step S35 includes: Calculate the semantic similarity between the webpage title and the question. The webpage title text and the user input query are converted into high-dimensional vector representations respectively; the cosine similarity algorithm is used to calculate the cosine value between the title vector and the query vector to obtain the semantic similarity score between the title and the query. The score range is [0,1], and the higher the score, the higher the semantic matching degree. Analyze the matching density of question keywords in the summary information of web pages; The retrieved web pages are sorted based on a weighted score of semantic similarity and matching density. The weight coefficient W1 for the semantic similarity of the title and the weight coefficient W2 for the keyword matching density of the abstract are preset, and W1+W2=1; The overall score for each webpage is calculated using the formula: Weighted Score = Semantic Similarity Score × W1 + Matching Density × W2; all webpages are then sorted from highest to lowest based on their overall scores. Remove web pages whose weighted scores are below the threshold.

[0041] A preset relevance threshold T (T ranges from [0,1]) is set, which can be based on historical data or empirical values; the overall score of each webpage is compared with the threshold T, webpages with an overall score lower than T are removed, and webpages with an overall score ≥ T are retained for subsequent processing.

[0042] In some embodiments, the step of calculating the matching density of problem keywords in the summary includes: Extract the core keywords and their weights from the problem; Statistically analyze the frequency of occurrence of the core keywords in the webpage's summary information; Calculate matching density The formula is as follows:

[0043] In the formula, n is the total number of keywords extracted from the problem. For the first i The number of times each keyword appears in the webpage summary information. For the first i The semantic importance weight of each keyword. is the total word count of the summary (the total number of words in the webpage summary text after word segmentation). n is the total number of core keywords extracted from the query, extracted using methods including: (a) The semantic structure of the query is analyzed using a large model to identify named entities and core verb phrases; (b) The list of keywords retained after filtering out stop words.

[0044] The keyword weights are determined in the following way: Based on the semantic parsing of the query using a large model, core entities are identified and weights are assigned; or the TF-IDF algorithm is used to learn weights from historical search data.

[0045] For example, a question query: "How to fix a cracked phone screen?" Extract keywords: {"repair", "phone", "cracked screen"} (n=3); Weighting: {"Screen Crack": 0.6, "Phone": 0.3, "Repair": 0.1} (Generated from a large model); Abstract: "Methods to repair a cracked mobile phone screen...screen...replacement steps" (Total words: 20): Frequency: {"Screen cracked": 2, "Phone": 1, "Repair": 1}; Match density: (2×0.6+1×0.3+1×0.1) / 20=0.08.

[0046] In some embodiments, step S41 specifically includes: The parsed text of each webpage is divided into blocks, generating several content blocks, each containing consecutive sentences or paragraphs; these blocks are then divided according to HTML tag levels (e.g., ...). 、 The text within the tag is a block; or it can be divided by a fixed number of characters or sentences (e.g., every 200 characters or 3 sentences is a block).

[0047] The semantic relevance score between each content block and the question is calculated using a large model. Calculate the cosine similarity between the query and the content block using the large model embedding vector; or directly output the relevance score (e.g., 0-1 points) through the large model, with the prompt: "Please evaluate the relevance of the following text to the question '[query]' and output 0-1 points."

[0048] Content blocks with relevance scores exceeding a preset threshold are selected to form a preliminary set of relevant content. The preset threshold is dynamically adjusted in the following way: If the initial set of relevant content is empty, lower the threshold by 10% and re-filter; If the number of content blocks exceeds 5, the threshold is increased by 5% to focus on highly relevant content.

[0049] Redundancy removal is performed on the initial set of related content, and semantically overlapping content blocks are merged. Redundancy removal includes: Calculate the semantic similarity between content blocks. If the similarity exceeds the overlap threshold (e.g., 0.8), merge or retain the block with the higher score. Delete highly repetitive content blocks and retain at least one representative text.

[0050] The output is a final set of content, where each element is a text fragment highly relevant to the question. The output set of content has also been optimized as follows: Text compression is performed on the content extracted from the large model to retain core facts and conclusions; Add source tags (such as webpage title and link) for the subsequent answer generation module to reference.

[0051] In S42, the specific steps for sequentially concatenating the acquired content sets to form a complete prompt word include: S421. The content set {content1, content2, ..., content...} obtained in step S41 is... N Perform a validity check to confirm that each content is a valid extractable content that is not "null"; S422. Based on the ranking results of the webpage relevance assessment in step S3 (from high to low comprehensive score), determine the splicing order of each content in the content set. S423. Perform format preprocessing on each piece of content, including removing redundant line breaks, standardizing indentation format, and correcting special character garbled characters to ensure that the text format of each piece of content is standardized. S424. Following a predetermined order, concatenate the preprocessed content items sequentially using newline characters or delimiters to form a continuous string; S425. The concatenated string is used as the core content of the search result and embedded into a preset prompt word template to generate a complete search_content prompt word. The template includes task instructions for the large model, search result identifiers, and question association guidance. The large model in this invention is a language large model.

[0052] like Figure 2 As shown, this embodiment of the invention provides a large-scale model-enhanced thinking networked search and question-answering system, including: The question retrieval module is used to retrieve natural language questions input by the user. The network connectivity judgment module is used to determine whether the problem requires network search to obtain information; The search module is used to generate search keywords based on the question when the network connection judgment module determines that a network connection is required, and call the search engine to obtain relevant web pages. Each web page contains a title, link and summary information. The content parsing module is used to parse the content of the acquired web pages and extract relevant content, including: The content analysis unit is used to perform content analysis on each webpage, extract the content most relevant to the question, and obtain a content set; The prompt word generation unit is used to concatenate the acquired content set sequentially to form a complete prompt word; The answer generation module is used to generate a final answer based on the complete hint words and the question if the complete hint words exist; otherwise, it directly generates a final answer based on the question.

[0053] In some embodiments, the judgment criteria are as follows: Situations requiring internet connectivity include: real-time data, dynamic events, and personalized knowledge; Situations that do not require an internet connection include: scientific laws, historical common sense, and publicly available formulas.

[0054] In some embodiments, the search module specifically includes: The keyword generation unit is used to perform semantic analysis based on the user's input question and automatically generate search keywords that match the intent of the question; The search request unit is used to call the search engine API, take the search keywords as input, and initiate a search request. The result receiving unit is used to receive the response results returned by the search engine and extract the N relevant web pages contained therein; The data extraction unit is used to extract structured data from each webpage, including the title, webpage links, and summary information; The relevance assessment unit is used to perform a preliminary relevance assessment on the N obtained web pages based on the semantic matching degree between the title and summary of the web pages and the question, and to filter out the set of web pages with the highest relevance to the question.

[0055] In some embodiments, the specific steps for extracting summary information in the data extraction unit include: Obtain the summary directly from the metadata returned by the search engine; or If there is no digest in the metadata, the first preset characters of the webpage text will be used as the digest.

[0056] In some embodiments, the preliminary correlation assessment step in the correlation assessment unit includes: Calculate the semantic similarity between the webpage title and the question. Analyze the matching density of question keywords in the summary information of web pages; The retrieved web pages are sorted based on a weighted score of semantic similarity and matching density. Web pages with weighted scores below a threshold are removed. This threshold can be dynamically adjusted based on the actual application scenario; for example, it can be set to 0.6 in information-intensive scenarios and 0.4 in information-sparse scenarios to balance information coverage and accuracy.

[0057] In some embodiments, the step of calculating the matching density of question keywords in the relevance assessment unit includes: Extract the core keywords and their weights from the problem; Statistically analyze the frequency of occurrence of the core keywords in the webpage's summary information; The matching density is calculated using the following formula:

[0058] In the formula, n is the total number of keywords extracted from the problem. For the first i The number of times each keyword appears in the webpage summary information. For the first i The semantic importance weight of each keyword. This represents the total number of words in the abstract.

[0059] In some embodiments, the content analysis unit in the content parsing module specifically includes: The chunking unit is used to divide the parsed text of each webpage into chunks to generate several content blocks, each containing consecutive sentences or paragraphs. The relevance calculation unit is used to calculate the semantic relevance score between each content block and the question using a large model; The content filtering unit is used to filter content blocks whose relevance scores exceed a preset threshold, forming a preliminary set of relevant content. The redundancy processing unit is used to perform redundancy removal on the initial set of related content and merge semantically overlapping content blocks. The content output unit is used to output the final content set, where each element in the set is a text fragment that is highly relevant to the question.

[0060] This invention also provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus. The communication bus is used for information transmission between the processor, communication interface, and memory. The processor can call logical instructions in the memory to execute the following method: S1, obtaining a natural language question input by the user; S2, determining whether the question requires an online search for information; if an online search is required, proceeding to step S3; if no online search is required, proceeding to step S5; S3, generating search keywords based on the question, and calling a search engine to obtain relevant web pages, each web page containing a title, link, and summary information; S4, performing content parsing and relevant content extraction on the obtained web pages, including: S41, performing content analysis on each web page, extracting the content most relevant to the question, and obtaining a content set; S42, sequentially concatenating the obtained content set to form a complete prompt word; S5, if the complete prompt word exists, generating a final answer based on the complete prompt word and the question; if the complete prompt word does not exist, directly generating a final answer based on the question.

[0061] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] This invention provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to execute the method provided in the above-described method embodiments. For example, the instructions include: S1, obtaining a natural language question input by a user; S2, determining whether the question requires an online search for information; if an online search is required, proceeding to step S3; if no online search is required, proceeding to step S5; S3, generating search keywords based on the question, and calling a search engine to obtain relevant web pages, each web page containing a title, link, and summary information; S4, performing content parsing and relevant content extraction on the obtained web pages, including: S41, performing content analysis on each web page, extracting the content most relevant to the question, and obtaining a content set; S42, sequentially concatenating the obtained content set to form a complete prompt word; S5, if the complete prompt word exists, generating a final answer based on the complete prompt word and the question; if the complete prompt word does not exist, directly generating a final answer based on the question.

[0063] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A networked search and question-answering method for large-scale model-enhanced thinking, characterized in that, include: S1. Obtaining natural language input from the user; S2. Determine whether the problem requires online searching to obtain information. If online search is required, proceed to step S3. If no internet connection is required, proceed to step S5; S3. Generate search keywords based on the question, call the search engine to obtain relevant web pages, each web page containing title, link and summary information; S4. Perform content parsing and relevant content extraction on the acquired web pages, including: S41. Perform content analysis on each webpage, extract the content most relevant to the question, and obtain a content set; S42. Concatenate the acquired content sets sequentially to form a complete prompt word; S5. If the complete prompt word exists, input the complete prompt word and the question into the language model to generate the final answer; if the complete prompt word does not exist, directly input the question into the language model to generate the final answer.

2. The networked search and question-answering method for large-scale model-enhanced thinking according to claim 1, characterized in that, The judgment criteria in step S2 are: Situations requiring internet connectivity include real-time data, dynamic events, and personalized knowledge; Situations that do not require an internet connection include scientific laws, historical common sense, and publicly available formulas.

3. The networked search and question-answering method for large-scale model-enhanced thinking according to claim 2, characterized in that, Step S3 specifically includes: S31. Based on the user's input question, perform semantic analysis and automatically generate search keywords that match the intent of the question; S32. Call the search engine API, using the search keywords as input, to initiate a search request; S33. Receive the response results returned by the search engine and extract the N related web pages contained therein; S34. Extract structured data for each webpage, including title, webpage link, and summary information; S35. Based on the semantic matching degree between the title and summary of the webpage and the question, perform a preliminary relevance assessment on the N obtained webpages, and select the set of webpages with the highest relevance to the question.

4. The networked search and question-answering method for large-scale model-enhanced thinking according to claim 3, characterized in that, In step S34, the specific steps for extracting summary information include: The summary is obtained directly from the metadata returned by the search engine; or if there is no summary in the metadata, the first preset characters of the webpage text are extracted as the summary.

5. The networked search and question-answering method for large-scale model-enhanced thinking according to claim 4, characterized in that, The preliminary relevance assessment steps in step S35 include: Calculate the semantic similarity between the webpage title and the question. Analyze the matching density of question keywords in the summary information of web pages; The retrieved web pages are sorted based on a weighted score of semantic similarity and matching density. Remove web pages whose weighted scores are below the threshold.

6. The networked search and question-answering method for large-scale model-enhanced thinking according to claim 5, characterized in that, The steps for determining the matching density of question keywords in a statistical summary include: Extract the core keywords and their weights from the problem; Statistically analyze the frequency of occurrence of the core keywords in the webpage's summary information; Calculate matching density The formula is as follows: In the formula, n is the total number of keywords extracted from the problem. For the first i The number of times each keyword appears in the webpage summary information. For the first i The semantic importance weight of each keyword. This represents the total number of words in the abstract.

7. The networked search and question-answering method for large-scale model-enhanced thinking according to claim 6, characterized in that, Step S41 includes: The parsed text of each webpage is divided into blocks to generate several content blocks, each containing consecutive sentences or paragraphs; Calculate the semantic relevance score of each content block to the question; Content blocks with relevance scores exceeding a preset threshold are selected to form a preliminary set of relevant content. Redundancy removal is performed on the initial set of related content, and semantically overlapping content blocks are merged. Output the final set of content, where each element in the set is a text fragment that is highly relevant to the question.

8. A large-scale model-based networked search and question-answering system, characterized in that, include: The question retrieval module is used to retrieve natural language questions input by the user. The network connectivity judgment module is used to determine whether the problem requires network search to obtain information; The search module is used to generate search keywords based on the question when the network connection judgment module determines that a network connection is required, and call the search engine to obtain relevant web pages. Each web page contains a title, link and summary information. The content parsing module is used to parse the content of the acquired web pages and extract relevant content, including: The content analysis unit is used to perform content analysis on each webpage, extract the content most relevant to the question, and obtain a content set; The prompt word generation unit is used to concatenate the acquired content set sequentially to form a complete prompt word; The answer generation module is used to generate a final answer based on the complete hint words and the question if the complete hint words exist; otherwise, it directly generates a final answer based on the question.

9. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions executable by the at least one processor, the computer program instructions being executed by the at least one processor to enable the at least one processor to perform the large-scale model-enhanced thinking networked search question-answering method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to execute the networked search and question-answering method for large-scale model-enhanced thinking as described in any one of claims 1 to 7.

Images