Information processing method and device based on large model, equipment and storage medium

Abstract

This application discloses an information processing method, apparatus, device, and storage medium based on a large model, applicable to the field of artificial intelligence. The method includes: upon receiving a first instruction to construct knowledge documents, acquiring the content of multiple preset web pages; performing semantic analysis on the content of each preset web page using a large model, and aggregating semantically related content from each preset web page into a single content fragment based on the semantic analysis results; for each content fragment, performing content analysis on the content fragment using the large model, and expanding the content fragment with different knowledge dimensions based on the content analysis results, generating multiple knowledge documents that are knowledge-related to the content fragment. Using this application embodiment can improve the efficiency of knowledge document construction and has high applicability.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to an information processing method, apparatus, device, and storage medium based on a large model. Background Technology

[0002] With the continuous development of artificial intelligence technology, large-scale models, based on their massive neural network architecture and powerful language understanding and information processing capabilities, have demonstrated outstanding information processing performance in many fields. To improve their information processing capabilities and output more accurate results, existing large-scale models often refer to knowledge documents for information processing. For example, in existing question-answering systems, large-scale models typically retrieve corresponding reference knowledge documents based on the question statement and answer the question based on those documents.

[0003] In existing technologies, knowledge documents are usually directly searched from the Internet or manually constructed, resulting in insufficient knowledge documents, narrow coverage, and messy semantic content. Summary of the Invention

[0004] This application provides an information processing method, apparatus, device, and storage medium based on a large model, which can improve the efficiency of knowledge document construction and has high applicability.

[0005] On the one hand, embodiments of this application provide an information processing method based on a large model, the method comprising: When the first instruction for building a knowledge document is received, the web page content of multiple preset web pages is retrieved; The large model is used to perform semantic analysis on the webpage content of each of the above-mentioned preset webpages, and webpage content with semantic relationships is aggregated into the same content fragment based on the semantic analysis results. For each of the above content fragments, the content fragment is analyzed using the aforementioned large model, and based on the results of the content analysis, the content fragment is expanded with knowledge in different knowledge dimensions to generate multiple knowledge documents that are knowledge-related to the content fragment.

[0006] On the other hand, embodiments of this application provide an information processing apparatus based on a large model, the apparatus comprising: The content acquisition module is used to acquire the web page content of multiple preset web pages when the first instruction for building a knowledge document is received. The information processing module is used to perform semantic analysis on the webpage content of each of the above-mentioned preset webpages through a large model, and to aggregate the webpage content with semantic relationships into the same content fragment based on the semantic analysis results. The aforementioned information processing module is used to perform content analysis on each of the aforementioned content fragments using the aforementioned large model, and to expand the knowledge of the content fragments in different knowledge dimensions based on the content analysis results, thereby generating multiple knowledge documents that are knowledge-related to the content fragments.

[0007] On the other hand, embodiments of this application provide an electronic device, including a processor and a memory, which are interconnected; The aforementioned memory is used to store computer programs; The processor described above is used to execute the information processing method based on a large model provided in the embodiments of this application when the computer program described above is invoked.

[0008] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the large-model-based information processing method provided in embodiments of this application.

[0009] On the other hand, embodiments of this application provide a computer program product, which includes a computer program that, when executed by a processor, implements the information processing method based on a large model provided in embodiments of this application.

[0010] In this embodiment, by accessing multiple preset web pages and obtaining their content, a wide range of information from different sources can be quickly gathered. This helps to build a comprehensive knowledge document foundation, providing rich material for the subsequent generation of knowledge documents. Using a large model to segment web page content according to semantics allows for more accurate identification and understanding of content fragments with different semantic meanings within the web page. This not only improves the efficiency of information processing but also enables the generation of knowledge documents for single semantic meanings, enhancing the relevance of knowledge documents. For each content fragment, knowledge expansion through the large model generates knowledge documents that are knowledge-related to that content fragment. This step not only enriches the information content of the original web page but also enhances the depth and breadth of the content fragments, making the constructed knowledge document set more comprehensive and detailed, more effectively catering to various information processing scenarios, and also improving the accuracy of the large model's search for knowledge documents. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1This is a schematic diagram of a scenario for the information processing method based on a large model provided in an embodiment of this application; Figure 2 This is a flowchart illustrating an information processing method based on a large model provided in an embodiment of this application; Figure 3 This is a schematic diagram of a process for accessing a preset webpage provided in an embodiment of this application; Figure 4 This is one of the scenario illustrations provided in the embodiments of this application for obtaining web page content based on the document object model node tree; Figure 5 This is the second schematic diagram of a scenario for obtaining web page content based on a document object model node tree, provided in an embodiment of this application. Figure 6 This is a schematic diagram of another access process for accessing a preset page provided in an embodiment of this application; Figure 7 This is another flowchart illustrating the information processing method based on a large model provided in this application embodiment; Figure 8 This is a schematic diagram of the structure of the information processing device based on a large model provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] This application provides an information processing method based on a large model, which can construct rich knowledge documents for different content fragments through the large model, so as to achieve more accurate question responses in the field of artificial intelligence question answering and improve the accuracy of human-computer interaction.

[0015] Large models, also known as large artificial intelligence models, mainly refer to machine learning models with a large number of parameters and computational power. Based on deep learning algorithms, large models are trained using large datasets and powerful computing capabilities. They possess a vast number of parameters and complex network structures, enabling them to process and analyze massive amounts of data. Furthermore, they can have multimodal data processing capabilities, such as simultaneously processing data from multiple modalities like text, images, and speech, achieving cross-modal information understanding and generation.

[0016] Large models possess high versatility and generalization capabilities, applicable to fields such as natural language processing, image recognition, and speech recognition. They can be categorized into large language models, large visual models, multimodal large models, and basic large models, among others. For example, in natural language processing, large models can be applied to text generation, machine translation, sentiment analysis, and question-answering systems, achieving accurate language understanding and generation by understanding the semantics and contextual information of natural language text. Furthermore, in computer vision, large models can be applied to image recognition, image generation, and image enhancement, achieving high-precision image processing and recognition by capturing details and features from images.

[0017] In this application, a knowledge document refers to a document or data set containing information about a specific domain or topic. These documents are typically used to train or optimize large models so that they can more accurately understand knowledge in a specific domain, or to provide a reference for large models so that they can generate more reliable and professional responses when performing tasks.

[0018] For example, knowledge documents can serve as training data for large models, helping them learn the language styles, terminology, and knowledge of specific domains. In the sports field, for instance, knowledge documents might include sports rules, event information, club history, and statistical data. Furthermore, after a large model has undergone general training, it can be fine-tuned using knowledge documents to better adapt it to specific domain applications. Knowledge documents can also function as external knowledge bases, used in conjunction with large models. When a model cannot directly answer a question, it can generate a more accurate answer by retrieving relevant content from knowledge documents. Alternatively, knowledge documents can be used to verify the accuracy of content generated by the large model, especially in fields with stringent requirements for answers (such as medicine, law, and scientific research).

[0019] See Figure 1 , Figure 1 This is a schematic diagram of a scenario for the information processing method based on a large model provided in an embodiment of this application. For example... Figure 1 As shown, when device 20 receives a first instruction to construct knowledge documents, it acquires the content of multiple preset web pages. Then, it performs semantic analysis on the content of each preset web page using a large model, and aggregates semantically related content from each preset web page into the same content fragment based on the semantic analysis results. Based on this, each content fragment represents a different semantic meaning. Furthermore, device 20 can perform content analysis on each content fragment using the large model, and expand each content fragment with different knowledge dimensions based on the content analysis results, generating multiple knowledge documents that are knowledge-related to that content fragment.

[0020] In this case, when any object sends a question statement to the device 20 through the device 10, the device 20 can obtain the target knowledge document associated with the question statement from the knowledge document collection, generate the response information for the question statement based on the target knowledge document, and return the generated response content to the device 10, thereby completing the question response.

[0021] Among them, device 20 can run large models and can access large models through fixed ports to realize the calling of large models.

[0022] Among them, device 20 can be a terminal device or server with data processing capabilities. The server can be an independent physical server, such as a model server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0023] The terminal device can be a smartphone, tablet, laptop, desktop computer (including monitor), smartwatch, vehicle terminal, aircraft, smart home appliance (such as smart TV) or wearable device, etc.

[0024] Among them, device 10 can be a terminal device with display function for human-computer interaction, which can be determined based on the actual application scenario requirements and is not limited here.

[0025] It should be noted that the information processing method based on a large model provided in this application embodiment can be executed by device 20 or device 10 alone, or by device 20 and device 10 together, without limitation. For example, when the information processing method based on a large model provided in this application embodiment can be executed by device 20 or device 10 alone, device 20 or device 10 can access multiple preset web pages to construct a knowledge document set, and when a question statement is obtained, retrieve the target knowledge document associated with the question statement from the knowledge document set, and then generate the response content for the question statement based on the target knowledge document. For example, when the information processing method based on the large model provided in this application embodiment can be jointly executed by device 20 and device 10, any one or more of the following processes can be executed by device 10: accessing preset web pages and obtaining the web page content of each preset web page; segmenting the web page content of each preset web page according to semantics using the large model to obtain multiple content fragments; expanding the knowledge of the content fragments using the large model to obtain knowledge documents; constructing a knowledge document set; and generating response content. The remaining processes can be executed by device 20. The specific process can be determined based on the actual application scenario requirements and is not limited here.

[0026] See Figure 2 , Figure 2 This is a flowchart illustrating an information processing method based on a large model provided in an embodiment of this application. For example... Figure 2 As shown in the embodiments of this application, the information processing method based on a large model can be described using server execution as an example. The method may include the following steps: S21, when the first instruction for constructing a knowledge document is received, the web page content of multiple preset web pages is obtained.

[0027] In some feasible implementations, each preset webpage is a webpage that includes the content required to construct a knowledge document, such as a webpage that includes text information, image information, etc. required to construct a knowledge document.

[0028] The types of preset web pages can be academic websites, sports websites, government information websites, media websites, e-commerce websites, social networking websites, and various professional database websites, etc. The specific types can be determined based on the actual application scenario requirements, and there are no restrictions here.

[0029] The preset webpages can be selected by searching for relevant keywords through a search engine and identifying the webpages that rank highly in the search results. Alternatively, preset webpages can be webpages with high traffic and engagement in the relevant field. Alternatively, preset webpages can be identified by analyzing existing text data using natural language processing techniques to find frequently cited webpages and use them as preset webpages.

[0030] It should be noted that the method for determining the preset web pages can be based on the specific needs of the application scenario and is not limited here. For example, the preset web pages can be multiple web pages pre-specified before constructing the knowledge document collection.

[0031] The first instruction can be triggered based on time conditions, such as a preset schedule or cycle, to periodically generate knowledge documents.

[0032] Alternatively, the first instruction can be triggered when a specific event occurs. For example, when a hot topic event occurs, a preset webpage related to the hot topic event can be accessed immediately to obtain the webpage content, and then a corresponding knowledge document can be generated based on the obtained webpage content.

[0033] Alternatively, the first command can be triggered by a regular touch button, without any restrictions.

[0034] S22, semantic analysis is performed on the webpage content of each preset webpage using a large model, and the semantically related webpage content in each preset webpage is aggregated into the same content fragment based on the semantic analysis results.

[0035] In some feasible implementations, for each webpage, the webpage content typically describes different content in different paragraphs, or different content in different places within the same paragraph. That is, the webpage content may contain multiple paragraphs with different semantics, or a single paragraph may contain multiple semantic elements. Therefore, after obtaining the webpage content of each preset webpage, it is necessary to perform semantic analysis on the webpage content of each preset webpage using a large model. Based on the semantic analysis results, the semantically related webpage content in each preset webpage is aggregated into the same content fragment.

[0036] Among them, web page content with semantic relevance can be understood as web page content with the same or related semantics, or web page content with semantic similarity higher than a certain threshold, or web page content containing the same semantic keywords, etc. The specific definition can be determined based on the actual application scenario requirements, and no restrictions are imposed here.

[0037] In some feasible implementations, existing technologies typically segment web page content based on typographical features such as font size, blank lines, and indentation rules, or cut web page content according to features such as font size, blank lines, and titles. This can lead to problems with unclear semantics in the segmented paragraphs, such as web page content used to describe the same semantic content being divided into different segments.

[0038] Based on this, after obtaining the webpage content of each preset webpage, a first prompt message can be generated according to the webpage content of the preset webpage.

[0039] The first prompt message is used to instruct the large model to segment the webpage content of the preset webpage according to semantics.

[0040] In this case, based on the first prompt information, the large model can be invoked to perform semantic analysis on the webpage content of the preset webpage, and then the webpage content of the preset webpage can be segmented according to the semantic analysis results to obtain at least one content fragment.

[0041] In other words, by processing the webpage content of the preset webpage using a large model, multiple content fragments with different semantics can be obtained.

[0042] Specifically, after determining the first prompt information, the webpage content of the preset webpage can be embedded into the first prompt information. The large model interface is called so that the large model performs semantic analysis on the webpage content of the preset webpage according to the instructions of the first prompt information, and aggregates the semantically related webpage content in the preset webpage into the same content fragment based on the semantic analysis results.

[0043] The first prompt message can be a preset prompt template. It can call the large model interface to segment the web page content input into the large model according to semantics, so that the web page content can be segmented simply and efficiently without considering pagination, spacing and layout issues. On the other hand, it can avoid segmentation errors caused by unreasonable human editing and layout.

[0044] In the field of artificial intelligence, a prompt is a text or statement that guides a machine learning model to generate output of a specific type, topic, or format. In practical applications, a prompt can be regarded as a tool for launching and guiding large models. With well-designed prompts, the output quality and diversity of the model can be significantly improved, thus playing an important role in various tasks.

[0045] Prompts have wide applications in fields such as natural language processing and image recognition. They can be used to launch models in the form of questions or task descriptions, generating corresponding answers or results. Effective prompts should include clear instructions, provide reference text, break down complex tasks into simpler subtasks, give the model time to "think," and allow the use of external tools. For example, when using prompts for text generation, specific descriptions or topics can be given, such as "Write an article about the development of artificial intelligence," or in question-answering systems, prompts can be in the form of questions, such as "Explain what machine learning is."

[0046] To obtain better model responses, prompts should provide as much detailed information and context as possible, specifying the steps required to complete the task, providing examples, and specifying the output length. Additionally, specific strategies can be employed, such as allowing the model to find its own solutions before reaching a conclusion, or using external tools to assist the model's "thinking" process.

[0047] The process of aggregating semantically related web page content into the same content segment based on semantic analysis results can be regarded as knowledge cohesive segmentation.

[0048] Knowledge cohesion, in this context, describes the tightness and relevance of knowledge or information in terms of organization, structure, or function. In this application, knowledge cohesion refers to the tightness and relevance of knowledge elements (such as concepts, facts, rules, images, semantics, etc.) within webpage content in terms of logic, function, or structure. Knowledge elements with high cohesion typically have a clear theme or function and are strongly interconnected, forming an organic whole.

[0049] In other words, all the content in each content fragment obtained through the large model has the same semantics.

[0050] By using large models to segment web page content according to semantics, we can more accurately identify and understand content fragments with different semantic meanings on the web page. This not only improves the efficiency of information processing, but also allows us to generate knowledge documents for individual semantic meanings in the future, thereby enhancing the relevance of knowledge documents.

[0051] S23. For each content segment, the content segment is analyzed using a large model, and knowledge expansion is performed on the content segment in different knowledge dimensions based on the content analysis results, generating multiple knowledge documents that are knowledge-related to the content segment.

[0052] In some feasible implementations, after obtaining content fragments of web page content, since the content fragments only partially describe some facts, such as introducing a single function of a device or the usage method of a tool, if the content fragments are directly used as knowledge documents, it is impossible to combine the content fragments with actual application scenarios. As a result, when answering questions based on knowledge documents, it is not easy to find knowledge documents related to the question statements, and it is also easy to cause the problem of not being able to find knowledge documents.

[0053] Based on this, for each content fragment, a large model can be used to perform content analysis on the content fragment, and then the content fragment can be expanded with knowledge in different knowledge dimensions according to the content analysis results, thereby generating multiple knowledge documents that are knowledge-related to the content fragment.

[0054] In this context, "knowledge dimension" refers to the way content fragments are classified and expanded from different angles or levels. This allows for a more systematic and comprehensive approach to refining, optimizing, and organizing content fragments to meet diverse needs and application scenarios. Each knowledge dimension is essentially a perspective for analyzing or processing a content fragment. Knowledge documents that are knowledge-related to the content fragments are collections of knowledge obtained after multi-dimensional expansion of the content fragments. These include answers to questions, application scenarios, problem-solving techniques, topic overviews, areas of difficulty in understanding, and content rewriting, aiming to provide comprehensive knowledge support and comprehension guidance.

[0055] Optionally, the above-mentioned indication dimensions include, but are not limited to, at least one of the following: The content browsing target audience includes potential questions and corresponding answers related to the content segment. Information about the context in which the content fragment is applied by the content browsing object; The problem addressed by this content fragment; The main theme information of this content segment; This causes the content viewer to be unable to understand the key information in the content fragment; The content after rewriting and / or polishing the content fragment.

[0056] This section addresses potential questions and answers that may arise when browsing content snippets, focusing on potential points of confusion and clarification. When reading content snippets, users may encounter points they don't understand or that require further explanation. Providing these questions and their answers helps users resolve doubts and more accurately grasp the core meaning of the content snippets.

[0057] For example, the content fragment is: in HTML Tags and <section>The tags are used to define a section within a document. Therefore, the content browsing object may encounter the following problems when viewing content fragments: Tags and <section>What are the differences between the labels? The answer is: The tag is a general container with no semantic meaning, while <section>Tags are semantic and used to define independent parts of a document. In this case, the question-and-answer content mentioned above can be considered a knowledge document that has an informative connection to the content fragment.

[0058] The context information describing how the content fragment is applied by the content browsing user describes the environment and conditions under which the content fragment is used in practice. By understanding these contexts, the content browsing user can better comprehend how the content fragment is applied or its applicable scope, enabling them to associate the content with its actual uses and thus apply it more effectively in real life.

[0059] For example, the content fragment could be an `if` statement used for conditional judgment. The scenario information where this content fragment is applied to the content browsing object can be used during login verification to determine whether a visitor has access permissions.

[0060] Among them, the problem solved by content fragments is that they help content viewers quickly grasp the main content and purpose of the content fragments, thereby obtaining the information they need more efficiently.

[0061] For example, the content snippet might state: "CSS's flexbox layout can control the arrangement of child elements within a container." This snippet addresses the question: how to achieve flexible layout in a webpage, ensuring that child elements adapt to different layouts.

[0062] The theme of a content segment is its core and essence. By understanding the theme, the viewer can quickly establish a holistic cognitive framework for the content segment, laying the foundation for further in-depth understanding and application.

[0063] For example, if the content snippet states: "JavaScript provides various event handling mechanisms, such as onclick," then the main information of this content snippet is: JavaScript event handling.

[0064] The key information that prevents users from understanding a content fragment can identify the difficulties and crucial points within the fragment that may hinder comprehension. By identifying and explaining these difficulties, users can overcome these obstacles and improve their understanding efficiency.

[0065] For example, if the content fragment is: "Generators in Python are a special type of iterator, defined using the `yield` keyword," then the key information that causes the browsing object to be unable to understand this content fragment could be: the difference between generators and iterators. It should be further explained that generators are a convenient way to implement iterators, improving efficiency through lazy evaluation.

[0066] Rewriting and / or polishing content fragments is another form of expression. Different ways of expressing content can help viewers understand fragments from multiple perspectives, deepening their grasp and memory of key points. At the same time, rewriting and polishing also enhance readability and appeal, making the content easier to understand and accept.

[0067] For example, the content snippet is: HTML is a markup language used to define the structure of a webpage. The content snippet after rewriting and / or polishing is: HTML, a powerful markup language, is the cornerstone for describing the structure and content of webpages.

[0068] By leveraging large-scale models to generate knowledge documents corresponding to each content fragment, the analysis process of human understanding web page content can be simulated. This establishes a mapping relationship between web page content and relevant business knowledge. Then, the large-scale model learns from the text and / or images within the content fragments to gain a deeper understanding of the inherent knowledge logic and branching within the fragments. This allows for the identification of potential problems humans might encounter during content comprehension, thematic information about the content fragments, and relevant application scenarios. In other words, the large-scale model can determine the problems humans might face when encountering content fragments, their potential confusion, how they might ask questions, and how to provide answers.

[0069] In some feasible implementations, after acquiring the content fragments, for each content fragment, a second prompt message can be generated based on the content fragment. The second prompt message is used to instruct the large model to generate a knowledge document that has a knowledge-related association with the content fragment.

[0070] In this case, based on the second prompt, the large model can be invoked to perform content analysis on the content fragment, and then the content fragment can be expanded with knowledge in different knowledge dimensions based on the content analysis results to generate a knowledge document that has a knowledge relationship with the content fragment.

[0071] The second prompt message can be a preset prompt template, which can call the large model interface to perform content analysis on the content fragments input into the large model and generate knowledge documents.

[0072] In this application, knowledge-related documents refer to knowledge documents that are intrinsically linked to or associated with content fragments. These knowledge-related documents help content viewers to understand, apply, or expand their knowledge of content fragments more deeply, or guide them to effectively apply the information provided by the content fragments in specific scenarios. Knowledge-related association not only makes content fragments and knowledge documents more systematic, but also helps resolve potential questions in content fragments, or enhances the readability and application value of content fragments.

[0073] In this embodiment, by accessing multiple preset web pages and obtaining their content, a wide range of information from different sources can be quickly gathered, which helps to build a comprehensive knowledge document foundation and provides rich materials for the subsequent generation of knowledge documents. Using a large model to segment web page content according to semantics allows for more accurate identification and understanding of content fragments with different semantic meanings within the web page. This not only improves the efficiency of information processing but also enables the generation of knowledge documents for single semantic meanings, enhancing the relevance of knowledge documents. For each content fragment, knowledge expansion through the large model can generate knowledge documents that are knowledge-related to that content fragment. This step not only enriches the information content of the original web page but also enhances the depth and breadth of the content fragments, making the constructed knowledge document set more comprehensive and detailed, more effectively catering to various information processing scenarios, and also improving the accuracy of the large model's search for knowledge documents.

[0074] In some feasible implementations, when obtaining the web page content of multiple preset web pages, multiple web pages can be accessed and their content obtained based on random access. In this case, the randomly accessed web pages belong to the preset domain, such as sports, news, etc., and there are no restrictions here.

[0075] Optionally, when accessing multiple preset web pages, a configuration file can be obtained, which includes the URL information of the multiple preset web pages.

[0076] The URL information for each preset webpage can be a Uniform Resource Locator (URL).

[0077] Before accessing multiple preset web pages, the browser can be launched in a headless mode based on the headless parameter and the preset remote-debugging-port parameter, so that the preset web pages can be accessed without displaying the web pages.

[0078] In headless mode, a browser can run in an environment without a monitor, keyboard, or mouse. For servers, since they typically lack these hardware devices but still need to perform browser-based page access tasks, the headless parameter allows them to simulate these characteristics using their computing power, thus completing the page access task.

[0079] The main purpose of the headless parameter is to run the browser in a headless mode. This mode reduces reliance on a graphical interface and improves program efficiency in scenarios such as automated access and web crawling of web content. When the browser is launched using the headless parameter, it will not display a graphical interface, but will still perform all web page loading and rendering operations.

[0080] The `remote-debugging-port` parameter, which specifies a port number for remote browser debugging, allows the server to communicate with the browser through this port, view the browser's status, debug web pages, and more.

[0081] By combining the headless parameter and the preset remote-debugging-port parameter, the server can launch a browser instance that runs in the background and supports remote debugging, thereby enabling access to preset web pages.

[0082] Furthermore, after launching the browser in a headless mode, a preset communication protocol connection can be established with the browser, and the browser can be controlled to access the preset webpage corresponding to each URL information to obtain the webpage content of the corresponding preset webpage through this communication protocol connection.

[0083] The aforementioned preset communication protocol can be the WebSocket protocol or other communication protocols, and no restrictions are imposed here.

[0084] Specifically, when establishing a preset communication protocol connection with the browser, it can be determined whether the browser is listening for the preset communication protocol connection on the port specified by the preset remote debugging port parameter. If it is determined that the browser is listening for the preset communication protocol connection, a preset communication protocol connection can be established with the browser, and then the browser can be controlled to access the preset webpage corresponding to each URL information through the established communication protocol connection.

[0085] When the browser listens for connections using a preset communication protocol on the port specified by the preset remote debugging port parameter, it indicates that the browser is running in the background in a headless mode. At this time, the server can establish a connection with the browser using a preset communication protocol, such as establishing a Web socket connection.

[0086] Specifically, when the browser is listening for a connection to a preset communication protocol on the port specified by the preset remote debugging port parameter, a client can be created and used to attempt to establish a connection with the browser on the port specified by the preset remote debugging port parameter. If the connection is successfully established, the browser can be controlled to access each preset webpage in a headless mode through the communication protocol connection.

[0087] For example, when a browser is listening for Web socket connections on the port specified by the preset remote debugging port parameter, a Web socket client can be created to attempt to establish a Web socket connection with the browser on the port specified by the preset remote debugging port parameter. Once the Web socket connection is successfully established, the browser can be controlled through the Web socket connection to access a preset page in headless mode.

[0088] When accessing each URL in the configuration file through a browser, the CDP (Chrome Development Protocol) protocol can be used to access the URLs one by one. Specifically, this can be done through the proto.PageNavigate directive.

[0089] The proto.PageNavigate directive is used to navigate browser pages to specific URLs in automation scripts.

[0090] In some feasible implementations, when accessing various preset web pages and obtaining web page content through a browser, if any preset web page requires access permission verification, the access permission verification data of the preset web page can be obtained from the aforementioned configuration file, and then access permission verification can be performed based on the access permission verification data of the preset web page, so that the preset web page can be accessed after the access permission verification is passed.

[0091] The aforementioned permission verification data is associated with the permission verification method of the corresponding preset webpage. The permission verification methods of different preset webpages can be the same or different, and can be determined based on the actual webpage configuration. No restrictions are imposed here.

[0092] The aforementioned permission verification method can be login name and password verification, and the aforementioned permission verification data can be the login name and password used when registering on the webpage.

[0093] Alternatively, the above-mentioned permission verification method can be a token verification method, and the above-mentioned permission verification data can be a token. The token serves as the visitor's identity credential, typically an encrypted string containing information such as identity and validity period.

[0094] Alternatively, the above-mentioned permission verification method can be authorization authentication, and the above-mentioned permission verification data can be an authorization code. Here, authorization verification refers to logging in through the authentication mechanism of a third-party platform.

[0095] Alternatively, the above-mentioned permission verification method can be cookie verification or digital certificate verification, and the corresponding permission verification data can be cookie data or digital certificate.

[0096] Cookies are small pieces of data sent to a browser by the server corresponding to a webpage and stored locally. They are sent to the server with the browser's data every time it makes a subsequent request to the same webpage. Cookies are primarily used for session state management, helping webpage servers identify visitors and maintain their session state. For example, when a visitor logs into a webpage, the webpage server can generate a cookie containing the visitor's session information and send it to the browser. Subsequently, the browser will include this cookie in every request to the website, allowing the webpage server to identify the visitor and maintain their session state.

[0097] The following is combined with Figure 3 The process of accessing a preset webpage in the embodiments of this application will be further explained. Figure 3 This is a schematic diagram of a process for accessing a preset webpage provided in an embodiment of this application. Figure 3 The access process shown may include the following steps: S31, launch the browser in headless mode.

[0098] Specifically, before accessing a preset webpage, a configuration file needs to be obtained, which includes the URL information of multiple preset webpages.

[0099] Furthermore, the browser can be launched in a headless mode based on the headless parameter and the preset remote-debugging-port parameter, thereby allowing access to preset web pages without displaying the web page.

[0100] The headless parameters and preset remote debugging port parameters can be pre-configured or obtained from the configuration file; there are no restrictions here.

[0101] S32 determines whether the browser is listening for a connection to a preset communication protocol on the port specified by the preset remote debugging port parameter.

[0102] Specifically, when the browser is listening for a connection using the preset communication protocol on the port specified by the preset remote debugging port parameter, it indicates that the browser is running in the background in headless mode and is capable of accessing web pages. When the browser is not listening for a connection using the preset communication protocol on the port specified by the preset remote debugging port parameter, it is impossible to determine whether the browser is running in headless mode in the background and it is impossible to establish a connection using the preset communication protocol with the browser, thus making it impossible to control the browser to access preset web pages.

[0103] S33 establishes a connection with the browser via a preset communication protocol through a port specified by preset remote debugging parameters.

[0104] When a browser listens for connections using a preset communication protocol on the port specified by the preset remote debugging port parameter, a preset communication protocol connection can be established with the browser on the port specified by the preset remote debugging port parameter, such as establishing a Web socket connection. Specifically, a client can be created, and the client can be used to attempt to establish a preset communication protocol connection with the browser on the port specified by the preset remote debugging port parameter.

[0105] When the browser listens for connections using the preset communication protocol on the port specified by the preset remote debugging port parameter, the access process to the preset webpage ends.

[0106] S34 connects and controls the browser to access preset web pages via a preset communication protocol.

[0107] After establishing a connection with the browser using a preset communication protocol, you can control the browser through the established connection on the port specified by the preset remote debugging port parameter, thereby enabling access to preset web pages. In other words, you can control the browser to access each URL in the configuration file through the established connection on the port specified by the preset remote debugging port parameter.

[0108] S35, Preset whether the webpage needs access permission verification.

[0109] In controlling the browser to access any preset webpage through URL information, it is necessary to determine whether access to each preset webpage is restricted, that is, whether each preset webpage requires access permission verification, thereby determining whether the preset webpage can be accessed. For any preset webpage, if the preset webpage does not require access permission verification, step S36 is executed; if the preset webpage requires access permission verification, step S37 is executed.

[0110] S36, access the preset webpage and obtain the webpage content of the preset webpage.

[0111] S37, obtain the permission verification data of the preset webpage, access the preset webpage according to the permission verification data and obtain the webpage content.

[0112] For any given webpage that requires access permission verification, permission verification data can be loaded from the configuration file. This allows the browser to re-access the webpage based on the URL information and retrieve the webpage content.

[0113] In some feasible implementations, for each preset webpage, when obtaining the webpage content of the preset webpage, the document object model (DOM) node tree of the preset webpage can be obtained, and then the webpage content of the preset webpage can be obtained from the webpage content corresponding to each document object model node in the document object model node tree.

[0114] In other words, after obtaining the document object model node tree of the preset webpage, the webpage content corresponding to some or all of the document object model nodes in the document object model node tree can be obtained, thus obtaining the webpage content obtained from the preset webpage.

[0115] By retrieving webpage content from a pre-defined webpage using a document object model node tree, irrelevant content can be effectively removed, thereby obtaining the core content of the webpage and improving the efficiency and effective content retrieval rate.

[0116] In the Document Object Model (DOM) node tree, nodes represent objects on a webpage. These DOM nodes are of various types, including element nodes, text nodes, attribute nodes, and document nodes. They are organized hierarchically, forming a tree structure. Each DOM node can have multiple child nodes, and each child node has a parent node. This hierarchical structure allows browsers to navigate and easily access various content elements within the webpage.

[0117] Specifically, for each preset webpage, after obtaining the document object model node tree of the preset webpage, the document object model nodes of the document object model node tree can be traversed sequentially from the document node of the document object model node tree based on the preorder traversal method, and the webpage content corresponding to each traversed document object model node can be obtained.

[0118] Optionally, for each preset webpage, after obtaining the document object model node tree of the preset webpage, it can be determined whether the selector parameter of the preset webpage has been obtained.

[0119] The selector parameters can be included in the aforementioned configuration file. In the process of obtaining web page content in this application, the selector parameters are used to locate or select specific document object model nodes. Selector parameters are typically used to query and manipulate specified web page content, such as text content and image content.

[0120] Based on this, for each preset webpage, given the selector parameters of that webpage, the document object model (DOM) node specified by the selector parameters in the document object model (DOM) node tree can be determined, and the webpage content corresponding to the specified DOM node can be obtained. Specifically, the selector parameters can be injected into JavaScript to search for the specified DOM node, and the webpage content corresponding to the specified DOM node can be obtained through the proto.DOMGetOuterHTML instruction in the CDP protocol.

[0121] As an example, see Figure 4 , Figure 4 This is one of the schematic diagrams illustrating a scenario for retrieving webpage content based on a document object model node tree, as provided in this application embodiment. For example... Figure 4 As shown, suppose the document object model node tree of a predefined webpage is as follows: Figure 4 As shown, this includes Document Object Model (DOM) nodes B1 to B8. If the selector parameters for the preset webpage are obtained, and the DOM nodes specified by the selector parameters are DOM nodes B4 to B7, then the webpage content corresponding to DOM nodes B4 to B7 can be obtained.

[0122] If the selector parameters for the preset webpage are not available, a preorder traversal can be used to iterate through the Document Object Model (DOM) node tree, starting from the body node, and retrieve the webpage content corresponding to each traversed DOM node. Specifically, the proto.DOMGetOuterHTML directive in the CDP protocol can be used to obtain the webpage content corresponding to all DOM nodes starting from the body node.

[0123] In the Document Object Model (DOM) node tree, the body node is an important component of a webpage, representing the main content of the page. The body node and its child nodes contain all the content displayed in the browser, such as text, images, and tables.

[0124] As an example, see Figure 5 , Figure 5 This is the second schematic diagram of a scenario for retrieving webpage content based on a document object model node tree, provided in an embodiment of this application. Assume... Figure 4 This is a partial document object model (Document Object Model) node tree of a preset webpage. Document object model node A1 is the body node in the document object model node tree of this preset webpage. If the selector parameters for this preset webpage are not obtained, the process can start from document object model node A1, traversing A1, then A2, and then A5. If document object model node A5 has no child nodes, the process restarts from A1, traversing A3, then A4, thus completing the document object model node traversal. Furthermore, during the traversal, the webpage content corresponding to each traversed document object model node can be obtained in real time; that is, the final obtained webpage content includes the content corresponding to document object model nodes A1 to A4.

[0125] It is worth noting that when accessing each preset webpage, the webpage content can also be retrieved from the browser based on the CDP protocol. Furthermore, preset webpages based on URL information do not rely on third-party HTTP interfaces, which helps maintain the stability and efficiency of webpage content retrieval.

[0126] In some feasible implementations, since the web page content obtained based on the document object model node tree includes information unrelated to building knowledge documents, such as the layout information of the Cascading Style Sheets (CSS) and the color information of the web page content, in order to simplify the obtained web page content, while retaining only the font and title size and removing the layout, color and other information, the obtained web page content can be converted into lightweight markup language format after obtaining the web page content of each preset web page, that is, the final obtained web page content.

[0127] Cascading Style Sheets (CSS) is a language used to describe the presentation style of web page content. It is mainly used to control the layout, font, color, spacing and other visual effects of web pages to make them more aesthetically pleasing.

[0128] This includes converting the obtained webpage content into Markdown format.

[0129] Markdown is a lightweight markup language that allows documents to be written in an easy-to-read and easy-to-write plain text format. Markdown can preserve basic web page content such as different levels of headings, paragraph content, fonts (italic, bold, and bold italic, etc.), ordered and unordered lists, links, images, code blocks, and tables.

[0130] Based on the above methods for obtaining webpage content, even when the webpage does not provide a content retrieval interface, it is possible to effectively control the browser to retrieve webpage content at the document object model node level through a headless browser access method and a communication method based on the CDP protocol. Furthermore, these methods can serve as general-purpose webpage content retrieval methods, enabling a wider range of application scenarios.

[0131] The following is combined with Figure 6 The process of accessing a preset page is illustrated with a specific example. Figure 6 This is a schematic diagram of the process for accessing a preset page provided in an embodiment of this application. For example... Figure 6 As shown, the process of accessing the preset page may include the following steps: S61, when the first instruction for building the knowledge document is received, the configuration file is retrieved.

[0132] When the first instruction for building a knowledge document is received, configuration information can be obtained from the sender of the first instruction.

[0133] The configuration file includes headless parameters, preset remote debugging port parameters, and multiple URLs, with each URL corresponding to a preset webpage.

[0134] S62 launches the browser in headless mode based on the headless parameters and preset remote debugging port parameters in the configuration file.

[0135] Before accessing multiple preset web pages, the browser can be launched in headless mode based on the headless parameter and the preset remote-debugging-port parameter, allowing access to preset web pages without displaying them.

[0136] S63 determines whether the browser is listening for Web socket connections on the port specified by the preset remote debugging port parameter.

[0137] When a browser is listening for Web socket connections on the port specified by the preset remote debugging port parameter, it indicates that the browser is running in the background in headless mode, and the browser has the necessary conditions to establish a Web socket connection. When the browser is not listening for Web socket connections on the port specified by the preset remote debugging port parameter, the browser does not have the necessary conditions to establish a Web socket connection. Since a Web socket connection cannot be established with the browser, the default webpage access process can be terminated.

[0138] S64 establishes a Web socket connection with a browser via a port specified by preset remote debugging parameters.

[0139] When a browser is listening for Web socket connections on the port specified by the preset remote debugging port parameter, that is, when the browser has the conditions to establish a Web socket connection, a Web socket connection can be established with the browser through the port specified by the preset remote debugging port parameter.

[0140] S65 controls browser access to URL information in a configuration file via a Web socket connection.

[0141] Once a WebSocket connection is successfully established, the browser can be controlled via the WebSocket connection to access preset pages in a headless mode. For example, the proto.PageNavigate directive under the CDP protocol can be used to access each URL in the configuration file one by one.

[0142] S66, during the access process, determine whether the preset webpage requires access permission verification.

[0143] During the process of accessing each URL, it is necessary to determine whether the preset webpage corresponding to each URL requires access permission verification. If access permission verification is required, then execute S68. If access permission verification is not required, then the preset webpage corresponding to the URL can be accessed directly in the manner shown in S67.

[0144] S67, directly access the preset webpage corresponding to the URL information if the preset webpage does not require access permission verification.

[0145] S68: If a preset webpage requires access permission verification, retrieve cookie data from the configuration file, and access the preset webpage corresponding to the URL information after passing access permission verification based on the cookie.

[0146] When a preset webpage requires access permission verification, the cookie data corresponding to that preset webpage can be obtained from the configuration file. Access permission verification is then performed based on the cookie data. In other words, the cookie data enables the server corresponding to the preset webpage to identify the browser's identity and maintain the browser's session state, thereby enabling access to the corresponding preset webpage through access permission verification.

[0147] In some feasible implementations, after generating multiple knowledge documents based on a large language model, the association between the various knowledge documents corresponding to the same content fragment can be further established, so that when searching for and retrieving one of the knowledge documents, the other related knowledge documents can be searched or retrieved at the same time.

[0148] The relationships between different knowledge documents corresponding to the same content fragment can be established through unique identifiers or through associated storage, without any restrictions.

[0149] Optionally, when establishing the association between various knowledge documents corresponding to the same content fragment, retrieval-augmented generation (RAG) can be performed on various knowledge documents corresponding to the same content fragment. This maps the independent vector feature representations of each knowledge document into a set of multi-dimensional vector feature representations, where each dimension of the vector feature representation corresponds to different knowledge documents of the same content fragment.

[0150] Alternatively, through retrieval-enhanced generation (RAG) processing, knowledge documents corresponding to the same content fragment can be integrated to obtain the final knowledge document. In the field of artificial intelligence, RAG technology plays a role in various natural language processing tasks, including question-answering systems, document generation, intelligent assistants, information retrieval, and knowledge graph filling.

[0151] In this process, after establishing the relationships between instruction documents, or after integrating all knowledge documents corresponding to the same content fragment, a knowledge document set can be constructed based on the final knowledge documents.

[0152] For each content fragment, knowledge expansion through a large model can generate knowledge documents that are knowledge-related to that content fragment. This step not only enriches the information content of the original web page, but also enhances the depth and breadth of the content fragment, making the constructed knowledge document set more comprehensive and detailed, and more effectively catering to question-and-answer scenarios.

[0153] In some feasible implementations, after the knowledge document is generated, further information processing can be performed based on the knowledge document.

[0154] Specifically, when a question is obtained from the questioner to the large model, the target knowledge document that is knowledge-related to the question can be determined from the knowledge document set. That is, the target knowledge document that is knowledge-related to the question can be determined from the aforementioned knowledge documents.

[0155] Specifically, when a question is obtained from the questioner's statement to the large model, the large model can search for target knowledge documents related to the question and then generate response content based on the target knowledge documents.

[0156] Specifically, when generating the response content for the question statement based on the target knowledge document, a third prompt message can be generated based on the target knowledge document and the question statement. The third prompt message is used to prompt the large model to generate the response content for the question statement based on the target knowledge document.

[0157] Specifically, after determining the third prompt information, the target knowledge document can be embedded into the third prompt information. The large model interface is then called so that the large model performs semantic analysis on the question statement according to the instructions of the third prompt information, and answers the question statement based on the semantic analysis results and the target knowledge document to obtain the response content.

[0158] The third prompt message can be a preset prompt template. It can call the large model interface to perform semantic analysis on the question statement input into the large model, and generate the response content of the question statement based on the semantic analysis results and the target knowledge document.

[0159] In some feasible implementations, when obtaining target knowledge documents associated with the question statement from a knowledge document set, if no association is established between the knowledge documents corresponding to the same content fragment in the knowledge document set, the feature similarity between the question statement and each knowledge document in the knowledge document set can be determined, and then the knowledge documents with feature similarity higher than a preset threshold can be identified as target knowledge documents associated with the question statement.

[0160] Optionally, if the knowledge documents corresponding to the same content fragment in the knowledge document set have been associated, the feature similarity between the question statement and each knowledge document in the knowledge document set can be determined. The knowledge document with a feature similarity higher than a preset threshold is identified as the first knowledge document, and the second knowledge document associated with the first knowledge document in the knowledge document set is identified. Then, the first knowledge document and the second knowledge document are identified as the target knowledge document associated with the question statement, and the target knowledge document is obtained from the knowledge document set.

[0161] As an example, if the association between various knowledge documents corresponding to the same content fragment is achieved through a unique identifier or through associated storage, then after determining the knowledge document associated with the question statement from the knowledge document set based on feature similarity, the knowledge document and all other knowledge documents corresponding to the same unique identifier or associated storage are determined as the target knowledge document associated with the question statement, and the target knowledge document is obtained from the knowledge document set.

[0162] As an example, if the relationships between various knowledge documents corresponding to the same content fragment are represented as multi-dimensional vector features after retrieval enhancement processing, then when obtaining the target knowledge document, the knowledge documents corresponding to the multi-dimensional feature vectors whose feature similarity to the question statement in any dimension exceeds a preset threshold can be identified as the target knowledge document and obtained.

[0163] When a question statement is obtained, the system can quickly retrieve the target knowledge document associated with the question statement from the knowledge document collection, and generate the response content based on the target knowledge document. This helps improve the accuracy and relevance of the response and can also adapt to a wider range of information processing needs.

[0164] The following is combined with Figure 7 The information processing method based on large models provided in this application will be further explained. Figure 7 This is another schematic diagram of the information processing method based on a large model provided in the embodiments of this application. For example... Figure 7 As shown, the information processing method based on a large model provided in this application embodiment can also be described using server execution as an example. The method may include the following steps: S701: When the first instruction for building a knowledge document is received, the browser is launched in headless mode.

[0165] Upon receiving the first instruction to build a knowledge document, the browser can be launched in a headless mode, allowing control over access to preset web pages without displaying the webpage.

[0166] After obtaining the first instruction, a configuration file can be obtained. This configuration file can be pre-configured or carried in the first instruction; there is no restriction on this.

[0167] The configuration file includes the URL information of multiple preset web pages, and the URL information of each preset web page can be a URL.

[0168] The configuration file can also include a headless parameter and a preset remote-debugging-port parameter, which can be used to launch the browser in a headless mode.

[0169] S702: When the browser is listening for a connection using a preset communication protocol on the port specified by the preset remote debugging port parameter, a connection using the preset communication protocol is established with the browser through the port specified by the preset remote debugging parameter.

[0170] When a browser is listening for a connection using the preset communication protocol on the port specified by the preset remote debugging port parameter, it indicates that the browser is running in the background in headless mode and is capable of accessing web pages. When the browser is not listening for a connection using the preset communication protocol on the port specified by the preset remote debugging port parameter, it is impossible to determine whether the browser is running in headless mode in the background and it is impossible to establish a connection using the preset communication protocol, thus making it impossible to control the browser to access preset web pages.

[0171] S703 connects to and controls the browser to access preset web pages via a preset communication protocol.

[0172] When a browser listens for connections using a preset communication protocol on the port specified by the preset remote debugging port parameter, a preset communication protocol connection can be established with the browser on the port specified by the preset remote debugging port parameter, such as establishing a Web socket connection. Specifically, a client can be created, and the client can be used to attempt to establish a preset communication protocol connection with the browser on the port specified by the preset remote debugging port parameter.

[0173] When the browser listens for connections using the preset communication protocol on the port specified by the preset remote debugging port parameter, the access process to the preset webpage ends.

[0174] S704: When any preset webpage requires access permission verification, obtain the permission verification data of the preset webpage and access the preset webpage according to the permission verification data.

[0175] In controlling a browser to access any preset webpage using URL information, it's necessary to determine whether access to each preset webpage is restricted, i.e., whether access permission verification is required, to determine if access to the preset webpage is permissible. For any given preset webpage, if access permission verification is not required, the browser can access the webpage and retrieve its content. If access permission verification is required, permission verification data can be loaded from a configuration file, and the browser can then be controlled to re-access the preset webpage and retrieve its content based on the URL information using the permission verification data.

[0176] The aforementioned access permission verification method can be Cookie verification, and the corresponding access permission verification data can be Cookie data.

[0177] S705, if the selector parameter of the preset webpage is obtained, determine the document object model node specified by the selector parameter in the document object model node tree, and obtain the webpage content corresponding to the specified document object model node.

[0178] For each preset webpage, given the selector parameters for that webpage, the document object model (DOM) node specified by the selector parameters in the document object model (DOM) node tree can be determined, and the webpage content corresponding to the specified DOM node can be obtained. Specifically, the selector parameters can be injected into JavaScript to search for the specified DOM node, and the webpage content corresponding to the specified DOM node can be obtained through the proto.DOMGetOuterHTML directive in the CDP protocol.

[0179] S706, if the selector parameter of the preset webpage is not obtained, the node traversal is performed starting from the main node of the document object model node tree by preorder traversal, and the webpage content corresponding to each document object model node traversed is obtained.

[0180] If the selector parameters for the preset webpage are not available, a preorder traversal can be used to iterate through the Document Object Model (DOM) node tree, starting from the body node, and retrieve the webpage content corresponding to each traversed DOM node. Specifically, the proto.DOMGetOuterHTML directive in the CDP protocol can be used to obtain the webpage content corresponding to all DOM nodes starting from the body node.

[0181] S707 converts the acquired web page content into a lightweight markup language format.

[0182] Since the webpage content obtained based on the document object model node tree includes information such as webpage layout and color information that is not related to the construction of knowledge documents, in order to simplify the obtained webpage content, while retaining only the font and title size and removing layout, color and other information, the obtained webpage content can be converted into lightweight markup language format after obtaining the webpage content of each preset webpage, which is the final obtained webpage content.

[0183] The lightweight markup language format can be Markdown, and there are no restrictions on it.

[0184] S708, generate first prompt information based on the webpage content of each preset webpage, and based on the first prompt information, call the large model to perform semantic analysis on the webpage content of the preset webpage, and aggregate the semantically related webpage content in the preset webpage into the same content fragment based on the semantic analysis results.

[0185] After obtaining the webpage content of each preset webpage, a first prompt message can be generated based on the webpage content of that preset webpage.

[0186] The first prompt message is used to instruct the large model to segment the webpage content of the preset webpage according to semantics.

[0187] In this case, based on the first prompt information, the large model can be invoked to perform semantic analysis on the webpage content of the preset webpage, and then the webpage content of the preset webpage can be segmented according to the semantic analysis results to obtain at least one content fragment.

[0188] In other words, by processing the webpage content of the preset webpage using a large model, multiple content fragments with different semantics can be obtained.

[0189] Specifically, after determining the first prompt information, the webpage content of the preset webpage can be embedded into the first prompt information. The large model interface is called so that the large model performs semantic analysis on the webpage content of the preset webpage according to the instructions of the first prompt information, and aggregates the semantically related webpage content in the preset webpage into the same content fragment based on the semantic analysis results.

[0190] S709: Generate a second prompt message for each content fragment; based on the second prompt message, call the large model to perform content analysis on the content fragment; and based on the content analysis results, expand the knowledge of the content fragment in different knowledge dimensions to generate multiple knowledge documents that are knowledge-related to the content fragment.

[0191] After obtaining content fragments of web page content, a large model can be used to perform content analysis on these fragments. Then, based on the results of the content analysis, knowledge expansion can be performed on the content fragments in different knowledge dimensions, thereby generating multiple knowledge documents that are knowledge-related to the content fragments.

[0192] Optionally, the above-mentioned indication dimensions include, but are not limited to, at least one of the following: The content browsing target audience includes potential questions and corresponding answers related to the content segment. Information about the context in which the content fragment is applied by the content browsing object; The problem addressed by this content fragment; The main theme information of this content segment; This causes the content viewer to be unable to understand the key information in the content fragment; The content after rewriting and / or polishing the content fragment.

[0193] For each content fragment, a second prompt can be generated based on that content fragment. The second prompt is used to instruct the large model to generate a knowledge document that has a knowledge-related connection with that content fragment.

[0194] In this case, based on the second prompt, the large model can be invoked to perform content analysis on the content fragment, and then the content fragment can be expanded with knowledge in different knowledge dimensions based on the content analysis results to generate a knowledge document that has a knowledge relationship with the content fragment.

[0195] S710: Construct a knowledge document set based on the knowledge document corresponding to each content fragment.

[0196] After generating multiple knowledge documents based on the large language model, the relationships between the various knowledge documents corresponding to the same content fragment can be further established, such as by using unique identifiers or by using associated storage, without any restrictions.

[0197] Optionally, when establishing the association between various knowledge documents corresponding to the same content fragment, the various knowledge documents corresponding to the same content fragment can be subjected to RAG, which maps the independent vector feature representations of each knowledge document into a set of multi-dimensional vector feature representations, where each dimension of the vector feature representation corresponds to different knowledge documents of the same content fragment.

[0198] Alternatively, by using retrieval-enhanced generation processes, knowledge documents corresponding to the same content fragment can be integrated to obtain the final knowledge document.

[0199] In this process, after establishing the relationships between instruction documents, or after integrating all knowledge documents corresponding to the same content fragment, a knowledge document set can be constructed based on the final knowledge documents.

[0200] S711, when the question statement of the questioning object is obtained, the target knowledge document that has a knowledge relationship with the question statement is determined from the knowledge document set; through the large model, the response content of the question statement is generated according to the target knowledge document, and the response content is returned to the corresponding questioning object.

[0201] When the question statement of the questioner is obtained and asked by the questioner to the large model, the target knowledge document that has a knowledge relationship with the question statement can be determined from the knowledge document set. That is, the target knowledge document that has a knowledge relationship with the question statement can be determined from the aforementioned knowledge documents, and then the response content can be generated based on the target knowledge document.

[0202] Specifically, when generating the response content for the question statement based on the target knowledge document, a third prompt message can be generated based on the target knowledge document and the question statement. The third prompt message is used to prompt the large model to generate the response content for the question statement based on the target knowledge document.

[0203] Specifically, after determining the third prompt information, the target knowledge document can be embedded into the third prompt information. The large model interface is then called so that the large model performs semantic analysis on the question statement according to the instructions of the third prompt information, and answers the question statement based on the semantic analysis results and the target knowledge document to obtain the response content.

[0204] The third prompt message can be a preset prompt template. It can call the large model interface to perform semantic analysis on the question statement input into the large model, and generate the response content of the question statement based on the semantic analysis results and the target knowledge document.

[0205] In this embodiment, the information embedding process, such as embedding webpage content into the first prompt information or embedding the target knowledge document into the third prompt information, refers to the process of mapping objects (such as words, images, or any type of text) to real-valued vectors. Embedding is a representation learning technique used to transform high-dimensional or structured data into low-dimensional, dense vector forms. The purpose of this transformation is to preserve key information in the data while reducing dimensionality, so that machine learning models can process it more efficiently.

[0206] The embedding methods involved in the embodiments of this application include, but are not limited to, word embedding, deep learning embedding, autoencoders, and graph embedding. The specific method can be determined based on the actual application scenario requirements and is not limited here.

[0207] The knowledge document set provided in this application embodiment can be stored in a preset storage space, which includes but is not limited to cloud storage, database, blockchain, etc. The specific storage space can be determined based on the actual application scenario requirements and is not limited here.

[0208] It should be specifically noted that the collection and processing of relevant web page content in this application should strictly comply with the requirements of relevant national laws and regulations, obtain informed consent or separate consent from the relevant parties, and conduct subsequent data use and processing within the scope of laws, regulations, and the authorization of the relevant parties. It is understood that in specific embodiments of this application, when web page content is used in specific products or technologies, permission or consent from the content publisher is required, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0209] See Figure 8 , Figure 8 This is a schematic diagram of the structure of the large-model-based information processing device provided in this application embodiment. The large-model-based information processing device provided in this application embodiment includes: The content acquisition module 801 is used to acquire the web page content of multiple preset web pages when a first instruction for constructing a knowledge document is received. The information processing module 802 is used to perform semantic analysis on the web page content of each of the above-mentioned preset web pages through a large model, and to aggregate the web page content with semantic relationship into the same content fragment based on the semantic analysis results. The aforementioned information processing module is used to perform content analysis on each of the aforementioned content fragments using the aforementioned large model, and to expand the knowledge of the content fragments in different knowledge dimensions based on the content analysis results, thereby generating multiple knowledge documents that are knowledge-related to the content fragments.

[0210] In some feasible implementations, for each of the aforementioned preset web pages, when the information processing module 802 segments the web page content of the preset web page according to semantics using a large model to obtain multiple content fragments, it is used for: The first prompt information is generated based on the webpage content of the preset webpage. The first prompt information is used to instruct the large model to segment the webpage content of the preset webpage according to semantics. Based on the first prompt information mentioned above, the aforementioned large model is invoked to perform semantic analysis on the webpage content of the preset webpage, and based on the semantic analysis results, the webpage content with semantic relationships in the preset webpage is aggregated into the same content fragment.

[0211] In some feasible implementations, for each of the aforementioned content fragments, when the information processing module 802 performs content analysis on the content fragment using the aforementioned large model, and expands the content fragment with different knowledge dimensions based on the content analysis results to generate multiple knowledge documents that are knowledge-related to the content fragment, it is used for: A second prompt message is generated based on the content fragment. The second prompt message is used to instruct the large model to generate a knowledge document that is knowledge-related to the content fragment. Based on the second prompt information above, the above-mentioned large model is invoked to perform content analysis on the content fragment, and based on the content analysis results, knowledge expansion of the content fragment in different knowledge dimensions is performed to generate multiple knowledge documents that are knowledge-related to the content fragment.

[0212] In some feasible implementations, the information processing module 802 is further used for: When a question is asked by the questioner to the above-mentioned large model, the target knowledge document that has a knowledge-related relationship with the question is determined from the knowledge document set; wherein, the knowledge document set includes all knowledge documents generated based on each of the above-mentioned content fragments; Using the aforementioned large model, the response content for the aforementioned question is generated based on the aforementioned target knowledge document, and the response content is returned to the corresponding questioner.

[0213] In some feasible implementations, the information processing module 802 is further used for: For each of the above content fragments, a retrieval enhancement and generation process is performed on the knowledge documents to establish the relationship between the knowledge documents of that content fragment. When the information processing module 802 identifies a target knowledge document from the knowledge document set that has a knowledge-related association with the above-mentioned question statement, it is used for: Determine the feature similarity between the above-mentioned question statement and each knowledge document in the above-mentioned knowledge document set; Knowledge documents with feature similarity higher than a preset threshold are identified as first knowledge documents, and second knowledge documents associated with the first knowledge document in the knowledge document set are identified based on the association between the knowledge documents in the knowledge document set. The first knowledge document and the second knowledge document mentioned above are identified as target knowledge documents that have a knowledge-based association with the above question statement.

[0214] In some feasible implementations, when the information processing module 802 generates the response content for the question statement based on the target knowledge document using the large model, it is used for: Based on the aforementioned target knowledge document and the aforementioned question statement, a third prompt message is generated. The aforementioned third prompt message is used to instruct the aforementioned large model to generate the response content of the aforementioned question statement based on the aforementioned target knowledge document. Based on the third prompt information mentioned above, the aforementioned large model is invoked. The large model performs semantic analysis on the aforementioned question statement and generates the response content for the aforementioned question statement based on the semantic analysis results and the aforementioned target knowledge document.

[0215] In some feasible implementations, when the content acquisition module 801 acquires the webpage content of multiple preset webpages, it is used to: Obtain the configuration file, which includes multiple URLs. The browser is launched in a headless mode based on the headless parameters and the preset remote debugging port parameters. After establishing a preset communication protocol connection with the browser, the browser is controlled to access the preset webpages corresponding to each of the above-mentioned URLs and to obtain the webpage content of the corresponding preset webpages through the communication protocol connection.

[0216] In some feasible implementations, the content acquisition module 801 is further used for: In the process of obtaining the web page content of multiple preset web pages, when any of the preset web pages requires access permission verification, the permission verification data of the preset web page is obtained from the configuration file, and the browser is controlled to access the preset web page according to the permission verification data.

[0217] In some feasible implementations, for each of the aforementioned preset web pages, when the content acquisition module 801 acquires the web page content of the preset web page, it is used to: Obtain the document object model tree of the preset webpage; The webpage content of the preset webpage is obtained from the webpage content corresponding to each document object model node in the above document object model node tree.

[0218] In some feasible implementations, for each of the aforementioned preset web pages, when the content acquisition module 801 acquires the web page content of the preset web page from the web page content corresponding to each document object model node in the document object model node tree, it is used to: If the selector parameter of the preset webpage is obtained, the document object model node specified by the selector parameter in the document object model node tree is determined, and the webpage content corresponding to the specified document object model node is obtained. If the selector parameter of the preset webpage is not obtained, the nodes are traversed starting from the main node of the above document object model node tree by preorder traversal, and the webpage content corresponding to each document object model node traversed is obtained.

[0219] In some feasible implementations, the content acquisition module 801 is further used for: The obtained web page content is converted into web page content in a lightweight markup language format.

[0220] In specific implementation, the aforementioned information processing device based on a large model can perform the above-described functions through its built-in functional modules. Figure 2 and / or Figure 6 The implementation methods provided for each step are detailed in the above-mentioned implementation methods and will not be repeated here.

[0221] See Figure 9 , Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 9 As shown, the electronic device 900 in this embodiment may include: a processor 901, a network interface 904, and a memory 905. Furthermore, the electronic device 900 may also include: an object interface 903, and at least one communication bus 902. The communication bus 902 is used to implement communication between these components. The object interface 903 may include a display screen and a keyboard; optionally, the object interface 903 may also include a standard wired interface or a wireless interface. The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 905 may be a high-speed RAM or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory 905 may also be at least one storage device located remotely from the aforementioned processor 901. Figure 9 As shown, the memory 905, which is a computer-readable storage medium, may include an operating system, a network communication module, an object interface module, and a device control application.

[0222] exist Figure 9 In the illustrated electronic device 900, the network interface 904 provides network communication functionality; the object interface 903 is primarily used to provide an input interface for objects; and the processor 901 can be used to call the device control application program stored in the memory 905 to achieve: In some feasible implementations, the processor 901 described above is used for: When the first instruction for building a knowledge document is received, the web page content of multiple preset web pages is retrieved; The large model is used to perform semantic analysis on the webpage content of each of the above-mentioned preset webpages, and webpage content with semantic relationships is aggregated into the same content fragment based on the semantic analysis results. For each of the above content fragments, the content fragment is analyzed using the aforementioned large model, and based on the results of the content analysis, the content fragment is expanded with knowledge in different knowledge dimensions to generate multiple knowledge documents that are knowledge-related to the content fragment.

[0223] In some feasible implementations, for each of the aforementioned preset web pages, the processor 901 performs semantic analysis on the web page content of the preset web page using a large model, and when aggregating semantically related web page content in the aforementioned preset web page into the same content fragment based on the semantic analysis results, it is used for: The first prompt information is generated based on the webpage content of the preset webpage. The first prompt information is used to instruct the large model to segment the webpage content of the preset webpage according to semantics. Based on the first prompt information mentioned above, the aforementioned large model is invoked to perform semantic analysis on the webpage content of the preset webpage, and based on the semantic analysis results, the webpage content with semantic relationships in the preset webpage is aggregated into the same content fragment.

[0224] In some feasible implementations, for each of the aforementioned content fragments, when the processor 901 performs content analysis on the content fragment using the aforementioned large model, and expands the content fragment with different knowledge dimensions based on the content analysis results to generate multiple knowledge documents that are knowledge-related to the content fragment, it is used for: A second prompt message is generated based on the content fragment. The second prompt message is used to instruct the large model to generate a knowledge document that is knowledge-related to the content fragment. Based on the second prompt information above, the above-mentioned large model is invoked to perform content analysis on the content fragment, and based on the content analysis results, knowledge expansion of the content fragment in different knowledge dimensions is performed to generate multiple knowledge documents that are knowledge-related to the content fragment.

[0225] In some feasible implementations, the processor 901 is further configured to: When a question is asked by the questioner to the above-mentioned large model, the target knowledge document that has a knowledge-related relationship with the question is determined from the knowledge document set; wherein, the knowledge document set includes all knowledge documents generated based on each of the above-mentioned content fragments; Using the aforementioned large model, the response content for the aforementioned question is generated based on the aforementioned target knowledge document, and the response content is returned to the corresponding questioner.

[0226] In some feasible implementations, the processor 901 is further configured to: For each of the above content fragments, a retrieval enhancement and generation process is performed on the knowledge documents to establish the relationship between the knowledge documents of that content fragment. Determine the feature similarity between the above-mentioned question statement and each knowledge document in the above-mentioned knowledge document set; Knowledge documents with feature similarity higher than a preset threshold are identified as first knowledge documents, and second knowledge documents associated with the first knowledge document in the knowledge document set are identified based on the association between the knowledge documents in the knowledge document set. The first knowledge document and the second knowledge document mentioned above are identified as target knowledge documents that have a knowledge-based association with the above question statement.

[0227] In some feasible implementations, when the processor 901 generates the response content for the question statement based on the target knowledge document using the large model, it is used for: Based on the aforementioned target knowledge document and the aforementioned question statement, a third prompt message is generated. The aforementioned third prompt message is used to instruct the aforementioned large model to generate the response content of the aforementioned question statement based on the aforementioned target knowledge document. Based on the third prompt information mentioned above, the aforementioned large model is invoked. The large model performs semantic analysis on the aforementioned question statement and generates the response content for the aforementioned question statement based on the semantic analysis results and the aforementioned target knowledge document.

[0228] In some feasible implementations, when the processor 901 acquires the webpage content of multiple preset webpages, it is used to: Obtain the configuration file, which includes multiple URLs. The browser is launched in a headless mode based on the headless parameters and the preset remote debugging port parameters. After establishing a preset communication protocol connection with the browser, the browser is controlled to access the preset webpages corresponding to each of the above-mentioned URLs and to obtain the webpage content of the corresponding preset webpages through the communication protocol connection.

[0229] In some feasible implementations, the processor 901 is further configured to: In the process of obtaining the web page content of multiple preset web pages, when any of the preset web pages requires access permission verification, the permission verification data of the preset web page is obtained from the configuration file, and the browser is controlled to access the preset web page according to the permission verification data.

[0230] In some feasible implementations, for each of the aforementioned preset web pages, when the processor 901 acquires the web page content of the preset web page, it is used to: Obtain the document object model tree of the preset webpage; The webpage content of the preset webpage is obtained from the webpage content corresponding to each document object model node in the above document object model node tree.

[0231] In some feasible implementations, for each of the aforementioned preset web pages, when the processor 901 obtains the web page content of the preset web page from the web page content corresponding to each document object model node in the document object model node tree, it is used to: If the selector parameter of the preset webpage is obtained, the document object model node specified by the selector parameter in the document object model node tree is determined, and the webpage content corresponding to the specified document object model node is obtained. If the selector parameter of the preset webpage is not obtained, the nodes are traversed starting from the main node of the above document object model node tree by preorder traversal, and the webpage content corresponding to each document object model node traversed is obtained.

[0232] In some feasible implementations, the processor 901 is further configured to: The obtained web page content is converted into web page content in a lightweight markup language format.

[0233] It should be understood that in some feasible implementations, the processor 901 described above may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0234] In specific implementation, the aforementioned electronic device 900 can perform the above-described actions through its built-in functional modules. Figure 2 and / or Figure 6 The implementation methods provided for each step are detailed in the above-mentioned implementation methods and will not be repeated here.

[0235] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement... Figure 2 and / or Figure 6 The methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.

[0236] The aforementioned computer-readable storage medium can be an internal storage unit of the large-model-based information processing apparatus or electronic device provided in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. The aforementioned computer-readable storage medium can also include magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0237] This application provides a computer program product, which includes a computer program that is executed by a processor. Figure 2 and / or Figure 6 The methods provided for each step in the process.

[0238] The terms "first," "second," etc., used in the claims, description, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or electronic devices. References to "embodiment" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The presentation of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" as used in this application's description and appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0239] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

[0240] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.< / section> < / section> < / section>

Claims

1. An information processing method based on a large model, characterized in that, The method includes: When the first instruction for building a knowledge document is received, the web page content of multiple preset web pages is retrieved; The large model is used to perform semantic analysis on the webpage content of each preset webpage, and webpage content with semantic relationships is aggregated into the same content fragment based on the semantic analysis results. For each content fragment, the large model is used to perform content analysis on the content fragment, and based on the content analysis results, knowledge expansion is performed on the content fragment in different knowledge dimensions to generate multiple knowledge documents that are knowledge-related to the content fragment.

2. The method according to claim 1, characterized in that, For each of the preset web pages, the process involves performing semantic analysis on the web page content using a large model, and aggregating semantically related web page content into the same content fragment based on the semantic analysis results, including: The first prompt information is generated based on the webpage content of the preset webpage. The first prompt information is used to instruct the large model to segment the webpage content of the preset webpage according to semantics. Based on the first prompt information, the large model is invoked to perform semantic analysis on the webpage content of the preset webpage, and based on the semantic analysis results, the webpage content with semantic relationships in the preset webpage is aggregated into the same content fragment.

3. The method according to claim 1 or 2, characterized in that, For each content fragment, the large model is used to perform content analysis on the content fragment, and based on the content analysis results, knowledge expansion is performed on the content fragment in different knowledge dimensions to generate multiple knowledge documents that are knowledge-related to the content fragment, including: A second prompt message is generated based on the content fragment, and the second prompt message is used to instruct the large model to generate a knowledge document that has a knowledge association with the content fragment; Based on the second prompt, the large model is invoked to perform content analysis on the content fragment, and based on the content analysis results, knowledge expansion of the content fragment is performed on different knowledge dimensions to generate multiple knowledge documents that are knowledge-related to the content fragment.

4. The method according to claim 1, characterized in that, The method further includes: When a question is asked by the questioner to the large model, target knowledge documents that have a knowledge-related relationship with the question are determined from the knowledge document set; wherein, the knowledge document set includes all knowledge documents generated based on each content fragment; Using the large model, a response to the question is generated based on the target knowledge document, and the response is returned to the corresponding questioner.

5. The method according to claim 4, characterized in that, The method further includes: For each content fragment, a retrieval enhancement generation process is performed on the knowledge document to establish the association between the knowledge documents of the content fragment; The step of identifying target knowledge documents from the knowledge document set that have a knowledge-based association with the question statement includes: Determine the feature similarity between the question statement and each knowledge document in the knowledge document set; A knowledge document with a feature similarity higher than a preset threshold is identified as a first knowledge document, and a second knowledge document associated with the first knowledge document in the knowledge document set is identified based on the association relationship between the knowledge documents in the knowledge document set. The first knowledge document and the second knowledge document are identified as target knowledge documents that have a knowledge-based association with the question statement.

6. The method according to claim 4, characterized in that, The step of generating the response content for the question statement based on the target knowledge document using the large model includes: A third prompt message is generated based on the target knowledge document and the question statement. The third prompt message is used to instruct the large model to generate a response to the question statement based on the target knowledge document. The large model is invoked based on the third prompt information. The large model performs semantic analysis on the question statement and generates a response to the question statement based on the semantic analysis results and the target knowledge document.

7. The method according to claim 1, characterized in that, The step of obtaining the webpage content of multiple preset webpages includes: Obtain the configuration file, which includes multiple URLs; The browser is launched in a headless mode based on the headless parameters and the preset remote debugging port parameters. After establishing a preset communication protocol connection with the browser, the browser is controlled to access the preset webpages corresponding to each URL information and obtain the webpage content of the corresponding preset webpages through the communication protocol connection.

8. The method according to claim 7, characterized in that, The method further includes: During the process of acquiring the webpage content of multiple preset webpages, when any of the preset webpages requires access permission verification, the access permission verification data of the preset webpage is obtained from the configuration file, and the browser is controlled to access the preset webpage according to the access permission verification data.

9. The method according to claim 1 or 7, characterized in that, For each preset webpage, obtain the webpage content of that preset webpage, including: Obtain the document object model tree of the preset webpage; The webpage content of the preset webpage is obtained from the webpage content corresponding to each document object model node in the document object model node tree.

10. The method according to claim 9, characterized in that, For each of the preset web pages, obtaining the web page content of the preset web page from the web page content corresponding to each document object model node in the document object model node tree includes: If the selector parameter of the preset webpage is obtained, the document object model node specified by the selector parameter in the document object model node tree is determined, and the webpage content corresponding to the specified document object model node is obtained. If the selector parameters of the preset webpage are not obtained, the nodes are traversed starting from the main node of the document object model node tree through preorder traversal, and the webpage content corresponding to each traversed document object model node is obtained.

11. The method according to claim 1 or 10, characterized in that, The method further includes: The obtained web page content is converted into web page content in a lightweight markup language format.

12. An information processing device based on a large model, characterized in that, The device includes: The content acquisition module is used to acquire the web page content of multiple preset web pages when the first instruction for building a knowledge document is received. The information processing module is used to perform semantic analysis on the webpage content of each preset webpage through a large model, and to aggregate the webpage content with semantic relationship into the same content fragment based on the semantic analysis results. The information processing module is used to perform content analysis on each content fragment using the large model, and to expand the content fragment with knowledge in different knowledge dimensions based on the content analysis results, thereby generating multiple knowledge documents that are knowledge-related to the content fragment.

13. An electronic device, characterized in that, It includes a processor and a memory, which are interconnected; The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 11 when the computer program is invoked.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method of any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images