Enterprise information processing method and device based on large language model, equipment and storage medium

Abstract

The invention relates to the technical field of enterprise product analysis, in particular to an enterprise information processing method and device based on a large language model, equipment and a storage medium. The method comprises the steps of obtaining a target page group of a to-be-analyzed enterprise; performing data cleaning on the parent page and the child page of the same target page group, and obtaining page information of the target page of the to-be-analyzed enterprise after data cleaning; constructing an enterprise product tree based on the page information of all the target pages; performing link analysis on the enterprise product tree to obtain a page source code block corresponding to the to-be-analyzed enterprise; and inputting the page source code block into a preset large language model, and performing product information extraction on the page source code block through the preset large language model to obtain enterprise product information corresponding to the to-be-analyzed enterprise. According to the method, the enterprise product information of the to-be-analyzed enterprise can be obtained, so that the problem that an enterprise product library is difficult to construct is solved, the blank of enterprise product library resources on the market is filled, and the beneficial effects of enabling enterprises, industries and industry analysis are finally realized.

Description

Technical Field

[0001] The present application relates to the technical field of enterprise product analysis, and in particular to an enterprise information processing method, apparatus, device and storage medium based on a large language model. Background Art

[0002] Industry research and industrial chain analysis conducted by financial and government-related institutions are of great significance in promoting economic development, guiding policy formulation, optimizing resource allocation, enhancing international competitiveness, and serving the real economy. The enterprise product library is crucial for industry research and industrial chain analysis. It is used to evaluate the diversity and coverage of enterprise products, provide researchers with comprehensive enterprise knowledge, and help researchers deeply analyze the potential and development prospects of enterprises, thereby providing more detailed enterprise information for government investment promotion.

[0003] Although the construction of enterprise product libraries is crucial for industry research and industry chain analysis, there are difficulties in the acquisition process. From a macro perspective, the resistance comes from the following two aspects: 1. Information is not disclosed in a centralized manner: Whether it is a listed or unlisted company, there is a lack of a unified organization requiring them to disclose detailed product information, resulting in scattered data that is difficult to integrate.

[0004] 2. Diverse official website styles: The designs of corporate official websites vary greatly, making it difficult to extract all product information through unified crawler technology.

[0005] From a technical perspective, building an enterprise product library also faces many obstacles: 1. Limitations of general crawlers: Existing general crawlers are difficult to accurately extract product information directly from complex web page structures and require further analysis and processing based on the web page text level.

[0006] 2. Redundancy of web page text information: When extracting web page text, it often contains a large amount of content that is irrelevant to product information, such as navigation bars, footers, etc. These redundant information not only increases the difficulty of data processing, but may also interfere with the accurate extraction of information.

[0007] 3. Difficulty in locating product pages: In the process of crawling sub-pages step by step starting from the homepage of the company's official website, only a small part of the sub-pages contain the required valid product information. How to accurately locate the page containing valid product information is a tricky problem.

[0008] 4. Limitations of large language models: Although large language models make it possible to generate structured enterprise information, research has shown that some LLMs (Large Language Models) are sensitive to prompts and have the ability to output structured results, but their corresponding understanding and analysis capabilities are insufficient, making it difficult to obtain corresponding structured results simply through prompts.

[0009] Based on this, how to solve the problem of difficulty in building an enterprise product library and fill the gap in enterprise product library resources on the market, so as to empower enterprises, industries and industry analysis, is a technical problem that needs to be solved urgently. Summary of the invention

[0010] In order to overcome the shortcomings of the prior art, the present application provides an enterprise information processing method, device, equipment and storage medium based on a large language model, which is used to solve the problem of difficulty in building an enterprise product library and fill the gap in enterprise product library resources on the market, thereby achieving the beneficial effect of enabling enterprise, industry and industry analysis.

[0011] The technical solution adopted by this application to solve its technical problem is: In a first aspect, the present application provides an enterprise information processing method based on a large language model, the method comprising: Obtain a target page group of the enterprise to be analyzed; one target page group includes a parent page and a child page; Performing data cleaning on the parent page and the child page of the same target page group, and obtaining page information of the target page of the enterprise to be analyzed after data cleaning; the target page is the parent page and the child page; Building an enterprise product tree based on the page information of all the target pages; Performing link analysis on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed; The page source code group is input into a preset large language model, and product information is extracted from the page source code group through the preset large language model to obtain enterprise product information corresponding to the enterprise to be analyzed.

[0012] In a second aspect, the present application provides an enterprise information processing device based on a large language model, comprising: A page cleaning and information acquisition module, used to perform data cleaning on the parent page and the child page of the same target page group, and obtain page information of the target page of the enterprise to be analyzed after data cleaning; the target page is the parent page and the child page; An enterprise product tree construction module, used to construct an enterprise product tree based on the page information of all the target pages; A page source code group acquisition module is used to perform link analysis on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed; The enterprise product information output module is used to input the page source code group into a preset large language model, extract product information from the page source code group through the preset large language model, and obtain the enterprise product information corresponding to the enterprise to be analyzed.

[0013] In a third aspect, the present application provides an electronic device, including: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions, which, when executed by the one or more processors, enable the electronic device to perform the above method.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, in which a program or instruction is stored. When the program or instruction is executed, the above method is implemented.

[0015] This application crawls the target page on an enterprise's official website through crawler technology, and performs data cleaning on the crawled page to reduce data redundancy. Based on the parent-child link relationship between the target pages after data cleaning, the page information of each target page is used to build an enterprise product tree. In this way, the target page that needs to be analyzed in an enterprise can be represented in the form of an enterprise product tree, and the hierarchical relationship between each target page can be clearly displayed. In order to enable the large language model to process the data in the enterprise product tree, this application performs link analysis on the enterprise product tree, and can effectively retrieve the product information of each product from the source to obtain the page source code group, and combine the prompt learning method to enable the large language model to parse the information in the page source code group and output structured enterprise product information. In this way, while reducing the amount of data input into the large language model, it is possible to quickly build an enterprise product library based on structured enterprise product information, fill the gap in enterprise product library resources on the market, and ultimately enable enterprise, industry and industry analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of an enterprise information processing method based on a large language model provided in an embodiment of the present application; Figure 2 It is an example diagram of a web page of an enterprise to be analyzed for explaining the data cleaning step in the enterprise information processing method based on a large language model provided in an embodiment of the present application; Figure 3It is an example diagram of a web page of an enterprise to be analyzed provided in an embodiment of the present application for explaining a data cleaning step when parallel sub-pages exist in an enterprise information processing method based on a large language model; Figure 4 It is a schematic diagram of the output result of the step of outputting the enterprise product information corresponding to the structured prompt words provided in the embodiment of the present application; Figure 5 It is a virtual structural diagram of an enterprise information processing device based on a large language model provided by the present application; Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The present application is further described below in conjunction with the accompanying drawings and embodiments.

[0018] The following will clearly and completely describe the concept, specific structure and technical effects of the present application in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments of the present application, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present application. In addition, all the connection / link relationships involved in the patent do not simply refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features created in this application can be combined interactively without conflicting with each other.

[0019] Reference Figure 1 , Figure 1 : is a flow chart of the enterprise information processing method based on the large language model provided in the embodiment of the present application, which specifically includes the following steps, which are specifically described below: In step S1, a target page group of the enterprise to be analyzed is obtained; one target page group includes a parent page and a child page.

[0020] Specifically, the target page group refers to the set of all target pages under the official website of the enterprise to be analyzed, wherein one target page group includes a parent page and a child page, and the target page refers to all parent pages and child pages. The association between parent pages and child pages is based on the nested hierarchy of HTML (HyperText Markup Language). For example, for the enterprise to be analyzed, it is assumed that there is a complete link of "official website page-product category page-product A page-product A1 page". Through this link, the product information page of "product A1" can be reached based on the link relationship of the web page. Then, there are three target page groups in this link, which are: Target page group 1: the parent page is the official website page, and the child page is the product category page; Target page group 2: the parent page is the product category page, and the child page is the product A page; Target page group 3: The parent page is product A page, and the child page is product A1 page.

[0021] Based on the above examples, it can be seen that the parent page and child page exist independently in each target page group. A child page in one target page group can be the parent page of another target page group, and vice versa. However, it is worth noting that target pages such as "official website page" that do not have corresponding parent pages in the nested level can only be used as parent pages in the target page group. Similarly, target pages such as "product A1 page" that do not have child pages that continue to link downward can only be used as child pages in the target page group.

[0022] More specifically, the embodiment of the present application provides an implementation step of obtaining a target page group, which specifically includes: The official website page of the enterprise to be analyzed is located, and the official website page is used as the parent page in the first target page group.

[0023] Specifically, the official website of the company to be analyzed is found, and the homepage of the official website of the company to be analyzed is used as the official website page. The official website page is then used as the parent page in the first target page group. Since there is no corresponding parent page for the official website page, the first target page group is the starting target page group of the entire process. In layman's terms, the official website page is used as the starting page for subsequent data cleaning and data analysis, so as to gradually clean and process other pages within the official website of the company to be analyzed along its links from the starting page.

[0024] Furthermore, the page corresponding to the page link containing the preset keyword in the official website page is used as a child page in the first target page group and a parent page in the new target page group.

[0025] Specifically, the official website page is usually designed with a navigation menu, which may include link keywords such as products, services, about us, news, and contact information. For obtaining the enterprise product information of the enterprise to be analyzed, pages under links such as "about us", "contact information", and "news" are not applicable, so at least one preset keyword is required to further determine the required target page. Among them, the preset keywords include but are not limited to words such as "products" and "solutions". Specifically, the keywords can be appropriately adjusted according to the enterprise type to be analyzed. For example, for service-oriented enterprises, the preset keywords can be adjusted to "services".

[0026] More specifically, a link containing a preset keyword is found in the determined official website page, and the page pointed to by the link containing the preset keyword will become a subpage of the first target page group, that is, a subpage of the official website page. The page pointed to by the link containing the preset keyword is usually a category page of all products of the enterprise to be analyzed, that is, a product category page.

[0027] More specifically, there are usually multiple links in a product category page, which are used to point to product pages of multiple categories of products. That is, the product category page has a corresponding parent page upward, which forms the first target page with the official website page, and there may be corresponding child pages downward that form a new target page group with the product category page. Therefore, the product category page can be first used as the parent page of the new target page group, and it can be determined in the following steps whether the new target page group is formed.

[0028] Furthermore, it is detected whether the parent page in the new target page group has a corresponding child page.

[0029] Specifically, it is determined whether there is a corresponding sub-page on the product category page to determine whether a new target page group is formed. Specifically, a link search is performed based on the parent page of the new target page group determined in the above steps, and it is determined whether the found link points to a valid sub-page by verifying the link target, etc. If there is at least one valid sub-page, that is, it is determined whether there is a corresponding sub-page on the parent page in the new target page group, and a new target page group can be formed.

[0030] Furthermore, when the parent page in the new target page group does not have a corresponding child page, the first target page group is used as the target page group of the enterprise to be analyzed.

[0031] Specifically, if the parent page in the new target page group does not have a corresponding child page, it means that the official website page does not have a corresponding product category page, and the product information may be displayed directly. In this case, a complete link such as "official website homepage-product page" has been formed, and only the first target page group is obtained.

[0032] Furthermore, when a corresponding child page exists for a parent page in a new target page group, the parent page in the new target page group and the detected child page are combined into a new target page group, and the child page in the new target page group is used as the parent page in the next new target page group, and it is detected whether the parent page in the next new target page group has a corresponding child page, until the parent page in the next new target page group does not have a corresponding child page, then the first target page group and all new target page groups are used as the target page groups of the enterprise to be analyzed.

[0033] Specifically, if the parent page in the new target page group has a corresponding child page, the new target page group composition is determined, and then the next new target page group is constructed. The steps for constructing the next new target page group are the same as the above steps. First, the child page in the newly constructed target page group is used as the parent page of the next new target page group, and then a link search is performed based on the newly determined parent page. If there is a valid link, the newly determined parent page and the page pointed to by the newly found link can be used as the next new target page. Repeat this process until there is no corresponding child page for the parent page in the newly formed target page group, that is, the parent page is the bottom-level page in this new target page group. In this case, all the constructed target page groups including the first target page group are used as the target page groups of the enterprise to be analyzed.

[0034] For example, suppose that a product A under the first product category is crawled from the official website of an enterprise, and there is no next page to jump to after the page of product A, then the page of the product does not have a corresponding sub-page. However, if there is a next page to jump to for the page of another product B (for example, a sub-product C of product B), the page of product B and the page of sub-product C will form a new target page group, and continue to detect whether there is a next page to jump to after the page of sub-product C. It can be seen that for the detection of parent pages and sub-pages, a parent page may correspond to multiple sub-pages (for example, if a company's products have multiple product categories, then the total product page will form a target page group with each product category page). Even if there is no corresponding sub-page for one of the sub-pages, it is still necessary to detect whether there are corresponding sub-pages for other sub-pages, so that all pages about products on the official website of the company can be completely crawled, which is convenient for the subsequent construction of the company's product tree.

[0035] In step S2, data cleaning is performed on the parent page and the child page of the same target page group to obtain page information of the target page of the enterprise to be analyzed after data cleaning.

[0036] Specifically, based on the above step S1, at least one target page group can be obtained, each target page group includes a parent page and a corresponding child page. Since the method is intended to obtain product information of the enterprise to be analyzed, each target page may contain links to the same resource or content. If these paths are not uniformly cleaned, redundancy and inconsistency in the data may occur, affecting subsequent data processing and analysis. By cleaning the data, the same paths of the parent page and the child page between the same target page group are removed, which can ensure data consistency and improve processing efficiency. Then, the page information of all target pages after data cleaning is completed is obtained, wherein the page information includes the page source code of the target page and the file resources downloaded from the target page. File resources may include product brochures, technical manuals, factory instructions, etc. for further introducing the product.

[0037] In one embodiment provided by the present application, the following steps are used to clean the data of the parent page and the child page of the same target page group, specifically including: The path data of all leaf nodes in the document object model tree corresponding to the parent page are crawled to obtain a first path set corresponding to the parent page.

[0038] Specifically, the Document Object Model Tree (Dom Tree) is a tree-like data structure, and a leaf node is a node without subsequent child nodes. The Document Object Model Tree includes a root node and possible intermediate nodes, as well as one or more leaf nodes. Starting from the root node, all nodes are arranged in a hierarchical manner of link relationships to obtain one or more link paths, where each path is "root node-intermediate node-leaf node" or "root node-leaf node".

[0039] More specifically, in this step, first find the document object model tree corresponding to the parent page in the target page group, take the page as the root node, and find the paths of all leaf nodes along the link relationship of its pages. Since there may be multiple link paths in the page, all the paths obtained in this step are defined as the first path set in this application. Generally speaking, one leaf node corresponds to one path.

[0040] Furthermore, the path data of all leaf nodes in the document object model tree corresponding to the sub-page are crawled to obtain a second path set corresponding to the sub-page.

[0041] Specifically, in the same manner as obtaining the first path set corresponding to the parent page, the child page in the same target page group as the parent page as the root node is used as the root node, and all leaf nodes in the corresponding document object model tree are crawled to obtain all link paths corresponding to the child page, which is defined as the second path set in this application.

[0042] Furthermore, invalid path data of sub-pages in each target page group is cleaned.

[0043] Specifically, the invalid path data is the same path data between the first path set and the second path set in the same target page group, and the link part that is the same as its parent page in the child page of each target page group is removed, so as to remove the pages pointed to by the links shared by the parent page and the child page in the child page, and obtain the page source code of all parent pages and all child pages after the cleaning is completed. For example, both the parent page and the child page may have a link to the "About Us" page, but this page is not the page required for subsequent acquisition of product information. In the process of data cleaning, the link path of this page will be cleaned as an invalid path.

[0044] More specifically, all target page groups obtained in the above steps perform the above path cleaning between the parent page and the child page, until the child pages in all target page groups are cleaned, and then the page source code corresponding to each target page is obtained.

[0045] In the above embodiment, a data cleaning method is proposed that first determines all target page groups and then performs path cleaning on the sub-pages in each target page group. In another embodiment provided in the present application, it is also possible to first determine a target page group and then clean the invalid path part in the sub-pages of the target page group. The cleaned sub-page is used as the parent page of the next target page group, and the sub-page corresponding to the next target page group is found at the same time. The above cleaning process is repeated until the sub-page that is finally cleaned has no corresponding sub-page, and the data cleaning is terminated to achieve the effect of performing data cleaning on the parent page and the sub-page of the same target page group.

[0046] For example, refer to Figure 2 , Figure 2 is an example diagram of a web page of an enterprise to be analyzed for explaining the data cleaning step in the enterprise information processing method based on a large language model provided in an embodiment of the present application. Figure 2 In the example shown, the target page groups are the "Tapered Roller" page and the "Roller Series" page, where the "Roller Series" page is the parent page, and the "Tapered Roller" page is the child page corresponding to the "Roller Series" page. In this case, the part of the second path set corresponding to the "Roller Series" page that needs to be retained is the content selected in the box in the above figure.

[0047] More specifically, in the embodiments provided in the present application, there may be a situation where a parent page has multiple corresponding child pages, for example, a product category page has multiple product pages. In this case, each product page will be a child page of the product category page, forming multiple target page groups corresponding to the number of child pages. The product category is used as the parent page of each product page, that is, considering the parallel situation of multiple child pages contained in a parent page of a first path set, the cleaning steps are only repeated between the parent page and the child page, thereby avoiding incorrect cleaning and improving the reliability of data cleaning.

[0048] For example, refer to Figure 3 , Figure 3 This is an example diagram of a web page of an enterprise to be analyzed provided by an embodiment of the present application for explaining the data cleaning step when there are parallel sub-pages in the enterprise information processing method based on a large language model. It can be seen that there are three types of products under the "chip mounting" category in the figure, and the current "chip mounting" page is used as the parent page. The page pointed to by the link corresponding to each type of product map will be used as the sub-page corresponding to the current "chip mounting" page. Since there are three groups, that is, there are three sub-pages respectively, forming three target page groups. In response to this situation, the sub-pages in each target page group respectively perform the cleaning step based on the repeated path part with the parent page "chip mounting".

[0049] In step S3, an enterprise product tree is constructed based on the page information of all the target pages.

[0050] Specifically, the enterprise product tree is a hierarchical structure diagram for displaying the products of the enterprise to be analyzed, which includes multiple product nodes, each of which includes a product or a product category. In this step, the page source code of all target pages is obtained after the above cleaning step, and then all the page source codes are captured as page information, and the enterprise product tree is constructed based on the captured product information or product category information.

[0051] In an embodiment provided in the present application, the following specific steps may be used to implement the step of building an enterprise product tree based on the page information of all the target pages, specifically including: Get the basic product tree.

[0052] Specifically, the basic product tree may be a preset product tree framework, which includes a plurality of product nodes, wherein each of the product nodes corresponds to a node level. The product nodes in the basic product tree need to be filled according to the crawling results obtained by crawling the page information subsequently.

[0053] According to the association relationship between the parent page and the child page in each target page group, the page level of the target page of the enterprise to be analyzed is determined.

[0054] Specifically, before filling in each product in the enterprise product tree, first determine the hierarchy through the link relationship between the parent page and the child page in each target page group. For example, assuming there are two target page groups, "Product Category Page - Product A Page" and "Product A Page - Product A1 Page", the hierarchy obtained based on these two target page groups is "Product Category Page - Product A Page - Product A1 Page".

[0055] Based on the node level and the page level, the parent page information corresponding to the parent page and the child page information corresponding to the child page are inserted into the basic product tree to obtain an enterprise product tree.

[0056] Specifically, after determining the page level of the enterprise product tree of the enterprise to be analyzed, the basic product tree is adaptively adjusted to facilitate the subsequent insertion of product information of each product node. Then, based on all the acquired page information, the basic product tree that has been adjusted through the page level and node level is inserted to obtain the enterprise product tree of the enterprise to be analyzed.

[0057] In another embodiment provided by the present application, the basic product tree may not be obtained, and multiple links may be directly determined based on the link relationship between the parent page and the child page in all target page groups, and then the same nodes in the multiple links may be merged and summarized, and finally the enterprise product tree may be integrated. It can be understood that when executing step S1, when crawling the official website of the enterprise to be analyzed, the jump relationship between each page and its corresponding next page is already known. The jump relationship is the link relationship between the parent page and the child page, so that the enterprise product tree can be constructed based on the link relationship between the parent page and the child page in all target page groups.

[0058] In step S4, link analysis is performed on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed.

[0059] Specifically, the enterprise product tree is formed by integrating the links formed by the link relationship between the parent page and the child page in multiple target page groups. In the integration process of the enterprise product tree, the links that may have duplicate parts are merged. Then, each branch (that is, each link) in the tree structure of the enterprise product tree is analyzed, and all the page source codes on each link are crawled to finally form a page source code group.

[0060] More specifically, a parent page may have multiple corresponding child pages, while a target page group contains only one parent page and one child page. When analyzing each branch of the enterprise product tree, you may encounter a link fork, such as "Product Category Page - Product A / Product B". In this example, the parent page has two child pages, forming two target page groups: "Product Category Page - Product A" and "Product Category Page - Product B".

[0061] In the case of link branching, in the embodiment of the present application, the node corresponding to the parent page at the top of the hierarchy of the enterprise product tree is used as the root node, and the node corresponding to the page at the bottom that does not have a corresponding child page is used as the leaf node, and a depth-first search is adopted, specifically including: Starting from the starting node, select a child node as the next node to be explored, and then deeply explore this child node and all its subordinate child nodes until reaching the leaf node. When reaching the leaf node, backtrack to the previous node and continue to select another unexplored child node for in-depth exploration, and repeat this process until all links are traversed. That is, after completing the exploration of all vertical links, the other sub-pages in the parallel direction are refined and analyzed to ensure that all nodes of the entire enterprise product tree are visited and analyzed.

[0062] In one embodiment provided by the present application, the following specific steps may be used to implement link analysis on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed, specifically including: The page information of each product node in the enterprise product tree is obtained, and a link analysis is performed on the page information to determine the initial page source code corresponding to each page information.

[0063] Specifically, the enterprise product tree includes multiple product nodes; one product node contains page information of one target page. The enterprise product tree includes multiple branches, each branch includes multiple product nodes, and each product node corresponds to the page information of a target page. In this step, for each branch of the enterprise product tree, each product node on the branch is analyzed and the page source code of all analyzed product nodes is obtained, and this part of the page source code is defined as the initial page source code.

[0064] Furthermore, each of the initial page source codes is cleaned to obtain a cleaned page source code corresponding to each of the product nodes.

[0065] Specifically, the initial page source code obtained includes a lot of content that is not related to product information. This part of the content will affect the length of the page source code that is subsequently input into the preset large language model. An overly long page source code may not meet the input restrictions of the preset large language model, and may confuse the model judgment and cause the key points to be unclear. Based on this, it is necessary to clean the initial page source codes one by one, so as to obtain the cleaned page source code after each page source code is reduced.

[0066] In an embodiment provided by the present application, the following specific steps can be used to implement source code cleaning of each of the initial page source codes to obtain the cleaning page source code corresponding to each of the product nodes, specifically including: A first cleaning is performed on each of the initial page source codes to clean all product nodes that do not contain text information in each of the initial page source codes to obtain multiple intermediate page source codes.

[0067] Specifically, the first cleaning refers to clearing all product nodes that do not contain text information in the initial page source code, thereby removing those parts that do not contribute to the understanding of the content, reducing the irrelevant information that the model needs to process, and improving the efficiency and accuracy of the model processing. In addition, when the first cleaning is performed first, since the entire product node and the label in the node are removed together, the amount of data required to be processed in the second cleaning described below is also reduced. In this step, the initial page source code remaining after the first cleaning process is called the intermediate page source code, which is convenient for subsequent distinction.

[0068] Furthermore, a second cleaning is performed on each of the intermediate page source codes to clean non-structural tags and other pre-specified tags in each of the intermediate page source codes to obtain a plurality of the cleaned page source codes.

[0069] Furthermore, the page source code group is determined based on all the cleaned page source codes.

[0070] Specifically, the second cleaning refers to the non-structural tags and other pre-specified tags in all intermediate page source codes, wherein non-structural tags (such as , <script>等）可以简化文档的结构，这是因为非结构标签通常用于提供额外的功能或样式信息，而不是承载内容本身，通过移除非结构标签，可以将注意力集中在主要内容上，同时减少了模型需要解析的标签数量；在本申请实施例中，还预指定删除可以被展开显示的标签，这是由于标签的内容已经被展开（即显示了其内部的内容），那么在爬取下来的页面源码中，这个标签及其内容已经存在。在这种情况下，需要删除的是该标签及其内部的具体内容。随即，完成第二清洗得到了标签长度缩减后的适用于输入预设大语言模型的多个清洗页面源码。

[0071] 进一步地，在完成对每一所述初始页面源码进行源码清洗，得到每一所述产品节点对应的清洗页面源码的步骤之后，还包括以下步骤：将每一所述产品节点对应的清洗页面源码替换所述企业产品树中对应产品节点的页面信息，得到所述页面源码组。

[0072] 具体地，基于清洗后得到的清洗页面源码对企业产品树中对应产品节点的页面信息进行替换，随即基于替换后的企业产品树的每一分支进行分析，并获取分析过每一产品节点对应的页面源码，形成页面源码组。

[0073] 在步骤S5中，将所述页面源码组输入至预设大语言模型，通过所述预设大语言模型对所述页面源码组进行产品信息提取，得到所述待分析企业对应的企业产品信息。

[0074] 具体地，将所述页面源码组输入至预设大语言模型，并结合预设的结构化提示词引导预设大语言模型对页面源码组进行产品信息提取，最终得到待分析企业的产品信息。其中，预设的结构化提示词可以由相关人员根据实际需求进行设定，包括但不限于产品名称、产品型号、产品详情、产品类别等产品信息，在本申请实施例中，结构化提示词包括产品名称、产品型号和功能描述，即"请根据该页面源码，找到产品名称、产品型号和功能描述”，预设大语言模型会结合本实施例提供的引导词，对页面源码组进行处理得到企业产品信息。

[0075] 示例性的，参照图4，图4是本申请实施例提供的输出与结构化提示词相对应的企业产品信息步骤的输出结果示意图，包括提取的产品名称、产品型号及功能描述。

[0076] 进一步地，企业产品信息采用结构化的输出结果展示，而结构化由输入至预设大语言模型的结构化提示词调整得到，但实际情况中，可能存在从页面源码组中无法得到某些产品对应的产品详细信息的情况或者在执行数据清洗 / 源码清洗时剔除了某些有效信息，从而产生结构化输出的企业产品信息中存在一个或多个产品对应的产品信息不完整的情况，示例性的，存在产品信息存在结构空缺可以参照图4中"无”的部分。

[0077] 为了完善企业产品信息的内容，提高本方法的可用性，本申请还提出在得到所述待分析企业对应的企业产品信息的步骤之后，还包括以下步骤：对所述企业产品信息进行完整性检测，以从所述企业产品信息中筛选出空缺产品信息。

[0078] 具体地，在得到企业产品信息的结构化输出（以下简称为结构化输出）后，对结构化输出进行完整性检测，完整性检测的过程中扫描并确认每一产品涉及的产品信息是否包含所有需求的产品信息。若否，则确定对应的产品的所有产品信息为空缺产品信息，所缺少的需求的产品信息为空缺信息项。在本申请实施例中，统一将结构化输出中的空缺信息项用同一关键词表示，例如图4中的"无”，随即进行关键词筛选，即可定位企业产品信息中的所有空缺产品信息，并确定每一空缺产品信息对应的空缺信息项。

[0079] 进一步地，获取所述空缺产品信息对应的页面信息中的文件资源。

[0080] 具体地，文件资源包含至少一个页面资源信息。可以理解地，文件资源可以是某个产品对应的产品介绍书、技术手册、出厂说明书等。而产品介绍书、技术手册或者出厂说明书都可以作为包含产品进一步介绍的页面资源信息。在本步骤中，根据空缺产品信息对应的产品，到与该产品相对应的页面信息中进行检索，得到页面信息中包括的所有页面资源信息。此外，结合上述对于步骤S2的阐述可知，在完成步骤S2涉及的数据清洗之后，对每一目标页面的文件资源进行保存，若存在空缺产品信息则可进行调用。

[0081] 进一步地，检测所述文件资源信息是否存在与所述空缺信息项对应的页面资源信息；进一步地，若存在所述空缺信息项对应的页面资源信息，则将所述页面资源信息补充至所述空缺信息项并更新所述企业产品信息。

[0082] 具体地，在对应页面信息中得到的所有页面资源信息后，需要进一步检测得到的所有页面资源信息中是否存在可填补空缺信息项的页面资源信息。举例而言，若产品A涉及的产品信息为空缺产品信息，其空缺信息项为"产品规格”，获取的页面资源信息包括资源A、资源B和资源C，那么在此步骤中需要对这三个页面资源信息进行检测，以判断其中是否存在关于产品A的"产品规格”的信息。若否，则保留对应空缺信息项，否则，则基于对应的页面资源信息中关于产品A的"产品规格”所涉及的内容对该空缺信息项进行补充并更新。

[0083] 更具体地，对每一空缺信息项执行上述步骤后，得到最终的企业产品信息的结构化输出。而在本申请提出的一个实施例中，还可以将最终的结构化输出中的空缺产品信息设置为待补充产品信息，并获取与待补充产品信息对应页面信息的更新状态，若对应页面信息存在更新，则获取页面信息中新的文件资源信息，并返回检测所述文件资源信息是否存在与所述空缺信息项对应的页面资源信息的步骤，以持续优化完善待分析企业的企业产品信息。

[0084] 参照图5，图5是本申请提供的基于大语言模型的企业信息处理装置的虚拟结构示意图，该装置包括：目标页面组获取模块100，用于获取待分析企业的目标页面组；一个所述目标页面组包含一个父页面以及一个子页面；页面清洗及信息获取模块200，用于对同一所述目标页面组的所述父页面和所述子页面进行数据清洗，获取数据清洗后所述待分析企业的目标页面的页面信息；所述目标页面为所述父页面和所述子页面；企业产品树构建模块300，用于基于所有所述目标页面的页面信息，构建企业产品树；页面源码组获取模块400，用于对所述企业产品树进行链路分析，得到所述待分析企业对应的页面源码组；企业产品信息输出模块500，用于将所述页面源码组输入至预设大语言模型，通过所述预设大语言模型对所述页面源码组进行产品信息提取，得到所述待分析企业对应的企业产品信息。

[0085] 本申请实施例所述的基于大语言模型的企业信息处理装置可执行上述实施例所提供的企业信息处理及分析方法，所述企业信息处理及分析装置具备上述实施例所述企业信息处理及分析方法相应的功能步骤以及有益效果，具体请参阅上述企业信息处理及分析方法的实施例，本申请实施例在此不再赘述。

[0086] 本申请实施例还提供了一种电子设备，请参照图6，图6是本申请实施例提供的电子设备的结构示意图，该电子设备可以包括处理器和存储器，其中处理器和存储器可以通过总线或者其他方式连接。处理器可以为中央处理器（CentralProcessing Unit，CPU）。处理器还可以为其他通用处理器、数字信号处理器（DigitalSignal Processor，DSP）、专用集成电路（Application Specific Integrated Circuit，ASIC）、现场可编程门阵列（Field-Programmable Gate Array，FPGA）或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等芯片，或者上述各类芯片的组合。存储器作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块，如本申请实施例中的企业信息处理及分析方法对应的程序指令 / 模块。处理器通过运行存储在存储器中的非暂态软件程序、指令以及模块，从而执行处理器的各种功能应用以及数据处理，即实现上述方法实施例中的企业信息处理及分析方法。

[0087] 存储器可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储处理器所创建的数据等。此外，存储器可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。所述一个或者多个模块存储在所述存储器中，当被所述处理器执行时，执行如上述方法实施例中的企业信息处理及分析方法。上述电子设备具体细节可以对应上述方法实施例中对应的相关描述和效果进行理解，此处不再赘述。本领域技术人员可以理解，实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述存储介质可为只读存储记忆体（Read-Only Memory，ROM）、随机存储记忆体（Random AccessMemory，RAM）、快闪存储器（FlashMemory）、硬盘（Hard Disk Drive，缩写：HDD）或固态硬盘（Solid-State Drive，SSD)等；所述存储介质还可以包括上述种类的存储器的组合。

[0088] 在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本申请的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。

[0089] 类似地，应当理解，为了精简本公开并帮助理解各个发明方面中的一个或多个，在上面对本申请的示例性实施例的描述中，本申请的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本申请要求比在每个权利要求中所明确记载的特征更多的特征。更确切地说，如权利要求书所反映的那样，发明方面在于少于前面公开的单个实施例的所有特征。因此，遵循具体实施方式的权利要求书由此明确地并入该具体实施方式，其中每个权利要求本身都作为本申请的单独实施例。

[0090] 应该注意的是上述实施例对本申请进行说明而不是对本申请进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。< / script>

Claims

1. An enterprise information processing method based on a large language model, characterized in that: The method comprises: Obtain a target page group of the enterprise to be analyzed; one target page group includes a parent page and a child page; Performing data cleaning on the parent page and the child page of the same target page group, and obtaining page information of the target page of the enterprise to be analyzed after data cleaning; the target page is the parent page and the child page; Building an enterprise product tree based on the page information of all the target pages; Performing link analysis on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed; The page source code group is input into a preset large language model, and product information is extracted from the page source code group through the preset large language model to obtain enterprise product information corresponding to the enterprise to be analyzed.

2. The enterprise information processing method based on a large language model according to claim 1, characterized in that: The step of obtaining the target page group of the enterprise to be analyzed includes: Locate the official website page of the enterprise to be analyzed, and use the official website page as the parent page in the first target page group; The page corresponding to the page link containing the preset keyword on the official website page is used as a child page in the first target page group and a parent page in the new target page group; Check whether the parent page in the new target page group has a corresponding child page; When the parent page in the new target page group does not have a corresponding child page, the first target page group is used as the target page group of the enterprise to be analyzed; When there is a corresponding child page for the parent page in the new target page group, the parent page in the new target page group and the detected child page are combined into a new target page group, and the child page in the new target page group is used as the parent page in the next new target page group, and it is detected whether there is a corresponding child page for the parent page in the next new target page group, until there is no corresponding child page for the parent page in the next new target page group, then the first target page group and all new target page groups are used as the target page groups of the enterprise to be analyzed.

3. The enterprise information processing method based on a large language model according to claim 1, characterized in that: The step of performing data cleaning on the parent page and the child page of the same target page group includes: Crawling the path data of all leaf nodes in the document object model tree corresponding to the parent page to obtain a first path set corresponding to the parent page; Crawling the path data of all leaf nodes in the document object model tree corresponding to the subpage to obtain a second path set corresponding to the subpage; Invalid path data of sub-pages in each target page group are cleaned, wherein the invalid path data is the same path data between the first path set and the second path set in the same target page group.

4. The enterprise information processing method based on a large language model according to claim 1, characterized in that: The step of constructing the enterprise product tree based on the page information of all the target pages includes: Obtain a basic product tree; the basic product tree includes multiple product nodes; one product node corresponds to one node level; Determine the page level of the target page of the enterprise to be analyzed according to the association relationship between the parent page and the child page in each target page group; Based on the node level and the page level, the parent page information corresponding to the parent page and the child page information corresponding to the child page are inserted into the basic product tree to obtain an enterprise product tree.

5. The enterprise information processing method based on a large language model according to claim 1, characterized in that: The enterprise product tree includes a plurality of product nodes; one of the product nodes includes page information of one of the target pages; the step of performing link analysis on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed includes: Obtaining page information of each of the product nodes in the enterprise product tree, and performing link analysis on the page information to determine the initial page source code corresponding to each of the page information; Perform source code cleaning on each of the initial page source codes to obtain a cleaned page source code corresponding to each of the product nodes; The page source code corresponding to each of the product nodes is used to replace the page information of the corresponding product node in the enterprise product tree to obtain the page source code group.

6. The enterprise information processing method based on a large language model according to claim 5, characterized in that: The step of performing source code cleaning on each of the initial page source codes to obtain the cleaned page source code corresponding to each of the product nodes comprises: Performing a first cleaning on each of the initial page source codes to clean all product nodes that do not contain text information in each of the initial page source codes to obtain a plurality of intermediate page source codes; Performing a second cleaning on each of the intermediate page source codes to clean non-structural tags and other pre-specified tags in each of the intermediate page source codes to obtain a plurality of the cleaned page source codes; The page source code group is determined according to all the cleaned page source codes.

7. The enterprise information processing method based on a large language model according to claim 1, characterized in that: After the step of obtaining the enterprise product information corresponding to the enterprise to be analyzed, the method further includes: Performing integrity check on the enterprise product information to filter out missing product information from the enterprise product information; the missing product information contains at least one missing information item; Acquire the file resource in the page information corresponding to the missing product information; the file resource includes at least one page resource information; Detecting whether the file resource information contains page resource information corresponding to the missing information item; If there is page resource information corresponding to the missing information item, the page resource information is added to the missing information item and the enterprise product information is updated.

8. An enterprise information processing device based on a large language model, characterized in that: include: A target page group acquisition module is used to acquire the target page group of the enterprise to be analyzed; One of the target page groups includes a parent page and a child page; A page cleaning and information acquisition module, used to perform data cleaning on the parent page and the child page of the same target page group, and obtain page information of the target page of the enterprise to be analyzed after data cleaning; the target page is the parent page and the child page; An enterprise product tree construction module, used to construct an enterprise product tree based on the page information of all the target pages; A page source code group acquisition module is used to perform link analysis on the enterprise product tree to obtain a page source code group corresponding to the enterprise to be analyzed; The enterprise product information output module is used to input the page source code group into a preset large language model, extract product information from the page source code group through the preset large language model, and obtain the enterprise product information corresponding to the enterprise to be analyzed.

9. An electronic device, characterized in that: include: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a program or an instruction, and when the program or the instruction is executed, the method according to any one of claims 1 to 7 is implemented.

Images