Knowledge extraction method, electronic device, and program product

Abstract

The present disclosure provides a knowledge extraction method, an electronic device and a program product. The method of the present disclosure comprises: obtaining screen recording data of a current page; the screen recording data comprises user operation logs and page metadata; the user operation logs contain code path markers of source code location information associated with user operations; based on the screen recording data, determining a project name to which the current page belongs and a page path of the current page; according to the project name and the page path, obtaining a stored historical knowledge tree; based on the screen recording data, constructing a context sequence containing business operation information and code verification information; using the context sequence to perform an incremental update operation on the nodes in the historical knowledge tree, generating a new knowledge tree, and storing the new knowledge tree to a knowledge base.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to knowledge extraction methods, electronic devices, and program products. Background Technology

[0002] With the rapid development of artificial intelligence technology, using large language models (LLM) to assist front-end code development and improve R&D efficiency has become a major direction for AI empowerment.

[0003] In existing technologies, the acquisition of business knowledge mainly relies on manual sorting and document maintenance, while automated extraction faces numerous technical challenges. Some solutions employ knowledge extraction methods from a single data source, which have limited coverage of business information. If the business logic is complex or the data sources are scattered and difficult to fully acquire, even if acquisition is achieved, it incurs high time and manpower costs. Although some solutions adopt multi-source data collection measures, the lack of a unified semantic association mechanism or code location capability easily leads to knowledge fragmentation or misunderstanding, resulting in incomplete and inaccurate extracted knowledge. Furthermore, most existing solutions use a full-coverage update approach, which means that when business logic changes, relevant knowledge cannot be updated in a timely manner, resulting in high maintenance costs and poor timeliness. Therefore, a solution that can extract and maintain knowledge at a low cost is needed. Summary of the Invention

[0004] This disclosure provides knowledge extraction methods, electronic devices, and program products.

[0005] According to a first aspect of this disclosure, a knowledge extraction method is provided. The method specifically includes: acquiring screen recording data of the current page; the screen recording data includes user operation logs and page metadata; the user operation logs contain code path markers with source code location information associated with user operations; based on the screen recording data, determining the project name to which the current page belongs and the page path of the current page; acquiring a stored historical knowledge tree based on the project name and page path; constructing a context sequence containing business operation information and code verification information based on the screen recording data; performing incremental update operations on the nodes in the historical knowledge tree using the context sequence to generate a new knowledge tree, and storing the new knowledge tree in a knowledge base.

[0006] According to the above solution, by receiving screen recording data with code path markers collected by browser plugins through the server, the automatic association between user operations and source code locations is realized, solving the problem of the disconnect between business knowledge and code implementation. In addition, by obtaining historical knowledge trees based on project names and page paths and performing incremental update operations, full coverage and repeated extraction of knowledge are avoided, significantly reducing knowledge maintenance costs. By constructing context sequences containing business operation information and code verification information, the accuracy and completeness of knowledge generation are improved.

[0007] According to at least one embodiment of this disclosure, the screen recording data further includes: network interface logs and page screenshots; wherein, the network interface logs are used to record interface requests and response data triggered by user operations; and the page screenshots are visual snapshots of the current page at the end of recording.

[0008] According to at least one embodiment of this disclosure, the method for generating code path markers containing source code location information associated with user operations in the user operation log includes: during the compilation phase of the front-end business system, traversing the code abstract syntax tree and injecting code path markers containing the relative path of the source code file and the row and column numbers into the DOM elements; when the user operation triggers the current page, capturing the code path markers corresponding to the triggered elements in the current page and recording them in the user operation log.

[0009] According to at least one embodiment of this disclosure, determining the project name and page path of the current page based on screen recording data includes: matching the page URL in the page metadata with the path format field in the pre-configured mapping file using regular expressions; if the match is successful, extracting the project name and page path corresponding to the current page; if the match fails, traversing the code path markers in the user operation log, extracting the project identifier as the project name from the relative path of the source code file, and querying the pre-configured mapping file to obtain the page path.

[0010] According to at least one embodiment of this disclosure, the construction method of the historical knowledge tree includes: taking the current page as the root node, constructing child nodes according to the hierarchical nesting relationship of business elements in the current page; recording the node name, node type, function description, code location information and child node relationship field for each node; wherein, the node type is used to distinguish the page, component or field level; the code location information field is generated based on the code path markup and is used to store the path and row and column number information of the node in the source code file.

[0011] According to at least one embodiment of this disclosure, a context sequence containing business operation information and code verification information is constructed based on screen recording data, including: combining user operation logs, response data from network interface logs, page screenshots, and page paths into a business operation context subarray; combining source code fragments obtained based on code path tag parsing into a code verification context subarray; and concatenating them in order of priority from business operation context to code verification context to form a context sequence.

[0012] According to at least one embodiment of this disclosure, an incremental update operation is performed on nodes in a historical knowledge tree using a context sequence to generate a new knowledge tree, including: inputting the historical knowledge tree and the context sequence into a large language model; calculating the semantic similarity between each node in the historical knowledge tree and the context sequence using the large language model; if the semantic similarity is lower than a preset threshold and the context sequence contains new business elements, then a node addition operation is performed; if the semantic similarity is lower than a preset threshold and the context sequence describes the same business element as an existing node but the functional logic has changed, then a node update operation is performed; if the semantic similarity is higher than a preset threshold, then the original node is reused.

[0013] According to at least one embodiment of this disclosure, storing a new knowledge tree in a knowledge base includes: generating a unique file identifier based on the project name, page path, and timestamp; serializing the new knowledge tree into a structured data file, uploading it to a knowledge base service, and obtaining the file identifier returned by storage; and establishing a mapping record between the file identifier and the page path in the database.

[0014] After obtaining the screen recording data of the current page according to at least one embodiment of this disclosure, the method further includes: generating a globally unique task identifier, binding the screen recording data with the task identifier and adding it to an asynchronous processing queue; periodically triggering queue consumption through a timed scheduling mechanism, retrieving tasks from the asynchronous processing queue and executing knowledge extraction tasks.

[0015] According to at least one embodiment of this disclosure, the method further includes: a knowledge retrieval step: receiving a knowledge query request initiated by the front end, the knowledge query request containing a page path and a project name; querying mapping records in the database based on the page path and project name to obtain the corresponding file identifier; loading the corresponding knowledge tree from the knowledge base based on the file identifier and returning it to the front end for assisting code development or business understanding.

[0016] According to a second aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs a first aspect of any embodiment of this disclosure.

[0017] According to a third aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement a first aspect of any embodiment of this disclosure.

[0018] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a first aspect of any embodiment of this disclosure. Attached Figure Description

[0019] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0020] Figure 1 This is a flowchart illustrating the knowledge extraction method provided in an embodiment of the present disclosure.

[0021] Figure 2 This is a flowchart illustrating the code path tag generation method provided in an embodiment of the present disclosure.

[0022] Figure 3 This is a flowchart illustrating the method for determining current page information provided in an embodiment of this disclosure.

[0023] Figure 4 A flowchart illustrating the historical knowledge tree generation method provided in this embodiment of the disclosure.

[0024] Figure 5 This is a flowchart illustrating the context sequence generation method provided in an embodiment of the present disclosure.

[0025] Figure 6 A schematic flowchart illustrating the new knowledge tree update method provided in this embodiment of the disclosure.

[0026] Figure 7 This is a flowchart illustrating the knowledge base storage method provided in an embodiment of this disclosure.

[0027] Figure 8 This is a flowchart illustrating the task generation method provided in an embodiment of the present disclosure.

[0028] Figure 9 This is a flowchart illustrating the knowledge retrieval method provided in an embodiment of this disclosure.

[0029] Figure 10 This is a schematic diagram illustrating the knowledge extraction and storage process as an example of this disclosure.

[0030] Figure 11 This is a schematic block diagram of the structure of a knowledge extraction device according to one embodiment of the present disclosure.

[0031] Figure 12 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0032] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0033] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0034] Figure 1 This is a flowchart illustrating the knowledge extraction method provided in an embodiment of this disclosure. Figure 1 The method shown includes steps S101 to S105. This method can be executed by a server (local server, cloud server, etc.).

[0035] Specifically, Figure 1 The method shown includes step S101: obtaining screen recording data of the current page; the screen recording data includes user operation logs and page metadata; the user operation logs contain code path markers of source code location information associated with user operations, and the code path markers indicate the path and row and column numbers of the code of the element operated by the user in the source code file.

[0036] It should be noted that the screen recording data mentioned here refers to the data packet generated after the browser plugin finishes recording the user's actions on the page, used to reconstruct the user's operation process and the page content context. The code path markers mentioned here refer to the identification information embedded in the user operation log, used to establish the mapping relationship between user interface operations and backend source code files.

[0037] In practical applications, screen recording data is obtained by executing screen recording tasks on the browser side, which can be done through browser plugins (such as Clipchamp, Screenity, etc.). The server receives the screen recording data sent by the browser plugin. The browser plugin can run on the user's end (such as a mobile client, desktop client, etc.). When the user performs actions such as clicking or typing on the current page, the browser plugin can capture the operation event and the operated element (i.e., the element triggered by the user). Since the front-end business system has injected markers containing source code location information into the DOM elements during the compilation phase, the browser plugin reads this marker on the target element while capturing the operation event and records it as a code path marker in the user operation log. Therefore, each operation record in the user operation log obtained by the server carries the specific path and line / column number of the UI element corresponding to the operation in the source code file through the code path marker. This realizes the association between user behavior data and code implementation data. The server can know the code location behind each operation without manual annotation, providing an accurate data foundation for subsequent knowledge extraction.

[0038] Step S102: Based on the screen recording data, determine the project name and page path of the current page.

[0039] After receiving the screen recording data, the server parses the page metadata (such as the URL) and the code path markers in the user operation log. The server uses a recognition algorithm to identify the identity of the front-end business system to which the current page belongs, i.e., the project name, based on the URL in the page metadata or the code path markers in the user operation log; and it also identifies the logical route address of the current page within the front-end business system, i.e., the page path. Through this recognition method, the server can accurately pinpoint the business system and specific page to which the screen recording data belongs, ensuring that subsequent knowledge updates can locate the correct knowledge tree file and avoiding recognition errors caused by dynamic changes in page routing.

[0040] Step S103: Obtain the stored historical knowledge tree based on the project name and page path.

[0041] Based on the project name and page path determined in the previous step, the server constructs a unique file identifier and queries the knowledge base to see if a corresponding mapping record exists. If it exists, the server loads the stored historical knowledge tree from the knowledge base according to the file identifier in the mapping record; otherwise, it initializes an empty knowledge tree structure. The historical knowledge tree stores the previously extracted business knowledge nodes for this page in a tree structure, with each node containing information such as function description and code location. This ensures the continuity of the knowledge extraction process, allowing the server to reuse existing knowledge results when extracting new knowledge, avoiding duplication of effort, and providing a comparison benchmark for subsequent incremental updates.

[0042] Step S104: Construct a context sequence containing business operation information and code verification information based on the screen recording data.

[0043] The server processes and fuses multi-source information from screen recording data in parallel. It combines business-level information such as user operation logs and API response data into business operation information, representing the user's intent and operational flow at the business level. Simultaneously, based on code path markers in the user operation logs, the server reads corresponding source code fragments and combines them into code verification information, which provides details of the business operation's implementation at the code level. The server then concatenates these two elements into a context sequence, prioritizing business operation information and supplementing it with code verification information. This hierarchical context construction approach allows subsequent processing engines to prioritize understanding the business logic before using code information for verification, effectively improving the accuracy and completeness of knowledge generation.

[0044] Step S105: Use the context sequence to perform incremental update operations on the nodes in the historical knowledge tree to generate a new knowledge tree, and store the new knowledge tree in the knowledge base.

[0045] It should be noted that the historical knowledge tree mentioned here refers to the collection of page business knowledge already stored in the knowledge base, organized in a tree structure, including page components, fields, and their corresponding code location information. The new knowledge tree mentioned here refers to the latest business knowledge tree generated after this update. The knowledge base mentioned here refers to the storage system used to persistently store knowledge tree files and maintain the mapping relationship between page paths and file identifiers.

[0046] The server inputs the historical knowledge tree and context sequence into the processing engine. The engine traverses each node in the historical knowledge tree and performs semantic comparisons between the node information and the business operation information and code verification information in the context sequence. If the context sequence contains a new business element that is not present in the historical knowledge tree, the engine performs a node addition operation; if the information in the context sequence describes the same element as an existing node but the functional logic or code location has changed, the engine performs a node update operation; if the node information has not changed substantially, the original node is reused. Through this incremental update operation, the server generates a new knowledge tree containing the latest business logic, serializes it, and uploads it to the knowledge base, while simultaneously updating the mapping records of page paths and file identifiers. This enables the dynamic evolution of business knowledge, ensuring that the content in the knowledge base is always synchronized with online business, without requiring a full rewrite, significantly reducing maintenance costs.

[0047] Based on the publicly available solutions described above, by receiving screen recording data with code path markers collected by browser plugins via the server, the system automatically associates user operations with source code locations, solving the problem of disconnect between business knowledge and code implementation. Furthermore, by obtaining historical knowledge trees based on project names and page paths and performing incremental update operations, the system avoids full coverage and repeated extraction of knowledge, significantly reducing knowledge maintenance costs. By constructing context sequences containing business operation information and code verification information, the accuracy and completeness of knowledge generation are improved.

[0048] In one or more embodiments of this disclosure, the screen recording data further includes: network interface logs and page screenshots; wherein, the network interface logs are used to record interface requests and response data triggered by user operations; and the page screenshots are visual snapshots of the current page when recording ends.

[0049] During operation, the browser plugin not only captures user operation logs and page metadata, but also monitors network communication events and the page's visual status. The browser plugin packages the captured multi-source data into screen recording data and sends it to the server. After parsing the screen recording data, the server obtains not only user operation logs and page metadata, but also network interface logs and page screenshots.

[0050] Specifically, the browser plugin intercepts network request interfaces of the front-end business system and automatically records the generated interface requests and corresponding response data when users perform operations that lead to data interaction. The browser plugin timestamps the interface request and response data with the operation records in the user operation log, ensuring that the network interface log accurately reflects the data flow process triggered by user operations. After the server obtains the network interface log, it can understand the details of data changes behind each operation, such as which fields the interface returned after clicking the query button. This provides real data samples for subsequent code verification, improving the accuracy of knowledge extraction.

[0051] When a user completes a business operation and triggers the stop recording command, the browser plugin immediately captures an image of the visible area of ​​the current page, generating a screenshot. This screenshot serves as a visual snapshot of the current page at the end of recording, fully preserving the page's final state after the operation, including component layout, data display content, and style information. After obtaining the screenshot, the server can intuitively verify whether the business state described in the user operation logs and network interface logs matches the actual page display, providing a visual basis for subsequent incremental update operations on nodes in the historical knowledge tree using context sequences.

[0052] Based on the publicly available solutions described above, by adding network interface logs and page screenshots to the screen recording data, a three-dimensional complementarity of operational behavior, data flow, and visual state is achieved. Network interface logs provide the data logic support behind business operations, solving the problem that the interface data structure cannot be determined solely from operation logs; page screenshots provide the visual results after business operations, solving the problem that the actual rendering effect of the page cannot be determined solely from code. The combination of these two allows the server to integrate multimodal information when constructing a context sequence containing business operation information and code verification information, significantly improving the completeness and credibility of the generated new knowledge tree.

[0053] In one or more embodiments of this disclosure, such as Figure 2 This is a flowchart illustrating the code path marker generation method provided in an embodiment of this disclosure. Figure 2 As shown, the generation method of code path markers containing source code location information associated with user operations in the user operation log includes: Step S201: During the compilation phase of the front-end business system, the code abstract syntax tree is traversed through a plugin to inject code path markers containing the relative path and row / column number of the source code file into the DOM element. Step S202: When a user operation triggers the current page, the code path marker corresponding to the triggered element in the current page is captured and recorded in the user operation log.

[0054] It's important to note that the front-end business system mentioned here refers to the user interface program running in a browser. The source code of the front-end business system can be converted into executable code before deployment. The code abstract syntax tree (CTL) refers to the tree-like structure generated during the compilation process of the front-end business system's source code. The DOM element refers to the Document Object Model (DOM) nodes that constitute the current page; the relative path of the source code file refers to the path address of the front-end business system's source code file relative to the project's root directory. The row and column numbers refer to the specific line and column numbers of the code within the source code file. The triggered element refers to the specific UI component that the user interacts with on the current page.

[0055] In practical applications, the code path markers in the user operation logs received by the server are pre-generated and captured during the development and operation of the front-end business system. This ensures that the data received by the server naturally contains code location information, eliminating the need for the server to perform additional code matching calculations after receiving the data, thus effectively improving the efficiency and accuracy of knowledge extraction.

[0056] When building and deploying a front-end business system, plugins (such as Babel plugins) scan all source code files and generate a code abstract syntax tree. The plugin traverses each node in the code abstract syntax tree, identifying the node corresponding to the interface display, i.e., the DOM element. For each identified DOM element, the plugin reads its source code location in the front-end business system, generates a string containing the relative path and row / column number of the source code file, and injects it as an attribute into the compiled DOM element. For example, the plugin can inject the attribute "data-src-position= / src / components / Button.tsx:20:10" into a button component. This achieves static binding of source code location information to DOM elements, providing a data foundation for subsequent runtime capture, and the plugin automatically completes the injection without requiring developers to manually add markers.

[0057] When a user performs a click or input action on the current page, the browser plugin listens to the event stream of that page. When an event occurs, the browser plugin locates the target node of the event, i.e., the triggered element. The browser plugin reads the pre-injected code path marker on the triggered element and packages this marker along with information such as the type of user action and timestamp to form a record in the user action log. By dynamically associating the static location information injected at compile time with the specific user action, the user action log received by the server can accurately reflect the code location behind each action.

[0058] Based on the publicly available solutions described above, generating code path markers through a combination of compile-time injection and runtime capture achieves a precise correlation between user actions and source code locations. During the compilation phase, a plugin traverses the code abstract syntax tree and injects markers, ensuring comprehensive coverage and automated generation of source code location information without the need for manual maintenance of mapping relationships. At runtime, the browser plugin captures the code path markers corresponding to triggered elements, ensuring real-time correspondence between user actions and code locations. This enables the server-acquired user operation logs to possess high-precision code tracing capabilities, providing a reliable data source for subsequent code verification information construction based on code path markers, and significantly improving the accuracy of knowledge extraction.

[0059] In one or more embodiments of this disclosure, such as Figure 3 This is a flowchart illustrating the method for determining current page information provided in an embodiment of this disclosure. Figure 3As shown, based on screen recording data, the project name and page path of the currently recorded page are determined, including: Step S301: Matching the page URL in the page metadata with the path format field in the pre-configured mapping file using regular expressions. Step S302: If the match is successful, extracting the project name and page path corresponding to the current page. Step S303: If the match fails, traversing the code path markers in the user operation log, extracting the project identifier as the project name from the relative path of the source code file, and querying the pre-configured mapping file to obtain the page path.

[0060] It should be noted that the page URL mentioned here refers to the Uniform Resource Locator (URL) displayed in the browser's address bar for the current page. The pre-configured mapping file refers to a configuration file that pre-stores the mapping relationship between page routes and code structure in the front-end business system. The path format field refers to the field in the pre-configured mapping file used to store regular expressions for page paths. The project identifier refers to a string extracted from the relative path of the source code file to identify the business system to which the page belongs, such as the root directory name in the source code directory structure. The project name refers to the identity identifier of the front-end business system to which the current page belongs. The page path refers to the logical route address of the current page within the front-end business system. The relative path of the source code file refers to the path address of the source code file of the front-end business system relative to the project root directory.

[0061] The server parses the page metadata from the screen recording data to obtain the page URL. The server reads the pre-configured mapping file, iterates through each mapping record in the file, and compares the page URL with the regular expression stored in the path format field using a regular expression matching algorithm.

[0062] When a page URL matches a path format field in a mapping record, the server directly reads the pre-configured project name and page path from that mapping record. For example, if the page URL is " / bill-management / supplier", and a record with the path format "^ / bill-management / supplier$" is found, then the corresponding project name "bill-management" and page path " / bill-management / supplier" are extracted from that record. This ensures that, within the routing specifications, the target business system and page can be accurately and quickly located, providing a precise index key for subsequent retrieval of the historical knowledge tree.

[0063] When a page URL fails to match via regular expressions due to dynamic routing, parameterized paths, or a Single-Page Application (SPA) architecture, the server initiates a fallback identification strategy. The server iterates through each user operation log, reading the code path markers and parsing out the relative paths of the source code files. The server extracts the project identifier from the directory structure of the source code file's relative path; for example, it extracts "bill-management" from " / src / projects / bill-management / components / List.tsx" as the project identifier and identifies it as the project name. Subsequently, the server uses the extracted project name or the relative path of the source code file to query the pre-configured mapping file, matching the corresponding code entry path to obtain the page path. This leverages the stability of the code structure to compensate for the variability of page routing, ensuring accurate page identification even in complex routing scenarios.

[0064] Based on the publicly available solutions described above, the dual-strategy identification method combining regular expression matching and code path mapping significantly improves the robustness of determining project names and page paths. Regular expression matching is efficient and simple to configure in standard routing scenarios; code path mapping is highly reliable and less prone to failure in dynamic routing scenarios. The combination of these two methods solves the problem of page recognition failure caused by front-end route reconstruction or dynamic parameters, ensuring that the server can always accurately locate the business system and logical path to which the current page belongs. This provides a reliable prerequisite for subsequently retrieving the stored historical knowledge tree based on the project name and page path, avoiding knowledge tree loading failures or knowledge corruption due to recognition errors.

[0065] In one or more embodiments of this disclosure, such as Figure 4 This is a flowchart illustrating the historical knowledge tree generation method provided in this embodiment of the disclosure. Figure 4 As shown, the construction method of the historical knowledge tree includes: Step S401: Taking the current page as the root node, construct child nodes according to the hierarchical nesting relationship of business elements in the current page; record the node name, node type, function description, code location information, and child node relationship fields for each node; wherein, the node type is used to distinguish the page, component, or field level. Step S402: The code location information field is generated based on the code path markup and is used to store the path and row / column number information of the node in the source code file.

[0066] It's important to clarify that the historical knowledge tree mentioned here refers to the collection of page business knowledge stored in the knowledge base, organized in a tree structure. The root node refers to the top-level node of the historical knowledge tree, corresponding to the current page. Child nodes refer to the subordinate nodes of the parent node in the historical knowledge tree, corresponding to the business elements on the current page. Business elements refer to the functional units that constitute the current page, including the page itself, UI components, or data fields. There is a correspondence between DOM elements and business elements; DOM elements are the technical implementation carriers of business elements on the front-end page, and a business element is typically composed of one or more DOM elements. For example, the business element "submit button" technically corresponds to a DOM element with a specific code path marker. <button>DOM elements. Code path tags establish a mapping between DOM elements and the source code implementation behind business elements, enabling the server to capture technical tags and update business-level knowledge. The nested hierarchy mentioned here refers to the parent-child relationship of business elements within the current page, such as a page containing components, and components containing fields. The node name mentioned here refers to the text used to identify the business meaning of the node, such as the page title or component label. The node type mentioned here refers to the identifier used to distinguish node levels, including page, component, or field. The functional description mentioned here refers to the natural language description of the node's business function.

[0067] When initializing the knowledge tree, the server first creates a root node, recording the current page's path and metadata within it. Then, the server parses the current page's structure, identifying all business elements contained within. Based on the visual inclusion relationships of these business elements in the user interface or the component nesting relationships in the code, the server determines the hierarchical nesting relationships. For example, if a form component contains multiple input fields, the form component node is the parent node, and the input field nodes are the child nodes. The server attaches child nodes to their corresponding parent nodes according to the hierarchical nesting relationships, forming a tree structure. This ensures that the generated knowledge tree accurately reflects the current page's business logic structure, providing a structural foundation for subsequent node-based incremental updates.

[0068] When constructing each node in the historical knowledge tree, the server extracts the display label of the business element as the node name, such as "Submit Button" or "Username Input Box". The server sets the node type based on the nature of the business element: "Page" for the entire page, "Component" for a UI control, and "Field" for a data item. The server also generates a functional description to record the business purpose of the node. The server stores an array of child nodes through a child node relationship field to maintain the hierarchical structure of the tree. This ensures that each node in the historical knowledge tree possesses complete semantic and structural information, enabling the server to accurately compare the functional and structural changes of each node when performing incremental update operations on nodes in the historical knowledge tree using context sequences.

[0069] When building nodes, the server reads code path markers from user operation logs or obtains the corresponding code location from the front-end source code parsing results. The server writes the relative path and row / column number of the source file indicated by the code path marker into the node's code location information field. For example, if the code path marker for a component node is " / src / components / Login.tsx:20:10", then the code location information field stores that path and row / column number. Establishing a precise mapping between business nodes in the historical knowledge tree and the front-end source code allows the server to trace the specific code implementation through the code location information field, providing a reliable basis for subsequent code verification information.

[0070] Based on the publicly available solutions described above, the structured storage and code-level traceability of business knowledge are achieved by standardizing the construction method of the historical knowledge tree. Using the current page as the root node and constructing child nodes according to hierarchical nesting relationships ensures that the knowledge tree structure is consistent with the page's business structure, facilitating understanding and maintenance. Furthermore, by recording node type, function description, and code location information fields, the information dimensions of the nodes are enriched, enabling the server to perform semantic comparisons based on multi-dimensional information. The code location information field is generated based on code path markers, ensuring that each node in the knowledge tree can be accurately associated with the path and row / column number information in the source code file.

[0071] In one or more embodiments of this disclosure, such as Figure 5 This is a flowchart illustrating the context sequence generation method provided in an embodiment of this disclosure. Figure 5 As shown, a context sequence containing business operation information and code verification information is constructed based on screen recording data, including: Step S501: Combining user operation logs, network interface log response data, page screenshots, and page paths into a business operation context subarray. Step S502: Combining source code fragments obtained from parsing code path markers into a code verification context subarray. Step S503: Concatenating the context sequences according to the priority of business operation contexts over code verification contexts to form a context sequence.

[0072] It should be noted that the business operation information refers to a data set reflecting the user's business intent and operation results on the current page; the code verification information refers to a data set reflecting the code implementation details behind the business operation; the context sequence refers to the input data sequence provided to the processing engine for generating a new knowledge tree; the business operation context subarray refers to the data structure in the context sequence specifically storing the business operation information; the code verification context subarray refers to the data structure in the context sequence specifically storing the code verification information; the source code fragment refers to the code content within a specific line range read from the source code file; and the priority refers to the order in which the data is arranged in the context sequence, used to indicate the order in which the processing engine focuses on the data.

[0073] The server extracts user operation logs from screen recording data to obtain user action sequences such as clicks and inputs; it extracts response data from network interface logs to obtain backend data return results triggered by the operation; it extracts page screenshots to obtain the visual state of the page after the operation; and it extracts page paths to obtain the routing identifier of the current business scenario. The server packages these four types of data into a structured format to form a business operation context subarray. By integrating business information from four dimensions—behavior, data, visual, and routing—the server enables the business operation information to fully reconstruct the business scenario of the user's operation, providing rich contextual basis for subsequent understanding of business logic.

[0074] Furthermore, the server iterates through each record in the user operation log, reading the code path markers within them. Based on the relative path and line / column information of the source code file carried in the code path markers, the server accesses the source code repository or code library storing the front-end business system, reads the code content within a specified line range of the corresponding source code file, and forms a source code fragment. The server collects the source code fragments corresponding to all operation records, forming a code verification context subarray. This achieves a mapping from runtime behavior to static code implementation, ensuring that the code verification information accurately reflects the code logic corresponding to each business operation.

[0075] The server places the business operation context subarray at the beginning of the context sequence and the code verification context subarray at the end. This order ensures that when the processing engine consumes the context sequence, it receives and processes business operation information first, followed by code verification information. This arrangement allows the processing engine to prioritize understanding the business intent based on user operation logs, network interface log response data, and page screenshots, forming an initial business understanding. It then uses source code snippets to verify and correct this understanding at the code level. This effectively avoids the problem of the processing engine focusing excessively on implementation details and neglecting business logic due to directly inputting code, ensuring the business-oriented nature of knowledge generation.

[0076] Based on the publicly available solutions described above, hierarchical fusion of multimodal data is achieved by constructing context sequences containing business operation information and code verification information. User operation logs, network interface log response data, page screenshots, and page paths are combined into a business operation context subarray, ensuring the integrity of business scenario information. Source code fragments parsed based on code path markers are combined into a code verification context subarray, ensuring the accuracy of code implementation information. Furthermore, concatenating these subarrays according to the priority of business operation contexts over code verification contexts optimizes the information consumption order of the processing engine. This construction method effectively improves the accuracy of understanding subsequent incremental update operations, ensuring that the generated new knowledge tree conforms to both business logic and code implementation, effectively solving the knowledge fragmentation problem caused by the lack of semantic associations among multi-source data.

[0077] In one or more embodiments of this disclosure, such as Figure 6 This is a flowchart illustrating the new knowledge tree update method provided in this embodiment of the disclosure. Figure 6 As shown, incremental updates are performed on nodes in the historical knowledge tree using context sequences to generate a new knowledge tree, including: Step S601: Input the historical knowledge tree and context sequences into a large language model. Step S602: Calculate the semantic similarity between each node in the historical knowledge tree and the context sequence using the large language model. Step S603: If the semantic similarity is lower than a preset threshold and the context sequence contains new business elements, then a node addition operation is performed. Step S604: If the semantic similarity is lower than a preset threshold and the context sequence describes the same business element as an existing node but the functional logic has changed, then a node update operation is performed. Step S605: If the semantic similarity is higher than a preset threshold, then the original node is reused.

[0078] It should be noted that the "large language model" mentioned here refers to a natural language processing model based on deep learning technology, possessing semantic understanding, logical reasoning, and text generation capabilities. The semantic similarity mentioned here refers to the quantified value of the degree of matching in business meaning between each node in the historical knowledge tree and the context sequence. The preset threshold mentioned here refers to the numerical limit used to determine whether the semantic similarity meets the reuse conditions. The business element mentioned here refers to the functional unit that constitutes the current page, including the page itself, interface components, or data fields. The functional logic mentioned here refers to the operational rules and data processing flow executed by business elements in a business scenario.

[0079] The server serializes the historical knowledge tree into structured text data and formats the context sequence into prompt word input data. The server then inputs both of this data into the large language model, enabling the model to simultaneously acquire existing business knowledge benchmarks (the historical knowledge tree) and the latest business operation evidence (the context sequence). This provides the large language model with a complete comparison context, allowing it to make incremental judgments based on historical knowledge rather than generating data from scratch, thus ensuring the continuity and consistency of knowledge updates.

[0080] The large language model traverses each node in the historical knowledge tree, extracting the node name, functional description, and code location information, and semantically compares them with the business operation information and code verification information in the context sequence. The large language model calculates the semantic similarity between the two, for example, by quantifying the degree of matching of business meanings through a vector space model or attention mechanism. This achieves automatic knowledge comparison without manual verification, accurately identifying subtle changes in business logic and providing a basis for subsequent execution decisions.

[0081] When the large language model determines that the semantic similarity between a certain part of the context sequence and all nodes in the historical knowledge tree is below a preset threshold, and identifies the unrecorded business element corresponding to that part of the context sequence, the large language model generates new node data, including node name, node type, and functional description, and attaches the new node to the appropriate level in the historical knowledge tree. This allows the new knowledge tree to promptly absorb new knowledge generated by business iterations and avoids knowledge omissions.

[0082] When the large language model determines that the semantic similarity between the context sequence and an existing node in the historical knowledge tree is below a preset threshold, but identifies that both point to the same business element, and the functional logic in the context sequence is inconsistent with the functional description of the existing node, the large language model modifies the functional description and code location information fields of the existing node to reflect the latest business logic. This achieves dynamic synchronization of business knowledge, ensuring that the node information in the new knowledge tree always remains consistent with the functional logic actually running online.

[0083] When the large language model determines that the semantic similarity between the context sequence and an existing node in the historical knowledge tree is higher than a preset threshold, it indicates that the business element corresponding to that node has not undergone substantial changes. The large language model directly retains all field information of the existing node without performing any modification operations, and directly incorporates it into the new knowledge tree. This effectively avoids unnecessary repeated calculations and storage, significantly reduces the resource consumption of knowledge updates, and maintains the stability of the historical knowledge structure, preventing knowledge tree structure chaos caused by frequent changes.

[0084] Based on the publicly available solutions described above, intelligent incremental update operations achieved through a large language model significantly improve the efficiency and accuracy of knowledge maintenance. By comparing semantic similarity calculations with preset thresholds, automated identification of node changes is achieved without manual intervention. By differentiating between three strategies—node addition, node update, and reuse of existing nodes—refined evolution of the knowledge tree is realized, ensuring timely inclusion of new business information while avoiding the ineffective overwriting of old knowledge.

[0085] In one or more embodiments of this disclosure, such as Figure 7 This is a flowchart illustrating the knowledge base storage method provided in an embodiment of this disclosure. Figure 7 As shown, storing the new knowledge tree in the knowledge base includes: Step S701: Generating a unique file identifier based on the project name, page path, and timestamp. Step S702: Serializing the new knowledge tree into a structured data file, uploading it to the knowledge base service, and obtaining the file identifier returned by storage. Step S703: Establishing a mapping record between the file identifier and the page path in the database.

[0086] It's important to note that the unique file identifier mentioned here refers to a string used to uniquely identify a new knowledge tree file, generated by combining the project name, page path, and timestamp. The timestamp records the time the new knowledge tree was generated, used to distinguish different versions of the same page. A structured data file is a data file organized according to a predetermined format and parsable by a program, such as a JSON file. The knowledge base service refers to the backend service interface that provides knowledge tree file storage and retrieval functions. The file identifier is a unique identifier returned by the knowledge base service after storing the file, used for subsequent file loading. The database refers to the data storage system used to store mapping relationships. A mapping record is a data entry used to associate a page path with a file identifier, supporting quick lookup of the corresponding file identifier using the page path.

[0087] In practical applications, after generating a new knowledge tree, the server extracts the corresponding project name, page path, and current system time timestamp. The server then concatenates or hashes these three elements according to predetermined rules to generate a unique file identifier. For example, the project name "bill-management", page path " / supplier", and timestamp "20260313103000" are combined into "bill-management_supplier_20260313103000". This ensures that each version of the new knowledge tree has a globally unique identifier, avoiding storage overwriting due to file naming conflicts. Furthermore, the introduction of timestamps supports version tracking of the knowledge tree, facilitating subsequent rollback or comparative analysis.

[0088] The server converts the tree-like data structure of the new knowledge tree into a structured data file conforming to a predetermined pattern, for example, serializing it into a JSON string and writing it to a temporary file. The server then sends a request to the knowledge base service's upload interface via HTTP, encapsulating the structured data file in a form-based data format and uploading it. Upon receiving the structured data file, the knowledge base service stores it in a distributed file system or object storage, generating a file identifier as a return parameter of the storage result. The server parses the returned result to obtain the corresponding file identifier. This achieves persistent storage of the knowledge tree, facilitates cross-system transmission and parsing of the structured data file, and ensures the high availability and scalability of the knowledge tree file through the knowledge base service's specialized storage capabilities.

[0089] The server combines the file identifier, project name, and page path obtained in the previous step into a mapping record and writes it to the database. The mapping record structure includes a file identifier field, a project name field, and a page path field, supporting quick location of the corresponding file identifier by querying the combination of the project name and page path. An index relationship is established between page routes and stored files, enabling the server to directly query the database to obtain the file identifier by project name and page path when retrieving stored historical knowledge trees. Then, based on the file identifier, the server loads the corresponding knowledge tree file from the knowledge base service, significantly improving the efficiency of knowledge retrieval and avoiding a full scan of the storage system.

[0090] Based on the publicly available solutions described above, persistent management and efficient retrieval of the knowledge tree are achieved by standardizing the storage process of the new knowledge tree. A unique file identifier is generated based on the project name, page path, and timestamp, ensuring the uniqueness and traceability of the knowledge tree version. The new knowledge tree is serialized into a structured data file and uploaded to the knowledge base service, leveraging the high reliability of professional storage services. A mapping record between file identifiers and page paths is established in the database, constructing a fast indexing channel from page routing to the stored file. This storage solution enables the server to manage massive numbers of knowledge tree versions at low cost, supporting rapid loading and incremental updates of knowledge.

[0091] In one or more embodiments of this disclosure, such as Figure 8 This is a flowchart illustrating the task generation method provided in an embodiment of this disclosure. Figure 8 As shown, after obtaining the screen recording data of the current page, the process further includes: Step S801: Generating a globally unique task identifier, binding the screen recording data with the task identifier, and adding it to the asynchronous processing queue. Step S802: Periodically triggering queue consumption through a timed scheduling mechanism, retrieving tasks from the asynchronous processing queue, and executing knowledge extraction tasks. The knowledge extraction task includes the following steps: determining the project name and page path, obtaining the historical knowledge tree, constructing the context sequence, performing incremental update operations, and storing the new knowledge tree.

[0092] It should be noted that the globally unique task identifier mentioned here refers to a string or numeric code used to uniquely identify a single knowledge extraction request, ensuring the traceability of the task throughout the system. The task identifier mentioned here refers to the same identification information as the globally unique task identifier, used for binding and querying tasks in subsequent processes; the asynchronous processing queue refers to a data structure used to temporarily store tasks to be processed, supporting first-in-first-out or priority scheduling; the timed scheduling mechanism refers to a control system that periodically executes specific operations according to preset time intervals. Queue consumption mentioned here refers to the process of retrieving tasks from the asynchronous processing queue and processing them.

[0093] Upon receiving screen recording data from the browser plugin, the server immediately generates a globally unique task identifier, for example, using a unique string generated using the UUID algorithm. The server binds the screen recording data to the task identifier, forming a task object, and adds the task object to the asynchronous processing queue. This decouples data reception from data processing, allowing the server to respond to the browser plugin without waiting for processing to complete, significantly improving system response speed and concurrent processing capabilities. Simultaneously, the task identifier ensures the uniqueness and traceability of each screen recording data point throughout the entire processing chain.

[0094] The server is deployed with a timed scheduling mechanism, such as a scheduler based on Cron expressions or time wheel algorithms. This mechanism periodically triggers queue consumption at preset time intervals, such as every 5 seconds. When triggered, the server retrieves one or more tasks from the asynchronous processing queue and initiates the processing flow. This achieves controllable allocation of processing resources, preventing server resource exhaustion due to sudden high concurrency requests, and ensuring system stability and reliability through peak shaving and valley filling.

[0095] First, the process of determining the project name and page path is executed to identify the target business system. Next, the process of retrieving the historical knowledge tree is executed to load the existing knowledge base. Then, the process of constructing the context sequence is executed to prepare for data updates. Following this, the incremental update operation is executed to generate the latest knowledge. Finally, the process of storing the new knowledge tree is executed to complete persistence. This step encapsulates the above multiple steps into a knowledge extraction task, ensuring that each task in the asynchronous processing queue has complete business processing capabilities, thus guaranteeing the standardization and automation of the knowledge extraction process.

[0096] Based on the publicly available solutions described above, generating a globally unique task identifier and binding screen recording data to the task identifier before adding it to the asynchronous processing queue decouples request reception from business processing, supporting traffic buffering in high-concurrency scenarios. Furthermore, periodically triggering queue consumption through a timed scheduling mechanism enables controllable adjustment of the processing rhythm, avoiding resource contention. Encapsulating the core process into knowledge extraction tasks ensures the integrity and consistency of each task's execution. This asynchronous task scheduling mechanism enables the server to stably and efficiently handle massive amounts of page knowledge extraction requests, guaranteeing the continuous availability of the knowledge update service.

[0097] In one or more embodiments of this disclosure, such as Figure 9 This is a schematic flowchart illustrating the knowledge retrieval method provided in an embodiment of this disclosure. Figure 9 As shown, the process also includes a knowledge retrieval step: Step S901: Receive a knowledge query request initiated by the front-end, which includes the page path and project name. Step S902: Query the mapping record in the database based on the page path and project name to obtain the corresponding file identifier. Step S903: Load the corresponding knowledge tree from the knowledge base based on the file identifier and return it to the front-end to assist in code development or business understanding.

[0098] It should be noted that the knowledge retrieval step mentioned here refers to the processing flow where the server responds to the front-end query request and returns a knowledge tree. The knowledge query request mentioned here refers to the request message initiated by the front-end to the server to obtain business knowledge. The mapping record mentioned here refers to the data entry stored in the database used to associate file identifiers with page paths and project names.

[0099] The server listens for requests from the frontend through a predefined application programming interface (API). When the frontend needs to retrieve business knowledge for a specific page—for example, when a developer needs to refer to a component definition on a page during coding—the frontend constructs a knowledge query request, sending the target page's path and project name as request parameters to the server. Upon receiving the request, the server parses the request parameters and extracts the page path and project name. This establishes a knowledge exchange channel between the frontend and the server, enabling the frontend to obtain business knowledge on demand and providing a clear query target for subsequent knowledge retrieval.

[0100] The server uses the extracted page path and project name as the joint query conditions to retrieve mapping records from the database. These mapping records store the association between file identifiers and page paths and project names, supporting quick location of stored files via page routing. If the query is successful, the server extracts the corresponding file identifier from the mapping record; if the query fails, it returns an empty result or an initialization prompt. Utilizing a pre-built indexing mechanism, millisecond-level location from page routing to stored file identifiers is achieved, avoiding a full scan of the knowledge base and significantly improving the efficiency of knowledge retrieval.

[0101] The server uses the extracted file identifier to send a loading request to the knowledge base service. The knowledge base service retrieves the corresponding knowledge tree file based on the file identifier and returns it. The server serializes the loaded knowledge tree into a data format that the front-end can parse, such as a JSON object, and returns it to the front-end via a response message. After receiving the knowledge tree, the front-end can use it to assist in code development, such as providing information on component field definitions, event types, and code locations in an integrated development environment; or for business understanding, such as showing product managers the hierarchy of business elements and functional descriptions of a page.

[0102] Based on the publicly available solutions described above, the on-demand loading and application of the knowledge tree are achieved through a knowledge retrieval process. File identifiers are obtained by querying mapping records based on page paths and project names, utilizing a pre-established indexing mechanism to significantly improve knowledge retrieval efficiency. Loading the knowledge tree from the knowledge base based on the file identifier decouples knowledge storage from knowledge consumption, supporting independent evolution and version management of the knowledge tree. Returning the knowledge tree to the front end assists in code development or business understanding, enabling business knowledge to directly empower the development process.

[0103] To facilitate understanding, the process of knowledge extraction and storage will be explained below through specific examples. For instance... Figure 10 This is a schematic diagram illustrating the knowledge extraction and storage process as exemplified in this disclosure. For example... Figure 10 As shown, The first step is the data collection phase. During this phase, users perform business operations on the system page, triggering the generation of recorded data. This specifically includes: The screen recording plugin is responsible for recording operations, capturing user clicks, inputs and other interactive behaviors, and generating screen recording data that includes user operation logs, network interface logs, page screenshots and page metadata.

[0104] The Babel plugin is responsible for injecting code locations. During the compilation phase of the front-end business system, it traverses the code abstract syntax tree and injects code path markers containing the relative path of the source file and the row and column numbers into the DOM elements.

[0105] Through the collaborative work of the two plugins mentioned above, the user operation logs in the screen recording data include code path markers that contain source code location information associated with the user operations, thus achieving automatic association between user operations and code implementation.

[0106] The second step is the knowledge processing stage. This stage is the core processing flow and specifically includes: File ID retrieval using project + URL: After receiving the screen recording data, the server determines the project name and page path of the current page based on the page URL in the page metadata or the code path marker in the user operation log.

[0107] Determining the existence of a historical page knowledge tree: Based on the project name and page path, the server initiates a retrieval request to the RAG (Knowledge Base) to query whether a stored historical page knowledge tree exists.

[0108] Obtain the historical knowledge tree: If a historical knowledge tree exists, load it from RAG; otherwise, initialize an empty knowledge tree structure.

[0109] Context list construction: The server constructs a context sequence containing business operation information and code verification information based on screen recording data. Specifically: Business operation information includes user operation logs, network interface log response data, page screenshots, and page paths. Code verification information is obtained by retrieving associated code, i.e., reading the corresponding source code fragment from the project code repository based on the code file path. Knowledge file path and field mapping storage: The server maintains mapping records between file identifiers and page paths in the database, supporting fast retrieval.

[0110] The third step is the knowledge storage stage.

[0111] The knowledge storage phase completes the generation and persistence of the knowledge tree: LLM (Large Language Model) takes a historical knowledge tree and a context sequence as input and performs incremental update operations: Knowledge Tree: The input historical knowledge tree serves as the initial knowledge base. Context: The constructed context sequence serves as the basis for updates. Traversal Generation: LLM calculates the semantic similarity between each node in the historical knowledge tree and the context sequence, and performs corresponding operations based on the calculation results: If the semantic similarity is lower than a preset threshold and the context sequence contains new business elements, a node addition operation is performed. If the semantic similarity is lower than the preset threshold and the context sequence describes the same business element as an existing node but the functional logic has changed, a node update operation is performed. If the semantic similarity is higher than the preset threshold, the original node is reused.

[0112] Termination Check: After LLM has traversed all context elements, it checks whether the knowledge tree generation is complete. If not, it continues the loop; if complete, it outputs the final new knowledge tree.

[0113] Knowledge tree written to file: The server serializes the new knowledge tree into a structured data file.

[0114] Store RAG file ID: The server uploads the structured data file to the RAG service, obtains the file ID returned by the storage, and establishes a mapping record between the file ID and the page path in the database to complete the persistent storage of the knowledge tree.

[0115] Based on any of the above embodiments, this disclosure also provides a knowledge extraction device. Figure 11 This is a schematic block diagram of the structure of a knowledge extraction device according to one embodiment of this disclosure. Figure 11 As shown, the knowledge extraction device includes: an acquisition module 1101, used to acquire screen recording data of the current page; the screen recording data includes user operation logs and page metadata; the user operation logs contain code path markers of source code location information associated with user operations. A determination module 1102, used to determine the project name and page path of the current page based on the screen recording data. An acquisition module 1103, used to acquire a stored historical knowledge tree based on the project name and page path. A construction module 1104, used to construct a context sequence containing business operation information and code verification information based on the screen recording data. A generation module 1105, used to perform incremental update operations on the nodes in the historical knowledge tree using the context sequence, generate a new knowledge tree, and store the new knowledge tree in the knowledge base.

[0116] Optionally, the screen recording data may also include: network interface logs and page screenshots; wherein, the network interface logs are used to record the interface requests and response data triggered by user operations; and the page screenshots are visual snapshots of the current page at the end of the recording.

[0117] The generation module 1105 is used to traverse the code abstract syntax tree during the compilation phase of the front-end business system, inject code path markers containing the relative path of the source code file and the row and column numbers into the DOM elements; when the user operation triggers the current page, it captures the code path markers corresponding to the triggered elements in the current page and records them in the user operation log.

[0118] The determination module 1102 is used to match the page URL in the page metadata with the path format field in the pre-configured mapping file using regular expressions. If the match is successful, the project name and page path corresponding to the current page are extracted. If the match fails, the code path markers in the user operation log are traversed, the project identifier is extracted from the relative path of the source code file as the project name, and the page path is obtained by querying the pre-configured mapping file.

[0119] The generation module 1105 is used to construct child nodes based on the hierarchical nesting relationship of business elements in the current page, with the current page as the root node; for each node, it records the node name, node type, function description, code location information and child node relationship fields; among them, the node type is used to distinguish the page, component or field level; the code location information field is generated based on the code path mark and is used to store the path and row and column number information of the node in the source code file.

[0120] Module 1104 is used to combine user operation logs, network interface log response data, page screenshots and page paths into a business operation context subarray; combine source code fragments obtained from code path tag parsing into a code verification context subarray; and concatenate them into a context sequence according to the order in which the business operation context has a higher priority than the code verification context.

[0121] The generation module 1105 is used to input the historical knowledge tree and context sequence into the large language model; the large language model calculates the semantic similarity between each node in the historical knowledge tree and the context sequence; if the semantic similarity is lower than the preset threshold and the context sequence contains new business elements, a node addition operation is performed; if the semantic similarity is lower than the preset threshold and the context sequence describes the same business element as an existing node but the functional logic has changed, a node update operation is performed; if the semantic similarity is higher than the preset threshold, the original node is reused.

[0122] The generation module 1105 is used to generate a unique file identifier based on the project name, page path, and timestamp; serialize the new knowledge tree into a structured data file, upload it to the knowledge base service and obtain the file identifier returned by the storage; and establish a mapping record between the file identifier and the page path in the database.

[0123] The generation module 1105 is used to generate a globally unique task identifier, bind the screen recording data with the task identifier and add it to the asynchronous processing queue; the queue consumption is periodically triggered through a timed scheduling mechanism, the task is retrieved from the asynchronous processing queue and the knowledge extraction task is executed.

[0124] Optionally, it also includes a retrieval module 1106, which is used to receive a knowledge query request initiated by the front end, the knowledge query request containing a page path and a project name; query the mapping record in the database based on the page path and project name to obtain the corresponding file identifier; load the corresponding knowledge tree from the knowledge base based on the file identifier, and return it to the front end to assist in code development or business understanding.

[0125] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0126] The subject executing the knowledge extraction method in the specific embodiments of this disclosure can be an electronic device such as a server (including a local server or a cloud computing platform).

[0127] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the knowledge extraction method of any of the embodiments described above.

[0128] Figure 12 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure.

[0129] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0130] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Component (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one connection line is used in this diagram, but this does not imply that there is only one bus or only one type of bus.

[0131] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. "Readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of readable storage media include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0132] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0133] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0134] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] It is understood that before using the technical solutions disclosed in the embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained. Simultaneously, sensitive information should be de-identified (e.g., de-identification, anonymization, etc.). For example, in response to receiving a user's active request, a prompt message can be sent to the user to clearly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the operations of this disclosure's technical solutions, based on the prompt message. As an optional but non-limiting implementation, the method of sending a prompt message to the user in response to receiving a user's active request can be, for example, a pop-up window, where the prompt message can be presented in text form. Furthermore, the pop-up window can also include a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0139] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, or characteristic described in connection with that embodiment / mode or example is included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0140] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0141] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.< / button>

Claims

1. A knowledge extraction method, characterized in that, The method includes: Obtain the screen recording data of the current page; the screen recording data includes user operation logs and page metadata; the user operation logs contain code path markers of source code location information associated with user operations; Based on the screen recording data, determine the project name to which the current page belongs and the page path of the current page; Based on the project name and the page path, retrieve the stored historical knowledge tree; Construct a context sequence containing business operation information and code verification information based on the screen recording data; The nodes in the historical knowledge tree are incrementally updated using the context sequence to generate a new knowledge tree, which is then stored in the knowledge base.

2. The method according to claim 1, characterized in that, The generation method of the code path marker includes: During the compilation phase of the front-end business system, the code abstract syntax tree is traversed to inject code path markers containing the relative path of the source file and the row and column numbers into the DOM elements; When a user action triggers the current page, the code path marker corresponding to the triggered element on the current page is captured and recorded in the user action log.

3. The method according to claim 1, characterized in that, Determining the project name and page path of the current page based on the screen recording data includes: The page URL in the page metadata is matched with the path format field in the pre-configured mapping file using regular expressions. If a match is found, the project name and page path corresponding to the current page are extracted. If the match fails, the code path markers in the user operation log are traversed, the project identifier is extracted from the relative path of the source code file as the project name, and the page path is obtained by querying the pre-configured mapping file.

4. The method according to claim 1, characterized in that, The construction methods of the historical knowledge tree include: Using the current page as the root node, construct child nodes according to the hierarchical nesting relationship of business elements in the current page; Record the node name, node type, function description, code location information, and child node relationship fields for each node; The node type is used to distinguish between page, component, or field levels; The code location information field is generated based on the code path marker and is used to store the path and row / column number information of the node in the source code file.

5. The method according to claim 1, characterized in that, The screen recording data also includes: network interface logs and page screenshots; The network interface log is used to record interface requests and response data triggered by user operations; the page screenshot is a visual snapshot of the current page when recording ends.

6. The method according to claim 5, characterized in that, The construction of a context sequence containing business operation information and code verification information based on the screen recording data includes: The user operation log, the response data of the network interface log, the page screenshot, and the page path are combined into a business operation context subarray; The source code fragments parsed based on the code path markers are combined into a code verification context subarray; The context sequence is formed by concatenating the contexts in order of priority: business operation contexts take precedence over code verification contexts.

7. The method according to claim 1, characterized in that, The step of performing incremental update operations on nodes in the historical knowledge tree using the context sequence to generate a new knowledge tree includes: Input the historical knowledge tree and the context sequence into the large language model; The semantic similarity between each node in the historical knowledge tree and the context sequence is calculated using the large language model. If the semantic similarity is lower than a preset threshold and the context sequence contains new business elements, then a node addition operation is performed; If the semantic similarity is lower than a preset threshold and the context sequence describes the same business element as an existing node but the functional logic has changed, then a node update operation is performed. If the semantic similarity is higher than a preset threshold, the original node is reused.

8. The method according to claim 1, characterized in that, The step of storing the new knowledge tree in the knowledge base includes: A unique file identifier is generated based on the project name, the page path, and the timestamp. The new knowledge tree is serialized into a structured data file, uploaded to the knowledge base service, and the file identifier returned by the storage is obtained; Establish a mapping record between the file identifier and the page path in the database.

9. The method according to claim 1, characterized in that, The method further includes: a knowledge retrieval step: Receive a knowledge query request initiated by the front end, wherein the knowledge query request includes the page path and project name; Based on the page path and project name, query the mapping record in the database to obtain the corresponding file identifier; Based on the file identifier, the corresponding knowledge tree is loaded from the knowledge base and returned to the front end to assist in code development or business understanding.

10. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 9.

Images