Knowledge data processing method, device, equipment, medium and program product

Abstract

This application discloses a method, apparatus, device, medium, and program product for processing knowledge data, belonging to the field of artificial intelligence. The method includes: segmenting the knowledge data according to the structural features of Markdown to obtain multiple structural fragmentation units; performing adaptive semantic segmentation based on the semantic density parameters and contextual association parameters of the knowledge data in the structural fragmentation units to obtain a first semantic fragmentation unit; based on the segmented first semantic fragmentation unit, using semantic similarity calculation and edit distance calculation to obtain association weights and knowledge difference points; integrating and processing the first semantic fragmentation unit according to the association weights and knowledge difference points through a large model to generate an optimized Markdown; converting the optimized Markdown into a first knowledge card, where the knowledge difference points in the first knowledge card are displayed in a visually enhanced manner. According to the embodiments of this application, the efficiency of knowledge data retrieval can be improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, and in particular relates to a method, apparatus, device, medium and program product for processing knowledge data. Background Technology

[0002] In the fields of customer service and knowledge base management, knowledge documents are typically stored in the knowledge base as unstructured, flat text. Customer service representatives or knowledge base administrators need to search for corresponding response information from the vast amount of knowledge data in the knowledge base according to their needs. However, the knowledge data in the knowledge base may involve complex product terms and business rules from multiple regions, resulting in high similarity between different product terms and business rules from different regions. The key differences may lie in details such as terms or parameters. Based on this highly similar but subtly different knowledge data, it takes a lot of time to identify, whether manually or by machine, which greatly affects the data search efficiency of customer service representatives or knowledge base administrators. Summary of the Invention

[0003] This application provides a method, apparatus, device, medium, and program product for processing knowledge data, which can improve the efficiency of knowledge data retrieval for customer service or knowledge base administrators.

[0004] In a first aspect, embodiments of this application provide a method for processing knowledge data, comprising: segmenting the knowledge data in Markdown according to the structural features of Markdown to obtain multiple structural fragmentation units; adaptively semantically segmenting the structural fragmentation units based on the semantic density parameters and context association parameters of the knowledge data in the structural fragmentation units to obtain first semantic fragmentation units; obtaining the association weights and knowledge difference points between the first semantic fragmentation units based on the segmented first semantic fragmentation units using semantic similarity calculation and edit distance calculation; integrating and processing the first semantic fragmentation units according to the association weights and knowledge difference points through a large model to generate optimized Markdown, wherein the optimized Markdown includes content representing knowledge difference points; converting the optimized Markdown into a first knowledge card, wherein the first knowledge card is used to display the content of the optimized Markdown, and the knowledge difference points in the first knowledge card are displayed in a visually enhanced manner.

[0005] Secondly, embodiments of this application provide a knowledge data processing apparatus, comprising: a structure segmentation module, used to segment the knowledge data in Markdown according to the structural features of Markdown to obtain multiple structure segmentation units; a semantic segmentation module, used to perform adaptive semantic segmentation of the structure segmentation units based on the semantic density parameters and context association parameters of the knowledge data in the structure segmentation units to obtain a first semantic segmentation unit; an analysis module, used to obtain the association weights and knowledge difference points between the first semantic segmentation units based on the segmented first semantic segmentation units using semantic similarity calculation and edit distance calculation; an integration module, used to integrate the first semantic segmentation units according to the association weights and knowledge difference points through a large model to generate an optimized Markdown, wherein the optimized Markdown includes content representing the knowledge difference points; and a card generation module, used to convert the optimized Markdown into a first knowledge card, wherein the first knowledge card is used to display the content of the optimized Markdown, and the knowledge difference points in the first knowledge card are displayed in a visually enhanced manner.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the knowledge data processing method of the first aspect.

[0007] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the knowledge data processing method of the first aspect.

[0008] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, implements the knowledge data processing method of the first aspect.

[0009] This application provides a method, apparatus, device, medium, and program product for processing knowledge data. It can record knowledge data using Markdown. Based on the structural features of Markdown, the knowledge data in Markdown is initially and coarsely segmented to obtain structural fragment units. Based on the semantic density parameters and context parameters of the knowledge data in the structural fragment units, the structural fragment units are dynamically segmented at the semantic level to obtain first semantic fragment units that are topic-focused and semantically self-consistent. Using semantic similarity and edit distance calculations, the association weights and knowledge difference points between the first semantic fragment units can be obtained, thereby determining the associations and differences between the first semantic fragment units. Guided by the associations and differences between the first semantic fragment units, a large language model integrates and processes the first semantic fragment units, completing a deep logical reconstruction of knowledge. The reconstructed knowledge data is displayed in the form of knowledge cards, and the knowledge difference points between semantic fragment units can be displayed in the knowledge cards through enhanced visualization. This allows for rapid identification of differences in highly similar knowledge data, thereby significantly improving the efficiency of knowledge data retrieval for customer service or knowledge base administrators. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a knowledge data processing method provided in an embodiment of this application; Figure 2 A schematic diagram illustrating an example of Markdown and knowledge cards provided in an embodiment of this application; Figure 3 A schematic diagram illustrating another example of a knowledge card provided in an embodiment of this application; Figure 4 A schematic diagram illustrating an example of the processing flow of a semantic clustering agent provided in an embodiment of this application; Figure 5 A schematic diagram illustrating an example of intelligent association of knowledge data provided in an embodiment of this application; Figure 6 A schematic diagram of the structure of a knowledge data processing apparatus provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the purpose, technical solution, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of the details in these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application. It should be noted that the acquisition, storage, use, and processing of information and data in the embodiments of this application are all authorized by users or relevant organizations and comply with the relevant provisions of national laws and regulations. In the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or necessarily used such a solution.

[0013] In the fields of customer service and knowledge base management, knowledge documents are typically stored in the knowledge base as unstructured, flat text. Customer service representatives or knowledge base administrators need to retrieve corresponding response information from the vast amount of knowledge data in the knowledge base based on their needs. However, the knowledge data in the knowledge base may involve complex product terms and business rules from multiple regions, leading to a high degree of similarity between different product terms and business rules from different regions. The key differences may lie in details such as terms or parameters. Based on this highly similar but subtly different knowledge data, it takes a significant amount of time, whether identified manually or by machine, greatly impacting the data retrieval efficiency of customer service representatives or knowledge base administrators. For example, the rules for consumer vouchers in different regions may only differ in category, with the rules being essentially the same. A knowledge document may record a dozen or more highly similar consumer voucher rules, making it difficult to quickly identify them. The diverse formats and extremely high information density of knowledge documents make it difficult to accurately parse and fully extract the knowledge data, hindering the construction of accurate and traceable knowledge graphs and knowledge bases. The credibility of the output of large models with the assistance of knowledge graphs and / or knowledge bases will also decrease. Moreover, in scenarios with limited computing resources, large models struggle to achieve lossless processing of long documents. The semantic integrity of long documents is significantly reduced during the processing of long documents by large models, thereby reducing the accuracy of large models in understanding the knowledge data in long documents.

[0014] This application provides a method, apparatus, device, medium, and program product for processing knowledge data. It can record knowledge data using Markdown, perform initial coarse-grained segmentation of Markdown based on its structural characteristics, and then dynamically segment the structural fragments obtained from the initial coarse-grained segmentation at the semantic level to obtain semantically focused and semantically self-consistent fragments. By analyzing the relationships and differences between semantic fragments, the fragments are integrated to complete a deep logical reconstruction of knowledge. The reconstructed knowledge data is then displayed in the form of knowledge cards. Enhanced visualization can be used to show the differences between semantic fragments within the knowledge cards, enabling rapid identification of differences in highly similar knowledge data, thereby significantly improving the data retrieval efficiency for customer service or knowledge base administrators. By segmenting and anchoring structural fragmentation units, dynamically segmenting semantic fragmentation units, and integrating with contextual relationships, the reconstruction of themes and semantics based on knowledge data can be achieved. While ensuring the semantic integrity of the knowledge data, more coherent, accurate, and concise knowledge data is obtained. Semantic fragmentation units, as input to the large model, avoid resource limitations caused by directly inputting long documents during the large model's parsing phase. Multiple semantic fragmentation units do not reduce semantic integrity during large model processing, ensuring the accuracy of the large model's understanding of the knowledge data. This application can also generate and display mind maps of the knowledge data, allowing bidirectional switching between mind maps and knowledge cards, and arbitrarily switching between a global overview and knowledge details. Entities and relationships are extracted from the knowledge data after deep logic reconstruction from the mind map, and a rule base ensures the compliance and credibility of entities and relationships. When new rules are generated by entities and relationships, the rule base achieves self-learning evolution, and the extracted entities and relationships can be used to update the knowledge graph of the knowledge data, thereby achieving real-time linkage and expansion of knowledge data. This provides the large model with a more accurate knowledge graph and knowledge base, thus improving the credibility of the output of the large model with the assistance of the knowledge graph and / or knowledge base.

[0015] The following describes the processing methods, apparatus, equipment, media, and program products of the knowledge data provided in this application.

[0016] This application provides a method for processing knowledge data, which can be applied to scenarios involving knowledge data processing in fields such as customer service, knowledge base management, education and training, and media dissemination. This method can be executed by a knowledge data processing device, electronic equipment, etc., and is not limited thereto. Figure 1 A flowchart of a knowledge data processing method provided in an embodiment of this application is shown below. Figure 1 As shown, the method for processing this knowledge data may include steps S101 to S105.

[0017] In step S101, the knowledge data in Markdown is segmented according to the structural characteristics of Markdown to obtain multiple structural segmentation units.

[0018] Markdown is a lightweight markup language for documents that can carry knowledge data. In some examples, knowledge documents can be converted to Markdown first, and then processed. Markdown's structural features may include, but are not limited to, one or more of the following: table of contents, heading levels, list items, table items, image items, code block start and end markers, quotation block identifiers, statement boundaries, image layout, image columns, and image title areas. These features can be used to segment knowledge data in Markdown according to its structure, resulting in multiple structural fragment units. These structural fragment units are preliminary, coarse-grained, and semantically complete fragment units, which can define the scope for subsequent, more refined semantic understanding of the data. Markdown segmentation can be performed using a structure recognition agent.

[0019] In some examples, when Markdown has explicit structural features, a segmentation model can be used to identify these features and segment the Markdown according to them, resulting in multiple structural fragments. Explicit structural features are explicit markers in Markdown that represent structure. These features may include, but are not limited to, one or more of the following: table of contents, heading levels, list items, table items, image items, code block start and end markers, reference block identifiers, image layout, image columns, and image title areas. For example, if the knowledge data in the Markdown is the content of a product manual, it can be segmented into multiple structural fragments such as product overview, features, user guide, and troubleshooting based on the table of contents, chapter titles, and figure / table numbers. Similarly, if the Markdown includes images, it can be segmented according to the visual structural features of the images, such as layout, columns, and title areas, resulting in multiple structural fragments. Segmentation models may include, but are not limited to, small models and large models. A small model is a model with a relatively small number of parameters; for example, a small language model (SLM). The large model is the Large Language Model (SLM).

[0020] In other examples, when Markdown lacks explicit structural features, a segmentation model uses syntactic analysis algorithms to identify sentence boundaries and segment the Markdown according to these boundaries, resulting in multiple structural fragment units. When Markdown lacks explicit structural features, syntactic analysis algorithms can be used to identify sentence boundaries in continuous text. For example, punctuation marks and paragraph separators can be used to identify sentence boundaries, thereby segmenting continuous text without explicit structural features into structural fragment units. Specific details of the segmentation model can be found in the relevant explanations in the examples above and will not be repeated here.

[0021] In some examples, before segmenting the knowledge data in Markdown to obtain structural fragment units, the knowledge data in Markdown can be preprocessed using standardization. At least two large models are then used to parse the structural features of the standardized Markdown, and the parsing comparison results are obtained. Due to the high information density of Markdown, traditional flat layouts are prone to problems such as disordered tables and misformatted code blocks. Standardization preprocessing can help obtain correctly formatted Markdown. Standardization preprocessing may include, but is not limited to, one or more of the following: special character cleaning, custom syntax extension, and metadata extraction. Special character cleaning can delete or correct non-content-related characters that do not conform to standards or interfere with parsing. Custom syntax extension can introduce user-defined syntax in addition to standard Markdown syntax, converting it into content that subsequent parsing can understand. Markdown metadata represents the structured information of the Markdown document's own attributes. For example, Markdown metadata may include, but is not limited to, the document name, author name, creation date, keywords, knowledge data source, document category, summary, and storage location. When needed in subsequent operations, Markdown metadata can be used for traceability, ensuring the integrity of the data source. Different large models are invoked to parse and extract structural features such as heading levels, code blocks, and metadata from the standardized preprocessed Markdown. Each large model outputs its extracted structural features, and the structural features extracted by different large models can be compared to obtain parsing comparison results. These results characterize whether the structural features extracted by different large models are the same and identify the differences. The parsing comparison results are used to assist in segmenting the knowledge data in the Markdown. They can also serve as input to the segmentation model, guiding it to focus more attention on the differences in the structural features output by different models. By extracting the structural features of the same Markdown using different large models, cross-validation can be used to eliminate single-model bias, prevent information distortion, and thus reduce or even avoid the adverse effects of a single model's "illusion" or "bias" on subsequent structural unit segmentation, ensuring that the structural features of the Markdown are accurately understood by the segmentation model.

[0022] In step S102, based on the semantic density parameters and context association parameters of the knowledge data in the structural segmentation unit, the structural segmentation unit is adaptively semantically segmented to obtain the first semantic segmentation unit.

[0023] The segmentation of structural data units is based on the topic and semantics of the knowledge data, achieving a deeper level of semantic and boundary understanding, thus obtaining the first semantic segment unit. For a structural segment unit, based on the knowledge data within that unit, semantic density parameters that characterize the semantic density of the knowledge data and contextual correlation parameters that characterize the relationships between knowledge data can be obtained. In some examples, the semantic density parameter may include keyword frequency; keyword frequency is positively correlated with the semantic density of the knowledge data. The higher the keyword frequency in the knowledge data, the higher the semantic density and the closer the semantic connections. In some examples, the contextual correlation parameter may include relevance and topic similarity; the higher the relevance of the knowledge data, the closer the connections between that part of the knowledge data, and therefore, that part of the knowledge data is not suitable for segmentation. The topic similarity of the knowledge data characterizes the degree of similarity between the topics of the knowledge data; the higher the topic similarity, the more coherent the topics of that part of the knowledge data, and therefore, that part of the knowledge data is not suitable for segmentation. Based on semantic density and contextual relevance parameters, semantically close and highly relevant knowledge data are retained within the same semantic segmentation unit. Segmentation is then performed from semantically less close and less relevant positions to obtain the first semantic segmentation unit. Adaptive semantic segmentation of structural segmentation units can be achieved by pre-setting segmentation rules based on semantic density and contextual relevance parameters; alternatively, prompt words can be constructed to guide the large model to achieve adaptive semantic segmentation of structural segmentation units based on semantic density and contextual relevance parameters. Structural segmentation units can be segmented into multiple first semantic segmentation units through this adaptive semantic segmentation method. Each first semantic segmentation unit is a topic-focused, semantically close, coherent, and semantically self-consistent semantic segmentation unit. In other words, the first semantic segmentation unit is a topic-focused, semantically close, coherent, and complete semantic segmentation unit.

[0024] In step S103, based on the first semantic segmentation unit obtained by segmentation, semantic similarity operation and edit distance operation are used to obtain the association weight and knowledge difference points between the first semantic segmentation units.

[0025] The first semantic fragments obtained from the segmentation are discrete, but the entire Markdown often integrates multiple information points. Therefore, it is necessary to reconstruct and strengthen the connections between knowledge data in multiple first semantic fragments. For multiple first semantic fragments obtained from a structural fragmentation, semantic similarity calculations can be used to obtain parameters that characterize the similarity between different first semantic fragments. The similarity between different first semantic fragments is related to the correlation between them. The higher the similarity between different first semantic fragments, the higher the correlation represented by the correlation weight. If the correlation between different first semantic fragments is high, the content of different first semantic fragments may overlap, cross-reference, or have other logical relationships. If the correlation between different first semantic fragments is low, there are differences between them. The correlation weight between first semantic fragments can be obtained from the results of semantic similarity calculations. The correlation weight can characterize the correlation between different first semantic fragments. In some examples, the correlation weight and correlation are positively correlated; the larger the correlation weight, the higher the correlation. For multiple first semantic segments obtained by dividing a structural segmentation unit, edit distance calculation can be used to help quantify the differences and overlaps between different first semantic segments at the character or structural level, and can also help identify the logical relationships between different first semantic segments. Based on the calculation results of association weights and edit distance, knowledge differences in different first semantic segments can be determined. Knowledge differences may include knowledge data that differs in different first semantic segments.

[0026] In step S104, the first semantic segmentation unit is integrated and processed according to the association weight and knowledge difference points through the large model to generate optimized Markdown.

[0027] Association weights and knowledge difference points reflect the connections and differences between different first semantic units. Guided by association weights and knowledge difference points, the large model can integrate multiple first semantic units to obtain optimized Markdown. In some examples, the integration process may include at least one of the following: deduplication, completion, logical sorting, and merging. Deduplication removes redundant content from first semantic units. Completion supplements incomplete content in first semantic units. Logical sorting arranges first semantic units in a logical order. Merging merges the content in first semantic units. Through integration, a logically coherent, accurate, and concise optimized Markdown can be obtained. Optimized Markdown includes content representing knowledge difference points. Optimized Markdown may include knowledge difference points or objects generated based on these knowledge difference points. The form of objects generated based on knowledge difference points is not limited here and may include, but is not limited to, images, videos, tables, etc., specifically generated by the large model based on the knowledge difference points.

[0028] In step S105, the optimized Markdown is converted into the first knowledge card.

[0029] The first knowledge card is a knowledge card obtained from optimized Markdown conversion and can be used to display the content of optimized Markdown. A knowledge card is an information unit that structurally encapsulates knowledge data. It can be in card form, organizing core viewpoints, key data, related attributes, and other textual content, as well as multimodal materials, into a knowledge block that can be independently understood, reused, and displayed. In some examples, interface parameters can be extracted from user-input interface design information. An Abstract Syntax Tree (AST) rendering engine is then used to convert the optimized Markdown into the first knowledge card based on the interface parameters and a preset knowledge card template. Users can upload interface design information to trigger the generation of the first knowledge card. Interface design information can be in the form of design specification files; the format is not limited here. For example, interface design information can include, but is not limited to, JSON format design documents, Sketch source files, and Cascading Style Sheets (CSS) in a specific format. Interface parameters can be extracted from the interface design information through a structure recognition agent. Interface parameters include, but are not limited to, core color values, font hierarchy specifications, corner radius, and shadow effects. Knowledge card templates can be selected by users from a knowledge card template library. The abstract syntax tree rendering engine automatically adjusts the layout ratio of the content in the first knowledge card based on the knowledge card template, interface parameters, the screen size of the device displaying the first knowledge card, and the interaction method, thereby generating and displaying the first knowledge card. For example, Figure 2 A schematic diagram illustrating an example of Markdown and knowledge cards provided in an embodiment of this application, as shown below. Figure 2 As shown, the Markdown 21 on the left can be converted into the knowledge card 22 on the right. The readability and aesthetics of knowledge card 22 are significantly better than Markdown 21. By using different knowledge card templates, knowledge cards on different topics can be generated. For example, Figure 3 This is a schematic diagram illustrating another example of a knowledge card provided in an embodiment of this application. Figure 3 The topic of knowledge card 23 in the middle is Figure 2 The knowledge cards in this embodiment have different themes and layouts. By parsing the interface design information, converting tokens, rendering abstract syntax trees, and loading the knowledge card templates of the themes, knowledge cards that meet the style requirements can be automatically generated, achieving cross-platform adaptability of knowledge cards. The knowledge cards in this embodiment include metadata of the Markdown in the above embodiments. Users do not need to read lengthy Markdowns; they can quickly obtain core knowledge through standardized knowledge cards. They can also trace the Markdown that carries the knowledge data based on the metadata, ensuring that the knowledge cards generated by the large model are based on evidence. In customer service scenarios, whether it is human or machine customer service, core information can be quickly obtained and fed back to the user through knowledge cards. This embodiment can generate knowledge cards that meet the theme according to business needs to adapt to web pages, knowledge bases, mobile applications, large model dialogue windows, etc. As reusable and composable "knowledge atoms," knowledge cards can be called by external systems and can also be embedded in business systems for use. Business systems can correspond to application scenarios. For example, business systems can include, but are not limited to, customer service systems, training platforms, business push systems, etc. Knowledge cards can be used as MCP Servers or Skills, and are not limited here.

[0030] In this design, the knowledge differences in the first knowledge card are displayed using enhanced visualization. Enhanced visualization refers to presenting these differences through text styling, graphics, images, and dynamic means, thereby highlighting them and allowing for quick differentiation even among highly similar knowledge data. In some examples, enhancement methods may include text styling enhancement and / or media format conversion. Text styling enhancement modifies the visual attributes of the text representing the knowledge differences to emphasize them; for example, it may include, but is not limited to, bolding, italics, underlining, highlighting, and changing text color. Media format conversion transforms the text representing the knowledge differences into one or more media formats, altering how these differences are perceived to emphasize them; for example, it may include images, videos, and tables. A large model can be used to convert knowledge differences into images, videos, tables, etc., and then display these media formats to highlight the differences. By visualizing knowledge differences, humans or machines can quickly identify knowledge differences in highly similar knowledge data, thereby improving the database search efficiency for customer service or knowledge base administrators.

[0031] In this embodiment, knowledge data can be recorded using Markdown. Based on the structural features of Markdown, the knowledge data is initially and coarsely segmented to obtain structural fragmentation units. Based on the semantic density parameters and contextual parameters of the knowledge data within these structural fragmentation units, dynamic semantic segmentation is performed to obtain first semantic fragmentation units that are topic-focused and semantically consistent. Using semantic similarity and edit distance calculations, the association weights and knowledge difference points between the first semantic fragmentation units can be obtained, thus determining the associations and differences between them. Guided by the associations and differences between the first semantic fragmentation units, the large language model integrates and processes these units, completing a deep logical reconstruction of knowledge. The reconstructed knowledge data is then displayed as knowledge cards, and the knowledge difference points between semantic fragmentation units can be displayed in the knowledge cards through enhanced visualization. This allows for rapid identification of differences in highly similar knowledge data, significantly improving the efficiency of knowledge data retrieval for customer service or knowledge base administrators. First, the long Markdown is segmented based on structural features to obtain structural fragment units. Then, these structural fragment units are further segmented semantically. Compared with related techniques that brute-force truncation by character, this embodiment can significantly improve the retention rate of semantic integrity of knowledge data in long documents. The first semantic fragment unit serves as input to a large model, enabling the large model to accurately understand the knowledge data even with limited computing resources. Integrating the first semantic fragment unit can automatically simplify, refine, and restructure lengthy, unstructured knowledge data, resulting in accurate, coherent, and concise optimized Markdown. This improves the quality of the Markdown and reduces the automatic layout design time for converting the optimized Markdown into the first knowledge card to the minute level, increasing the speed and efficiency of knowledge card generation. Furthermore, deep logical reconstruction of knowledge can avoid the risk of knowledge cards not conforming to design specifications.

[0032] In some embodiments, the correlation of knowledge data, the semantic coherence of text, topic turning points, and contextual dependencies in the structural fragmentation units can be analyzed. A segmentation strategy combining sliding windows and semantic density is adopted to further refine the structural fragmentation units into first semantic fragmentation units with smaller semantic granularity. The first semantic fragmentation unit is a semantically coherent and complete semantic fragmentation unit that is topic-focused. Specifically, in the structural fragmentation unit, a preset sliding window can be slid sentence by sentence to obtain multiple fragmentation units; the semantic density parameter of the fragmentation unit and the correlation and topic similarity of adjacent fragmentation units are calculated; knowledge data in fragmentation units with close semantic density parameters are retained without being segmented, and segmentation is performed from adjacent fragmentation units where the correlation is lower than the correlation threshold and / or the topic similarity is lower than the similarity threshold to obtain the first semantic fragmentation unit.

[0033] A sliding window can be used to simulate fragmentation, resulting in multiple fragment units. By calculating the semantic density parameters of the knowledge data within each fragment unit, the semantic density of that unit can be determined, serving as the basis for segmenting structural fragment units. Calculating the relevance of fragment units determines the connections and dependencies between them; calculating the topic similarity identifies the coherence and topic transition points between different fragment units. Relevance and topic similarity can also be used as the basis for segmenting structural fragment units. Structural fragment units can be segmented using a large language model based on their semantic density parameters, relevance, and topic similarity with adjacent fragment units. Alternatively, they can be segmented based on the semantic density parameters, relevance, and topic similarity of adjacent fragment units, along with predefined rules.

[0034] The higher the semantic density parameter of a segment unit, the tighter the semantics at that segment unit, and the greater the probability that the core concept will be truncated when segmenting from that segment unit; the lower the semantic density parameter of a segment unit, the looser the semantics at that segment unit, and the smaller the probability that the core concept will be truncated when segmenting from that segment unit. In some examples, a first semantic density range and a second semantic density range can be pre-defined, with values ​​within the first range being greater than those within the second range. The first semantic density range is the range where segmentation would result in the truncation of the core concept. If the semantic density parameter represents a semantic density within the first range, it indicates that the semantics at that segment unit are tightly bound, and segmentation from that segment unit would result in the truncation of the core concept, making it unsuitable for segmentation. The second semantic density range is the range where segmentation would not result in the truncation of the core concept. If the semantic density parameter represents a semantic density within the second range, it indicates that the semantics at that segment unit are relatively loose, and segmentation from that segment unit would not result in the truncation of the core concept, making it suitable for segmentation. In other examples, the semantic density parameter of a segment unit can be used as one of the bases for the large model to segment structural segment units, assisting the large model in segmenting structural segment units.

[0035] The higher the correlation between two adjacent fragment units, the stronger the correlation and dependency between them, making them unsuitable for segmentation. Conversely, the lower the correlation between two adjacent fragment units, the weaker the correlation and dependency, making them more suitable for segmentation. In some examples, a correlation threshold can be set, which is a limit on the correlation level that will not affect the semantic association of fragment units during segmentation. If the correlation between two adjacent fragment units is higher than or equal to this threshold, it indicates a high correlation and strong dependency, and segmentation should not be performed between them. If the correlation between two adjacent fragment units is lower than this threshold, it indicates a low correlation and weak dependency, and segmentation can be performed between them. In other examples, the correlation between adjacent fragment units can be used as one of the criteria for segmenting structural fragment units in the larger model, assisting the larger model in segmenting structural fragment units.

[0036] The similarity between two adjacent fragment units can be defined as topic similarity. Alternatively, the topic information of each fragment unit can be summarized using a large model, and the similarity of the topic information between two adjacent fragment units can be defined as topic similarity; neither is limited here. A higher topic similarity between two adjacent fragment units indicates higher semantic coherence and a lower probability of topic shifts. Conversely, a lower topic similarity indicates lower semantic coherence and a higher probability of topic shifts. In some examples, a similarity threshold can be pre-set. This threshold is a limit on the topic similarity that will not affect the semantic coherence of the fragment unit components during segmentation. If the topic similarity between two adjacent fragment units is higher than or equal to this threshold, it indicates high semantic coherence and a low probability of topic shifts, making segmentation between these two fragment units unsuitable. If the topic similarity between two adjacent fragment units is lower than this threshold, it indicates low semantic coherence and a high probability of topic shifts, making segmentation between these two fragment units appropriate. In other examples, the topic similarity between adjacent fragment units can be used as one of the bases for the large model to segment structural fragment units, thus assisting the large model in segmenting structural fragment units.

[0037] The above-mentioned segmentation of structural units can be performed by a semantic clustering agent. For example, Figure 4 A schematic diagram illustrating an example of the processing flow of the semantic clustering agent provided in the embodiments of this application, as shown below. Figure 4As shown, the structural segmentation unit enters the data output layer 31. The core intelligent agent control center 32 includes a semantic clustering intelligent agent 33, which can call the large model 34 for assistance in processing. The semantic clustering intelligent agent 33 can obtain output 36 through the policy and algorithm engine 34 and the decision and optimization logic 35. Output 36 is the first semantic segmentation unit, which can also be called a semantically self-consistent independent unit, i.e., Topic-Focused Snippets. In the policy and algorithm engine 34, context dependency analysis, keyword frequency evaluation, core concept relevance evaluation, and topic similarity evaluation can be achieved through the calculation of sliding window, semantic density parameter, and context association parameter. Context dependency analysis can be performed based on the relevance and topic similarity in the above embodiments and can be used to determine topic turning points. Keyword frequency evaluation can be performed based on the keyword frequency in the semantic density parameter. Core concept relevance evaluation can be performed based on the relevance in the above embodiments. Topic similarity evaluation can be performed based on the topic similarity in the above embodiments and can be used to evaluate the coherence across segmentation units. In the decision and optimization logic 35, the semantics of the segmented unit can be determined based on semantic density parameters, relevance, topic similarity, etc. If the semantics are split, the adjacent segmented units with low relevance are split to ensure the integrity of the segmented units with high semantic density and obtain the output of the first semantic segmented unit. If the semantics are not split, the segmented unit is kept in its original state and is used as the output of the first semantic segmented unit.

[0038] Segmenting structural units based on semantic density parameters, relevance, and topic similarity ensures the integrity and semantic coherence of high-semantic-density units, guaranteeing that the first semantic unit obtained is a topic-focused, semantically self-consistent, and independent semantic unit.

[0039] In some embodiments, knowledge difference points can be obtained by jointly analyzing the relevance and difference of the first semantic segmentation units. Specifically, the topic information of each first semantic segmentation unit can be summarized by a large model; semantic similarity calculation is performed based on the topic information of the first semantic segmentation units to obtain the relevance weight of the first semantic segmentation units; edit distance calculation is performed on different first semantic segmentation units to obtain the edit distance of different first semantic segmentation units; knowledge difference points are obtained by analyzing the relevance weight and edit distance of the first semantic segmentation units using a large model.

[0040] Having obtained the topic information for each first semantic segment unit, similarity calculations can be performed on the topic information of multiple first semantic segment units to obtain the similarity between the topic information of the first semantic segment units. Based on the similarity between the topic information of the first semantic segment units, the association weight of the first semantic segment unit can be determined. The association weight of a first semantic segment unit can characterize the association between that first semantic segment unit and other first semantic segment units. In some examples, a context summary header can be added to each first semantic segment unit according to its arrangement order. The context summary header includes the topic information of the previous first semantic segment unit and the topic information of the current first semantic segment unit. Semantic similarity calculations can be performed based on the context summary header of the first semantic segment unit to obtain the similarity between the topic information of each first semantic segment unit and the topic information of its predecessor. The similarity between the topic information of the predecessor and the topic information of the current first semantic segment unit can be determined as the association weight of the current first semantic segment unit. For ease of acquisition, the similarity between the topic information of each first semantic segment unit and the topic information of its predecessor can be stored in the form of a preorder semantic similarity matrix. In other examples, the similarity of topic information between any two first semantic segment units can be calculated. For the topic information of a single first semantic segment unit, multiple similarities to the topic information of other first semantic segment units can be calculated. Integration algorithms, such as weighted algorithms, can be used to integrate these multiple similarities to obtain the association weight of that first semantic segment unit. The edit distance, obtained by performing distance editing operations on different first semantic segment units, represents the minimum number of single-character editing operations required to transform one first semantic segment unit into another. Edit distance can help quantify the differences and overlaps between first semantic segment units at the character or structural level. Larger models can understand the relationships and differences between first semantic segment units based on association weights and edit distances, thereby outputting knowledge difference points.

[0041] In some embodiments, after obtaining the optimized Markdown, a mind map can be generated based on the optimized Markdown to enable users to acquire knowledge data more efficiently. Specifically, the optimized Markdown can be segmented to obtain multiple second semantic fragment units; content semantic features can be extracted from the second semantic fragment units to construct the association between the second semantic fragment units and the content semantic features; a mind map can be generated by a large model based on the association between the second semantic fragment units and the content semantic features, and the nodes in the mind map represent the second semantic fragment units.

[0042] The second semantic segmentation unit is a topic-focused, semantically coherent, and complete semantic segmentation unit obtained by segmenting the optimized Markdown. In some examples, a large model can be used to segment the optimized Markdown. By constructing prompt words, the large model is guided to segment the optimized Markdown into multiple topic-focused, semantically coherent, and complete second semantic segmentation units. In other examples, the optimized Markdown can be segmented into second semantic segmentation units in the same way as the first semantic segmentation unit. Specifically, the knowledge data in the optimized Markdown can be segmented according to its structural features to obtain multiple structural segmentation units. Based on the semantic density parameters and contextual association parameters of the knowledge data in the structural segmentation units, adaptive semantic segmentation is performed on the structural segmentation units to obtain the second semantic segmentation units. For details, please refer to the relevant descriptions in the above embodiments, which will not be repeated here. After obtaining the second semantic segmentation units, they can be stored in a knowledge base for later retrieval.

[0043] Content semantic features are the semantic features that embody the knowledge data content within the second semantic segmentation unit. These features can be extracted from the second semantic segmentation unit using a large model. In some examples, content semantic features may include topic features, and may also include, but are not limited to, one or more of user features, business features, and rule features. Topic features are the essential features of the topic within the second semantic segmentation unit. For example, if the knowledge data in the optimized Markdown is knowledge data in the domain of consumer voucher usage, then the topic features of the second semantic segmentation unit may include, but are not limited to, "receiving," "verifying," and "refund eligibility." User features include features representing users and user behavior. For example, if the knowledge data in the optimized Markdown is knowledge data in the domain of payment transactions, then user features may include, but are not limited to, the user's spending power level, transaction preferences, user age, and user location assessed based on transaction records. Business features include features representing the business, and may include, but are not limited to, product features, activity features, and service features. For example, if the knowledge data in the optimized Markdown is knowledge data in the domain of discount strategies, then business features may include, but are not limited to, the usage methods of consumer vouchers and a comparison of the advantages of overlapping subsidies from multiple discount institutions. Rule features may include, but are not limited to, the features of the designed policy terms and the features of the platform rules. To establish the association between the second semantic segmentation unit and the content semantic features, for example, a multi-dimensional feature matrix of the structural metadata and content semantic features of the second semantic segmentation unit can be constructed. The structural metadata of the second semantic segmentation unit can be the knowledge data of the second semantic segmentation unit itself, or the metadata extracted from the second semantic segmentation unit, which is not limited here. The element in the i-th row and j-th column of the multi-dimensional feature matrix can be the structural metadata of the i-th second semantic segmentation unit and the j-th dimension of the content semantic features of the i-th second semantic segmentation unit.

[0044] The large model can deduce the logical relationships between second semantic segment units based on the association between them and the semantic features of the content. Based on these logical relationships, it extends branches from the nodes of the second semantic segment units to generate a mind map. Logical relationships can include, but are not limited to, causal, progressive, and parallel relationships. Branches in the mind map can include, but are not limited to, current situation analysis, rule matching, and feasible solutions. Each node on each branch in the mind map is the smallest interconnected and effective knowledge node. During the mind map generation process, the large model can also extract core keywords from the second semantic segment units, add association tags to the nodes of the second semantic segment units in the mind map, and identify key nodes in the mind map through a large language model, displaying these key nodes in a visually enhanced manner.

[0045] In some examples, nodes in the mind map can be associated with knowledge cards corresponding to the core keywords of the second semantic segmentation unit represented by the node. This association can be achieved by adding card identifiers or content link fields to the knowledge cards in the mind map. The mind map can be displayed, and when a node's switching control is triggered, the associated knowledge card is displayed. The mind map can be displayed directly; alternatively, the first knowledge card can be displayed first, and the display can be treated as a mind map in response to the user's switching operation on the first knowledge card. Each node in the mind map can be configured with a switching control. When the switching control is triggered, the associated knowledge card is displayed, thus achieving bidirectional anchor switching between knowledge cards and the mind map, i.e., bidirectional anchor jump between macro-level overview and micro-level details. In some examples, nodes in the mind map can be specifically associated with semantic segments in knowledge cards corresponding to the core keywords of that node. This allows the display of the associated knowledge data paragraph to switch when the node's switching control is triggered. Similarly, the second semantic segment in the first knowledge card can also be configured with a switching control, which, when triggered, can switch the display of the corresponding node in the mind map. By switching between mind maps and knowledge cards, the cognitive load on administrators or knowledge base managers can be reduced, and knowledge retrieval efficiency can be improved.

[0046] In some examples, the system can also receive user adjustments to branches in the mind map. By using a large language model to learn user preference behaviors based on these adjustments, the system can continuously improve the ability and quality of mind map generation, making the generated mind maps more in line with users' reading habits.

[0047] In some embodiments, semantic feature networks can be extracted from optimized Markdown and mind maps to generate a self-learning and evolving rule base. The semantic feature networks are then compliantly integrated into the knowledge graph to achieve real-time linkage and expansion of knowledge data. Specifically, a large model can be used to extract relationship link information from optimized Markdown and mind maps to represent the relationships between knowledge data in the second semantic segmentation unit. The large model matches the extracted relationship link information with relationship rules in the rule base. If a relationship rule matching the relationship link information exists in the rule base, the relationship link information is converted into a first knowledge link segment based on the matching link rule. The first knowledge link segment is then fused with the corresponding knowledge path in the knowledge graph to update the knowledge graph. The knowledge graph corresponds to the knowledge base, which stores the second semantic segmentation unit, obtained by splitting the optimized Markdown using the large model. If no relationship rule matching the relationship link information exists in the rule base, the large model generates new relationship rules and a second knowledge link segment based on the relationship link information. The new relationship rules are stored in the rule base, and the second knowledge link segment is added to the knowledge graph.

[0048] Relationship link information can include entities and relationships. Entities represent knowledge data in the second semantic segmentation unit, and relationships are the connections between entities, i.e., the relationships between knowledge data. For example, the relationship link information could be "Enter Activity → Trigger Rule → View Rule," where "Enter Activity" and "View Rule" are entities, and "Trigger Rule" is the relationship; or "Refund → Trigger Process → Return Qualification," where "Refund" and "Return Qualification" are entities, and "Trigger Process" is the relationship. Relationship link information can transform real-time data into a network structure with semantic relationships. The relationships between entities reflected in the relationship link information can distinguish the meaning of entities with the same name and their corresponding knowledge data in different contexts. The rule base includes compliant and trustworthy relationship rules. By matching the relationship link information with the relationship rules in the rule base, it can be determined whether the relationship between the knowledge data represented by the relationship link information is a known, compliant, and trustworthy relationship. If the rule base has relationship rules that match the associated link information, it indicates that the relationship between the knowledge data represented by the relationship link information is a known, compliant, and trustworthy relationship. The associated link information can then be standardized into a first knowledge link segment, facilitating the updating of the knowledge graph. The first knowledge link segment follows the same structural form as the schema in the knowledge graph. In some examples, the first knowledge link segment can be implemented as a triple, which includes a head entity, a relation, and a tail entity. For example, if the second semantic segment unit is "If a user claims a coupon but does not actually use it, they can apply for a refund," then the head entity in the resulting triple is "refund," the relation is "trigger rule," and the tail entity is "refundable for actual unused use." Integrating the first knowledge link segment with the corresponding knowledge path in the knowledge graph updates the knowledge graph, enriching the knowledge along that path. If the rule base does not have a relation rule matching the associated link information, it means the relationship between the knowledge data represented by the link information is not a known, compliant, and reliable relationship, and a new relation rule needs to be created. To ensure the credibility and compliance of the new relation rule, after generating the new relation rule, a prompt message can be sent to the user, prompting them to review the new relation rule and the second knowledge link segment. After receiving feedback from the user confirming the compliance of the new relation rule, the new relation rule is stored in the rule base, and the second knowledge link segment is added to the knowledge graph. The second knowledge link segment has a structure consistent with the schema in the knowledge graph. In some examples, the second knowledge link segment can be implemented as a triple, which includes a head entity, a relation, and a tail entity.For example, if the second semantic segment is "Digital RMB Red Packet Verification," and there are no matching relationship rules in the rule base, the large model can initiate self-learning to generate new relationship rules and a second knowledge link segment. This second knowledge link segment is a triple, with the head entity being "Marketing Tool," the relationship being "Rule," and the tail entity being "Red Packet Verification." After user confirmation, the new relationship rule can be stored in the rule base, and the second knowledge link segment can be added to the knowledge graph, completing the knowledge graph update. The knowledge graph can be used for logical reasoning related to knowledge data. Logical reasoning related to knowledge data can be achieved by querying and traversing the knowledge graph. For example, a knowledge path in the knowledge graph might be "Action: Refund → Reason: User Dissatisfaction → Rule: Refundable Qualification Not Actually Used → User Preference: Sensitive → Suggestion: Refund." Querying along this knowledge path will yield the final solution for the user's refund action, namely "Suggestion: Refund."

[0049] Through the techniques employed in the above embodiments, such as the linked generation of knowledge cards and multimodal elements, the extraction of correlation link information from optimized Markdown and mind maps, and the dynamic updating of knowledge graphs, intelligent association of knowledge data can be achieved. For example, Figure 5 A schematic diagram illustrating an example of intelligent association of knowledge data provided in an embodiment of this application, such as... Figure 5 As shown, in the visualization layer 41, knowledge data can be acquired. This knowledge data can be the core knowledge data in the knowledge cards. Based on the knowledge data, a dynamic mapping model between text semantics and multimodal requirements is constructed to achieve intelligent linkage generation of knowledge data and multimodal elements. For example, multimodal elements such as images, videos, and tables representing knowledge differences can be generated through artificial intelligence and displayed through knowledge cards, thereby ensuring a high degree of semantic consistency between the multimodal elements and the text of the knowledge cards, and improving the visualization effect of knowledge. In the extraction and standardization layer 42, entities and relationships can be extracted from the knowledge data. The reasoning function of the large model is used to perform standardized relationship mapping to obtain relationship rules to construct a rule base 44. The extracted entities and relations can be converted into knowledge link segments by the format conversion engine in the knowledge graph reasoning layer 43. If there are no relation rules in the rule base 44 that match the relation link information containing entities and relations, the large model generates relation rules, that is, the large model recommends relation rules. After manual review, if the newly generated relation rules and knowledge link segments are deemed compliant and reliable, the newly generated relation rules are stored in the rule base 44, and the converted knowledge link segments are connected to the knowledge graph database 45 to update the knowledge graph in the knowledge graph database 45.

[0050] In some embodiments, knowledge cards that meet the user's knowledge query needs can be displayed to the user based on the user's query request. Specifically, in response to the user's query request, a second semantic segmentation unit matching the query request is retrieved from the knowledge base. The knowledge base stores the second semantic segmentation units, which are obtained by splitting optimized Markdown using a large model. Based on the matched second semantic segmentation units, semantic similarity calculation and edit distance calculation are used to obtain the association weights and knowledge differences between the matched second semantic segmentation units. The first knowledge card corresponding to the matched second semantic segmentation unit is displayed, and the knowledge differences in the first knowledge card corresponding to the matched second semantic segmentation unit are displayed in a visually enhanced manner. Alternatively, in response to a user's query request, the system retrieves a second semantic segment unit that matches the query request from the knowledge base. The knowledge base stores these second semantic segment units, which are obtained by splitting the optimized Markdown using a large model. Based on the matched second semantic segment units, semantic similarity and edit distance calculations are used to obtain the association weights and knowledge differences between the matched second semantic segment units. Using the large model, the matched second semantic segment units are integrated according to the association weights and knowledge differences to generate an integrated Markdown. The integrated Markdown is then converted into a second knowledge card and displayed, with the knowledge differences in the second knowledge card presented in a visually enhanced manner.

[0051] Each time Markdown is generated and optimized, and then segmented into second semantic units, these second semantic units can be stored in a knowledge base, gradually building the knowledge base, which can be dynamically updated. Matching a query request with the second semantic units' topic information or core keywords can be achieved. Semantic similarity calculations can be performed based on the topic information of the matched second semantic units to obtain their association weights. Edit distance calculations can be performed on different matched second semantic units to obtain their edit distances. A large model can then analyze the association weights and edit distances of the matched second semantic units to obtain their knowledge differences. This process of obtaining the knowledge differences of the matched second semantic units is similar to the process of obtaining the association weights and knowledge differences between the first semantic units in the above embodiments; for details, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0052] In some examples, after obtaining the association weights and knowledge differences between the matched second semantic segments, the optimized Markdown first knowledge cards corresponding to each matched second semantic segment can be displayed; that is, all pre-generated first knowledge cards related to the query request can be displayed. When there are multiple matched second semantic segments, multiple first knowledge cards can be displayed simultaneously, and the user can switch between multiple first knowledge cards and their corresponding mind maps based on user input. Multiple first knowledge cards can be arranged in descending order of matching degree between the second semantic segments and the query request. Even if the user makes a mistake, they can quickly view and jump to the first knowledge cards and their corresponding mind maps according to the matching degree.

[0053] In other examples, after obtaining the association weights and knowledge differences between the matched second semantic segments, the matched second semantic segments can be integrated to generate a new Markdown, i.e., the integrated Markdown. The process of generating the integrated Markdown is similar to the process of generating the optimized Markdown in the above embodiments; for details, please refer to the relevant content in the above embodiments, which will not be repeated here. The second knowledge card is the knowledge card obtained by converting the integrated Markdown. When there are multiple matched second semantic segments, a new second knowledge card is generated based on the multiple matched second semantic segments. The knowledge data of multiple second semantic segments related to the user's query request is displayed to the user through a single second knowledge card. The knowledge differences can be displayed in a visually enhanced manner within a single second semantic segment. For details on displaying knowledge differences in a visually enhanced manner, please refer to the above embodiments, which will not be repeated here. In this example, content semantic features can be extracted from the matched second semantic segments to construct the association relationship between the matched second semantic segments and the content semantic features. Based on the association relationship between the matched second semantic segments and the content semantic features, a mind map is generated by a large model. This mind map can be switched between the second knowledge card and the other two.

[0054] This application also provides a knowledge data processing apparatus, which corresponds to the knowledge data processing method in the above embodiments. Figure 6 This is a schematic diagram of the structure of a knowledge data processing apparatus provided in an embodiment of this application, as shown below. Figure 6 As shown, the knowledge data processing device 500 may include a structure segmentation module 501, a semantic segmentation module 502, an analysis module 503, an integration module 504, and a card generation module 505.

[0055] The structure segmentation module 501 can be used to segment knowledge data in Markdown according to the structural features of Markdown, and obtain multiple structural segmentation units.

[0056] The semantic segmentation module 502 can be used to perform adaptive semantic segmentation on the structural segmentation unit based on the semantic density parameter and context association parameter of the knowledge data in the structural segmentation unit, so as to obtain the first semantic segmentation unit.

[0057] The analysis module 503 can be used to obtain the association weights and knowledge differences between the first semantic segmentation units based on the segmentation, using semantic similarity calculation and edit distance calculation.

[0058] The integration module 504 can be used to integrate the first semantic segmentation unit through a large model, based on the association weights and knowledge differences, to generate an optimized Markdown, which includes content representing knowledge differences.

[0059] The card generation module 505 can be used to convert optimized Markdown into first knowledge cards, which are used to display the content of optimized Markdown. The knowledge differences in the first knowledge cards are displayed in a visually enhanced manner.

[0060] In some embodiments, the structure segmentation module 501 can be used to: when the Markdown has explicit structure features, identify the explicit structure features through a segmentation model, segment the Markdown according to the explicit structure features, and obtain multiple structure fragmentation units; when the Markdown does not have explicit structure features, identify the statement boundaries through a segmentation model using a syntactic analysis algorithm, segment the Markdown according to the statement boundaries, and obtain multiple structure fragmentation units.

[0061] In some embodiments, the knowledge data processing apparatus 500 may further include a preprocessing module. The preprocessing module may be used to: perform standardized preprocessing on the knowledge data in Markdown; call at least two large models to parse the structural features of the standardized preprocessed Markdown, and obtain parsing comparison results, which are used to assist in segmenting the knowledge data in Markdown.

[0062] In some embodiments, the context association parameters include relevance and topic similarity. The semantic segmentation module 502 can be used to: slide a preset sliding window sentence by sentence in the structural segmentation unit to obtain multiple segmentation units; calculate the semantic density parameter of the segmentation unit and the relevance and topic similarity of adjacent segmentation units; retain the knowledge data in the semantically close segmentation units represented by the semantic density parameter without segmentation, and segment from adjacent segmentation units where the relevance is lower than the relevance threshold and / or the topic similarity is lower than the similarity threshold to obtain the first semantic segmentation unit.

[0063] In some embodiments, the analysis module 503 can be used to: summarize the topic information of each first semantic segment unit through a large model; perform semantic similarity calculation based on the topic information of the first semantic segment unit to obtain the association weight of the first semantic segment unit; perform edit distance calculation on different first semantic segment units to obtain the edit distance of different first semantic segment units; and perform analysis based on the association weight and edit distance of the first semantic segment unit through the large model to obtain knowledge difference points.

[0064] In some embodiments, the knowledge data processing apparatus 500 may further include a retrieval module. The retrieval module may be used to: in response to a user's query request, retrieve a second semantic fragment unit that matches the query request from a knowledge base, the knowledge base storing the second semantic fragment unit, which is obtained by splitting optimized Markdown using a large model.

[0065] Analysis module 503 can also be used to: based on the matched second semantic segmentation unit, use semantic similarity operation and edit distance operation to obtain the association weight and knowledge difference points between the matched second semantic segmentation units.

[0066] The card generation module 505 is also used to: display the first knowledge card corresponding to the matched second semantic segmentation unit, and display the knowledge difference points in the first knowledge card corresponding to the matched second semantic segmentation unit in a visually enhanced manner.

[0067] Alternatively, the integration module 504 can also be used to: integrate the matched second semantic segmentation units based on the association weights and knowledge differences between the matched second semantic segmentation units using a large model, generating an integrated Markdown. The card generation module 505 can also be used to: convert the integrated Markdown into second knowledge cards and display them, with the knowledge differences in the second knowledge cards displayed in a visually enhanced manner.

[0068] In some examples, visualization enhancements include text style enhancements and / or media format transformations.

[0069] In some embodiments, the knowledge data processing apparatus 500 may further include a mind map generation module. The mind map generation module can be used to: segment the optimized Markdown to obtain multiple second semantic fragment units; extract content semantic features from the second semantic fragment units, and construct the association between the second semantic fragment units and the content semantic features; and generate a mind map based on the association between the second semantic fragment units and the content semantic features using a large model, where nodes in the mind map represent the second semantic fragment units.

[0070] In some embodiments, nodes in the mind map are associated with knowledge cards corresponding to the core keywords of the second semantic segmentation units represented by the nodes. The mind map generation module can also be used to: display the mind map; and, when the switching control of a node in the mind map is triggered, switch the display of the knowledge cards associated with the node.

[0071] In some embodiments, the knowledge data processing apparatus 500 may further include a knowledge graph association module. The knowledge graph association module can be used to: extract relationship link information from optimized Markdown and mind maps using a large model to represent the relationships between knowledge data in the second semantic segmentation unit; match the extracted relationship link information with relationship rules in a rule base using the large model; if a relationship rule matching the relationship link information exists in the rule base, convert the relationship link information into a first knowledge link segment according to the matched link rule, and fuse the first knowledge link segment with the knowledge path corresponding to the matched relationship rule in the knowledge graph to update the knowledge graph. The knowledge graph corresponds to a knowledge base, which stores the second semantic segmentation unit, which is obtained by splitting the optimized Markdown using the large model; if no relationship rule matching the relationship link information exists in the rule base, generate new relationship rules and a second knowledge link segment based on the relationship link information using the large model, store the new relationship rule in the rule base, and add the second knowledge link segment to the knowledge graph.

[0072] In some embodiments, the card generation module 505 can be used to: extract interface parameters from the interface design information input by the user, and use an abstract syntax tree rendering engine to convert optimized Markdown into a first knowledge card based on the interface parameters and a preset knowledge card template.

[0073] It should be noted that the knowledge data processing device 500 is a device corresponding to the knowledge data processing method described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0074] This application also provides an electronic device. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, as shown below. Figure 7As shown, the electronic device 600 includes a memory 601, a processor 602, and a computer program stored in the memory 601 and executable on the processor 602.

[0075] In some examples, the processor 602 described above may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that may be configured to implement the embodiments of this application.

[0076] Memory 601 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the knowledge data processing method according to embodiments of this application.

[0077] The processor 602 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 601, so as to implement the knowledge data processing method in the above embodiments.

[0078] In some examples, the single device 600 may also include a communication interface 603 and a bus 604. For example, Figure 7 As shown, the memory 601, processor 602, and communication interface 603 are connected through bus 604 and complete communication with each other.

[0079] The communication interface 603 is mainly used to enable communication between various modules, devices, units, and / or equipment in the embodiments of this application. Input devices and / or output devices can also be connected through the communication interface 603.

[0080] Bus 604 includes hardware, software, or both, that couples components of electronic device 600 together. For example, and not as a limitation, bus 604 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 604 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0081] This application also provides a computer-readable storage medium storing computer program instructions. When these computer program instructions are executed by a processor, they can implement the knowledge data processing method described in the above embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The aforementioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., and is not limited thereto.

[0082] This application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the knowledge data processing method in the above embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0083] It should be clarified that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. For the device embodiments, equipment embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description section of the method embodiments. This application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application. Furthermore, for the sake of brevity, detailed descriptions of known methods and techniques are omitted here.

[0084] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0085] Those skilled in the art will understand that the above embodiments are exemplary and not restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, specification, and claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other means or steps; the quantifier "a" does not exclude a plurality; the terms "first" and "second" are used to identify names and not to indicate any particular order. No reference numerals in the claims should be construed as limiting the scope of protection. The functionality of multiple parts appearing in the claims can be implemented by a single hardware or software module. The appearance of certain technical features in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A method for processing knowledge data, characterized in that, include: Based on the structural characteristics of Markdown, the knowledge data in the Markdown is segmented to obtain multiple structural fragmentation units; Based on the semantic density parameters and context association parameters of the knowledge data in the structural segmentation unit, the structural segmentation unit is adaptively semantically segmented to obtain the first semantic segmentation unit; Based on the first semantic segmentation unit obtained from the segmentation, semantic similarity calculation and edit distance calculation are used to obtain the association weight and knowledge difference points between the first semantic segmentation units; The first semantic segmentation unit is integrated and processed using a large model based on the association weights and knowledge differences to generate an optimized Markdown, which includes content representing knowledge differences. The optimized Markdown is converted into a first knowledge card, which is used to display the content of the optimized Markdown. The knowledge differences in the first knowledge card are displayed in a visually enhanced manner.

2. The method according to claim 1, characterized in that, The knowledge data in the Markdown is segmented according to the structural features of Markdown to obtain multiple structural fragmentation units, including: When the Markdown has explicit structural features, the explicit structural features are identified by the segmentation model, and the Markdown is segmented according to the explicit structural features to obtain multiple structural fragmentation units; In the case where the Markdown does not have explicit structural features, a segmentation model is used to identify statement boundaries using a syntactic analysis algorithm, and the Markdown is segmented according to the statement boundaries to obtain multiple structural fragmentation units.

3. The method according to claim 1, characterized in that, Before segmenting the knowledge data in the Markdown according to its structural features to obtain multiple structural fragment units, the method further includes: The knowledge data in the Markdown is standardized and preprocessed. At least two large models are invoked to parse the structural features of the standardized preprocessed Markdown, and the parsing comparison results are obtained. The parsing comparison results are used to assist in the segmentation of knowledge data in the Markdown.

4. The method according to claim 1, characterized in that, Contextual parameters include relevance and topic similarity; The first semantic segmentation unit is obtained by adaptively semantically segmenting the structural segmentation unit based on the semantic density parameters and context association parameters of the knowledge data in the structural segmentation unit, including: In the structural segmentation unit, multiple segmentation units are obtained by sliding a preset sliding window sentence by sentence, taking sentences as the unit; Calculate the semantic density parameters of the fragmented units, as well as the relevance and topic similarity of adjacent fragmented units; Knowledge data in semantically close segments, represented by semantic density parameters, are not segmented. Instead, segments are made from adjacent segments with relevance below the relevance threshold and / or topic similarity below the similarity threshold to obtain the first semantic segment.

5. The method according to claim 1, characterized in that, The first semantic segmentation unit obtained based on segmentation uses semantic similarity calculation and edit distance calculation to obtain the association weights and knowledge difference points between the first semantic segmentation units, including: The main topic information of each first semantic segment unit is summarized by a large model; Based on the topic information of the first semantic segmentation unit, semantic similarity calculation is performed to obtain the association weight of the first semantic segmentation unit; Edit distance calculations are performed on different first semantic fragment units to obtain the edit distances of different first semantic fragment units; Knowledge discrepancies are identified by analyzing the association weights and edit distances of the first semantic segmentation unit using a large model.

6. The method according to claim 1, characterized in that, Also includes: In response to a user's query request, a second semantic segmentation unit matching the query request is retrieved from the knowledge base. The knowledge base stores the second semantic segmentation units, which are obtained by splitting the optimized Markdown using a large model. Based on the matched second semantic segmentation units, semantic similarity calculation and edit distance calculation are used to obtain the association weights and knowledge differences between the matched second semantic segmentation units; The system displays the first knowledge card corresponding to the matched second semantic segment unit, and the knowledge differences in the first knowledge card corresponding to the matched second semantic segment unit are displayed in a visually enhanced manner; or, through a large model, the matched second semantic segment units are integrated according to the association weights and knowledge differences between the matched second semantic segment units to generate an integrated Markdown, and the integrated Markdown is converted into a second knowledge card and displayed, and the knowledge differences in the second knowledge card are displayed in a visually enhanced manner.

7. The method according to claim 1 or 6, characterized in that, Visual enhancement methods include text style enhancement methods and / or media format transformation methods.

8. The method according to claim 1, characterized in that, Also includes: The optimized Markdown is segmented to obtain multiple second semantic fragment units; Extract content semantic features from the second semantic segmentation unit and construct the association between the second semantic segmentation unit and the content semantic features; A mind map is generated based on the relationship between the second semantic segmentation unit and the content semantic features using a large model. The nodes in the mind map represent the second semantic segmentation unit.

9. The method according to claim 8, characterized in that, The nodes in the mind map are associated with knowledge cards corresponding to the core keywords of the second semantic segmentation unit represented by the nodes. The method further includes: Show the mind map; When the switching control of a node in the mind map is triggered, the knowledge card associated with the node is switched and displayed.

10. The method according to claim 8, characterized in that, Also includes: The large model extracts relational link information from the optimized Markdown and the mind map to reflect the relationships between knowledge data in the second semantic segmentation unit; The extracted relationship link information is matched with the relationship rules in the rule base using a large model; If there are relational rules in the rule base that match the relational link information, the relational link information is converted into a first knowledge link segment according to the matching link rule. The first knowledge link segment is then fused with the knowledge path corresponding to the matching relational rule in the knowledge graph to update the knowledge graph. The knowledge graph corresponds to the knowledge base. The knowledge base stores a second semantic segmentation unit, which is obtained by splitting the optimized Markdown using a large model. If there are no relationship rules in the rule base that match the relationship link information, a new relationship rule and a second knowledge link segment are generated by the large model based on the relationship link information. The new relationship rule is stored in the rule base, and the second knowledge link segment is added to the knowledge graph.

11. The method according to claim 1, characterized in that, The step of converting the optimized Markdown into a first knowledge card includes: The interface parameters are extracted from the user-input interface design information, and the optimized Markdown is converted into the first knowledge card using the abstract syntax tree rendering engine based on the interface parameters and the preset knowledge card template.

12. A knowledge data processing device, characterized in that, include: The structure segmentation module is used to segment the knowledge data in the Markdown according to the structural features of Markdown, and obtain multiple structure segmentation units; The semantic segmentation module is used to perform adaptive semantic segmentation of the structural segmentation unit based on the semantic density parameter and context association parameter of the knowledge data in the structural segmentation unit, so as to obtain the first semantic segmentation unit. The analysis module is used to obtain the association weights and knowledge differences between the first semantic segment units obtained by segmentation, using semantic similarity calculation and edit distance calculation. An integration module is used to integrate the first semantic segmentation unit through a large model based on the association weights and knowledge difference points to generate an optimized Markdown, wherein the optimized Markdown includes content representing knowledge difference points; The card generation module is used to convert the optimized Markdown into a first knowledge card. The first knowledge card is used to display the content of the optimized Markdown, and the knowledge differences in the first knowledge card are displayed in a visually enhanced manner.

13. An electronic device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the knowledge data processing method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the knowledge data processing method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the knowledge data processing method according to any one of claims 1 to 11.

Images