Knowledge slicing analysis processing method and system based on context window semantic clustering

Abstract

The application provides a knowledge slicing analysis processing method and system based on context window semantic clustering, relates to the technical field of analysis processing, obtains text feature field determination data of input text data, determines a corresponding preset field vocabulary library, analyzes the turning density proportion of the input text data and the preset field vocabulary library, determines a text field density sequence, analyzes the density interval difference and the density fault, obtains a hierarchical density sequence, and further determines text field density weight data; the input text data is segmented by a fixed window, the hierarchical density sequence is combined by the fixed window, dynamic window data is generated according to segmentation and combination processing, the dynamic window data is clustered by the fixed window size, and then knowledge information is sliced, knowledge slicing processing data is obtained, the semantic fixed and dynamic linkage analysis processing of knowledge are realized by the application, and the processing precision efficiency is improved.

Description

Technical Field

[0001] This invention proposes a knowledge slice analysis and processing method and system based on context window semantic clustering, which relates to the field of analysis and processing technology, specifically to the field of knowledge slice analysis and processing technology based on context window semantic clustering. Background Technology

[0002] Traditional knowledge slicing methods often rely on fixed window partitioning or single semantic feature clustering. Fixed window sizes lack domain adaptability, and general windows easily fragment complete knowledge units within a specialized domain, leading to semantically incoherent slices. Existing methods frequently ignore differences in text domain density distribution, determining domain affiliation solely through keyword matching. This makes it difficult to accurately quantify the semantic contribution of each domain, easily resulting in cross-domain mixed text clustering and reduced slice purity. Traditional techniques fail to effectively utilize the correlation between text structural features and domain lexicons, failing to identify discontinuities in density distribution. Consequently, weight allocation lacks hierarchical basis, and window segmentation and combination are highly arbitrary. Summary of the Invention

[0003] This invention provides a knowledge slicing analysis and processing method and system based on context window semantic clustering to solve the above-mentioned problems:

[0004] The present invention proposes a knowledge slicing analysis and processing method and system based on context window semantic clustering, the method comprising:

[0005] S1. Perform similarity analysis between the features of the input text data and the preset domain to obtain text feature domain determination data;

[0006] S2. Based on the text feature domain, determine the corresponding preset domain vocabulary, perform transition density ratio analysis between the input text data and the preset domain vocabulary, determine the text domain density sequence, perform density interval difference analysis and density tomography analysis to obtain the hierarchical density sequence, and then determine the text domain density weight data.

[0007] S3. Set a fixed window for the input text data, segment the fixed window using text domain density weight data, combine the fixed window using hierarchical density sequence, generate dynamic window data based on the segmentation and combination processing, cluster the dynamic window data using a fixed window size, and then slice the knowledge information to obtain knowledge slice processing data.

[0008] Further, S1 includes:

[0009] The process involves: acquiring input text data; extracting text features from the input text data to obtain text feature data; acquiring a preset domain vocabulary; and calculating the similarity between the text feature data and the preset domain vocabulary to obtain text feature domain similarity.

[0010] The text feature domain similarity is compared with a preset feature domain similarity threshold to obtain a similarity comparison result;

[0011] Based on the similarity comparison results, the domain information of the text feature data is determined to obtain text feature domain determination data.

[0012] Further, S2 includes:

[0013] Obtain a preset domain vocabulary corresponding to the text feature domain determination data, perform transition density analysis on the preset domain vocabulary, and obtain domain transition density analysis data;

[0014] Based on text feature data, perform transition density analysis on the input text data to obtain text transition density analysis data;

[0015] Obtain the proportion of text transition density analysis data in domain transition density analysis data to obtain text domain density proportion data;

[0016] Obtain text transition density analysis data and multiple text domain closure ratio data in all preset domain vocabularies, and determine text domain density weight data based on the multiple text domain density ratio data.

[0017] Further, the acquisition of text transition density analysis data includes the proportion of multiple text domain closures in all preset domain vocabularies, and the determination of text domain density weight data based on the proportion of multiple text domain density data, including:

[0018] The density percentage data of multiple text domains are sorted to obtain the text domain density sequence;

[0019] Obtain the difference between adjacent density proportions in the density sequence of the text domain to obtain the density adjacent difference;

[0020] The density adjacent difference is compared with a preset density adjacent threshold to obtain the density adjacent comparison result;

[0021] Density tomography analysis is performed based on the density adjacency comparison results and the density sequence of the text domain to obtain density tomography analysis data.

[0022] The density weight data of the text neighborhood is determined based on the density tomography analysis data.

[0023] Furthermore, based on the density adjacency comparison results and the density sequence of the text domain, density tomography analysis is performed to obtain density tomography analysis data, including:

[0024] Obtain at least one density adjacency difference for each text domain density percentage data and compare it with a preset density adjacency threshold.

[0025] When the density adjacent comparison result is that the density adjacent difference is greater than the preset density adjacent threshold, the density adjacent difference is determined to be the fault difference.

[0026] The text domain density sequence is broken according to the tomographic difference to obtain multiple hierarchical density sequences;

[0027] The average value of the text domain density proportion data for each density sequence is obtained to obtain density tomography analysis data.

[0028] Further, S3 includes:

[0029] Determine the fixed window size based on the input text data and text feature data, and obtain multiple fixed windows for the input text data;

[0030] Based on the text domain density weight data, multiple fixed windows are segmented to obtain fixed window segmentation data;

[0031] Based on the multiple hierarchical density sequences, multiple fixed windows are combined to obtain fixed window combination data;

[0032] Dynamic window data is generated by combining the fixed window segmentation data with the fixed window combination data. Dynamic semantic clustering is then performed based on the dynamic window data and the fixed window size to obtain dynamic semantic clustering information.

[0033] Based on the dynamic semantic clustering information, the preset knowledge information is sliced ​​to obtain knowledge slice processing data.

[0034] Further, based on the text density weight data, multiple fixed windows are segmented to obtain fixed window segmentation data, including:

[0035] Obtain text domain density weight data for multiple text feature data in each fixed window;

[0036] A fixed segmentation window is obtained by segmenting a fixed window based on the density weight data of the text domain.

[0037] Obtain the window segmentation data for each fixed segmentation window to obtain the fixed window segmentation data;

[0038] The fixed segmentation windows are sorted according to the text feature data order to obtain the window segmentation sequence, and then the fixed window segmentation data sequence is obtained.

[0039] Further, based on the multiple hierarchical density sequences, multiple fixed windows are combined to obtain fixed window combination data, including:

[0040] Obtain the fixed segmentation window corresponding to the same level of the window segmentation sequence based on the hierarchical sequence density;

[0041] Determine whether multiple fixed segmentation windows of the same level in a window segmentation sequence are adjacent to each other, and obtain window adjacency determination information;

[0042] Based on the window adjacency determination information, adjacent fixed segmented windows at the same level are combined to obtain fixed window combination data.

[0043] Furthermore, dynamic window data is generated based on the fixed window segmentation data and the fixed window combination data. Dynamic semantic clustering is then performed based on the dynamic window data and the fixed window size to obtain dynamic semantic clustering information, including:

[0044] Dynamic window data is obtained by updating the fixed window segmented data by combining data from fixed windows.

[0045] Get the size of the dynamic window data;

[0046] The fixed window size is compared with the dynamic window size to obtain the window size comparison result;

[0047] Based on the comparison results of window sizes, dynamic semantic clustering is performed on the windows to obtain dynamic semantic clustering information.

[0048] Furthermore, the system includes:

[0049] The feature domain analysis module is used to perform similarity analysis between the features of the input text data and the preset domain to obtain data to determine the text feature domain;

[0050] The transition density ratio analysis module is used to obtain the corresponding preset domain vocabulary based on the data determined by the text feature domain, perform transition density ratio analysis between the input text data and the preset domain vocabulary, determine the text domain density sequence, perform density interval difference analysis and density tomography analysis to obtain the hierarchical density sequence, and then determine the text domain density weight data.

[0051] The window dynamic processing module is used to set fixed windows for input text data, perform fixed window segmentation using text domain density weight data, combine fixed windows using hierarchical density sequences, generate dynamic window data based on segmentation and combination processing, perform clustering processing on dynamic window data using fixed window size, and then slice knowledge information to obtain knowledge slice processing data.

[0052] The beneficial effects of this invention are as follows: This method breaks away from the traditional fixed-window single-processing mode. By combining domain weights and dynamic windows, it achieves a transformation in text processing from indiscriminate segmentation to precise domain adaptation. In the domain localization stage, it ensures accurate matching between text and the domain; in the density weight calculation stage, it refines the domain hierarchy through tomographic analysis, making the weight allocation more closely aligned with the actual contribution of the domain; in the window processing stage, it balances the stability of fixed windows with the flexibility of dynamic windows, resulting in more accurate semantic clustering results. The final generated knowledge slices can completely retain the core knowledge units of each domain, effectively avoiding semantic fragmentation and information redundancy, and are suitable for knowledge extraction scenarios involving multi-domain mixed text. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a knowledge slicing analysis and processing method based on context window semantic clustering. Detailed Implementation

[0054] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0055] In one embodiment of the present invention, the knowledge slicing analysis and processing method and system based on context window semantic clustering proposed by the present invention includes:

[0056] S1. Perform similarity analysis between the features of the input text data and the preset domain to obtain text feature domain determination data;

[0057] S2. Based on the text feature domain, determine the corresponding preset domain vocabulary, perform transition density ratio analysis between the input text data and the preset domain vocabulary, determine the text domain density sequence, perform density interval difference analysis and density tomography analysis to obtain the hierarchical density sequence, and then determine the text domain density weight data.

[0058] S3. Fixed window settings are applied to the input text data. Fixed window segmentation is performed using text domain density weight data. Fixed window combination is then performed using hierarchical density sequences. Dynamic window data is generated based on the segmentation and combination processing. This dynamic window data is then clustered using a fixed window size, thereby slicing the knowledge information to obtain knowledge slice processing data, such as… Figure 1 As shown.

[0059] The working principle and technical effects of the above technical solution are as follows: First, a preliminary domain determination is completed for the input text. Based on the similarity matching between text features and a preset domain vocabulary, the core domain to which the text belongs is identified. Second, domain density weights are calculated in depth. Using transition density as the core indicator, the density ratio between the input text and the corresponding domain vocabulary is analyzed. Through density sequence sorting, adjacent difference comparison, and tomographic analysis, hierarchical density sequences are divided and the weights of each domain are determined. Third, dynamic window processing and knowledge slicing are implemented. First, fixed windows are set based on text features. Window segmentation is completed by combining domain weights. Then, dynamic windows are generated by combining adjacent windows of the same level according to the hierarchical density sequence. Semantic clustering is carried out based on window size differences, and finally, knowledge slicing is completed.

[0060] This method breaks away from the traditional fixed-window single-processing mode. By combining domain weights and dynamic windows, it transforms text processing from indiscriminate segmentation to precise domain adaptation. In the domain localization stage, it ensures accurate matching between text and the domain; in the density weight calculation stage, it refines the domain hierarchy through tomographic analysis, making the weight allocation more closely aligned with the actual contribution of the domain; in the window processing stage, it balances the stability of fixed windows with the flexibility of dynamic windows, resulting in more accurate semantic clustering results. The final generated knowledge slices can completely retain the core knowledge units of each domain, effectively avoiding semantic fragmentation and information redundancy, making it suitable for knowledge extraction scenarios involving multi-domain mixed text.

[0061] In one embodiment of the present invention, S1 includes:

[0062] The input text data is acquired, and text features are extracted from the input text data to obtain text feature data; the feature data includes keywords, domain information, and word meaning information, etc.

[0063] The process involves acquiring a predefined domain vocabulary, calculating the similarity between text feature data and the predefined domain vocabulary, and obtaining the text feature domain similarity. The predefined vocabulary includes legal vocabulary knowledge bases, medical vocabulary knowledge bases, and educational vocabulary knowledge bases, etc., acquired through big data methods.

[0064] The text feature domain similarity is compared with a preset feature domain similarity threshold to obtain a similarity comparison result;

[0065] Based on the similarity comparison results, the domain information of the text feature data is determined to obtain text feature domain determination data.

[0066] Those with a similarity greater than a threshold are identified as belonging to the corresponding domain.

[0067] The working principle and technical effect of the above technical solution are as follows: Focusing on text domain determination, feature extraction is first performed on the input text to extract core feature data such as keywords, domain-specific information, and word meanings to construct the feature representation of the text; a preset domain vocabulary library is called, which covers a collection of professional terms from multiple fields such as law, medicine, and education collected and organized through big data. Similarity calculation algorithms (such as cosine similarity and Jaccard similarity coefficient) are used to compare the matching degree between the text feature data and the vocabulary libraries of each domain to obtain the feature similarity between the text and different domains; a preset feature domain similarity threshold is set, and the calculated similarity is compared with the threshold. If the similarity of a certain domain exceeds the threshold, the text is determined to belong to that domain, and the text feature domain determination data is output.

[0068] This method improves the accuracy of text domain determination by extracting multi-dimensional features and matching them with a professional domain vocabulary, avoiding misjudgments caused by single features. The professionally designed pre-defined domain vocabulary ensures adaptability to texts in various domains, especially for professional fields such as law and medicine, accurately identifying domain-specific features. The introduction of a similarity threshold provides a clear standard for domain determination, reducing the interference of subjective factors. The final output domain-determined data provides precise domain guidance for subsequent density weight calculations, window segmentation, and other steps, ensuring that the entire knowledge slicing process revolves around the core domain of the text, improving the targeting and efficiency of the processing.

[0069] In one embodiment of the present invention, S2 includes:

[0070] A preset domain vocabulary library corresponding to the text feature domain determination data is obtained, and a transition density analysis is performed on the preset domain vocabulary library to obtain domain transition density analysis data; the transition density analysis data mainly includes the occurrence rate percentage of some words that represent transition meanings.

[0071] Based on text feature data, perform transition density analysis on the input text data to obtain text transition density analysis data;

[0072] Obtain the proportion of text transition density analysis data in domain transition density analysis data to obtain text domain density proportion data;

[0073] Obtain text transition density analysis data and multiple text domain closure ratio data in all preset domain vocabularies, and determine text domain density weight data based on the multiple text domain density ratio data.

[0074] The working principle and technical effect of the above technical solution are as follows: Based on the data determined by the output text feature domain, the corresponding preset domain lexicon is called, and a transition density analysis is performed on the lexicon. The frequency and proportion of words representing logical transitions (such as words with unrelated meanings, words with opposite meanings, however, in addition, in summary, etc.) in the lexicon are statistically analyzed to generate domain transition density analysis data. Based on the extracted text feature data, the input text is scanned, and the number and distribution of similar transition words in the text are identified and statistically analyzed. The proportion of the occurrence rate of transition words in the text is calculated to obtain text transition density analysis data. Through proportional calculation, the proportion of text transition density analysis data in the corresponding domain transition density analysis data is obtained, which is the text domain density proportion data. The text transition density analysis data is compared with all preset domain lexicons to calculate the density proportion data of the text in multiple domains.

[0075] This method establishes a connection between text and a domain lexicon using the quantitative indicator of transition density, thereby accurately characterizing the degree of text domain affiliation. The analysis of transition density focuses on the logical structural features of the text, effectively reflecting the structural fit between the text and the domain lexicon, supplementing the shortcomings of single semantic features. The acquisition of multi-domain density proportion data breaks through the limitations of single-domain determination, providing multi-dimensional data support for hierarchical density sequence division and weight calculation, ensuring a more comprehensive and objective subsequent weight allocation, and avoiding bias caused by a single-domain perspective.

[0076] In one embodiment of the present invention, the step of obtaining text transition density analysis data of multiple text domain closure ratios in all preset domain vocabularies, and determining text domain density weight data based on the multiple text domain density ratios, includes:

[0077] The density percentage data of multiple text domains are sorted to obtain the text domain density sequence;

[0078] Obtain the difference between adjacent density proportions in the density sequence of the text domain to obtain the density adjacent difference;

[0079] The density adjacent difference is compared with a preset density adjacent threshold to obtain the density adjacent comparison result;

[0080] Density tomography analysis is performed based on the density adjacency comparison results and the density sequence of the text domain to obtain density tomography analysis data.

[0081] The text neighborhood density weight data is determined based on density tomography data. The average text neighborhood density percentage of each density sequence is the text neighborhood density weight data for one or more neighborhoods corresponding to the density sequence.

[0082] The working principle and technical effect of the above technical solution are as follows: Multiple text domain density proportion data are sorted according to their numerical values ​​to form an ordered text domain density sequence; the difference between the density proportion data of two adjacent domains in the sequence is calculated sequentially to obtain the density adjacent difference; each density adjacent difference is compared with a preset density adjacent threshold to determine whether the density difference between adjacent domains reaches the tomographic standard, generating a density adjacent comparison result; density tomographic analysis is carried out by combining the comparison result with the density sequence; if the adjacent difference exceeds the threshold, the position is determined to be a density tomographic, and the original sequence is split into multiple hierarchical density sequences based on this threshold; the average value of all domain density proportion data within each hierarchical density sequence is calculated, and this average value is the text domain density weight data of the corresponding hierarchical domain, realizing the quantitative determination of the weight.

[0083] This method transforms disordered density data into a structured hierarchical sequence through density sequence sorting and tomographic analysis, making the calculation of domain weights more logical and hierarchical. Comparison of density adjacency differences with thresholds accurately identifies key boundary points in the domain density distribution, ensuring that the hierarchical division conforms to the density distribution pattern. Using the average value of each level as the weight balances the contribution of each domain within a level, avoiding excessive influence of density anomalies in a single domain on the weights, while simplifying the weight calculation process and improving the stability and reliability of the weight data. The final output density weight data provides a scientific basis for subsequent fixed-window segmentation, ensuring that the window segmentation accurately reflects the weight proportions of each domain.

[0084] In one embodiment of the present invention, density tomography analysis is performed based on the density adjacency comparison results and the density sequence of the text domain to obtain density tomography analysis data, including:

[0085] Obtain at least one density adjacency difference for each text domain density percentage data and compare it with a preset density adjacency threshold.

[0086] When the density adjacent comparison result is that the density adjacent difference is greater than the preset density adjacent threshold, the density adjacent difference is determined to be the fault difference.

[0087] The text domain density sequence is broken according to the tomographic difference to obtain multiple hierarchical density sequences;

[0088] The average value of the text domain density proportion data for each density sequence is obtained to obtain density tomography analysis data.

[0089] The working principle and technical effect of the above technical solution are as follows: For each domain density proportion data in the text domain density sequence, the difference between it and the density data of the adjacent domains before and after it is extracted (if it is the data at the beginning and end of the sequence, only the difference between adjacent domains on one side is taken), and each adjacent difference is compared with a preset density adjacent threshold. The comparison result of each difference (exceeding the threshold or not exceeding the threshold) is recorded. When the comparison result of a certain density adjacent difference exceeds the preset threshold, the difference is determined as a fault difference, and the adjacent position is marked as a potential density fault. The original text domain density sequence is split into multiple continuous subsequences, i.e., multiple hierarchical density sequences, using the positions corresponding to all fault differences as dividing points. The arithmetic mean of all domain density proportion data in each hierarchical density sequence is calculated to obtain the average density proportion of each level. The average density proportion and the corresponding hierarchical sequence together constitute the density fault analysis data.

[0090] This method achieves precise localization of density sequence faults through point-by-point difference analysis and threshold determination, ensuring the objectivity and accuracy of hierarchical division. The clear criteria for determining fault differences avoid subjective biases caused by manual hierarchical division; splitting the sequence into multiple hierarchical density sequences reveals a clear hierarchical structure from the potentially chaotic density distribution, facilitating weight calculation and window processing; the calculation of hierarchical average density assigns a unified quantitative index to each level, simplifying subsequent data application processes while ensuring that the characteristics of each level can be accurately represented, providing a clear hierarchical basis for window combination and clustering.

[0091] In one embodiment of the present invention, S3 includes:

[0092] Determine the fixed window size based on the input text data and text feature data, and obtain multiple fixed windows for the input text data;

[0093] Based on the text domain density weight data, multiple fixed windows are segmented to obtain fixed window segmentation data;

[0094] Based on the multiple hierarchical density sequences, multiple fixed windows are combined to obtain fixed window combination data;

[0095] Dynamic window data is generated by combining the fixed window segmentation data with the fixed window combination data. Dynamic semantic clustering is then performed based on the dynamic window data and the fixed window size to obtain dynamic semantic clustering information.

[0096] Based on the dynamic semantic clustering information, the preset knowledge information is sliced ​​to obtain knowledge slice processing data.

[0097] The working principle and technical effect of the above technical solution are as follows: Combining the length of the input text and the information density in the text feature data (such as keyword density and domain terminology distribution), the size of a fixed window is determined. The input text is then divided into multiple continuous fixed windows according to this size. The output text domain density weight data is called, and each fixed window is segmented according to the weight ratio of each domain, ensuring that the proportion of text in each domain within the window matches the weight, thus obtaining fixed window segmentation data. Then, based on the generated multiple hierarchical density sequences, fixed segmentation windows belonging to the same level in the window segmentation sequence are selected. It is determined whether these windows of the same level are consecutively adjacent in the text, and adjacent windows of the same level are merged to generate fixed window combination data. Based on the fixed window segmentation data and combination data, the window structure is updated to generate dynamic window data. The size of the dynamic window is obtained and compared with the fixed window size. Based on the size difference, an appropriate semantic clustering algorithm is selected (e.g., K-Means is selected when sizes are similar, and DBSCAN is selected when size differences are large) to perform dynamic semantic clustering, obtaining dynamic semantic clustering information. Based on the clustering information, preset knowledge information is sliced, and the core knowledge content corresponding to each cluster is extracted to obtain knowledge slice processing data.

[0098] By combining fixed and dynamic windows, the system balances stability and flexibility in text processing. Personalized settings for fixed window sizes ensure that windows adapt to the characteristics of different texts, avoiding information fragmentation or redundancy caused by uniform window sizes. Weighted window segmentation and hierarchical window combination enable window structures to conform to domain density distribution patterns, improving domain consistency of text within the window. Dynamic semantic clustering selects algorithms based on window size differences, ensuring the accuracy of clustering results. The generated knowledge slices fully preserve core knowledge from various domains, with clear slice structures and balanced information density, meeting the needs of knowledge extraction and application in multiple scenarios.

[0099] In one embodiment of the present invention, window segmentation is performed on multiple fixed windows based on the text density weight data to obtain fixed window segmentation data, including:

[0100] Obtain text domain density weight data for multiple text feature data in each fixed window;

[0101] A fixed segmentation window is obtained by segmenting a fixed window based on the density weight data of the text domain.

[0102] Obtain the window segmentation data for each fixed segmentation window to obtain the fixed window segmentation data;

[0103] The fixed segmentation windows are sorted according to the text feature data order to obtain the window segmentation sequence, and then the fixed window segmentation data sequence is obtained.

[0104] The working principle and technical effect of the above technical solution are as follows: For each generated fixed window, extract multiple text feature data (such as keywords and terms from different fields) contained within the window, and call the output text field density weight data to determine the field weight corresponding to each text feature data; according to the weight ratio of each field, spatially segment the fixed window, allocating more window space to fields with higher weight ratios to ensure that each sub-window (i.e., fixed segmentation window) after segmentation mainly corresponds to a single field or a field dominated by weight ratio, resulting in multiple fixed segmentation windows; record the start position, end position, corresponding field, and weight of each fixed segmentation window to form window segmentation data; sort all fixed segmentation windows according to the order of text feature data corresponding to each fixed segmentation window in the input text to form an ordered window segmentation sequence, which, together with the corresponding segmentation data, constitutes the fixed window segmentation data sequence.

[0105] This method combines text features and domain weights for window segmentation, ensuring that the segmented sub-windows accurately correspond to different domains (or ranges), improving the domain purity of the text within each window, and preventing text from multiple domains from mixing in the same window. This provides high-quality window data for semantic clustering. Detailed recording of window segmentation data facilitates window tracking and adjustment; the window segmentation sequence, ordered by text features, ensures that the window data maintains the logical order of the original text, avoiding textual logical confusion in subsequent processing. It also provides an ordered data foundation for window combination, improving the efficiency and accuracy of window combination.

[0106] In one embodiment of the present invention, multiple fixed windows are combined according to the multiple hierarchical density sequences to obtain fixed window combination data, including:

[0107] Obtain the fixed segmentation window corresponding to the same level of the window segmentation sequence based on the hierarchical sequence density;

[0108] Determine whether multiple fixed segmentation windows of the same level in a window segmentation sequence are adjacent to each other, and obtain window adjacency determination information;

[0109] Based on the window adjacency determination information, adjacent fixed segmented windows at the same level are combined to obtain fixed window combination data.

[0110] The working principle and technical effect of the above technical solution are as follows: Multiple hierarchical density sequences are invoked to clarify the domain scope and corresponding hierarchical features of each level; based on the hierarchical features, the window segmentation sequence output by the fixed window segmentation sub-implementation is scanned to identify the level to which each fixed segmentation window in the sequence belongs (i.e., to determine which hierarchical density sequence the domain of the window belongs to), and fixed segmentation windows belonging to the same level are filtered out to obtain a set of fixed segmentation windows corresponding to the same level; for each window in the set of fixed segmentation windows corresponding to the same level, based on its position information in the window segmentation sequence, it is determined whether two adjacent windows of the same level are continuous in the original input text (i.e., whether the end position of the previous window connects with the start position of the next window), generating window adjacency judgment information; when the judgment result is adjacent, the two adjacent fixed segmentation windows of the same level are merged to form a larger window unit, and this process is repeated until all adjacent windows of the same level are merged, finally generating fixed window combination data.

[0111] By employing hierarchical matching and adjacency judgment, precise combination of fixed windows at the same level is achieved. This integrates scattered small windows at the same level into larger window units, avoiding semantic fragmentation caused by overly fine window segmentation and ensuring that complete knowledge units within the same domain are preserved within the same combined window. Merging windows at the same level improves the semantic coherence and domain consistency of the text within the window, providing higher-quality window data for dynamic window generation and semantic clustering. The adjacency judgment mechanism ensures that combination only occurs between consecutive windows, avoiding textual logical breaks caused by combining non-consecutive windows, while simplifying the combination process and improving combination efficiency and accuracy.

[0112] In one embodiment of the present invention, dynamic window data is generated based on the fixed window segmentation data and the fixed window combination data, and dynamic semantic clustering is performed based on the dynamic window data and the fixed window size to obtain dynamic semantic clustering information, including:

[0113] Dynamic window data is obtained by updating the fixed window segmented data by combining data from fixed windows.

[0114] Get the size of the dynamic window data;

[0115] The fixed window size is compared with the dynamic window size to obtain the window size comparison result;

[0116] Based on the comparison results of window sizes, dynamic semantic clustering is performed on the windows to obtain dynamic semantic clustering information.

[0117] The working principle and technical effect of the above technical solution are as follows: This method completes dynamic window generation and semantic clustering. It merges the fixed window combination data output by the fixed window combination sub-implementation and the fixed window segmentation data output by the fixed window segmentation sub-implementation, updates the window units that have been combined in the segmentation data, replaces the original segmentation window data, and forms a new window structure, i.e., dynamic window data. It extracts the text length information corresponding to each dynamic window data and calculates the size of each dynamic window, i.e., the dynamic window size. It compares each dynamic window size with the preset fixed window size and analyzes the degree of difference between the two (such as the difference ratio and absolute difference) to generate window size comparison results. It selects the corresponding semantic clustering algorithm based on the size comparison results. If the difference between the dynamic window size and the fixed window size is small, a clustering algorithm suitable for uniform data distribution is selected; if the difference is large, a clustering algorithm suitable for non-uniform data distribution is selected. Based on the selected algorithm, semantic clustering is performed on all dynamic windows to mine the semantic associations between windows and form dynamic semantic clustering information.

[0118] This method generates dynamic windows that combine flexibility and structure by fusing segmented and combined data. This allows the window size to adapt to the actual semantic distribution of the text, avoiding the limitations of fixed window sizes. Comparing dynamic and fixed window sizes provides an objective basis for selecting clustering algorithms, ensuring that the algorithm can adapt to the distribution characteristics of the window data and improving the accuracy of clustering results. Semantic clustering can uncover potential semantic relationships between dynamic windows, grouping semantically similar windows into the same cluster. This provides a clear clustering basis for knowledge slices, ensuring that the final generated knowledge slices reflect the semantic logical structure of the text and improving the practicality and completeness of the knowledge slices.

[0119] According to one embodiment of the present invention, the system includes:

[0120] The feature domain analysis module is used to perform similarity analysis between the features of the input text data and the preset domain to obtain data to determine the text feature domain;

[0121] The transition density ratio analysis module is used to obtain the corresponding preset domain vocabulary based on the data determined by the text feature domain, perform transition density ratio analysis between the input text data and the preset domain vocabulary, determine the text domain density sequence, perform density interval difference analysis and density tomography analysis to obtain the hierarchical density sequence, and then determine the text domain density weight data.

[0122] The window dynamic processing module is used to set fixed windows for input text data, perform fixed window segmentation using text domain density weight data, combine fixed windows using hierarchical density sequences, generate dynamic window data based on segmentation and combination processing, perform clustering processing on dynamic window data using fixed window size, and then slice knowledge information to obtain knowledge slice processing data.

[0123] The working principle and technical effects of the above technical solution are as follows: First, a preliminary domain determination is completed for the input text. Based on the similarity matching between text features and a preset domain vocabulary, the core domain to which the text belongs is identified. Second, domain density weights are calculated in depth. Using transition density as the core indicator, the density ratio between the input text and the corresponding domain vocabulary is analyzed. Through density sequence sorting, adjacent difference comparison, and tomographic analysis, hierarchical density sequences are divided and the weights of each domain are determined. Third, dynamic window processing and knowledge slicing are implemented. First, fixed windows are set based on text features. Window segmentation is completed by combining domain weights. Then, dynamic windows are generated by combining adjacent windows of the same level according to the hierarchical density sequence. Semantic clustering is carried out based on window size differences, and finally, knowledge slicing is completed.

[0124] This method breaks away from the traditional fixed-window single-processing mode. By combining domain weights and dynamic windows, it transforms text processing from indiscriminate segmentation to precise domain adaptation. In the domain localization stage, it ensures accurate matching between text and the domain; in the density weight calculation stage, it refines the domain hierarchy through tomographic analysis, making the weight allocation more closely aligned with the actual contribution of the domain; in the window processing stage, it balances the stability of fixed windows with the flexibility of dynamic windows, resulting in more accurate semantic clustering results. The final generated knowledge slices can completely retain the core knowledge units of each domain, effectively avoiding semantic fragmentation and information redundancy, making it suitable for knowledge extraction scenarios involving multi-domain mixed text.

[0125] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A knowledge slicing analysis and processing method based on context window semantic clustering, characterized in that, The method includes: S1. Perform similarity analysis between the features of the input text data and the preset domain to obtain text feature domain determination data; S2. Based on the text feature domain, determine the corresponding preset domain vocabulary, perform transition density ratio analysis between the input text data and the preset domain vocabulary, determine the text domain density sequence, perform density interval difference analysis and density tomography analysis to obtain the hierarchical density sequence, and then determine the text domain density weight data. S3. Set a fixed window for the input text data, segment the fixed window using text domain density weight data, combine the fixed window using hierarchical density sequence, generate dynamic window data based on the segmentation and combination processing, cluster the dynamic window data using a fixed window size, and then slice the knowledge information to obtain knowledge slice processing data. Wherein, S2 includes: A preset domain vocabulary corresponding to the text feature domain determination data is obtained, and a transition density analysis is performed on the preset domain vocabulary to obtain domain transition density analysis data; the transition density analysis data includes the occurrence rate percentage of words representing transition meanings; Based on text feature data, perform transition density analysis on the input text data to obtain text transition density analysis data; Obtain the proportion of text transition density analysis data in domain transition density analysis data to obtain text domain density proportion data; Obtain text transition density analysis data and multiple text domain density percentage data in all preset domain vocabularies, and determine text domain density weight data based on the multiple text domain density percentage data.

2. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 1, characterized in that, S1 includes: Obtain input text data, extract text features from the input text data, and obtain text feature data; Obtain a predefined domain vocabulary, calculate the similarity between the text feature data and the predefined domain vocabulary, and obtain the text feature domain similarity. The text feature domain similarity is compared with a preset feature domain similarity threshold to obtain a similarity comparison result; Based on the similarity comparison results, the domain information of the text feature data is determined to obtain text feature domain determination data.

3. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 1, characterized in that, The process of obtaining text transition density analysis data includes multiple text domain density percentage data across all preset domain vocabularies, and determining text domain density weight data based on the multiple text domain density percentage data, including: The density percentage data of multiple text domains are sorted to obtain the text domain density sequence; Obtain the difference between adjacent density proportions in the density sequence of the text domain to obtain the density adjacent difference; The density adjacent difference is compared with a preset density adjacent threshold to obtain the density adjacent comparison result; Density tomography analysis is performed based on the density adjacency comparison results and the density sequence of the text domain to obtain density tomography analysis data. The density weight data of the text neighborhood is determined based on the density tomography analysis data.

4. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 3, characterized in that, Density tomography analysis is performed based on the density adjacency comparison results and the density sequence of the text domain to obtain density tomography analysis data, including: Obtain at least one density adjacency difference for each text domain density percentage data and compare it with a preset density adjacency threshold. When the density adjacent comparison result is that the density adjacent difference is greater than the preset density adjacent threshold, the density adjacent difference is determined to be the fault difference. The text domain density sequence is broken according to the tomographic difference to obtain multiple hierarchical density sequences; The average value of the text domain density proportion data for each density sequence is obtained to obtain density tomography analysis data.

5. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 4, characterized in that, S3 includes: Determine the fixed window size based on the input text data and text feature data, and obtain multiple fixed windows for the input text data; Based on the text domain density weight data, multiple fixed windows are segmented to obtain fixed window segmentation data; Based on the multiple hierarchical density sequences, multiple fixed windows are combined to obtain fixed window combination data; Dynamic window data is generated by combining the fixed window segmentation data with the fixed window combination data. Dynamic semantic clustering is then performed based on the dynamic window data and the fixed window size to obtain dynamic semantic clustering information. Based on the dynamic semantic clustering information, the preset knowledge information is sliced ​​to obtain knowledge slice processing data.

6. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 5, characterized in that, Based on the text domain density weight data, multiple fixed windows are segmented to obtain fixed window segmentation data, including: Obtain text domain density weight data for multiple text feature data in each fixed window; A fixed segmentation window is obtained by segmenting a fixed window based on the density weight data of the text domain. Obtain the window segmentation data for each fixed segmentation window to obtain the fixed window segmentation data; The fixed segmentation windows are sorted according to the text feature data order to obtain the window segmentation sequence, and then the fixed window segmentation data sequence is obtained.

7. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 5, characterized in that, Based on the multiple hierarchical density sequences, multiple fixed windows are combined to obtain fixed window combination data, including: Obtain the fixed segmentation window corresponding to the same level of the window segmentation sequence based on the hierarchical sequence density; Determine whether multiple fixed segmentation windows of the same level in a window segmentation sequence are adjacent to each other, and obtain window adjacency determination information; Based on the window adjacency determination information, adjacent fixed segmented windows at the same level are combined to obtain fixed window combination data.

8. The knowledge slicing analysis and processing method based on context window semantic clustering according to claim 5, characterized in that, Dynamic window data is generated based on the fixed window segmentation data and the fixed window combination data. Dynamic semantic clustering is then performed based on the dynamic window data and the fixed window size to obtain dynamic semantic clustering information, including: Dynamic window data is obtained by updating the fixed window segmented data by combining data from fixed windows. Get the size of the dynamic window data; The fixed window size is compared with the dynamic window size to obtain the window size comparison result; Based on the comparison results of window sizes, dynamic semantic clustering is performed on the windows to obtain dynamic semantic clustering information.

9. A knowledge slicing analysis and processing system based on context window semantic clustering, characterized in that, The system includes: The feature domain analysis module is used to perform similarity analysis between the features of the input text data and the preset domain to obtain data to determine the text feature domain; The transition density ratio analysis module is used to obtain the corresponding preset domain vocabulary based on the data determined by the text feature domain, perform transition density ratio analysis between the input text data and the preset domain vocabulary, determine the text domain density sequence, perform density interval difference analysis and density tomography analysis to obtain the hierarchical density sequence, and then determine the text domain density weight data. The window dynamic processing module is used to set a fixed window for the input text data, perform fixed window segmentation through text domain density weight data, perform fixed window combination through hierarchical density sequence, generate dynamic window data based on segmentation and combination processing, perform clustering processing on dynamic window data through fixed window size, and then slice knowledge information to obtain knowledge slice processing data. The transition density ratio analysis module includes: A preset domain vocabulary corresponding to the text feature domain determination data is obtained, and a transition density analysis is performed on the preset domain vocabulary to obtain domain transition density analysis data; the transition density analysis data includes the occurrence rate percentage of words representing transition meanings; Based on text feature data, perform transition density analysis on the input text data to obtain text transition density analysis data; Obtain the proportion of text transition density analysis data in domain transition density analysis data to obtain text domain density proportion data; Obtain text transition density analysis data and multiple text domain density percentage data in all preset domain vocabularies, and determine text domain density weight data based on the multiple text domain density percentage data.

Images