A knowledge base document analysis and retrieval system based on big data analytics

By constructing a set of manifold embedding vectors in the manifold space and introducing a path distribution reparameterization alignment mechanism, the problem of unstable retrieval results in multimodal data scenarios is solved, achieving more accurate semantic hierarchical structure expression and consistency of retrieval results, and improving the reliability of large-scale knowledge base document analysis and retrieval.

CN122221971BActive Publication Date: 2026-11-10WUHAN CHUYU TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610326504.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-03-17
Publication Date
2026-11-10
Estimated Expiration
2046-03-17

AI Technical Summary

Technical Problem

Existing knowledge base systems struggle to accurately depict the nesting relationships between concepts in multimodal data scenarios, and the retrieval results exhibit deviations in terms of structural consistency and semantic integrity. Traditional path alignment methods lack probability distribution modeling mechanisms, leading to unstable retrieval results.

Method used

We employ non-Euclidean manifold embedding, geodesic distance modeling, path distribution reparameterization alignment, and joint energy optimization inference mechanisms to construct a complete process from multimodal data access to structure index generation. By characterizing semantic hierarchical relationships in manifold space, we introduce an adjustable path probability distribution to control the alignment structure and achieve unified optimization inference with a joint energy function.

Benefits of technology

It achieves a more accurate semantic hierarchy structure expression, enhances the stability and consistency of semantic matching between files, and improves the reliability of large-scale knowledge base file analysis and retrieval.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122221971B_ABST
    Figure CN122221971B_ABST
Patent Text Reader

Abstract

This invention discloses a knowledge base file analysis and retrieval system based on big data analytics, comprising: a data access module for generating a unified feature sequence; a manifold embedding construction module for generating a manifold embedding vector set and a geodesic distance matrix; a path distribution reparameterization and alignment module for generating a path probability distribution function, a soft alignment matrix, and path energy values; a joint energy construction module for generating joint energy results; an energy optimization inference module for outputting file alignment results; a structure index generation module for generating a structure index; and a retrieval ranking module for outputting retrieval ranking results. This invention achieves structure alignment-optimized retrieval.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of semantic structure retrieval technology, and in particular to a knowledge base document analysis and retrieval system based on big data analysis. Background Technology

[0002] With the continuous advancement of information technology infrastructure, enterprises and organizations generate a large amount of text files, spreadsheets, image data, and speech-to-text content in their daily operations. These data types exhibit significant structural differences, complex semantic expressions, and scattered storage formats. Traditional knowledge base systems often employ keyword matching or vector similarity retrieval methods for information searching. Retrieval models are typically built upon Euclidean space representations, lacking effective representation capabilities for semantic hierarchical relationships, concept inclusion relationships, and hierarchical relationships. In multimodal data scenarios, differences in expression scale and inconsistent feature distributions exist between different data types. Directly concatenating feature vectors can easily lead to semantic confusion, and the retrieval results have limited ability to recognize complex structural relationships.

[0003] In existing technologies, most retrieval systems use fixed distance metrics for similarity calculation. These distance functions fail to consider the hierarchical curvature of semantic structures, making it difficult to accurately characterize the nesting relationships between concepts in high-dimensional semantic spaces. Furthermore, traditional path alignment or sequence matching methods often employ deterministic shortest path strategies, lacking probabilistic distribution modeling mechanisms for path selection, making it difficult to comprehensively evaluate the structural differences among multiple candidate paths. Regarding energy modeling, most methods construct loss functions only for a single feature space, failing to form a multi-constraint joint optimization structure, resulting in deviations in the retrieval results regarding structural consistency and semantic integrity.

[0004] Therefore, in a multimodal knowledge base environment, how to construct an embedded representation that can express a semantic hierarchical structure, how to build an adjustable path probability distribution model based on geodesic distance, and how to achieve optimization inference under structural constraints through a joint energy mechanism have become urgent problems to be solved in current technologies. Summary of the Invention

[0005] One objective of this invention is to propose a knowledge base file analysis and retrieval system based on big data analytics. This invention combines non-Euclidean manifold embedding, geodesic distance modeling, path distribution reparameterization alignment, and a joint energy optimization inference mechanism to construct a complete process from multimodal data access, semantic structure mapping, path probability modeling, to structural index generation. By characterizing semantic hierarchical relationships in manifold space, introducing an adjustable path probability distribution to control the alignment structure, and achieving unified optimization inference using a joint energy function, the semantic relationship expression of files becomes clearer, the structural consistency is stronger, and the retrieval ranking results are more stable, making it suitable for large-scale knowledge base file analysis and retrieval scenarios.

[0006] According to an embodiment of the present invention, a knowledge base document analysis and retrieval system based on big data analysis includes: The system comprises the following modules: Data Access Module: Receives text files, table files, image files, and speech-to-text results, and generates a unified feature sequence; Manifold Embedding Construction Module: Performs non-Euclidean geometric mapping on the unified feature sequence to generate a set of manifold embedding vectors and constructs a geodesic distance matrix. The geodesic distance is controlled by curvature parameters to manage semantic hierarchy; Path Distribution Reparameterization Alignment Module: Constructs a reparameterizable path probability distribution function based on the geodesic distance matrix to generate a soft alignment matrix. The path probability distribution is controlled by learnable parameters to manage path sparsity and concentration, and outputs path energy terms; Joint Energy Construction Module: Performs a weighted combination of manifold embedding energy terms and path energy terms to form a total energy function; Energy Optimization Inference Module: Performs gradient descent on the total energy function to generate file alignment results; Structure Index Generation Module: Maps the file alignment results to a semantic structure index; Retrieval and Ranking Module: Generates retrieval and ranking results based on the semantic structure index.

[0007] Optionally, the data access module specifically includes: The file type recognition unit reads the file header identifiers of text files, table files, image files, and speech-to-text results to generate file type labels; the text parsing unit performs segmentation, syntactic segmentation, and lexicalization on text files to generate text feature sequences; the table parsing unit performs cell coordinate positioning, field name extraction, and numerical field normalization encoding on table files to generate table feature sequences; the image parsing unit performs character region positioning, character recognition, and recognition result sorting on image files to generate image text feature sequences; the speech-to-text processing unit performs timestamp sorting and punctuation standardization on speech-to-text results to generate speech text feature sequences; the feature encoding unit performs unified vector encoding on text feature sequences, table feature sequences, image text feature sequences, and speech text feature sequences to generate multimodal feature vectors; and the sequence alignment unit performs time index sorting and position index rearrangement on multimodal feature vectors to generate unified feature sequences.

[0008] Optionally, the process of performing a non-Euclidean geometric mapping on the uniform feature sequence to generate a set of manifold embedding vectors in the manifold embedding construction module specifically includes: Read the unified feature sequence, extract multimodal feature vectors in time index order to form a feature vector set, and assign a vector number and time index identifier to each multimodal feature vector in the feature vector set; Set the curvature parameter value k, establish a mapping table between the curvature parameter and the vector number, and write a k tag for each multimodal feature vector in the feature vector set; Perform a linear projection operation on each multimodal feature vector in the feature vector set to generate a tangent space vector. The dimension of the linear projection matrix is ​​consistent with the dimension of the multimodal feature vector. The norm operation is performed on the tangent space vector to obtain the vector norm value. The vector norm value is compared with a preset truncation threshold. If the vector norm value exceeds the truncation threshold, the vector scaling operation is performed to generate a truncated tangent space vector. The truncation threshold value is determined by k. The exponential mapping operation is performed on the truncated tangent space vector to generate a manifold embedding vector. The exponential mapping operation calculates the geodesic coordinate components through the curvature parameter k and outputs the manifold embedding vector. The manifold embedding vector is then written into the manifold embedding vector set. Perform manifold constraint verification operation on each manifold embedding vector in the manifold embedding vector set. The verification operation includes embedding vector norm calculation and manifold equation consistency calculation. If the verification fails, perform rescaling operation to generate a corrected manifold embedding vector and overwrite the original embedding vector. Perform geodesic distance calculation on any two manifold embedding vectors in the manifold embedding vector set to generate geodesic distance values. Arrange the geodesic distance values ​​in order of vector number to form a geodesic distance matrix and write it into the geodesic distance matrix storage area.

[0009] Optionally, the geodesic distance calculation operation specifically includes: Select the first manifold embedding vector and the second manifold embedding vector from the set of manifold embedding vectors, and read the set of coordinate components of the first manifold embedding vector and the set of coordinate components of the second manifold embedding vector; The inner product operation is performed on the first manifold embedding vector and the second manifold embedding vector to generate the inner product value. The inner product calculation adopts the hyperbolic space inner product form, and the inner product value includes the sign weighted result of the time component and the spatial component. The curvature scaling operation is performed on the inner product value to generate a scaled inner product value. The curvature scaling operation transforms the inner product value proportionally through the curvature parameter k. Perform inverse hyperbolic cosine operation on the scaled inner product value to generate geodesic angle values. The input of the inverse hyperbolic cosine operation must meet the constraint condition of being greater than or equal to one. If the constraint condition is not met, perform a truncation correction operation on the scaled inner product value. The curvature normalization operation is performed on the geodesic angle values ​​to generate geodesic distance values. The curvature normalization operation is used to scale the geodesic angle values ​​proportionally by taking the square root of the absolute value of k. The generated geodesic distance values ​​are written into the corresponding row and column positions of the geodesic distance matrix. The row and column positions are determined by the first manifold embedding vector number and the second manifold embedding vector number. Repeat the aforementioned steps for all vector pairs in the manifold embedding vector set to generate a complete geodesic distance matrix.

[0010] Optionally, the process of constructing the execution path probability distribution function for the path distribution reparameterization alignment module specifically includes: Read the geodesic distance matrix, perform full matrix statistical operations on the geodesic distance matrix to generate the mean distance value and the variance distance value; perform interval mapping operation on the variance distance value to generate path sparsity control value, and perform scaling operation on the mean distance value to generate path concentration control value; Perform diagonal bandwidth statistical operations on the geodesic distance matrix to generate path bandwidth control values; write the path sparse control values, path concentrated control values, and path bandwidth control values ​​into the path distribution parameter set in a fixed order; perform negative value mapping processing on the geodesic distance matrix to generate the path cost matrix; The path cost matrix is ​​subjected to centralized scaling to generate a logarithmic weight matrix, and the centralized scaling magnitude is determined by the centralized control value of the path; the logarithmic weight matrix is ​​superimposed with perturbation values ​​to generate a perturbation weight matrix, and the perturbation magnitude is determined by the path sparsity control value. Temperature normalization is performed on the perturbation weight matrix to generate an initial path probability matrix; the allowable index band is calculated according to the path bandwidth control value, and matrix elements exceeding the allowable index band are written with zero values ​​to generate a bandwidth-limited path probability matrix. The path probability matrix with limited bandwidth is subjected to alternating row normalization and column normalization until the error is less than a preset threshold to generate the path probability distribution function.

[0011] Optionally, the path distribution reparameterization alignment module performs the soft alignment matrix generation and path energy term calculation process, including: Read the path probability matrix and geodesic distance matrix corresponding to the path probability distribution function; perform element-wise multiplication on the path probability matrix and geodesic distance matrix to generate a path weighting matrix; Perform full matrix summation on the path weighting matrix to generate the cumulative path cost; perform logarithmic transformation on the path probability matrix to generate the logarithmic probability matrix; perform element-wise multiplication on the path probability matrix and the logarithmic probability matrix to generate the path entropy matrix; perform summation on the path entropy matrix to generate the path entropy value. Read the path centralized control value and path sparse control value from the path distribution parameter set; perform proportional weighting on the path cumulative cost value and path entropy value to generate the path energy value; Perform forward accumulation processing on the path probability matrix to generate a forward accumulation matrix; perform backtracking processing on the forward accumulation matrix to generate a path index sequence; Perform position writing processing on the path index sequence to generate a hard alignment matrix; perform weighted fusion processing on the path probability matrix and the hard alignment matrix to generate a soft alignment matrix.

[0012] Optionally, the joint energy construction module specifically includes: Read the path energy values ​​and geodesic distance matrix, and read the manifold embedding vector set; perform neighborhood index statistical processing on the manifold embedding vector set to generate a local neighborhood relation set; Perform geodesic distance difference statistical processing on the local neighborhood relation set to generate manifold smoothing constraint values; perform full matrix distribution statistical processing on the geodesic distance matrix to generate distance structure dispersion values; Piecewise normalization is performed on the path energy values, manifold smoothing constraint values, and distance structure dispersion values ​​to generate a standardized set of energy components. A weighted combination process is performed on the standardized energy component set in a fixed order to generate a joint energy value; a stability interval verification process is performed on the joint energy value, and a proportional compression process is performed when the joint energy value exceeds the preset interval; the joint energy value is written into the energy storage area to form the joint energy result. The joint energy results are smoothed using a historical sliding window to generate the final joint energy output.

[0013] Optionally, the energy optimization inference module specifically includes: Read the joint energy result and the soft alignment matrix, and construct the current state index set; perform gradient direction estimation on the joint energy result to generate energy change direction values. The gradient direction estimation process uses a finite difference method to calculate the energy change amplitude point by point in the neighborhood index set. A candidate update state set is generated based on the energy change direction and numerical value; energy recalculation is performed on the candidate update state set to generate a candidate energy set; The minimum value selection process is performed on the candidate energy set to generate the optimal candidate state; the convergence determination process is performed on the optimal candidate state, and the convergence determination process calculates the difference between the current energy and the previous energy. When the difference is less than a preset threshold, the optimization is terminated. When the difference is greater than the preset threshold, the optimal candidate state is written into the current state index set and the gradient direction estimation process is repeated. After the optimization is terminated, the path consistency verification process is performed on the final state index set to generate the inference result. The inference result is written into the retrieval result storage area to form the final output.

[0014] Optionally, the structure index generation module specifically includes: Read the inference results and the soft alignment matrix, and extract the path index sequence retained in the inference results; perform sequential traversal processing on the path index sequence to generate a set of node connection pairs; The set of node connections is deduplicated to generate a set of unique connections; the set of unique connections is then partitioned into hierarchical numbers to generate a set of hierarchical numbers, which are sorted and calculated according to path depth and the number of neighboring connections. A similarity threshold comparison is performed on the manifold embedding vector set to generate a set of similar node groups; a cross-mapping process is performed on the hierarchical number set and the set of similar node groups to generate a set of structural nodes; and a hash encoding process is performed on the set of structural nodes to generate a set of structural index key values. Perform inverted index mapping on the structure index key-value set to generate a structure inverted index table; perform consecutive numbering on the structure inverted index table to generate a structure index sequence; write the structure index sequence into the index storage area to form the structure index result.

[0015] The beneficial effects of this invention are: (1) This invention constructs a set of manifold embedding vectors in a non-Euclidean manifold space and uses geodesic distance matrix to characterize semantic hierarchy relationships, so that the inclusion relationship and hierarchical relationship between concepts form a distinguishable geometric structure in the embedding space. Compared with the traditional Euclidean vector representation, it can more accurately express complex semantic hierarchy structures and reduce the impact of structural distortion in high-dimensional semantic space on retrieval results.

[0016] (2) This invention introduces a path distribution reparameterization alignment mechanism, which constructs an adjustable path probability distribution function based on the geodesic distance matrix. The alignment structure is constrained by the path sparse control value and the path concentrated control value, so that the alignment process has continuous and adjustable characteristics, avoids the structural deviation caused by single path selection, and enhances the stability and consistency of semantic matching between files.

[0017] (3) This invention constructs a joint energy function and completes structural consistency verification and index generation through energy optimization inference. It models the manifold embedded energy term and the path energy term in a unified manner to achieve overall optimization under multiple constraints. This ensures that the generated structural index and the retrieval ranking results are consistent in terms of semantic expression and structural integrity, thereby improving the overall reliability of large-scale knowledge base file analysis and retrieval. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a knowledge base document analysis and retrieval system based on big data analysis proposed in this invention. Figure 2 This is a schematic diagram illustrating the manifold embedding and geodesic distance construction process of a knowledge base document analysis and retrieval system based on big data analysis proposed in this invention.

[0019] Figure 3 This is a schematic diagram illustrating the path distribution reparameterization and soft alignment generation process of a knowledge base file analysis and retrieval system based on big data analysis proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figures 1-3 A knowledge base document analysis and retrieval system based on big data analytics, comprising: The system comprises the following modules: Data Access Module: Receives text files, table files, image files, and speech-to-text results, and generates a unified feature sequence; Manifold Embedding Construction Module: Performs non-Euclidean geometric mapping on the unified feature sequence to generate a set of manifold embedding vectors and constructs a geodesic distance matrix. The geodesic distance is controlled by curvature parameters to manage semantic hierarchy; Path Distribution Reparameterization Alignment Module: Constructs a reparameterizable path probability distribution function based on the geodesic distance matrix to generate a soft alignment matrix. The path probability distribution is controlled by learnable parameters to manage path sparsity and concentration, and outputs path energy terms; Joint Energy Construction Module: Performs a weighted combination of manifold embedding energy terms and path energy terms to form a total energy function; Energy Optimization Inference Module: Performs gradient descent on the total energy function to generate file alignment results; Structure Index Generation Module: Maps the file alignment results to a semantic structure index; Retrieval and Ranking Module: Generates retrieval and ranking results based on the semantic structure index.

[0022] In this embodiment, the data access module specifically includes: The file type recognition unit reads the file header identifiers of text files, table files, image files, and speech-to-text results to generate file type labels; the text parsing unit performs segmentation, syntactic segmentation, and lexicalization on text files to generate text feature sequences; the table parsing unit performs cell coordinate positioning, field name extraction, and numerical field normalization encoding on table files to generate table feature sequences; the image parsing unit performs character region positioning, character recognition, and recognition result sorting on image files to generate image text feature sequences; the speech-to-text processing unit performs timestamp sorting and punctuation standardization on speech-to-text results to generate speech text feature sequences; the feature encoding unit performs unified vector encoding on text feature sequences, table feature sequences, image text feature sequences, and speech text feature sequences to generate multimodal feature vectors; and the sequence alignment unit performs time index sorting and position index rearrangement on multimodal feature vectors to generate unified feature sequences.

[0023] In this embodiment, the process of performing a non-Euclidean geometric mapping on the uniform feature sequence to generate a set of manifold embedding vectors in the manifold embedding construction module specifically includes: Read the unified feature sequence, extract multimodal feature vectors in time index order to form a feature vector set, and assign a vector number and time index identifier to each multimodal feature vector in the feature vector set; Set the curvature parameter value k, establish a mapping table between the curvature parameter and the vector number, and write a k tag for each multimodal feature vector in the feature vector set; Perform a linear projection operation on each multimodal feature vector in the feature vector set to generate a tangent space vector. The dimension of the linear projection matrix is ​​consistent with the dimension of the multimodal feature vector. The norm operation is performed on the tangent space vector to obtain the vector norm value. The vector norm value is compared with a preset truncation threshold. If the vector norm value exceeds the truncation threshold, the vector scaling operation is performed to generate a truncated tangent space vector. The truncation threshold value is determined by k. The exponential mapping operation is performed on the truncated tangent space vector to generate a manifold embedding vector. The exponential mapping operation calculates the geodesic coordinate components through the curvature parameter k and outputs the manifold embedding vector. The manifold embedding vector is then written into the manifold embedding vector set. Perform manifold constraint verification operation on each manifold embedding vector in the manifold embedding vector set. The verification operation includes embedding vector norm calculation and manifold equation consistency calculation. If the verification fails, perform rescaling operation to generate a corrected manifold embedding vector and overwrite the original embedding vector. Perform geodesic distance calculation on any two manifold embedding vectors in the manifold embedding vector set to generate geodesic distance values. Arrange the geodesic distance values ​​in order of vector number to form a geodesic distance matrix and write it into the geodesic distance matrix storage area.

[0024] In this embodiment, performing the geodetic distance calculation specifically includes: Select the first manifold embedding vector and the second manifold embedding vector from the set of manifold embedding vectors, and read the set of coordinate components of the first manifold embedding vector and the set of coordinate components of the second manifold embedding vector; The inner product operation is performed on the first manifold embedding vector and the second manifold embedding vector to generate the inner product value. The inner product calculation adopts the hyperbolic space inner product form, and the inner product value includes the sign weighted result of the time component and the spatial component. The curvature scaling operation is performed on the inner product value to generate a scaled inner product value. The curvature scaling operation transforms the inner product value proportionally through the curvature parameter k. Perform inverse hyperbolic cosine operation on the scaled inner product value to generate geodesic angle values. The input of the inverse hyperbolic cosine operation must meet the constraint condition of being greater than or equal to one. If the constraint condition is not met, perform a truncation correction operation on the scaled inner product value. The curvature normalization operation is performed on the geodesic angle values ​​to generate geodesic distance values. The curvature normalization operation is used to scale the geodesic angle values ​​proportionally by taking the square root of the absolute value of k. The generated geodesic distance values ​​are written into the corresponding row and column positions of the geodesic distance matrix. The row and column positions are determined by the first manifold embedding vector number and the second manifold embedding vector number. Repeat the aforementioned steps for all vector pairs in the manifold embedding vector set to generate a complete geodesic distance matrix.

[0025] In this embodiment, performing manifold constraint verification specifically includes: reading each manifold embedding vector one by one from the manifold embedding vector set and extracting all its coordinate components; performing norm calculation processing on the manifold embedding vector to generate a norm value.

[0026] The norm value is compared with the target norm lower bound and target norm upper bound values ​​determined by the curvature parameter. When the norm value is between the target norm lower bound and target norm upper bound values, the current manifold embedding vector remains unchanged; when the norm value is less than the target norm lower bound value, the current manifold embedding vector is scaled up to the length corresponding to the target norm lower bound value; when the norm value is greater than the target norm upper bound value, the current manifold embedding vector is scaled down to the length corresponding to the target norm upper bound value.

[0027] After completing the norm interval verification, the geodesic distance values ​​between the current manifold embedding vector and its neighboring vectors are read. The difference between adjacent geodesic distances is calculated and compared with a preset change threshold. When the difference is less than or equal to the preset change threshold, the structural relationship is deemed to meet the continuity condition; when the difference is greater than the preset change threshold, the average geodesic distance between the vector and its neighboring vectors is calculated, and the position coordinates of the current vector in the manifold space are recalculated based on the average value. The recalculated vector then overwrites the original manifold embedding vector.

[0028] In this embodiment, the process of constructing the execution path probability distribution function of the path distribution reparameterization alignment module specifically includes: Read the geodesic distance matrix, perform full matrix statistical operations on the geodesic distance matrix to generate the mean distance value and the variance distance value; perform interval mapping operation on the variance distance value to generate path sparsity control value, and perform scaling operation on the mean distance value to generate path concentration control value; Perform diagonal bandwidth statistical operations on the geodesic distance matrix to generate path bandwidth control values; write the path sparse control values, path concentrated control values, and path bandwidth control values ​​into the path distribution parameter set in a fixed order; perform negative value mapping processing on the geodesic distance matrix to generate the path cost matrix; The path cost matrix is ​​subjected to centralized scaling to generate a logarithmic weight matrix, and the centralized scaling magnitude is determined by the centralized control value of the path; the logarithmic weight matrix is ​​superimposed with perturbation values ​​to generate a perturbation weight matrix, and the perturbation magnitude is determined by the path sparsity control value. Temperature normalization is performed on the perturbation weight matrix to generate an initial path probability matrix; the allowable index band is calculated according to the path bandwidth control value, and matrix elements exceeding the allowable index band are written with zero values ​​to generate a bandwidth-limited path probability matrix. The path probability matrix with limited bandwidth is subjected to alternating row normalization and column normalization until the error is less than a preset threshold to generate the path probability distribution function.

[0029] In this embodiment, performing diagonal bandwidth statistical operations on the geodesic distance matrix to generate path bandwidth control values ​​specifically includes: Read the geodesic distance matrix, iterate through the positions of all elements whose geodesic distance values ​​are less than a preset distance threshold, and record the difference between the row index and column index of these elements. Perform absolute value processing on all differences to generate an offset value set. Then, perform maximum value statistics processing on the offset value set to generate the maximum offset value, and determine the maximum offset value or its proportional scaling result as the path bandwidth control value.

[0030] In this embodiment, performing full matrix statistical operations specifically includes: The complete geodesic distance matrix is ​​read from the geodesic distance matrix storage area. The geodesic distance matrix is ​​a square matrix structure, and each element in the matrix corresponds to the geodesic distance value between two manifold embedding vectors. The system traverses the matrix elements in row-major order to construct a distance value traversal sequence.

[0031] During the traversal, the geodesic distance values ​​for all non-diagonal positions are accumulated to generate a total cumulative distance value, and the number of elements involved in the accumulation is recorded. The geodesic distance values ​​for diagonal positions are zero and are not included in the statistical calculation. After the traversal is complete, the total cumulative distance value is divided by the number of elements involved in the statistics to generate the average distance value.

[0032] After generating the mean distance value, each off-diagonal element in the geodesic distance matrix is ​​read again in the same traversal order. For each geodesic distance value, a difference calculation is performed, subtracting the mean distance value from the current geodesic distance value to generate a deviation value. The deviation value is then squared to generate a squared deviation value, and all squared deviation values ​​are summed to generate a cumulative squared deviation value.

[0033] After traversing all elements, the cumulative squared deviation is divided by the number of elements involved in the statistics to generate the distance variance value. The distance mean value and the distance variance value are written into the input buffer of the path distribution parameter generation process, serving as the basic statistics for generating path sparsity control values ​​and path concentration control values.

[0034] In this embodiment, the temperature normalization process specifically includes: reading weight values ​​row by row from the perturbation weight matrix and constructing a set of weight values ​​for the current row. Then, the path sparsity control values ​​from the path distribution parameter set are read and mapped to temperature control values.

[0035] Before normalizing a row, each perturbation weight value in that row is divided by the temperature control value to generate a scaled weight value. The smaller the temperature control value, the more pronounced the weight differences after scaling; the larger the temperature control value, the smoother the weight differences after scaling. After scaling, an exponential mapping process is performed on the scaled weight values ​​to generate positive weight values.

[0036] The process involves summing all positive weight values ​​in the current row to generate the total weight value for that row. Then, each positive weight value in the current row is divided by the total weight value to obtain the probability value for that row, which is then written to the corresponding position in the path probability matrix. This process is repeated for each row of the perturbation weight matrix until all rows are traversed, ultimately generating the initial path probability matrix.

[0037] In this embodiment, calculating the allowed index band specifically includes: Read the number of rows and columns of the geodesic distance matrix, and denote them as the first index length and the second index length, respectively. Calculate the smaller of the two values ​​as the path baseline length. Then read the path bandwidth control value.

[0038] The path bandwidth control value is mapped to a bandwidth width value. The mapping method is as follows: the path bandwidth control value is proportionally scaled and converted into an integer width value, which does not exceed a certain percentage of the path baseline length. If the mapping result is less than the preset minimum width, the minimum width value is used; if it is greater than the maximum width limit, it is truncated to the maximum width value.

[0039] After the bandwidth value is determined, the system constructs an allowed index band according to the index position. For any element position in the path probability matrix, if the difference between its row index and column index is within the range determined by the bandwidth value, the position is marked as a valid index position; if the difference exceeds the range, the position is marked as an invalid index position.

[0040] After completing the full matrix indexing, a set of allowed index bands is formed. Then, bandwidth limiting processing is performed on the path probability matrix, writing zero values ​​to the matrix elements corresponding to all invalid index positions, and retaining only the probability values ​​within the allowed index bands, thereby generating a bandwidth-limited path probability matrix.

[0041] In this embodiment, the path distribution reparameterization alignment module performs the soft alignment matrix generation and path energy term calculation process, including: Read the path probability matrix and geodesic distance matrix corresponding to the path probability distribution function; perform element-wise multiplication on the path probability matrix and geodesic distance matrix to generate a path weighting matrix; Perform full matrix summation on the path weighting matrix to generate the cumulative path cost; perform logarithmic transformation on the path probability matrix to generate the logarithmic probability matrix; perform element-wise multiplication on the path probability matrix and the logarithmic probability matrix to generate the path entropy matrix; perform summation on the path entropy matrix to generate the path entropy value. Read the path centralized control value and path sparse control value from the path distribution parameter set; perform proportional weighting on the path cumulative cost value and path entropy value to generate the path energy value; Perform forward accumulation processing on the path probability matrix to generate a forward accumulation matrix; perform backtracking processing on the forward accumulation matrix to generate a path index sequence; forward accumulation processing refers to the process of calculating the path accumulation value position by position in the path probability matrix according to the index progression order; Perform position writing processing on the path index sequence to generate a hard alignment matrix; perform weighted fusion processing on the path probability matrix and the hard alignment matrix to generate a soft alignment matrix.

[0042] In this embodiment, the execution location writing process specifically includes: Create a zero-value matrix with the same dimensions as the path probability matrix as the initial structure of the hard alignment matrix. Then, read each index pair sequentially according to the path index sequence. For the current index pair, write the values ​​at the corresponding row and column indices in the hard alignment matrix to fixed-value markers. These fixed-value markers are typically a single value indicating that the position belongs to the optimal path structure. Except for positions appearing in the path index sequence, all other positions in the hard alignment matrix remain at zero.

[0043] After writing all index pairs, a hard-aligned matrix is ​​generated, where only the path positions have a value of one and the rest have a value of zero.

[0044] After generating the hard alignment matrix, a weighted fusion process is performed on the path probability matrix and the hard alignment matrix. Specifically, according to a preset fusion ratio, the value of each element in the path probability matrix is ​​proportionally combined with the corresponding element in the hard alignment matrix to generate the fused soft alignment matrix. After the fusion process, the path position retains both probability distribution information and enhanced optimal path structure information.

[0045] In this embodiment, the joint energy construction module specifically includes: Read the path energy values ​​and geodesic distance matrix, and read the manifold embedding vector set; perform neighborhood index statistical processing on the manifold embedding vector set to generate a local neighborhood relation set; Perform geodesic distance difference statistical processing on the local neighborhood relation set to generate manifold smoothing constraint values; perform full matrix distribution statistical processing on the geodesic distance matrix to generate distance structure dispersion values; Piecewise normalization is performed on the path energy values, manifold smoothing constraint values, and distance structure dispersion values ​​to generate a standardized set of energy components. A weighted combination process is performed on the standardized energy component set in a fixed order to generate a joint energy value; a stability interval verification process is performed on the joint energy value, and a proportional compression process is performed when the joint energy value exceeds the preset interval; the joint energy value is written into the energy storage area to form the joint energy result. The joint energy results are smoothed using a historical sliding window to generate the final joint energy output.

[0046] In this embodiment, the process of performing neighborhood index statistical processing to generate a local neighborhood relationship set specifically includes: Each manifold embedding vector is read one by one from the manifold embedding vector set, and its corresponding vector number is obtained. Then, the row of data corresponding to the current vector number in the geodesic distance matrix is ​​read. This row of data contains the geodesic distance values ​​between the current vector and all other vectors.

[0047] After acquiring the data for that row, a comparison process is performed on all geodesic distance values. Vectors whose geodesic distance values ​​are less than or equal to a preset neighborhood distance threshold are recorded as the neighborhood index set for the current vector. If the number of vectors meeting the condition in a row exceeds the preset maximum number of neighborhoods, they are sorted in ascending order of geodesic distance values, and only the first few vector numbers are retained as valid neighborhood indices.

[0048] The above process is repeated for each vector in the manifold embedding vector set, ultimately generating a corresponding neighborhood index set for each vector. All vectors and their neighborhood index sets are then aggregated to form a local neighborhood relation set.

[0049] In this embodiment, the specific steps for generating the distance structure discreteness value through full matrix distribution statistical processing include: Read the complete geodesic distance matrix from the storage area, iterate through all off-diagonal geodesic distance values, and detect the maximum and minimum distance values. Construct a distance interval range based on the maximum and minimum values, divide this range into several continuous sub-intervals, and count the number of corresponding geodesic distance values ​​in each sub-interval to generate an interval count set.

[0050] After completing the interval counting, a proportional conversion is performed on the interval count set to generate an interval distribution ratio set. Then, the deviation value between each interval ratio value and the average ratio is calculated, and the absolute value of the deviation value is accumulated to generate a cumulative distribution offset value, which is used to characterize the degree of concentration or dispersion of geodesic distance values ​​in the overall interval distribution.

[0051] Based on interval statistics, adjacent element difference statistical processing is performed along the row and column directions of the geodesic distance matrix, respectively. The absolute difference between adjacent geodesic distances is calculated and accumulated to generate structural fluctuation values. The cumulative distribution offset values ​​and structural fluctuation values ​​are combined to generate distance-structure dispersion values, which are then written into the joint energy construction module.

[0052] In this embodiment, the stable interval verification process specifically includes: In the joint energy construction module, a lower limit value and an upper limit value of energy are preset. The range of this range is determined based on the historical joint energy statistics or the energy distribution range recorded in the initial training phase.

[0053] After generating the joint energy value, a range comparison is performed between the joint energy value and the preset lower and upper energy limits. When the joint energy value is within the range, the original value remains unchanged; when the joint energy value is lower than the lower energy limit, the joint energy value is adjusted to the lower energy limit; when the joint energy value is higher than the upper energy limit, the joint energy value is adjusted to the upper energy limit.

[0054] In this embodiment, the energy optimization inference module specifically includes: Read the joint energy result and the soft alignment matrix, and construct the current state index set; perform gradient direction estimation on the joint energy result to generate energy change direction values. The gradient direction estimation process uses a finite difference method to calculate the energy change amplitude point by point in the neighborhood index set. A candidate update state set is generated based on the energy change direction and numerical value; energy recalculation is performed on the candidate update state set to generate a candidate energy set; The minimum value selection process is performed on the candidate energy set to generate the optimal candidate state; the convergence determination process is performed on the optimal candidate state, and the convergence determination process calculates the difference between the current energy and the previous energy. When the difference is less than a preset threshold, the optimization is terminated. When the difference is greater than the preset threshold, the optimal candidate state is written into the current state index set and the gradient direction estimation process is repeated. After the optimization is terminated, the path consistency verification process is performed on the final state index set to generate the inference result. The inference result is written into the retrieval result storage area to form the final output.

[0055] In this embodiment, the gradient direction estimation process specifically includes: Read the current state index set and determine the effective index positions to participate in optimization based on the soft alignment matrix. Construct a neighborhood state set for each effective index position in the current state index set. The neighborhood state set is generated by performing a single-step incremental or single-step decrement adjustment on the current index position, with the increment and decrement magnitude being a preset step size.

[0056] After constructing the neighborhood state set, joint energy recalculation is performed on each candidate state in the neighborhood state set to obtain the corresponding neighborhood energy value. The neighborhood energy value is compared with the joint energy value of the current state, and the energy difference in each neighborhood direction is calculated. A negative energy difference indicates a decreasing energy direction, and a positive energy difference indicates an increasing energy direction.

[0057] Then, the direction with the largest energy decrease is selected from all neighborhood directions, and the change in that direction is recorded as the energy change direction value. If the energy difference in all neighborhood directions is non-negative, the current state is determined to be in a locally stable state, and the direction is no longer updated.

[0058] In this embodiment, performing energy recalculation specifically includes: reading the candidate update state set and selecting candidate states one by one. For the current candidate state, updating the relevant element values ​​in the soft alignment matrix according to the index position corresponding to the state to generate a candidate soft alignment matrix.

[0059] After the candidate soft alignment matrix is ​​generated, the path energy values ​​and manifold structure constraint values ​​are recalculated based on the path probability matrix and the manifold embedding vector set. Specifically, the candidate soft alignment matrix and the geodesic distance matrix are multiplied element-wise to generate a new path weighting matrix, and the path weighting matrix is ​​accumulated to generate the path cumulative cost value; at the same time, the manifold smoothing constraint values ​​are recalculated based on the updated neighborhood index relationship.

[0060] After obtaining the cumulative path cost and manifold smoothing constraint values, a combination process is performed according to the predetermined weighting order in the joint energy construction module to generate the joint energy value corresponding to the current candidate state. This joint energy value is then written to the corresponding position in the candidate energy set. The above steps are repeated for all candidate states in the candidate updated state set to ultimately form a complete candidate energy set for subsequent optimal candidate state selection.

[0061] In this embodiment, the structure index generation module specifically includes: Read the inference results and the soft alignment matrix, and extract the path index sequence retained in the inference results; perform sequential traversal processing on the path index sequence to generate a set of node connection pairs; The set of node connections is deduplicated to generate a set of unique connections; the set of unique connections is then partitioned into hierarchical numbers to generate a set of hierarchical numbers, which are sorted and calculated according to path depth and the number of neighboring connections. A similarity threshold comparison is performed on the manifold embedding vector set to generate a set of similar node groups; a cross-mapping process is performed on the hierarchical number set and the set of similar node groups to generate a set of structural nodes; and a hash encoding process is performed on the set of structural nodes to generate a set of structural index key values. Perform inverted index mapping on the structure index key-value set to generate a structure inverted index table; perform consecutive numbering on the structure inverted index table to generate a structure index sequence; write the structure index sequence into the index storage area to form the structure index result.

[0062] In this embodiment, the similarity threshold comparison process specifically includes: Each manifold embedding vector is read one by one from the manifold embedding vector set, and its corresponding vector number is obtained. Then, the row of data corresponding to the current vector number in the geodesic distance matrix is ​​read. This row of data contains the geodesic distance values ​​between the current vector and all other vectors.

[0063] After acquiring the data for this row, a threshold comparison is performed on all geodesic distance values. Vectors with geodesic distance values ​​less than or equal to a preset similarity threshold are recorded as the set of similar nodes for the current vector. The similarity threshold is determined by the overall distribution range of the geodesic distance matrix, typically taking the range below the mean distance value as the similarity judgment interval.

[0064] To avoid duplicate grouping, the system performs a merging process after generating similar node sets. When the similar node sets of two vectors intersect, the corresponding sets are merged into the same group. This process is repeated for all vectors in the manifold embedding vector set, ultimately forming several similar node group sets. Each group set contains a set of node numbers whose semantic distance is within a threshold range.

[0065] In this embodiment, performing cross-mapping processing specifically includes: Read the set of hierarchical numbers, which contains the number of each node and its corresponding hierarchical number value. Then read the set of similar node groups, which contains a group of nodes.

[0066] The process iterates through the set of similar node groups, selecting all node numbers in the current group. For each node number in the current group, the corresponding level number value is found in the level number set, and the node number and level number value are paired to generate a node-level correspondence record. After pairing all nodes in the current group, node records with the same level number are aggregated to form a subset within the level.

[0067] Each subset within a level is combined with its corresponding similar group identifier to generate a structure node record. The structure node record contains three types of data: group identifier, level number, and set of node numbers. This process is repeated for all similar node groups to ultimately form a complete set of structure nodes.

[0068] In this embodiment, performing the inverted index mapping specifically includes: Iterate through the set of structure index key values, reading the file identifier corresponding to each structure index key. Then, check the structure inverted index table to see if an index entry corresponding to that structure index key already exists. If it does not exist, create a new index entry and write the file identifier under that entry; if it already exists, append the current file identifier to the file identifier list of that entry.

[0069] Example 1: To verify the feasibility of this invention in practice, it was applied to a large-scale knowledge management environment that has accumulated various forms of documents, including text reports, structured tabular data, image scans, and speech-to-text records. Due to the complexity of document sources and significant format differences, traditional retrieval methods often rely solely on keyword matching or simple vector similarity calculations when dealing with cross-modal content. This results in inaccurate expression of semantic hierarchy relationships, biased retrieval results, and insufficient structural consistency. Particularly, traditional systems struggle to identify internal semantic nesting relationships among multiple documents with similar content but different hierarchical structures, leading to poor retrieval ranking stability. In this embodiment, the data access module first performs unified processing on different types of documents. Text data is segmented and lexicalized to generate text feature vectors; tabular data has its field and numerical features extracted and encoded; image data is converted into text vectors through character recognition; and speech-to-text content is organized chronologically and encoded into speech-text feature vectors. After unified encoding, multimodal features form a unified feature sequence, ensuring that information from different sources resides in the same representation space. Subsequently, the manifold embedding construction module performs non-Euclidean geometric mapping on the unified feature sequence. By setting curvature parameters, feature vectors are mapped to manifold space, the manifold embedding vector set is calculated, and a geodesic distance matrix is ​​further generated. Compared to Euclidean distance, geodesic distance can characterize semantic hierarchical curvature relationships, allowing for a more reasonable expression of the inclusion relationships between concepts. In the path distribution reparameterization alignment stage, statistical operations are performed on the geodesic distance matrix to generate path sparsity control values, path concentration control values, and path bandwidth control values. Based on these, a path probability distribution function is constructed, and a soft alignment matrix is ​​generated. This mechanism allows multiple paths to participate in the matching calculation, avoiding structural offsets caused by single path selection. Subsequently, the joint energy construction module combines the path energy values ​​with the manifold structure constraint values ​​to form a joint energy result. The energy optimization inference module performs iterative update processing on the joint energy, gradually converging to a stable structural alignment state, and finally generating the file alignment result. The structure index generation module maps the alignment result to a semantic structure index, and the final ranking result is output through the retrieval and ranking module. In actual testing, a test set of 12,000 mixed-type files was constructed, including 6,800 text files, 2,100 table files, 1,900 image scan files, and 1,200 speech-to-text files. 200 randomly selected retrieval questions were used for comparative experiments, comparing the results with traditional Euclidean vector retrieval methods and fixed path matching methods. Average retrieval accuracy, structural consistency score, hit rate of the top ten results, and average response time were statistically analyzed. Experimental results show that this invention has significant advantages in complex semantic level recognition.

[0070] Table 1. Comparison of Multimodal Knowledge Base Retrieval Performance

[0071] A thorough review of the data in Table 1 reveals that the method of this invention demonstrates simultaneous improvements in accuracy, structural consistency, and result usability. Regarding average retrieval accuracy, traditional European search achieves 78.6%, fixed path matching 83.4%, while this invention reaches 91.8%. Using 200 retrieval questions as a statistical benchmark, 91.8% means approximately 184 questions returned the correct target in the first round of results, an increase of about 27 questions compared to approximately 157 questions in traditional European search, and an increase of about 17 questions compared to approximately 167 questions in fixed path matching. This difference is not accidental, but rather due to the fact that manifold embedding and geodesic distance can better characterize semantic hierarchical relationships, reducing mismatches caused by superficially similar words but different semantic structures.

[0072] In terms of structural consistency score, traditional European search scored 0.62, fixed path matching scored 0.71, while this invention achieved 0.86. The score differences between the two comparison methods and this invention are 0.24 and 0.15, respectively. From a structural performance perspective, 0.62 typically corresponds to paragraph breaks or field misalignments in the results, 0.71 indicates that the overall structure is basically formed but still has local offsets, while 0.86 reflects more complete alignment relationships and clearer semantic hierarchy and field relationships. This result provides a more stable foundation for subsequent structural index generation.

[0073] Regarding the hit rate of the top ten results, this invention achieved 88.9%, traditional European search achieved 65.3%, and fixed path matching achieved 72.8%. Out of 200 questions, approximately 178 questions had their target files appear in the top ten results, an increase of about 47 compared to approximately 131 questions in traditional European search, and an increase of about 32 compared to approximately 146 questions in fixed path matching. This metric directly reflects the user's actual search experience; the improved hit rate means users can locate their target content without scouring numerous irrelevant results, significantly improving efficiency.

[0074] Regarding response time, traditional Euclidean retrieval averages 420 milliseconds, fixed path matching 468 milliseconds, while this invention achieves 395 milliseconds. This represents a reduction of approximately 25 milliseconds compared to traditional Euclidean retrieval and approximately 73 milliseconds compared to fixed path matching. It can be seen that after introducing path probability distribution and joint energy mechanisms, the system does not incur additional time overhead. Instead, bandwidth constraints and normalization constraints reduce invalid path searches, making the overall computation process more focused.

[0075] Based on the above four data points, a consistent conclusion can be drawn: while maintaining response efficiency, the present invention demonstrates a steady improvement in retrieval accuracy, structural consistency, and result usability, verifying the practical application value of the present invention in multimodal knowledge base document analysis and retrieval scenarios.

[0076] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A knowledge base document analysis and retrieval system based on big data analytics, characterized in that, include: Data access module: used to receive text files, table files, image files and speech-to-text results and generate a unified feature sequence; Manifold Embedding Builder: Used to perform non-Euclidean geometric mapping on a uniform feature sequence to generate a set of manifold embedding vectors and construct a geodesic distance matrix. The geodesic distance controls the semantic hierarchy relationship through curvature parameters. The path distribution reparameterization alignment module is used to construct a reparameterizable path probability distribution function based on the geodesic distance matrix to generate a soft alignment matrix. The path probability distribution controls the path sparsity and path concentration through learnable parameters and outputs the path energy term. Joint Energy Construction Module: Used to perform a weighted combination of manifold embedded energy terms and path energy terms to form a total energy function; Energy optimization inference module: used to perform gradient descent operation on the total energy function to generate file alignment results; The structure index generation module is used to map file alignment results into semantic structure indexes. The retrieval and ranking module is used to generate retrieval and ranking results based on the semantic structure index. The process of constructing the execution path probability distribution function of the path distribution reparameterization alignment module specifically includes: Read the geodesic distance matrix, perform full matrix statistical operations on the geodesic distance matrix to generate the mean distance value and the variance distance value; perform interval mapping operation on the variance distance value to generate path sparsity control value, and perform scaling operation on the mean distance value to generate path concentration control value; Perform diagonal bandwidth statistical operations on the geodesic distance matrix to generate path bandwidth control values; write the path sparse control values, path concentrated control values, and path bandwidth control values ​​into the path distribution parameter set in a fixed order; perform negative value mapping processing on the geodesic distance matrix to generate the path cost matrix; The path cost matrix is ​​subjected to centralized scaling to generate a logarithmic weight matrix, and the centralized scaling magnitude is determined by the centralized control value of the path; the logarithmic weight matrix is ​​superimposed with perturbation values ​​to generate a perturbation weight matrix, and the perturbation magnitude is determined by the path sparsity control value. Temperature normalization is performed on the perturbation weight matrix to generate an initial path probability matrix; the allowable index band is calculated according to the path bandwidth control value, and matrix elements exceeding the allowable index band are written with zero values ​​to generate a bandwidth-limited path probability matrix. The path probability matrix with limited bandwidth is subjected to alternating row normalization and column normalization until the error is less than a preset threshold to generate the path probability distribution function.

2. The knowledge base document analysis and retrieval system based on big data analysis according to claim 1, characterized in that, The data access module specifically includes: The file type recognition unit reads the file header identifiers of text files, table files, image files, and speech-to-text results to generate file type labels; The text parsing unit performs segmentation, syntactic segmentation, and lexicalization on the text file to generate a text feature sequence; The table parsing unit performs cell coordinate positioning, field name extraction, and numerical field normalization encoding on the table file to generate a table feature sequence. The image parsing unit performs character region localization, character recognition, and sorting of recognition results to generate an image text feature sequence; The speech transcription processing unit performs timestamp sorting and punctuation standardization on the speech transcription results to generate a speech-text feature sequence. The feature encoding unit performs unified vector encoding on text feature sequences, table feature sequences, image text feature sequences, and speech text feature sequences to generate multimodal feature vectors; The sequence alignment unit performs time index sorting and position index rearrangement on the multimodal feature vectors to generate a unified feature sequence.

3. The knowledge base document analysis and retrieval system based on big data analysis according to claim 2, characterized in that, The manifold embedding construction module specifically includes performing a non-Euclidean geometric mapping on the uniform feature sequence to generate a manifold embedding vector set, which specifically includes: Read the unified feature sequence, extract multimodal feature vectors in time index order to form a feature vector set, and assign a vector number and time index identifier to each multimodal feature vector in the feature vector set; Set the curvature parameter value k, establish a mapping table between the curvature parameter and the vector number, and write a k tag for each multimodal feature vector in the feature vector set; Perform a linear projection operation on each multimodal feature vector in the feature vector set to generate a tangent space vector. The dimension of the linear projection matrix is ​​consistent with the dimension of the multimodal feature vector. The norm operation is performed on the tangent space vector to obtain the vector norm value. The vector norm value is compared with a preset truncation threshold. If the vector norm value exceeds the truncation threshold, the vector scaling operation is performed to generate a truncated tangent space vector. The truncation threshold value is determined by k. The exponential mapping operation is performed on the truncated tangent space vector to generate a manifold embedding vector. The exponential mapping operation calculates the geodesic coordinate components through the curvature parameter k and outputs the manifold embedding vector. The manifold embedding vector is then written into the manifold embedding vector set. Perform manifold constraint verification operation on each manifold embedding vector in the manifold embedding vector set. The verification operation includes embedding vector norm calculation and manifold equation consistency calculation. If the verification fails, perform rescaling operation to generate a corrected manifold embedding vector and overwrite the original embedding vector. Perform geodesic distance calculation on any two manifold embedding vectors in the manifold embedding vector set to generate geodesic distance values. Arrange the geodesic distance values ​​in order of vector number to form a geodesic distance matrix and write it into the geodesic distance matrix storage area.

4. The knowledge base document analysis and retrieval system based on big data analysis according to claim 3, characterized in that, The specific steps of performing the geodesic distance calculation include: Select the first manifold embedding vector and the second manifold embedding vector from the set of manifold embedding vectors, and read the set of coordinate components of the first manifold embedding vector and the set of coordinate components of the second manifold embedding vector; The inner product operation is performed on the first manifold embedding vector and the second manifold embedding vector to generate the inner product value. The inner product calculation adopts the hyperbolic space inner product form, and the inner product value includes the sign weighted result of the time component and the spatial component. The curvature scaling operation is performed on the inner product value to generate a scaled inner product value. The curvature scaling operation transforms the inner product value proportionally through the curvature parameter k. Perform inverse hyperbolic cosine operation on the scaled inner product value to generate geodesic angle values. The input of the inverse hyperbolic cosine operation must meet the constraint condition of being greater than or equal to one. If the constraint condition is not met, perform a truncation correction operation on the scaled inner product value. The curvature normalization operation is performed on the geodesic angle values ​​to generate geodesic distance values. The curvature normalization operation is used to scale the geodesic angle values ​​proportionally by taking the square root of the absolute value of k. The generated geodesic distance values ​​are written into the corresponding row and column positions of the geodesic distance matrix. The row and column positions are determined by the first manifold embedding vector number and the second manifold embedding vector number. Repeat the aforementioned steps for all vector pairs in the manifold embedding vector set to generate a complete geodesic distance matrix.

5. A knowledge base document analysis and retrieval system based on big data analysis according to claim 4, characterized in that, The path distribution reparameterization alignment module performs the soft alignment matrix generation and path energy term calculation process, including: Read the path probability matrix and geodesic distance matrix corresponding to the path probability distribution function; perform element-wise multiplication on the path probability matrix and geodesic distance matrix to generate a path weighting matrix; Perform full matrix summation on the path weighting matrix to generate the cumulative path cost; perform logarithmic transformation on the path probability matrix to generate the logarithmic probability matrix; perform element-wise multiplication on the path probability matrix and the logarithmic probability matrix to generate the path entropy matrix; perform summation on the path entropy matrix to generate the path entropy value. Read the path centralized control value and path sparse control value from the path distribution parameter set; perform proportional weighting on the path cumulative cost value and path entropy value to generate the path energy value; Perform forward accumulation processing on the path probability matrix to generate a forward accumulation matrix; perform backtracking processing on the forward accumulation matrix to generate a path index sequence; Perform position writing processing on the path index sequence to generate a hard alignment matrix; perform weighted fusion processing on the path probability matrix and the hard alignment matrix to generate a soft alignment matrix.

6. The knowledge base document analysis and retrieval system based on big data analysis according to claim 5, characterized in that, The joint energy construction module specifically includes: Read the path energy values ​​and geodesic distance matrix, and read the manifold embedding vector set; perform neighborhood index statistical processing on the manifold embedding vector set to generate a local neighborhood relation set; Perform geodesic distance difference statistical processing on the local neighborhood relation set to generate manifold smoothing constraint values; perform full matrix distribution statistical processing on the geodesic distance matrix to generate distance structure dispersion values; Piecewise normalization is performed on the path energy values, manifold smoothing constraint values, and distance structure dispersion values ​​to generate a standardized set of energy components. A weighted combination process is performed on the standardized energy component set in a fixed order to generate a joint energy value; a stability interval verification process is performed on the joint energy value, and a proportional compression process is performed when the joint energy value exceeds the preset interval; the joint energy value is written into the energy storage area to form the joint energy result. The joint energy results are smoothed using a historical sliding window to generate the final joint energy output.

7. A knowledge base document analysis and retrieval system based on big data analysis according to claim 6, characterized in that, The energy optimization inference module specifically includes: Read the joint energy result and the soft alignment matrix, and construct the current state index set; perform gradient direction estimation on the joint energy result to generate energy change direction values. The gradient direction estimation process uses a finite difference method to calculate the energy change amplitude point by point in the neighborhood index set. A candidate update state set is generated based on the energy change direction and numerical value; energy recalculation is performed on the candidate update state set to generate a candidate energy set; The minimum value selection process is performed on the candidate energy set to generate the optimal candidate state; the convergence determination process is performed on the optimal candidate state, and the convergence determination process calculates the difference between the current energy and the previous energy. When the difference is less than a preset threshold, the optimization is terminated. When the difference is greater than the preset threshold, the optimal candidate state is written into the current state index set and the gradient direction estimation process is repeated. After the optimization is terminated, the path consistency verification process is performed on the final state index set to generate the inference result. The inference result is written into the retrieval result storage area to form the final output.

8. A knowledge base document analysis and retrieval system based on big data analysis according to claim 7, characterized in that, The structure index generation module specifically includes: Read the inference results and the soft alignment matrix, and extract the path index sequence retained in the inference results; perform sequential traversal processing on the path index sequence to generate a set of node connection pairs; The set of node connections is deduplicated to generate a set of unique connections; the set of unique connections is then partitioned into hierarchical numbers to generate a set of hierarchical numbers, which are sorted and calculated according to path depth and the number of neighboring connections. A similarity threshold comparison is performed on the manifold embedding vector set to generate a set of similar node groups; a cross-mapping process is performed on the hierarchical number set and the set of similar node groups to generate a set of structural nodes; and a hash encoding process is performed on the set of structural nodes to generate a set of structural index key values. Perform inverted index mapping on the structure index key-value set to generate a structure inverted index table; perform consecutive numbering on the structure inverted index table to generate a structure index sequence; write the structure index sequence into the index storage area to form the structure index result.

Citation Information

Patent Citations

  • Multi-modal data alignment method and system based on iterative Riemannian manifold, terminal and storage medium

    CN120763861A

  • Intelligent hospital guide system and method based on multi-modal retrieval, electronic equipment and program product

    CN121331401A