Keyword-based file information identification method and system

Abstract

The invention provides a keyword-based file information identification method and system, and the method comprises the steps: obtaining existing files, building a file database based on the existing files, carrying out the clustering of the existing files in the file database according to the obtained preset industry term data, and obtaining a first clustering result of keywords; the method comprises the following steps: acquiring a correlation score of an existing file and a query keyword based on a BM25 model, performing optimization clustering on a first clustering result of the keyword based on the correlation score, acquiring a second clustering result of the keyword, constructing a keyword semantic association magnetic field according to the second clustering result of the keyword in combination with an electromagnetic field simulation mode, and acquiring historical query data of a user. A personalized dynamic weight is established based on historical query data of the user, a third clustering result of the keywords is obtained according to the personalized dynamic weight and the keyword semantic association magnetic field, and a corresponding existing file is recognized and matched in the file database for the user based on the third clustering result of the keywords. Through the method, the accuracy of querying the matched document file by the user can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of document file recognition. Specifically, it relates to a method and system for identifying file information based on keywords. Background Art

[0002] With the rapid development of information technology, in the fields of engineering design, enterprise knowledge bases, etc., how to quickly and accurately extract target information from a large number of document files has become a core problem to be solved urgently. Traditional methods mainly rely on exact string matching or simple regular expressions in information retrieval and text processing. Its core logic is to locate the target content through exact character-level consistency or fixed patterns. However, this mechanical matching mechanism has defects when facing the complexity of natural language.

[0003] On the one hand, the problem of synonym replacement is difficult to solve. For example, when a user searches for "bicycle", but the document uses synonyms such as "bike" or "pedal bike", traditional methods, due to the lack of semantic understanding ability, cannot recognize the equivalence of these words, resulting in the omission of relevant documents. Moreover, the problem of polysemy highlights its limitations. For example, "apple" can refer to both a fruit and a technology company. A simple character-based matching search may return a large number of irrelevant results and cannot combine the context to judge the user's true intention.

[0004] On the other hand, traditional methods for identifying file information only superficially understand long texts, resulting in the lack of context relevance of files. For example, the word "bank" has completely different meanings in "deposit interest rate" and "the bank on the riverbank". These defects directly or indirectly lead to low coverage and poor accuracy of search results. Especially in professional fields, the phenomenon of synonymous expressions and polysemy of terms is common. Traditional methods may ignore key information due to rigid matching rules and even lead to wrong decisions.

[0005] Therefore, in summary, traditional methods for identifying file information have poor accuracy when querying and matching document files. Summary of the Invention

[0006] In view of the problems mentioned above, in combination with the first aspect of the present invention, embodiments of the present invention provide a method for identifying file information based on keywords. The method includes: Obtain existing files and the user's query keywords, establish a file database based on the existing files, obtain preset industry term data, and establish an industry term database. Cluster the existing files in the file database based on the preset industry term data to obtain the first clustering result of keywords; Build a BM25 model, structure the first clustering result of keywords and import it into the BM25 model. Obtain key index information based on the BM25 model, and dynamically correct the saturation parameter and length adjustment parameter in the BM25 model according to the key index information. The saturation parameter is denoted as k1, and the length adjustment parameter is denoted as b. Obtain the relevance score between the existing documents and the query keywords based on the BM25 model; Perform optimized clustering based on the relevance score to obtain the second clustering result of keywords; Establish a keyword semantic association magnetic field based on the second clustering result of keywords and electromagnetic field simulation; Obtain the user's historical query data, establish a personalized dynamic weight based on the user's historical query data, and obtain the third clustering result of keywords according to the personalized dynamic weight and the keyword semantic association magnetic field; Identify and match the corresponding existing documents in the document database for the user based on the third clustering result of keywords.

[0007] On the other hand, the embodiment of the present invention also provides a file information recognition system based on keywords, including: An acquisition module for acquiring preset industry terms, existing documents and the user's query keywords; A database module for establishing a document database and an industry term database; A clustering module that can cluster the existing documents in the document database based on the preset industry terms and obtain the first clustering result of keywords, can perform optimized clustering based on the relevance score and obtain the second clustering result, and can obtain the third clustering result of keywords according to the personalized dynamic weight and the keyword semantic association magnetic field; A construction module for building a BM25 model and dynamically correcting the saturation parameter and length adjustment parameter in the BM25 model based on the key index information; A calculation module for obtaining the relevance score between the existing documents and the query keywords; A matching module for matching the corresponding existing documents.

[0008] Compared with the prior art, the present invention has the following beneficial effects: This method can perform hierarchical clustering on the existing files stored in the file database based on an industry term database, ensuring a strong association between the first clustering result of keywords and industry knowledge, reducing the interference of common vocabulary, and adapting to the characteristics of different document sets by dynamically correcting the saturation parameter k1 and the length adjustment parameter b in the BM25 model, solving the problem of scoring deviation often caused by fixed parameters in the traditional BM25 model. Moreover, it can perform secondary clustering based on the correlation scores output by the BM25 model to optimize the homogeneity of the files within the cluster, thus solving the problem of insufficient semantic depth understanding, and introducing electromagnetic field simulation and user historical data to fuse semantic associations and personalized preferences to achieve multi-dimensional semantic enhancement, thereby solving the problem of difficult adaptation to the dynamic context of different files. In summary, this file information recognition method improves the accuracy of users in querying and matching document files. Description of the Drawings

[0009] Figure 1 It is a flowchart of the steps of a keyword-based file information recognition method provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of a keyword-based file information recognition system provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed Embodiments

[0010] The present invention will be specifically described below with reference to the accompanying drawings of the specification. Figure 1 It is a schematic diagram of the execution process of a keyword-based file information recognition method provided by an embodiment of the present invention. The following will introduce this keyword-based file information recognition method in detail.

[0011] Step S1: Obtain the existing files and the user's query keywords, establish a file database based on the existing files, obtain the preset industry term data, establish an industry term database, and perform clustering on the existing files in the file database based on the preset industry term data to obtain the first clustering result of keywords.

[0012] In this embodiment, step S1 includes: Step S11: Extract core terms from industry standard documents, patent databases, and academic journal abstracts to establish an original term pool.

[0013] It is understandable that core terms are extracted from industry standard documents, patent databases, and academic journal abstracts. For professional documents such as laws and patents, named entity recognition is used to extract structured fields such as clause numbers, claim items, and embodiments, and a metadata index is established. For academic papers, abstracts, keywords, references, and chart titles are parsed to construct an associated graph of knowledge units. For image files, the CLIP model is used to generate text-image description vectors, which are stored in association with the text content. For table files, the table structure is recognized through the Tabulizer library and converted into a triple of "parameter name - value - unit" for storage.

[0014] Step S12: Use OpenNMT to translate and perform consistency checks on the data in the original term pool, and establish an industry term database.

[0015] Step S13: Based on the industry term database, perform hierarchical clustering on the existing files in the file database to obtain the first clustering result of keywords.

[0016] In this embodiment, step S13 includes: Step S131: Based on the industry term database, construct a multi-level regular expression. According to the multi-level regular expression, obtain the positions and frequencies of industry terms in the existing files, and mark the positions of industry terms in the existing files.

[0017] Specifically, classify the data in the industry term database. Classify the unique and irreplaceable vocabulary in the industry as core terms, such as "blockchain" and "smart contract". Classify the phrases composed of core terms and common words as combined terms, such as "distributed ledger technology". Classify the words related to the industry but with multiple meanings as marginal terms. For example, "node" has different meanings in the IT field and the biological field.

[0018] Furthermore, define the matching order according to the term classification. First, match the core terms, and then process the combined and marginal terms to avoid long phrases being truncated by short words. Use multi-threading to parallel process the existing files in the file database. For each file, record the start position, end position, and context fragment where the industry term appears, and count the global term frequency to generate a heat map of word frequency distribution, and identify high-frequency core terms and low-frequency marginal terms.

[0019] Step S132: Calculate TF-IDF based on the frequencies of industry terms in the existing files, and assign a semantic weight of 2 - 3 times to the industry terms in the existing files and a semantic weight of 0.5 times to the common vocabulary in the existing files to obtain the semantic weights of industry terms and common vocabulary in the existing files.

[0020] Step S133: Based on the semantic weights of industry terms and common vocabulary in the existing files, perform hierarchical clustering on the existing files in the file database to obtain the first clustering result of keywords.

[0021] Specifically, the hierarchical clustering algorithm is used to divide the data in the file database into 50 - 100 super - clusters. Each super - cluster focuses on a macro - field, such as "biomedicine", "new energy materials", "legal documents". The OPTICS algorithm is executed within each super - cluster to generate sub - clusters according to the term distribution density. For example, "biomedicine" is split into "gene **" and "drug delivery". The existing files across super - clusters are re - assigned. For example, if the term coverage rate of a file in sub - cluster A is 70% and in sub - cluster B is 30%, then the file is assigned to sub - cluster A. Calculate the TF - IDF mean, document coverage rate, and cross - cluster discrimination degree of the terms within the cluster, and select the top three terms within the cluster as the cluster label of this cluster. The cluster label includes a super - cluster label and a sub - cluster label. The super - cluster label is used to describe the macro - field, such as "advanced manufacturing", "new materials", and the sub - cluster label is used to refine the direction, such as "industrial robot - path planning - force sensing", "quantum computing - superconducting circuit - error correction coding".

[0022] Step S2: Establish a BM25 model, structure the first clustering result of keywords and import it into the BM25 model. Obtain key index information based on the BM25 model, and dynamically correct the saturation parameter and length adjustment parameter in the BM25 model. The saturation parameter is denoted as k1, and the length adjustment parameter is denoted as b. Obtain the correlation score between the existing file and the query keyword based on the BM25 model.

[0023] In this embodiment, step S2 includes: Step S21: Structure the first clustering result of keywords to generate a cluster ID - keyword list mapping table.

[0024] Specifically, assign a cluster ID to each cluster in the first clustering result of keywords, organize the keywords of each cluster into a keyword list, and at the same time bind the cluster ID of each cluster with the keyword list to generate a [cluster ID - keyword list] mapping table. For example, cluster C1 contains (blockchain, smart contract, consensus mechanism), and cluster C2 contains (neural network, convolutional layer, backpropagation).

[0025] It can be understood that the [cluster ID - keyword list] mapping table groups semantically similar keywords by the cluster ID, facilitating the BM25 model to understand the co - occurrence relationship of the keywords within the cluster, and providing a basis for dynamically adjusting the saturation parameter and length adjustment parameter of BM25. The saturation parameter is denoted as k1, and the length adjustment parameter is denoted as b.

[0026] Step S22: Establish a standard inverted index based on the cluster ID-keyword list mapping table, and integrate the first clustering result of keywords into the standard inverted index by adding the dimensional information of the clusters in the first clustering result to the standard inverted index.

[0027] Specifically, each index record contains a term, the cluster ID it belongs to, a list of document IDs and their corresponding term frequencies, the document length, and global statistical information. A multi-cluster association pointer is established for cross-cluster terms. For example, if the keyword "encryption" belongs to both the cryptography cluster and the blockchain cluster, a multi-cluster association pointer is established for the keyword "encryption". The BM25 model can quickly locate relevant keywords and their document sets through the cluster IDs in the standard inverted index, reducing the overhead of full traversal, and providing a basis for dynamically adjusting the saturation parameter and length adjustment parameter of BM25 according to the term frequency, document length, and global statistical information.

[0028] Step S23: Statistically analyze the key metric information for each cluster in the standard inverted index, and dynamically correct k1 and b according to the key metric information to obtain the corrected saturation parameter and the corrected document length parameter.

[0029] Specifically, the key metric information includes: The average term frequency within the cluster, which is used to calculate the average number of occurrences of all terms in the cluster in the existing documents; the variance of the term frequency within the cluster, which is used to measure the concentration degree of the term distribution in the existing documents; the overlap degree between clusters, which is used to count the penetration rate of terms in other clusters in the existing documents of this cluster; the proportion degree of terms, which is used to count the proportion of terms of the cluster in the existing documents.

[0030] Furthermore, the k1 value is dynamically corrected according to the key metric information, and different strategies are adopted for different types of clusters. For high-frequency and low-variance clusters, the k1 value is reduced to suppress the excessive influence of high-frequency words. For example, repeated technical terms in patent documents. For low-frequency and high-variance clusters, the k1 value is increased to strengthen the discrimination of low-frequency feature words. For example, the naming of new algorithms in scientific research papers. For highly overlapping clusters, a penalty coefficient is applied to the overlapping terms, such as k1×0.7, to avoid cross-cluster semantic interference.

[0031] Specifically, obtain the average term frequency within the cluster and the variance of the term frequency within the cluster, and dynamically correct k1 according to the k1 dynamic modification formula to obtain the corrected value of k1 The k1 dynamic modification formula can be expressed as: ; Where represents the default k1 value of 0.75, represents the average term frequency adjustment weight, represents the term frequency variance adjustment weight, Denoted as the global average word frequency, Denoted as the global word frequency variance.

[0032] Specifically, obtain the file length of each existing file , the average length within the cluster and the variance of the length within the cluster , modify the b value dynamically based on the cluster-level adaptive rule and the b value dynamic modification formula. The cluster-level adaptive rule means that for long document clusters, if the document information redundancy is high, the b value needs to be increased to strengthen the length penalty; if the information is dense, the b value is decreased. For short documents, the b value is decreased to avoid over-punishment. The b value dynamic modification formula can be expressed as: ; Among them, Denoted as the modified b value, Denoted as the global reference b value, with a default value of 0.75, Denoted as the average length adjustment weight, Denoted as the variance adjustment weight, Denoted as the global average document length, Denoted as the global length variance.

[0033] Furthermore, based on what is obtained in step S23 and obtain the relevance score according to the document-keyword relevance scoring formula. The document-keyword relevance scoring formula can be expressed as: ; Among them, Denoted as the query keyword, Denoted as the frequency of the term t appearing in the document d, Denoted as the inverse document frequency, Denoted as the global average document length, Denoted as the current document length.

[0034] Step S3, perform optimized clustering based on the relevance score to obtain the second clustering result of the keywords.

[0035] In this embodiment, step S3 includes: Step S31, construct a three-dimensional scoring tensor based on the relevance score of the existing file and the query keyword obtained by the BM25 model, and at the same time calculate the similarity matrix of the first clustering result of the keywords.

[0036] Specifically, the three-dimensional scoring tensor includes: Query dimension, which records the score distribution of each query keyword in different existing documents by extracting the mean, variance, and kurtosis of the query keyword scores; Document dimension, which summarizes the response of a single existing document to multiple query keywords by calculating the number of keyword clusters covered by the document and the score proportion of the highest-scoring cluster; Time dimension, which analyzes the scoring change rate of the same existing document at different time periods for the file database and identifies emerging hotspots based on the scoring change rate.

[0037] Step S32: Based on the similarity matrix, retain the semantic association basis of the first clustering result of the keywords. Based on the three-dimensional scoring tensor, perform hierarchical refinement and dynamic boundary adjustment on the first clustering result of the keywords to obtain the second clustering result of the keywords.

[0038] Specifically, generate a correlation matrix based on the correlation scores of the first clustering result of the keywords, perform agglomerative hierarchical clustering on the correlation matrix, preferentially merge clusters with higher correlations to generate a tree-shaped knowledge graph of keywords, and perform dynamic cutting on the tree-shaped knowledge graph based on the correlation scores.

[0039] Furthermore, screen the keywords in each cluster within the tree-shaped knowledge graph, remove the outliers whose correlation with the central keyword of the cluster is lower than the preset threshold, and merge multiple clusters with an average inter-cluster correlation higher than the preset threshold into one cluster. The preset threshold range is expressed as [0.3, 0.7]. For example, cluster A1 contains four keywords: "Artificial Intelligence", "Deep Learning", "Neural Network", and "Big Data", with the central keyword being "Artificial Intelligence". Among them, the correlation between "Big Data" and "Artificial Intelligence" is 0.62, so the keyword "Big Data" is retained. The correlation between "Data Mining" and "Artificial Intelligence" is 0.25, so the keyword "Data Mining" is removed from the cluster.

[0040] Generate the second clustering result of the keywords based on the dynamically cut tree-shaped knowledge graph. For example, cluster A involves the fields of technology and educational technology, and its intra-cluster keywords are [Artificial Intelligence, Deep Learning, Online Learning, Educational Technology]. Cluster B1 involves the vaccine development module in the medical field, and its intra-cluster keywords are [Vaccine R & D, Gene**, Clinical Trials]. Cluster C1 involves the teaching** module in the educational field, and its intra-cluster keywords are [Teaching**, Curriculum Design, Educational**].

[0041] Step S4: Establish a keyword semantic association magnetic field based on the second clustering result of the keywords and electromagnetic field simulation.

[0042] In this embodiment, step S4 includes: Step S41: Treat each keyword in the second clustering result of the keywords as a point charge with a specific electric charge, and the magnitude of the electric charge is determined by the semantic importance of the term in the field.

[0043] Specifically, the amount of electric charge distributed to the basic charge within the magnetic field of semantic association of keywords is calculated by the inverse document frequency. The amount of electric charge of common words is approximated to zero, and the amount of electric charge of scarce industry terms is set relatively higher than that of general industry terms. Dynamic polarization is used to endow the terms frequently clicked in the user's query session with induced charges, so as to form a local enhanced field strength area around the term. For example, if the user repeatedly browses documents related to "artificial intelligence", a strong field area that attracts "deep learning" and "neural network" will be formed around it. At the same time, a charge conservation mechanism is established to keep the total global charge constant through normalization constraints, avoiding individual hot words monopolizing the semantic space.

[0044] Step S42, according to the theory that there is an interaction force between charges in Coulomb's law, the interaction force between charges is corresponded to the semantic association strength between terms.

[0045] Specifically, according to the theory that there is an interaction force between charges in Coulomb's law, the interaction force between charges is corresponded to the semantic association strength between terms. The interaction force between charges can be divided into attractive force and repulsive force. The attractive force is used to represent the high co-occurrence probability between terms. The high co-occurrence probability indicates that there is a semantic association between terms, such as "machine learning" and "neural network", there is a functional dependence between terms, such as "mobile phone" and "battery", there is an event-driven relationship between terms, such as "football" and "shoot", there is a theme consistency between terms, such as "vaccine" and "virus", the same term points to different entities in different contexts, such as the ambiguous reference of "encryption" in different contexts, and the meaning of the same term is opposite in different fields, such as "virus" refers to the pathogen that infects organisms in the biological field and refers to the malicious code that destroys computer programs in the computer field.

[0046] It can be understood that the density of magnetic field lines and the Lorentz force in the electromagnetic field can be mapped to the optimal path navigation for multi-keyword joint queries. For example, when searching for "blockchain scalability", it guides the associated evolution chain of "sharding technology" - "state channel" - "zero-knowledge proof".

[0047] Specifically, by combining the electromagnetic field theory with multi-keyword query navigation, a semantic path optimization model is constructed. The density of magnetic field lines is used to map the semantic association strength between keywords. By simulating the distribution of magnetic dipoles, a high-density magnetic field area of core words is established to form a preferred navigation path, while edge words correspond to low-density areas, automatically weakening interference. The Lorentz force drive is introduced, regarding the user query as the motion of charged particles. The electric field forms a traction force from the similarity gradient of the query intention, and the magnetic field provides direction constraints through the association network generated by historical behavior data. The combined force generated by the cooperation of the two guides the query path to converge towards the optimal keyword semantic association magnetic field area. When performing a multi-keyword joint query, the path direction is dynamically adjusted based on the combined force of the query keywords related to the keyword semantic association magnetic field to ensure path efficiency.

[0048] Furthermore, a semantic magnetic moment vector is assigned to each term, and its direction is determined by its hierarchical position in the tree-shaped knowledge graph. For example, in the tree-shaped graph, the upper-level term (such as "means of transportation") is at the parent node layer, and the direction of its semantic magnetic moment is vertically downward. This direction simulates the polarization characteristics of the magnetic moment in the magnetic field, causing the upper-level term to exert a semantic attraction on the lower-level term. The lower-level term (such as "subway"), as a child node, has its magnetic moment direction pointing to the layer where the upper-level term is located, forming a traceability path.

[0049] When multiple terms co-occur, their magnetic moment vectors will perform vector superposition in the semantic space. For example, when querying "subway + new energy", the vertical component generated by the upward traceability of "subway" is fused with the horizontal component extended parallelly by "new energy", generating a synthetic magnetic field direction tilted at 45 degrees, guiding the system to extend towards cross-concepts such as "clean energy rail transit". This vector operation breaks through the path limitations of the traditional tree structure and realizes cross-subtree semantic transition.

[0050] Based on the concept of virtual particles in quantum field theory, the interaction of each term is regarded as the conduction of semantic influence through the exchange of virtual photons. The momentum carried by the photon is positively correlated with the semantic association strength, and the momentum direction is determined by the context dependence of the term. For example, the photon transmitted by "convolution" in computer vision points to "feature extraction", while in the field of mathematics, it points to "integral transform".

[0051] Step S43, establish a semantic gauge field based on Yang-Mills theory.

[0052] Specifically, different industry fields are first defined as independent gauge symmetry groups, and each group corresponds to a set of transformation rules for industry terms. Then, gauge bosons are established as semantic transmission media to perform normalization transformation of query words in the semantic gauge fields of different industries. For example, the semantic calibration of "option" in the financial field and "option clause" in the legal field.

[0053] Furthermore, set Dirichlet boundaries for the established keyword semantic association field, fix the positions of domain authoritative terms as the anchor points of the semantic field. For example, use the IEEE standard glossary as the anchor point of the semantic field in the electrical domain. At the same time, set periodic boundaries. For overly long tail words, such as compound words in patents, enable the mirror reflection mechanism to ensure that they are within the scope of the main semantic field. And set a field distortion correction mechanism for the keyword semantic association field. Detect abnormal repulsion phenomena through the manifold correction algorithm. For example, "data security" and "privacy protection" should not show strong repulsion. Add a Laplace smoothing term to eliminate the force line distortion that violates industry common sense.

[0054] Step S5: Obtain the user's historical query data, establish a personalized dynamic weight based on the user's historical query data, and obtain the third clustering result of keywords according to the personalized dynamic weight and the keyword semantic association magnetic field.

[0055] Step S51: Obtain the user's historical query data, calculate the user's query concentration, exploration tendency parameter, and emerging term query frequency within a fixed time period, construct a recent behavior vector based on the user's query concentration, exploration tendency parameter, and emerging term query frequency, and weight the recent behavior vector according to the time decay function to generate a dynamic interest vector.

[0056] Specifically, the user's historical query data includes query terms, clicked documents, dwell time, and timestamp information. Statistically calculate the query frequency, click times, and average dwell time of each keyword. For example, the user's query times for "deep learning" are 50 times, the number of clicked relevant documents is 20 times, and the average dwell time is 120 seconds.

[0057] It can be understood that based on the topic convergence of the user's historical query data, use a sliding window to analyze the topic distribution of continuously queried keywords, extract the hourly-level topic vector through the LDA model, and calculate the average cosine similarity of the vectors in each time period within the window to obtain the user's query concentration. For example, if the user continuously searches for terms in the same field such as "neural network training" and "gradient descent optimization", and the topic vector similarity between the search terms > 0.85, then set its concentration value to approach 1. If the user alternately searches for cross-domain words such as "quantum computing" and "baking skills", and the topic vector similarity between the search terms < 0.3, then set its concentration value to 0.2.

[0058] It is understandable that exploration tendency parameters are obtained based on exploration in horizontal and vertical dimensions. Among them, the horizontal dimension statistically analyzes the dispersion of query terms in the knowledge graph and calculates the average shortest path length between user-accessed nodes. The vertical dimension detects the cross-level jump frequency. For example, directly jumping from "machine learning" (parent node) to "federated learning privacy protection" (grandchild node) is regarded as a three-level jump. The two are weighted and fused. When the number of cross-three-level jumps per hour is greater than 5, the exploration parameter breaks through the 0.9 threshold, and the search growth rate of terms in each field is tracked in real time to obtain the query frequency of emerging terms.

[0059] Furthermore, the above three indicators are normalized to the interval [0, 1] and then composed into a three-dimensional vector V = (focus degree, exploration degree, emerging degree). An exponential decay factor λ = 0.05 is introduced to reduce the behavior weight 24 hours ago to 30% of the initial value, and the behavior at the current moment maintains a 100% weight.

[0060] Step S52, calculate the personalized dynamic weight based on the dynamic interest vector, obtain the user's core interest words and marginal interest words according to the personalized dynamic weight, enhance the field strength of the keyword semantic association magnetic field region where the user's core interest words are located, weaken the field strength of the keyword semantic association magnetic field region where the marginal interest words are located, form a steep semantic potential well to attract the aggregation of related terms with high similarity to the core keyword, and establish a user core interest word bag.

[0061] Step S53, based on the user core interest word bag, screen relevant clusters from the second keyword clustering result as the initial clustering center for dynamic clustering to obtain the third keyword clustering result.

[0062] Specifically, based on the keyword semantic association magnetic field, calculate the gravitational value between keyword terms within the cluster. For example, the association strength between "smart contract" and "decentralization" in cluster C is 0.92. Construct the PageRank ranking of keyword terms within the cluster to identify the core node. For example, the PageRank value of "deep learning" in cluster C5 is the highest, then "deep learning" is used as the core node of cluster C5. Simulate the user's jump probability between clusters through the random walk algorithm. For example, the probability of jumping from the "Internet of Things" cluster to the "edge computing" cluster is 0.65, generate preloading suggestions, and dynamically adjust the clustering weight and optimize the hierarchical merging strategy by integrating user behavior data and the keyword semantic association magnetic field, so as to obtain the third keyword clustering result.

[0063] Step S6, identify and match the corresponding existing files for the user in the file database based on the third keyword clustering result.

[0064] Specifically, based on the third clustering results of keywords, the corresponding existing files are identified and matched for the user in the file database. For simple queries with too short query information, the associated keywords in the third cluster are used to expand the original query. For example, if the user only enters the word "blockchain", "blockchain" is expanded to "blockchain smart contract consensus mechanism" to match more files for the user. For complex queries with too long query information, core phrase extraction is performed, and only the top three phrases in the query statement are retained, and they are used as matching anchor points to match files. For example, if the user enters the long query information of "selected Java online programming courses and practical project recommendations", the core phrase extraction is performed on the query information, and "Java", "programming courses" and "practical projects" are extracted as matching anchor points.

[0065] Furthermore, personalized matching settings are performed on user queries based on user historical query information, and the personalized matching settings include: In the user query matching results, up to 3 files of the same cluster are displayed on the result page, and they are comprehensively sorted by time, authority and user preference. At the same time, 10% of the recommendation positions are reserved for displaying low-relevance but high-popularity files, and user click behavior is monitored to explore potential interests. The results of high-frequency queries (>100 times) within 24 hours are pre-calculated and stored in Redis. The historical high-scoring file index is stored in Redis by user ID hash sharding. When the user query matches the pre-calculated result, personalized matching results are obtained from the cache first, and the association reward factor is increased for the cross-modal content that users click frequently, such as preference for patents. For users of attached drawings, a 20% image-text association reward factor is added when they search for patent documents. The interval between the document release time and the user's query time is analyzed, and the files with the most recent release time are matched to the user first. Based on the latest release time of the user's query file, a timeliness gain is applied to files with earlier release times. For example, for every week that the release time of a historical political document is earlier than the release time of the latest political document, the weight of the historical political document decreases by 0.5%. For the query keyword hitting the core node of a cluster, that is, the PageRank value of the query keyword is greater than the top 10% in the cluster, all files in the cluster will receive a 15% similarity bonus.

[0066] Figure 2 A schematic diagram of a keyword-based file information recognition system provided by some embodiments of the present application and capable of realizing the concept of the present application is shown. The keyword-based file information recognition system is introduced in detail below.

[0067] A document information recognition system based on keywords, comprising: The acquisition module is used to obtain preset industry terms, existing files and user query keywords.

[0068] A database module for establishing a file database and an industry term database.

[0069] A clustering module that can cluster existing files in the file database based on preset industry terms and obtain the first clustering result of keywords, can optimize clustering based on relevance scores and obtain the second clustering result of keywords, and can obtain the third clustering result of keywords based on personalized dynamic weights and keyword semantic association magnetic fields.

[0070] A construction module for establishing a BM25 model and dynamically correcting the saturation parameter and length adjustment parameter in the BM25 model based on key index information.

[0071] A calculation module for obtaining the relevance score between existing files and query keywords.

[0072] A matching module for matching corresponding existing files.

[0073] The following explains the specific usage and functions of this embodiment: First, obtain existing files through step S1, establish a file database based on the existing files, obtain preset industry term data to establish an industry term database, cluster the existing files in the file database based on the preset industry term data, and obtain the first clustering result of keywords. Through precise guidance of industry terms for clustering, rapid keyword positioning and intelligent file classification are achieved, thereby improving the accuracy of information retrieval and management efficiency. Then, establish a BM25 model through step S2, perform structured processing on the first clustering result of keywords and import it into the BM25 model, obtain key index information based on the BM25 model, dynamically correct the saturation parameter and length adjustment parameter according to the key index information. The saturation parameter is denoted as k1, and the length adjustment parameter is denoted as b, and obtain the relevance score between the existing files and the query keywords based on the BM25 model. Subsequently, based on step S3, optimize clustering based on the relevance score to obtain the second clustering result of keywords. By dynamically adjusting k1 and b in the BM25 model and combining the relevance score to optimize the clustering structure, the retrieval accuracy and semantic relevance are improved, and more precise keyword aggregation and file matching are achieved. Then, according to step S4, establish a keyword semantic association magnetic field based on the second clustering result of keywords and electromagnetic field simulation. Immediately following step S5, obtain the user's historical query data, establish personalized dynamic weights based on the user's historical query data, and obtain the third clustering result of keywords according to the personalized dynamic weights and keyword semantic association magnetic field. Finally, according to step S6, identify and match corresponding existing files for the user in the file database based on the third clustering result of keywords. This embodiment integrates user behavior and semantic field dynamic optimization to achieve personalized and precise matching, improving the retrieval efficiency of document files and the accuracy of user query matching.

[0074] Figure 3The figure shows a schematic diagram of an electronic device provided by some embodiments of the present application that can implement the idea of the present application. The following provides a detailed introduction to this electronic device.

[0075] An electronic device, comprising: At least one processor 10; and a memory 11 communicatively connected to the at least one processor; wherein, the memory 11 stores instructions executable by the at least one processor 10, and the instructions are executed by the at least one processor 10 so that the at least one processor 10 can execute the method proposed in Embodiment 1 of the present invention.

[0076] The following provides a specific introduction to each component of the electronic device: Among them, the processor 10 is the control center of the electronic device, which can be a single processor or a collective term for multiple processing elements. For example, the processor 10 is one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement Embodiment 1 of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0077] Among them, the processor 10 can execute various functions of the electronic device by running or executing software programs stored in the memory 11 and calling data stored in the memory 11.

[0078] The memory 11 is used to store the software program for implementing the solution of the present invention and is controlled by the processor 10 for execution. The specific implementation manner can refer to the above method embodiments and will not be elaborated here.

[0079] The memory 11 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 11 can be integrated with the processor 10 or can exist independently and be coupled to the processor through the interface circuit of the electronic device. The embodiments of the present invention do not make specific limitations thereto.

[0080] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0081] It should be understood that the term "and / or" in this text is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this text generally represents an "or" relationship between the preceding and following associated objects, but it may also represent an "and / or" relationship. The specific meaning can be understood by referring to the context before and after.

[0082] It should be understood that in the embodiments of the present invention, the magnitudes of the serial numbers of the above processes do not mean the sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0083] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A keyword-based file information recognition method, characterized in that It includes the following steps: Obtain existing documents and the user's query keywords, establish a document database based on the existing documents, obtain preset industry term data, and establish an industry term database. Cluster the existing documents in the document database based on the preset industry term data to obtain the first clustering result of the keywords; Establish a BM25 model, structurally process the first clustering result of the keywords and import it into the BM25 model. Obtain key metric information based on the BM25 model, and dynamically correct the saturation parameter and the length adjustment parameter in the BM25 model according to the key metric information. The saturation parameter is denoted as k1, and the length adjustment parameter is denoted as b. Obtain the relevance score between the existing documents and the query keywords based on the BM25 model; Perform optimized clustering based on the relevance score to obtain the second clustering result of the keywords; Establish a keyword semantic association magnetic field based on the second clustering result of the keywords and electromagnetic field simulation; Obtain the user's historical query data, establish a personalized dynamic weight based on the user's historical query data, and obtain the third clustering result of the keywords according to the personalized dynamic weight and the keyword semantic association magnetic field; Identify and match corresponding existing documents for the user in the document database based on the third clustering result of the keywords.

2. The method for identifying file information based on keywords according to claim 1, wherein The establishment of the BM25 model, structurally processing the first clustering result of the keywords and importing it into the BM25 model, obtaining key metric information based on the BM25 model, and dynamically correcting the saturation parameter and the length adjustment parameter according to the key metric information, includes: Structurally process the first clustering result of the keywords to generate a cluster ID-keyword list mapping table; Establish a standard inverted index based on the cluster ID-keyword list mapping table, and integrate the first clustering result of the keywords into the standard inverted index by adding the dimension information of the clusters in the first clustering result to the standard inverted index; Statistically analyze the key metric information for each cluster in the standard inverted index, and dynamically correct k1 and b according to the key metric information to obtain the corrected saturation parameter and the corrected document length parameter.

3. The method for identifying file information based on keywords according to claim 2, characterized in that, The key metric information includes: The average word frequency within the cluster, which is used to calculate the average number of occurrences of all terms in the existing documents within the cluster; The variance of the word frequency within the cluster, which is used to measure the concentration degree of the term distribution in the existing documents; The overlap degree between clusters, which is used to statistically analyze the penetration rate of terms in other clusters in the existing documents of this cluster; The proportion degree of terms, which is used to statistically analyze the proportion of terms of the cluster in the existing documents.

4. A keyword-based file information recognition method according to claim 1, characterized in that The establishment of the keyword semantic association magnetic field based on the second clustering result of the keywords and electromagnetic field simulation includes: Map the second clustering result of the keywords to charged field nodes, and allocate electric charges according to the semantic weights of the industry terms and general vocabulary included in the existing documents to generate the keyword semantic association magnetic field; The keyword semantic association magnetic field includes electrostatic force, magnetic moment, and Lorentz force; The electrostatic force is used to reflect the co-occurrence intensity of terms in the existing documents, the magnetic moment is used to capture the hierarchical relationship, and the Lorentz force is used to depict the evolution path of the user's query intention.

5. A method for identifying file information based on keywords according to claim 1, characterized in that The performing of optimized clustering based on the relevance score to obtain the second clustering result of the keywords includes: Construct a three-dimensional scoring tensor based on the relevance scores of existing documents and query keywords obtained from the BM25 model, and simultaneously calculate the similarity matrix of the first clustering result of the keywords; Based on the similarity matrix, retain the semantic association basis of the first clustering result of the keywords. Based on the three-dimensional scoring tensor, perform hierarchical refinement and dynamic boundary adjustment on the first clustering result of the keywords to obtain the second clustering result of the keywords.

6. The method for identifying file information based on keywords according to claim 5, characterized in that The three-dimensional scoring tensor includes: Query dimension. The query dimension records the score distribution of each query keyword in different existing documents by extracting the mean value, variance, and kurtosis of the scores of the query keywords. Document dimension. The document dimension summarizes the response of a single existing document to multiple query keywords by calculating the number of keyword clusters covered by the document and the score proportion of the highest-scoring cluster. Time dimension. The time dimension is used to analyze the scoring change rate of the same existing document at different time periods for the file database, and identify emerging hotspots based on the scoring change rate.

7. A method for identifying file information based on keywords according to claim 1, characterized in that, Obtain the user's historical query data, establish a personalized dynamic weight based on the user's historical query data, and obtain the third clustering result of the keywords according to the personalized dynamic weight and the keyword semantic association magnetic field, including: Obtain the user's historical query data, calculate the user's query focus, exploration tendency parameter, and emerging term query frequency within a fixed time period, construct a recent behavior vector based on the user's query focus, exploration tendency parameter, and emerging term query frequency, and weight the recent behavior vector according to the time decay function to generate a dynamic interest vector; Calculate the personalized dynamic weight based on the dynamic interest vector, obtain the user's core interest words and marginal interest words according to the personalized dynamic weight, enhance the field strength of the keyword semantic association magnetic field region where the user's core interest words are located, weaken the field strength of the keyword semantic association magnetic field region where the marginal interest words are located, form a steep semantic potential well to attract the aggregation of related terms with high similarity to the core keywords, and establish a user's core interest word bag; Based on the user's core interest word bag, screen relevant clusters from the second clustering result of the keywords as the initial clustering center for dynamic clustering to obtain the third clustering result of the keywords.

8. A method for identifying file information based on keywords according to claim 1, characterized in that, Obtain the preset industry term data, establish an industry term database, and cluster the existing documents in the file database based on the preset industry term data to obtain the first clustering result of the keywords, including: Extract core terms from industry standard documents, patent databases, and academic journal abstracts to establish an original term pool; Use OpenNMT to translate and perform consistency checks on the data in the original term pool to establish an industry term database; Perform hierarchical clustering on the existing documents in the file database based on the industry term database to obtain the first clustering result of the keywords.

9. The method for identifying file information based on keywords according to claim 8, wherein, The hierarchical clustering of the existing documents in the file database based on the industry term database to obtain the first clustering result of the keywords includes: Construct a multi-level regular expression based on the industry term database, obtain the positions and frequencies of industry terms in the existing documents according to the multi-level regular expression, and mark the positions of industry terms in the existing documents; Perform TF-IDF calculations based on the frequencies of industry terms in existing documents, assign semantic weights 2 to 3 times for industry terms in existing documents, assign a semantic weight of 0.5 times for general vocabulary in existing documents, and obtain the semantic weights of industry terms and general vocabulary in existing documents. Perform hierarchical clustering on the existing documents in the file database according to the semantic weights of industry terms and general vocabulary in the existing documents to obtain the first clustering result of keywords.

10. A keyword-based file information recognition system, characterized in that, Including: An acquisition module for acquiring preset industry terms, existing documents, and the user's query keywords. A database module for establishing a file database and an industry term database. A clustering module that can cluster the existing documents in the file database based on preset industry terms and obtain the first clustering result of keywords, can perform optimized clustering based on relevance scores and obtain the second clustering result of keywords, and can obtain the third clustering result of keywords based on personalized dynamic weights and the keyword semantic association magnetic field. A construction module for establishing a BM25 model and dynamically correcting the saturation parameter and length adjustment parameter in the BM25 model based on key indicator information. A calculation module for obtaining the relevance score between the existing document and the query keyword. A matching module for matching the corresponding existing document.

Images