A text recall system and method based on similarity retrieval

Abstract

The present invention discloses a text recall system and method based on similarity retrieval, S1, receiving a query text input by a user; S2, extracting a short text semantic feature vector for a short text, and extracting a long text semantic feature vector for a long text; S3, constructing an initial vector topological structure; S4, dynamically adjusting the initial vector topological structure respectively using an improved neural topological optimization algorithm in a vector library system; S5, generating a local topological feature representation of each short text and long text based on the optimized vector topological structure; S6, determining the potential associated fragments of the query request text; S7, sorting the query request text; S8, returning several query request texts with the highest similarity in the sorting results as retrieval result outputs. The present invention significantly improves the efficiency and accuracy of text recall.

Description

Technical Field

[0001] The present invention relates to the technical field of text recall, and in particular to a text recall system and method based on similarity retrieval. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing technologies, text retrieval and recall have become important technical support in the application fields of information retrieval, intelligent customer service, and intelligent question-answering systems. Through text recall technology systems, they can quickly extract content related to user query texts from massive text data and provide efficient and accurate retrieval results. However, existing technologies still face many challenges in practical applications and are difficult to meet the needs of efficient text retrieval in diverse scenarios.

[0003] In recent years, semantic matching technology has gradually replaced traditional keyword matching and become an important development direction in the field of text recall. The semantic vectorization of text is carried out through pre-training models, the query text and the document are mapped to the same vector space, and retrieval is performed based on vector similarity. This method solves the problem that keyword matching cannot capture semantic associations to a certain extent. However, the existing semantic matching technology has the following defects in practice:

[0004] Lack of topological information: Existing semantic matching technology is only based on similarity calculation in a single vector space, lacks the ability to model complex topological relationships between texts, and cannot effectively capture the correlation between text features in multi-semantic associations and dynamic scenarios.

[0005] Insufficient retrieval efficiency: In large-scale text libraries, semantic matching technology needs to calculate all documents, which requires a huge amount of calculations and the retrieval efficiency cannot meet real-time requirements. In addition, existing technologies lack adaptive capabilities in dynamic data update scenarios and cannot quickly respond to the processing needs of newly added text.

[0006] In summary, existing text recall technology has significant shortcomings in processing semantic differences between long and short texts, establishing complex topological associations between texts, and improving the efficiency of large-scale data retrieval. It is difficult to meet the application requirements of multiple scenarios, high precision and high efficiency. A new text recall method is urgently needed to solve the above problems. Summary of the invention

[0007] An object of the present invention is to provide a text recall system and method based on similarity retrieval, which significantly improves the efficiency and accuracy of text recall.

[0008] A text recall method based on similarity retrieval according to an embodiment of the present invention comprises the following steps:

[0009] S1, receiving a query text input by a user, and determining whether the query text is a short text or a long text according to the text length;

[0010] S2, extracting short text semantic feature vectors for short texts, and extracting long text semantic feature vectors for long texts;

[0011] S3, storing the short text semantic feature vector and the long text semantic feature vector in the vector library system respectively, and constructing an initial vector topology structure based on the spatial distance of the semantic feature vectors;

[0012] S4. In the vector library system, an improved neural topology optimization algorithm is used to dynamically adjust the initial vector topology structure, re-divide the adjacency relationship between the short text and long text semantic feature vectors, and optimize the spatial layout between the semantic feature vectors;

[0013] S5, generating local topological feature representations of each short text and long text based on the optimized vector topological structure;

[0014] S6, receiving a query request from a user, extracting a semantic feature vector of the query request text, and determining potential associated fragments of the query request text;

[0015] S7. In the potential associated shards, the similarity between the query request text and the semantic feature vector in the document library is calculated based on the local topological feature representation, and the query request text is sorted;

[0016] S8. Return several query request texts with the highest similarity in the sorted results as retrieval result output.

[0017] Optionally, S1 includes the following specific processes:

[0018] S11, receiving the query text Q input by the user, the query text length is ,in is the total number of characters in the query text Q;

[0019] S12, according to the preset text length threshold , determine the query text length Is it greater than the text length threshold? The text length threshold is based on the length distinction value set by the system optimization, which is used to distinguish long text from short text:

[0020] if , classify the query text Q into short text;

[0021] if , classify the query text Q into long text.

[0022] Optionally, S3 includes the following specific processes:

[0023] S31. Define a set of short text semantic feature vectors and long text semantic feature vector collection :

[0024] ;

[0025] ;

[0026] in, represents the i-th initial text input, Indicates the length of the initial text. To distinguish the length threshold of long and short text, and are the mapping functions of the short text and long text feature extraction models, respectively. m is the number of short texts, n is the number of long texts, and are the dimensions of the short text and long text feature vectors, respectively. is the set of real numbers;

[0027] S32, short text semantic feature vector collection and long text semantic feature vector collection They are stored in the vector library system respectively, forming an independent short text semantic feature vector storage area and a long text semantic feature vector storage area;

[0028] S33, short text semantic feature vector set and long text semantic feature vector collection The semantic space distance between any two vectors is calculated internally to establish the similarity basis between the initial vectors. and , and the long text semantic feature vector and , define the distance function as:

[0029] ;

[0030] ;

[0031] in, and are the components of the i-th and k-th short text feature vectors on the r-th feature dimension, and are the components of the p-th and q-th long text feature vectors on the h-th feature dimension respectively;

[0032] S34, introducing similarity threshold based on distance calculation results Construct the adjacency relationship between the short text semantic feature vector and the long text semantic feature vector to form a set of connecting edges between the short text feature vector sets The set of connecting edges between the long text feature vector set ;

[0033] S35. Constructing the initial vector topology structure of short text and long text initial vector topology .

[0034] Optionally, S4 includes the following specific processes:

[0035] S41. Initial vector topology structure for short text and long text initial vector topology Based on centrality threshold and distance threshold Layering is performed to divide the text semantic feature vector into a core layer and a peripheral layer:

[0036] ;

[0037] in, Indicates short text type or long text type, is the global centrality of the text semantic feature vector, defined as:

[0038] ;

[0039] in, Indicates whether the vector is connected. Represents the weight of the adjacency relationship, and n is the number of long texts;

[0040] S42. Based on the stratification results, the adjacency relationship is adjusted, the connection relationship within the core layer is optimized first, and the connection relationship within the peripheral layer is restricted. The adjacency update rules are defined:

[0041] ;

[0042] The internal connections of the peripheral layer are constrained, and the connection relationship between the core layer and the peripheral layer is based on the distance threshold. Sure;

[0043] S43. Introduce a dynamic weight update mechanism to independently update the adjacency relationship between the core layer and the peripheral layer, and set a higher update rate for the connection weight of the core layer:

[0044] ;

[0045] in, and are weight smoothing factors for the core layer and the peripheral layer, respectively, and , is the weight of the adjacency relationship before updating, is the weight of the updated adjacency relationship;

[0046] S45. Build an optimized short text topology structure based on the independently optimized adjacency relationship between the core layer and the peripheral layer and long text topology .

[0047] Optionally, S5 includes the following specific processes:

[0048] S51, based on the optimized short text topology and long text topology , extract the local neighborhood of each text semantic feature vector ;

[0049] S52. Calculate the local feature center of the text semantic feature vector and the local feature diffusion value :

[0050] ;

[0051] in, represents the number of neighborhood vectors, is the Euclidean norm of the vector, represents the weighted center of the semantic feature vector in the local neighborhood, Represents the degree of vector diffusion in the local neighborhood;

[0052] S53. Generate local topological feature representation of text semantic feature vector :

[0053] ;

[0054] in, Represents semantic feature vector The adjacency of:

[0055] ;

[0056] S54, for all short text semantic feature vector sets and a set of long text semantic feature vectors Batch generate corresponding local topological feature representation sets and .

[0057] Optionally, S6 includes the following specific processes:

[0058] S61, receiving a query request input by a user, parsing the query request text, and Determine whether the query request text type is short text or long text;

[0059] S62. For a short text query, a short text feature extraction model is used to extract a short text semantic feature vector of the query text; for a long text query, a long text feature extraction model is used to extract a long text semantic feature vector of the query text;

[0060] S63, selecting a corresponding optimized vector topology structure according to the type of the query request text, if it is a short text, selecting a short text optimized vector topology structure; if it is a long text, selecting a long text optimized vector topology structure;

[0061] S64. Calculate the similarity between the query text semantic feature vector and all nodes in the optimized vector topology structure, and introduce the shard center point Represents the central feature vector of each shard;

[0062] S65. Similarity based on calculation Introducing Sharding Center Point , calculate the similarity between the query text semantic feature vector and each shard center point ;

[0063] S66. Based on similarity , select the n shards with the highest similarity to form a set of potential associated shards for the query request text :

[0064] ;

[0065] in, It is the preset shard quantity threshold, which is used to limit the retrieval scope.

[0066] Optionally, the S7 includes the following specific processes:

[0067] S71. Potentially associated fragment set in the query request text Extract the document semantic feature vector set in each shard :

[0068] ;

[0069] in, Indicates short text or long text type, represents the set of feature vectors within a shard, is the set of semantic feature vectors in slice k;

[0070] S72. Calculate the semantic feature vector of the query request text and the semantic feature vector of each document in the shard The weighted similarity of local topological features ;

[0071] S73, according to the calculated weighted similarity Set of semantic feature vectors of documents in the shard Sort in descending order by similarity to get the sorting result :

[0072] ;

[0073] S74, select sort result The top m document semantic feature vectors with the highest similarity in the query text constitute the final retrieval result set ;

[0074] S75, collect the final search results Converted into document output and returned to the user end to complete dynamic text recall.

[0075] A text recall system based on similarity retrieval includes the following modules:

[0076] The query processing module is used to receive the query request input by the user, parse the query text, and determine whether the query text is a short text or a long text according to its length, and select the corresponding feature extraction process according to the different types;

[0077] A feature extraction module is used to load a corresponding short text feature extraction model or a long text feature extraction model according to the type of the query text, extract the semantic feature vector of the query text, and use it as input for subsequent processing;

[0078] The vector library management module is used to store text semantic feature vectors. It establishes an optimized topological structure of short texts and long texts, and stores semantic feature vectors in layers according to the core layer and the peripheral layer, while dynamically maintaining and updating the semantic vector library.

[0079] A shard association calculation module is used to calculate the similarity between the semantic feature vector of the query text and the shard center point in the storage vector library, and select several shards most relevant to the query text as a potential associated shard set according to the similarity value;

[0080] The similarity calculation and sorting module is used to comprehensively score and sort all documents in the potential associated shard set based on the semantic feature similarity and topological feature similarity between the query text and the document, and give priority to outputting the document that best matches the query text;

[0081] The result output module is used to select several documents with the highest similarity as the final search results according to the sorting results, convert the final search results into user-understandable content, and return them to the user end to complete the text recall;

[0082] The dynamic optimization module is used to periodically optimize the topology structure, semantic feature storage, and sharding management logic in the system to adapt to the dynamic changes in the size of the text library and optimize the recall efficiency and accuracy through adaptive strategies.

[0083] The beneficial effects of the present invention are:

[0084] (1) The present invention introduces a neural topology optimization method to dynamically adjust and hierarchically optimize the initial topology structure constructed by the text semantic feature vector. The text semantic feature vector is divided into a core layer and a peripheral layer, and the adjacency relationship within the core layer is optimized first, so that texts with higher semantic associations are more closely aggregated in the topological structure. At the same time, the connection within the peripheral layer is restricted, thereby improving the overall semantic representation ability of the topological structure. A dynamic weight adjustment mechanism is further introduced in the hierarchical optimization process, and the core layer connection weight update rate is higher, ensuring that the topological structure can quickly adapt to changes in new text data, thereby improving the efficiency and accuracy of dynamic text recall.

[0085] (2) The present invention combines the local topological features of the text, and characterizes the semantic position and association strength of the text in the topological structure by constructing the local feature center, diffusion and adjacency of the text. In the similarity calculation, the semantic feature similarity and the topological feature similarity are weightedly calculated to ensure that the deep semantic relationship between texts can be captured more accurately when processing complex text recall scenarios. Compared with the traditional single cosine similarity calculation method, the multi-dimensional similarity calculation strategy of the present invention significantly improves the accuracy of text recall, and performs better in multi-semantic association and dynamic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0087] Figure 1 The present invention provides a flowchart of a text recall system and method based on similarity retrieval. DETAILED DESCRIPTION

[0088] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0089] refer to Figure 1 , a text recall method based on similarity retrieval, comprising the following steps:

[0090] S1, receiving a query text input by a user, and determining whether the query text is a short text or a long text according to the text length;

[0091] S2, extracting short text semantic feature vectors for short texts, and extracting long text semantic feature vectors for long texts;

[0092] S3, storing the short text semantic feature vector and the long text semantic feature vector in the vector library system respectively, and constructing an initial vector topology structure based on the spatial distance of the semantic feature vectors;

[0093] S4. In the vector library system, an improved neural topology optimization algorithm is used to dynamically adjust the initial vector topology structure, re-divide the adjacency relationship between the short text and long text semantic feature vectors, and optimize the spatial layout between the semantic feature vectors;

[0094] S5, generating local topological feature representations of each short text and long text based on the optimized vector topological structure;

[0095] S6, receiving a query request from a user, extracting a semantic feature vector of the query request text, and determining potential associated fragments of the query request text;

[0096] S7. In the potential associated shards, the similarity between the query request text and the semantic feature vector in the document library is calculated based on the local topological feature representation, and the query request text is sorted;

[0097] S8. Return several query request texts with the highest similarity in the sorted results as retrieval result output.

[0098] In this implementation, S1 includes the following specific processes:

[0099] S11, receiving the query text Q input by the user, the query text length is ,in is the total number of characters in the query text Q;

[0100] S12, according to the preset text length threshold , determine the query text length Is it greater than the text length threshold? The text length threshold is based on the length distinction value set by the system optimization, which is used to distinguish long text from short text:

[0101] if , classify the query text Q into short text;

[0102] if , classify the query text Q into long text.

[0103] In this implementation, S3 includes the following specific processes:

[0104] S31. Define a set of short text semantic feature vectors and long text semantic feature vector collection :

[0105] ;

[0106] ;

[0107] in, represents the i-th initial text input, Indicates the length of the initial text. To distinguish the length threshold of long and short text, and are the mapping functions of the short text and long text feature extraction models, respectively. m is the number of short texts, n is the number of long texts, and are the dimensions of the short text and long text feature vectors, respectively. is the set of real numbers;

[0108] S32, short text semantic feature vector collection and long text semantic feature vector collection They are stored in the vector library system respectively, forming an independent short text semantic feature vector storage area and a long text semantic feature vector storage area;

[0109] S33, short text semantic feature vector set and long text semantic feature vector collection The semantic space distance between any two vectors is calculated internally to establish the similarity basis between the initial vectors. and , and the long text semantic feature vector and , define the distance function as:

[0110] ;

[0111] ;

[0112] in, and are the components of the i-th and k-th short text feature vectors on the r-th feature dimension, and are the components of the p-th and q-th long text feature vectors on the h-th feature dimension respectively;

[0113] S34, introducing similarity threshold based on distance calculation results Construct the adjacency relationship between the short text semantic feature vector and the long text semantic feature vector to form a set of connecting edges between the short text feature vector sets The set of connecting edges between the long text feature vector set ;

[0114] S35. Constructing the initial vector topology structure of short text and long text initial vector topology .

[0115] In this implementation, S4 includes the following specific processes:

[0116] S41. Initial vector topology structure for short text and long text initial vector topology Based on centrality threshold and distance threshold Layering is performed to divide the text semantic feature vector into a core layer and a peripheral layer:

[0117] ;

[0118] in, Indicates short text type or long text type, is the global centrality of the text semantic feature vector, defined as:

[0119] ;

[0120] in, Indicates whether the vector is connected. Represents the weight of the adjacency relationship, and n is the number of long texts;

[0121] S42. Based on the stratification results, the adjacency relationship is adjusted, the connection relationship within the core layer is optimized first, and the connection relationship within the peripheral layer is restricted. The adjacency update rules are defined:

[0122] ;

[0123] The internal connections of the peripheral layer are constrained, and the connection relationship between the core layer and the peripheral layer is based on the distance threshold. Sure;

[0124] S43. Introduce a dynamic weight update mechanism to independently update the adjacency relationship between the core layer and the peripheral layer, and set a higher update rate for the connection weight of the core layer:

[0125] ;

[0126] in, and are weight smoothing factors for the core layer and the peripheral layer, respectively, and , is the weight of the adjacency relationship before updating, is the weight of the updated adjacency relationship;

[0127] S45. Build an optimized short text topology structure based on the independently optimized adjacency relationship between the core layer and the peripheral layer and long text topology .

[0128] In this implementation, S5 includes the following specific processes:

[0129] S51, based on the optimized short text topology and long text topology , extract the local neighborhood of each text semantic feature vector ;

[0130] S52. Calculate the local feature center of the text semantic feature vector and the local feature diffusion value :

[0131] ;

[0132] in, represents the number of neighborhood vectors, is the Euclidean norm of the vector, represents the weighted center of the semantic feature vector in the local neighborhood, Represents the degree of vector diffusion in the local neighborhood;

[0133] S53. Generate local topological feature representation of text semantic feature vector :

[0134] ;

[0135] in, Represents semantic feature vector The adjacency of:

[0136] ;

[0137] S54, for all short text semantic feature vector sets and a set of long text semantic feature vectors Batch generate corresponding local topological feature representation sets and .

[0138] In this implementation, S6 includes the following specific processes:

[0139] S61, receiving a query request input by a user, parsing the query request text, and Determine whether the query request text type is short text or long text;

[0140] S62. For a short text query, a short text feature extraction model is used to extract a short text semantic feature vector of the query text; for a long text query, a long text feature extraction model is used to extract a long text semantic feature vector of the query text;

[0141] S63, selecting a corresponding optimized vector topology structure according to the type of the query request text, if it is a short text, selecting a short text optimized vector topology structure; if it is a long text, selecting a long text optimized vector topology structure;

[0142] S64. Calculate the similarity between the query text semantic feature vector and all nodes in the optimized vector topology structure, and introduce the shard center point Represents the central feature vector of each shard;

[0143] S65. Similarity based on calculation Introducing Sharding Center Point , calculate the similarity between the query text semantic feature vector and each shard center point ;

[0144] S66. Based on similarity , select the n shards with the highest similarity to form a set of potential associated shards for the query request text :

[0145] ;

[0146] in, It is the preset shard quantity threshold, which is used to limit the retrieval scope.

[0147] In this implementation, S7 includes the following specific processes:

[0148] S71. Potentially associated fragment set in the query request text Extract the document semantic feature vector set in each shard :

[0149] ;

[0150] in, Indicates short text or long text type, represents the set of feature vectors within a shard, is the set of semantic feature vectors in slice k;

[0151] S72. Calculate the semantic feature vector of the query request text and the semantic feature vector of each document in the shard The weighted similarity of local topological features ;

[0152] S73, according to the calculated weighted similarity Set of semantic feature vectors of documents in the shard Sort in descending order by similarity to get the sorting result :

[0153] ;

[0154] S74, select sort result The top m document semantic feature vectors with the highest similarity in the query text constitute the final retrieval result set ;

[0155] S75, collect the final search results Converted into document output and returned to the user end to complete dynamic text recall.

[0156] A text recall system based on similarity retrieval includes the following modules:

[0157] The query processing module is used to receive the query request input by the user, parse the query text, and determine whether the query text is a short text or a long text according to its length, and select the corresponding feature extraction process according to the different types;

[0158] A feature extraction module is used to load a corresponding short text feature extraction model or a long text feature extraction model according to the type of the query text, extract the semantic feature vector of the query text, and use it as input for subsequent processing;

[0159] The vector library management module is used to store text semantic feature vectors. It establishes an optimized topological structure of short texts and long texts, and stores semantic feature vectors in layers according to the core layer and the peripheral layer, while dynamically maintaining and updating the semantic vector library.

[0160] A shard association calculation module is used to calculate the similarity between the semantic feature vector of the query text and the shard center point in the storage vector library, and select several shards most relevant to the query text as a potential associated shard set according to the similarity value;

[0161] The similarity calculation and sorting module is used to comprehensively score and sort all documents in the potential associated shard set based on the semantic feature similarity and topological feature similarity between the query text and the document, and give priority to outputting the document that best matches the query text;

[0162] The result output module is used to select several documents with the highest similarity as the final search results according to the sorting results, convert the final search results into user-understandable content, and return them to the user end to complete the text recall;

[0163] The dynamic optimization module is used to periodically optimize the topology structure, semantic feature storage, and sharding management logic in the system to adapt to the dynamic changes in the size of the text library and optimize the recall efficiency and accuracy through adaptive strategies.

[0164] Example:

[0165] Example During the "Double Eleven" promotion in 2024, the intelligent customer service system of an e-commerce platform needs to process hundreds of millions of user query requests, including product promotion rules, return and exchange policies, logistics inquiries, and coupon usage content. In order to improve the efficiency of user query response, the platform applied a dynamic text recall method based on neural topology optimization and verified its effectiveness in actual operation.

[0166] At 19:30 on November 10, 2024, the customer service system monitored a surge in user inquiries related to "return rules", most of which were specific questions about the "Double Eleven Promotion Return" rules, such as "How to return buy one get one free" and "Do pre-sale products support 7-day unconditional return". At this time, 200,000 promotion-related return rule documents were added to the system knowledge base. Traditional retrieval methods produced a large number of irrelevant documents when querying "return rules", such as "general return policy" and "logistics query", which required users to screen multiple times, affecting the user experience.

[0167] The system imports the newly added 200,000 documents into the vector library in real time. Document features include:

[0168] 1. Document ID: such as "DOC20241110193501";

[0169] 2. Document type: such as "long text" or "short text";

[0170] 3. Semantic feature vector: extracted based on pre-trained model;

[0171] 4. Semantic feature centrality: calculated by neural topology optimization algorithm.

[0172] During the dynamic update process, the system automatically adjusts the topological structure of newly added documents, classifies documents related to the "Double Eleven Promotion" into the core layer and establishes a strong association adjacency relationship with existing "Promotion Activities" documents. At the same time, the system limits the connection of documents in the peripheral layer that are not related to the query, reducing interference.

[0173] At 00:03 on November 11, 2024, the user entered the query "Pre-sale Product Return Rules" through the mobile terminal. The following is the specific processing process of the system:

[0174] Text parsing: The system parses the query content entered by the user and determines it to be a short text type.

[0175] Semantic feature extraction: Load the short text feature extraction model, generate semantic feature vectors, and locate the optimized short text topology structure.

[0176] Shard selection: Calculate the similarity between the query text and the core layer shard center, and lock the two shards related to the "Double Eleven Promotion Return Rules".

[0177] In-shard sorting: In associated shards, the system sorts based on weighted similarity of local topological features. The results include "return policy for pre-sale products", "return process for pre-sale products", "special return conditions for Double Eleven", "scope of application for 7-day no-reason return", etc.

[0178] Result return: Finally, 5 highly relevant results are returned to the user end, with a response time of 220 milliseconds.

[0179] Case 1: Query "Return Rules"

[0180] The user entered the query "Return Rules" at 14:25 on November 11, 2024. According to the optimized topology structure, the system accurately returned documents related to "Double Eleven Promotion Returns". After completing the query through the intelligent customer service, the user clicked the operation step link in the "Return Process Details" document and jumped directly to the return application page. The whole operation took less than 30 seconds. Compared with the traditional method, users need to filter multiple irrelevant documents by themselves, which usually takes more than 2 minutes.

[0181] Case 2: Dynamically updating documents

[0182] At 10:10 on November 12, 2024, the customer service team added 5,000 "Pre-sale Product Special Return" documents, including "Pre-sale Product Combination Promotion Return Conditions" and "Pre-sale Product Return Restrictions". The system completed the semantic vector extraction and topological adjustment of the newly added documents within 5 seconds, divided the new content into the core layer and established an adjacency relationship. When the user queried "Pre-sale Product Return Rules" at 10:12, the system accurately returned the newly added documents, and the query response time was 210 milliseconds.

[0183] During the test period from November 11, 2024 to November 13, 2024, the platform collected 1 million user query data, and compared and analyzed the traditional keyword matching method, the existing semantic retrieval method and the method of the present invention. The results are shown in the following table:

[0184]

[0185] This embodiment verifies the effectiveness and applicability of the present invention through real-world scenario simulations, and in particular, performs significantly better than the prior art in terms of dynamically updating documents, improving retrieval efficiency, and improving user experience.

[0186] The present invention introduces a neural topology optimization method to dynamically adjust and hierarchically optimize the initial topology structure constructed by the text semantic feature vector. By dividing the text semantic feature vector into a core layer and a peripheral layer, and prioritizing the optimization of the adjacency relationship within the core layer, texts with higher semantic association are more closely aggregated in the topological structure, while limiting the connection within the peripheral layer, thereby improving the overall semantic representation ability of the topological structure. A dynamic weight adjustment mechanism is further introduced in the hierarchical optimization process, and the core layer connection weight update rate is higher, ensuring that the topological structure can quickly adapt to changes in newly added text data, thereby improving the efficiency and accuracy of dynamic text recall.

[0187] The present invention combines the local topological features of the text, and characterizes the semantic position and association strength of the text in the topological structure by constructing the local feature center, diffusion and adjacency of the text. In the similarity calculation, the semantic feature similarity and the topological feature similarity are comprehensively weighted for calculation, ensuring that the deep semantic relationship between texts can be captured more accurately when processing complex text recall scenarios. Compared with the traditional single cosine similarity calculation method, the multi-dimensional similarity calculation strategy of the present invention significantly improves the accuracy of text recall, and performs better in multi-semantic associations and dynamic scenarios.

[0188] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A text recall method based on similarity retrieval, characterized in that: The steps include: S1, receiving a query text input by a user, and determining whether the query text is a short text or a long text according to the text length; S2, extracting short text semantic feature vectors for short texts, and extracting long text semantic feature vectors for long texts; S3, storing the short text semantic feature vector and the long text semantic feature vector in the vector library system respectively, and constructing an initial vector topology structure based on the spatial distance of the semantic feature vectors; S4. In the vector library system, an improved neural topology optimization algorithm is used to dynamically adjust the initial vector topology structure, re-divide the adjacency relationship between the short text and long text semantic feature vectors, and optimize the spatial layout between the semantic feature vectors; S5, generating local topological feature representations of each short text and long text based on the optimized vector topological structure; S6, receiving a query request from a user, extracting a semantic feature vector of the query request text, and determining potential associated fragments of the query request text; S7. In the potential associated shards, the similarity between the query request text and the semantic feature vector in the document library is calculated based on the local topological feature representation, and the query request text is sorted; S8, returning several query request texts with the highest similarity in the sorting results as text recall result output; The S4 includes the following specific processes: S41. Initial vector topology structure for short text and long text initial vector topology Based on centrality threshold and distance threshold Layering is performed to divide the text semantic feature vector into a core layer and a peripheral layer: ； in, Indicates short text type or long text type, is the global centrality of the text semantic feature vector, defined as: ； in, Indicates whether the vector is connected. Represents the weight of the adjacency relationship, and n is the number of long texts; S42. Based on the stratification results, the adjacency relationship is adjusted, the connection relationship within the core layer is optimized first, and the connection relationship within the peripheral layer is restricted. The adjacency update rules are defined: ； The internal connections of the peripheral layer are constrained, and the connection relationship between the core layer and the peripheral layer is based on the distance threshold. Sure; S43. Introduce a dynamic weight update mechanism to independently update the adjacency relationship between the core layer and the peripheral layer, and set a higher update rate for the connection weight of the core layer: ； in, and are weight smoothing factors for the core layer and the peripheral layer, respectively, and , is the weight of the adjacency relationship before updating, is the weight of the updated adjacency relationship; S45. Build an optimized short text topology structure based on the independently optimized adjacency relationship between the core layer and the peripheral layer and long text topology .

2. A text recall method based on similarity retrieval according to claim 1, characterized in that: The S1 includes the following specific processes: S11, receiving the query text Q input by the user, the query text length is ,in is the total number of characters in the query text Q; S12, according to the preset text length threshold , determine the query text length Is it greater than the text length threshold? The text length threshold is based on the length distinction value set by the system optimization, which is used to distinguish long text from short text: if , classify the query text Q into short text; if , classify the query text Q into long text.

3. A text recall method based on similarity retrieval according to claim 1, characterized in that: The S3 includes the following specific processes: S31. Define a set of short text semantic feature vectors and long text semantic feature vector collection : ； ； in, represents the i-th initial text input, Indicates the length of the initial text. To distinguish the length threshold of long and short text, and are the mapping functions of the short text and long text feature extraction models, respectively. m is the number of short texts, n is the number of long texts, and are the dimensions of the short text and long text feature vectors, respectively. is the set of real numbers; S32, short text semantic feature vector collection and long text semantic feature vector collection They are stored in the vector library system respectively, forming an independent short text semantic feature vector storage area and a long text semantic feature vector storage area; S33, short text semantic feature vector set and long text semantic feature vector collection The semantic space distance between any two vectors is calculated internally to establish the similarity basis between the initial vectors. and , and the long text semantic feature vector and , define the distance function as: ； ； in, and are the components of the i-th and k-th short text feature vectors on the r-th feature dimension, and are the components of the p-th and q-th long text feature vectors on the h-th feature dimension respectively; S34, introducing similarity threshold based on distance calculation results Construct the adjacency relationship between the short text semantic feature vector and the long text semantic feature vector to form a set of connecting edges between the short text feature vector sets The set of connecting edges between the long text feature vector set ; S35. Constructing the initial vector topology structure of short text and long text initial vector topology .

4. The text recall method based on similarity retrieval according to claim 1, characterized in that: The S5 includes the following specific processes: S51, based on the optimized short text topology and long text topology , extract the local neighborhood of each text semantic feature vector ; S52. Calculate the local feature center of the text semantic feature vector and the local feature diffusion value : ； in, represents the number of neighborhood vectors, is the Euclidean norm of the vector, represents the weighted center of the semantic feature vector in the local neighborhood, Represents the degree of vector diffusion in the local neighborhood; S53. Generate local topological feature representation of text semantic feature vector : ； in, Represents semantic feature vector The adjacency of: ； S54, for all short text semantic feature vector sets and a set of long text semantic feature vectors Batch generate corresponding local topological feature representation sets and .

5. The text recall method based on similarity retrieval according to claim 1, characterized in that: The S6 includes the following specific processes: S61, receiving a query request input by a user, parsing the query request text, and Determine whether the query request text type is short text or long text; S62. For a short text query, a short text feature extraction model is used to extract a short text semantic feature vector of the query text; for a long text query, a long text feature extraction model is used to extract a long text semantic feature vector of the query text; S63, selecting a corresponding optimized vector topology structure according to the type of the query request text, if it is a short text, selecting a short text optimized vector topology structure; if it is a long text, selecting a long text optimized vector topology structure; S64. Calculate the similarity between the query text semantic feature vector and all nodes in the optimized vector topology structure, and introduce the shard center point represents the central feature vector of each shard; S65. Similarity based on calculation Introducing Sharding Center Point , calculate the similarity between the query text semantic feature vector and each shard center point ; S66. Based on similarity , select the n shards with the highest similarity to form a set of potential associated shards for the query request text : ； in, It is the preset shard quantity threshold, which is used to limit the retrieval scope.

6. A text recall method based on similarity retrieval according to claim 1, characterized in that: The S7 includes the following specific processes: S71. Potentially associated fragment set in the query request text Extract the document semantic feature vector set in each shard : ； in, Indicates short text or long text type, represents the set of feature vectors within a shard, is the set of semantic feature vectors in slice k; S72. Calculate the semantic feature vector of the query request text and the semantic feature vector of each document in the shard The weighted similarity of local topological features ; S73, according to the calculated weighted similarity Set of semantic feature vectors of documents in the shard Sort in descending order by similarity to get the sorting result : ； S74, select sort result The top m document semantic feature vectors with the highest similarity in the query text constitute the final retrieval result set ; S75, collect the final search results Converted into document output and returned to the user end to complete dynamic text recall.

7. A text recall system based on similarity retrieval, executing the text recall method based on similarity retrieval according to any one of claims 1 to 6, characterized in that: Includes the following modules: The query processing module is used to receive the query request input by the user, parse the query text, and determine whether the query text is a short text or a long text according to its length, and select the corresponding feature extraction process according to the different types; A feature extraction module is used to load a corresponding short text feature extraction model or a long text feature extraction model according to the type of the query text, extract the semantic feature vector of the query text, and use it as input for subsequent processing; The vector library management module is used to store text semantic feature vectors. It establishes an optimized topological structure of short texts and long texts, and stores semantic feature vectors in layers according to the core layer and the peripheral layer, while dynamically maintaining and updating the semantic vector library. A shard association calculation module is used to calculate the similarity between the semantic feature vector of the query text and the shard center point in the storage vector library, and select several shards most relevant to the query text as a potential associated shard set according to the similarity value; The similarity calculation and sorting module is used to comprehensively score and sort all documents in the potential associated shard set based on the semantic feature similarity and topological feature similarity between the query text and the document, and give priority to outputting the document that best matches the query text; The result output module is used to select several documents with the highest similarity as the final search results according to the sorting results, convert the final search results into user-understandable content, and return them to the user end to complete the text recall; The dynamic optimization module is used to periodically optimize the topology structure, semantic feature storage, and sharding management logic in the system to adapt to the dynamic changes in the size of the text library and optimize the recall efficiency and accuracy through adaptive strategies.

Images