Intelligent natural language information recall method and device and storage medium

Abstract

The invention discloses an intelligent natural language information recall method and device and a storage medium, and the method comprises the steps: segmenting and vectorizing an input document, storing slices in an elastic search database, and storing vectors in a vector database; for user query, the keyword index recall mode recall from the ES database to obtain a first recall set, and the vector recall mode recall from the vector database to obtain a second recall set; aggregating the recall results in the first recall set to obtain a first recall list, and aggregating the recall results in the second recall set to obtain a second recall list; and fusing and sorting the first recall list and the second recall list in a reciprocal ranking fusion mode to obtain a recall result queried by the user. By applying the scheme of the embodiment of the invention, two different recall modes are adopted for recall, and fusion is performed after respective aggregation, so that the defect of a single recall mode can be avoided, a more accurate recall result is obtained, and the query requirement of a user is met.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for recalling intelligent natural language information, a device for recalling intelligent natural language information, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of artificial intelligence (AI), existing technologies can recall information based on natural language for user reference. Applications include intelligent question answering, writing assistance, and document retrieval. However, existing information recall technologies typically employ a single recall strategy, resulting in inaccurate and incomplete results. Summary of the Invention

[0003] In response to the above-mentioned prior art, an embodiment of the present invention discloses a method for recalling intelligent natural language information, which can overcome the defects of a single recall method and improve the comprehensiveness and accuracy of the recall results.

[0004] In view of this, an embodiment of the present application proposes a method for recalling intelligent natural language information, the method comprising:

[0005] The input document is split and vectorized, the resulting slices are stored in the ElasticSearch (ES) database, and the resulting mapping vectors are stored in the vector database;

[0006] For user queries, a keyword index recall method is used to recall the first recall set from the ES database, and a vector recall method is used to recall the second recall set from the vector database;

[0007] Aggregating all recall results in the first recall set to obtain a first recall list, and aggregating all recall results in the second recall set to obtain a second recall list, wherein each element in the first recall list and the second recall list is an aggregated recall result, and the aggregated recall result is a triplet including a slice ID, a total slice recall score, and a slice recall frequency, and the elements in the first recall list and the second recall list are ranked according to the slice recall frequency and the total slice recall score;

[0008] The first recall list and the second recall list are fused and sorted using a reciprocal ranking fusion (RRF) method to obtain a recall result of the user query.

[0009] In response to the above-mentioned prior art, an embodiment of the present invention discloses an apparatus for an intelligent natural language information recall method, which can overcome the defects of a single recall method and improve the comprehensiveness and accuracy of the recall results.

[0010] In view of this, an embodiment of the present application proposes an intelligent natural language information recall device, which includes:

[0011] The preprocessing module is used to segment and vectorize the input document, save the resulting slices in the ElasticSearch (ES) database, and save the resulting mapping vectors in the vector database;

[0012] A recall module, for user queries, uses a keyword index recall method to recall a first recall set from the ES database, and uses a vector recall method to recall a second recall set from the vector database;

[0013] an aggregation processing module, configured to aggregate all the recall results in the first recall set to obtain a first recall list, and aggregate all the recall results in the second recall set to obtain a second recall list, wherein each element in the first recall list and the second recall list is an aggregated recall result, and the aggregated recall result is a triplet including a slice ID, a total slice recall score, and a slice recall frequency, and the elements in the first recall list and the second recall list are ranked according to the slice recall frequency and the total slice recall score;

[0014] The fusion module uses a reciprocal ranking fusion (RRF) method to fuse and sort the first recall list and the second recall list to obtain a recall result of the user query.

[0015] In response to the above-mentioned prior art, an embodiment of the present invention discloses a computer-readable storage medium, which can overcome the defects of a single recall method and improve the comprehensiveness and accuracy of the recall results.

[0016] A computer-readable storage medium stores computer instructions, which, when executed by a processor, can implement the steps of the above-mentioned intelligent natural language information recall method.

[0017] In response to the above-mentioned prior art, an embodiment of the present invention discloses a computer program product that can overcome the defects of a single recall method and improve the comprehensiveness and accuracy of the recall results.

[0018] A computer program product includes computer instructions, which, when executed by a processor, implement the above-mentioned intelligent natural language information recall method.

[0019] To sum up, the embodiment of the present application simultaneously utilizes the keyword index recall method and the vector recall method, aggregates the sets of the two recall results separately to obtain a recall list, and then merges and sorts the recall lists. Its recall results can avoid the limitations of a single recall method and improve the comprehensiveness and accuracy of the recall results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flowchart of Example 1 of the method for recalling intelligent natural language information implemented in this application.

[0022] Figure 2 This is a flow chart of the method for implementing document segmentation and vectorization in Example 2 of the method of this application.

[0023] Figure 3 This is a flowchart of the keyword index recall method in Example 3 of the present application.

[0024] Figure 4 This is a flowchart of the aggregation processing in Example 4 of the present application.

[0025] Figure 5 This is a flowchart of Example 5 of the method for recalling intelligent natural language information in this application.

[0026] Figure 6 This is a schematic diagram of the structure of the device for recalling intelligent natural language information implemented in this application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or apparatus.

[0029] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0030] In order to solve the problem of inaccurate recall caused by a single recall method in the existing technology, this application adopts a method of combining multiple recall methods to achieve it. Figure 1 This is a flow chart of the first embodiment of the method for recalling intelligent natural language information in this application. Figure 1 As shown, the method includes:

[0031] Step 101: Segment and vectorize the input document, save the resulting slices in an Elasticsearch (ES) database, and save the resulting mapping vectors in a vector database.

[0032] The amount of information in a document is usually large, while the amount of information in a user query is usually small. In order to recall user queries more accurately, the input document can be segmented in advance into several slices. Later, when recalling, the user query and the slice are compared. For example, by comparing the keywords in the user query with the keywords in the slice, keyword index recall is implemented to obtain the recall result. In addition, in order to avoid the defect of inaccurate single recall, the embodiment of the present application also vectorizes the slice and maps it to the vector space. Later, when recalling, the vector mapped by the user query and the vector mapped by the slice are compared to implement vector recall to obtain the recall result. This step saves the slice in the Elastic Search (ES, Elasticsearch) database, and the vector is saved in the vector database. The Elastic Search (ES, Elasticsearch) database is a database that is easy to scale horizontally and can be applied to data requirements of different scales. Vector Database is a database specially designed for efficient storage, management and retrieval of high-dimensional vector data. Unlike traditional databases (such as relational databases), it is optimized for the characteristics of vector data (such as high dimensionality and similarity calculation) and is particularly suitable for processing unstructured data in artificial intelligence (AI) and machine learning (ML) scenarios.

[0033] Step 102: For user queries, a keyword index recall method is used to recall from the ES database to obtain a first recall set, and a vector recall method is used to recall from the vector database to obtain a second recall set.

[0034] Two types of recall methods are used here, one is keyword index recall method and the other is vector recall method.

[0035] Keyword index recall is an important information retrieval method that uses keywords and index structures to retrieve relevant document slices from a database. Specifically, when a user enters a keyword, the system quickly finds document slices containing the keyword through a pre-established index and then returns these document slices as candidate results. This process is called recall.

[0036] Vector recall is a modern recall method in information retrieval and recommendation systems. It converts text features into vectors (Embedding) and uses similarity calculations between vectors to find relevant content.

[0037] For user queries, this step uses keyword indexing and vector recall to perform recall. Each recall method will obtain several recall results. In order to better distinguish, the recall result set obtained by the keyword index recall method is called the first recall set, and the recall result set obtained by the vector recall method is called the second recall set. In practical applications, no matter which recall method is used, the recall result can contain three types of data: slice ID, recall score, and recall ranking. Among them, the slice ID is the unique identifier of the recalled slice, the recall score refers to the score value of the recall result under a certain recall method, and the recall ranking refers to the sorting position of a certain recall result among all recall results. Generally speaking, the higher the recall score, the closer the relationship between the slice and the user query, and the higher its ranking will be.

[0038] Step 103: Aggregate all the recall results in the first recall set to obtain a first recall list, and aggregate all the recall results in the second recall set to obtain a second recall list. Each element in the first recall list and the second recall list is an aggregated recall result. The aggregated recall result is a triple, including the slice ID, the total slice recall score, and the slice recall frequency. The elements in the first recall list and the second recall list are ranked according to the slice recall frequency and the total slice recall score.

[0039] In actual applications, there may be duplication in the slices recalled for user queries. For example: the user query contains three keywords a, b, and c, and a slice D contains two keywords a and b. Therefore, if based on the keyword indexing method, the slice D may be recalled twice. In order to better analyze each recalled slice and recommend the slices that are most relevant to the user query to the user, this step aggregates the first recall set and the second recall set separately. It should be noted that the aggregation described here is to aggregate the first recall set and the second recall set separately, rather than aggregating the two recall sets into one.

[0040] Step 104: The first recall list and the second recall list are fused and sorted using a reciprocal ranking fusion (RRF) method to obtain a recall result of the user query.

[0041] The aggregated first and second recall lists are the result of two different recall methods, each with a ranking based on one of the recall methods. To further optimize the rankings, this step fuses them using Reciprocal Rank Fusion (RRF). Reciprocal Rank Fusion (RRF) is an algorithm that fuses multiple ranked lists, commonly used in scenarios such as information retrieval and recommendation systems. Its core concept is to generate a better overall ranked list by weightedly fusing the rankings of items in each list.

[0042] By applying the embodiments of the present application, two different recall methods are used for user queries, namely, the keyword index recall method and the vector recall method. After the two recall result sets are aggregated separately, they are fused and sorted using the reciprocal ranking fusion (RRF) method. This can avoid the defects of a single recall method, obtain more accurate recall results, and meet user query needs.

[0043] As mentioned above, in order to more accurately recall user queries, the input documents need to be segmented and vectorized in advance. The specific segmentation and vectorization schemes used in this application are described in detail below. Figure 2 This is a flow chart of the method for implementing document segmentation and vectorization in the second embodiment of the present invention. Figure 2 Said method comprises:

[0044] Step 201: The input document is represented in a structured format, and is segmented according to the structured format, and the obtained segments are stored in an ElasticSearch (ES) database.

[0045] A document in a structured format refers to a document that organizes and presents content through a specific format or markup language, so that it has a clear logical hierarchy and identifiable elements (such as titles, paragraphs, lists, tables, etc.). This type of document is not only easy for humans to read, but can also be parsed and processed by computer programs. The segmentation in this step is not an indiscriminate segmentation or forced segmentation according to a certain data volume, but segmentation according to a structured format. For example, a document is composed of structures such as main titles, subtitles, paragraphs, lists, and tables, and there is a certain hierarchy between these structures. For example, the main title is at the first level, followed by the subtitle, and then paragraphs, lists, and tables. According to this rule, a hierarchical traversal strategy and the maximum level L for processing titles are set in advance. In actual applications, how to formulate a hierarchical traversal strategy based on the structured format of the document can be determined according to needs. For example, the specific way of segmentation can be recursive segmentation driven by titles:

[0046] Step a1: Traverse the input document starting from the title according to the hierarchical traversal strategy to determine the candidate slices.

[0047] Specifically, the slicing process is expressed as:

[0048]

[0049] Where D represents the input document, S1, S2, ... S i ,…S m Indicates that it is divided into m candidate slices, S i Represents any candidate slice.

[0050] Specifically, you can first set the title of the input document to the current level l = 1, l∈[1,L], and traverse the entire text from top to bottom starting from the title of the input document to determine the candidate slices. If the candidate slice has a subtitle (i.e., l+1 layers), the candidate slice is processed recursively. If the current processing level has reached the maximum L levels, the recursion will no longer continue, but the length control logic processing needs to be performed on the candidate slice. At this point, the full text of the input document is divided into several candidate slices according to the above method. The hierarchical traversal strategy described here is to stratify according to the title of the input document, such as first-level title, second-level title, third-level title, and so on.

[0051] Step a2: Process the candidate slices according to a pre-set first length control logic to determine the slices.

[0052] Since the embodiment of the present application segments the document according to a structured format, some candidate slices may be too long and some may be too short. In order to make the slice length as balanced as possible, the minimum slice length min_chunk_size and the maximum slice length max_chunk_size can be set. The principle of the first length control logic processing is as follows:

[0053] 1) When the length of a candidate slice is less than the minimum slice length min_chunk_size, it is merged with the adjacent candidate slice (it may be merged with the right adjacent candidate slice first), and the length of the merged slice does not exceed the maximum slice length max_chunk_size.

[0054] 2) When the length of the candidate slice is greater than the maximum slice length max_chunk_size, it is split according to the natural sentence structure, for example, according to the sentence break mark of the natural sentence (such as comma or semicolon, etc.).

[0055] 3) If a candidate slice only contains a title but lacks a valid body, the candidate slice is discarded.

[0056] 4) The candidate slices should try to retain the complete structural format of the slice itself, such as retaining complete lists, code blocks, and tables.

[0057] In addition, in order to improve the rationality of the slice length in different languages, the document character distribution can also be dynamically determined. For example, the Chinese judgment threshold is set to r = the number of Chinese characters / the total number of characters. When r>0.3, the slice length magnification factor f is increased to 3. The slice length adjustment can be expressed as:

[0058]

[0059] Where chunk_size represents the adjusted slice length, base_chunk_size represents the base slice length, and f represents the scaling factor. In practice, if the candidate slice length is equal to the base slice length (base_chunk_size), the scaling factor f is set to 1, and the length does not need to be adjusted. If a candidate slice exceeds the corresponding decision threshold, the corresponding scaling factor f is adjusted.

[0060] Using the above method, this step can segment the input document according to the structured format and save the obtained slices in the Elastic Search (ES) database.

[0061] In summary, this embodiment segments the input document by traversing it according to a hierarchical traversal strategy and using the first length control logic as a constraint. This allows the input document to be segmented according to a structured format, generating slices with clear semantic boundaries and reproducible structure. Compared to existing methods of segmenting text using fixed lengths or sliding windows, the segmentation method of this embodiment has stronger structural and semantic preservation capabilities, which helps improve the relevance and coherence of recall results.

[0062] Step 202: further refine the slice into sub-slices.

[0063] The above step 201 is to divide the input document into slices according to a structured format. These slices are used for keyword index recall, so the slices usually have a certain size. However, in order to perform vector recall at the same time, the embodiment of the present application can also further refine these large slices into sub-slices to facilitate vectorization. Specifically, the specific method of the refinement process can be:

[0064] Step b1: Each slice is further refined and processed according to a pre-set second length control logic to determine sub-slices.

[0065] Specifically, the refinement process is expressed as:

[0066]

[0067] Among them, S i Represents any slice, C i1 ,C i2 ,…,C ij Represents the j sub-slices after refinement.

[0068] If the sub-slice length is fixed, it may cause the complete natural language sentence to be broken, which is unreasonable. In order to make the refinement reasonable, the minimum sub-slice length min_chunk_size can be set fine_slice and the maximum sub-slice length max_chunk_size fine_slice , and the principles for setting the second length control logic processing are as follows:

[0069] 1) The length of the slice is equal to or greater than the maximum sub-slice length max_chunk_size fine_slice When the maximum slice length is max_chunk_size fine_slice Search for a segmentation mark that matches the natural language syntax within the preset window range. If found, the segmentation mark that matches the natural language syntax is used as the cut point of the sub-slice; if not found, the maximum slice length max_chunk_size is directly increased. fine_slice As the tangent point of the sub-slice; if the length of the slice is less than the maximum sub-slice length max_chunk_size fine_slice , then directly use the slice as a subslice.

[0070] Here, when searching for sentence-ending markers that conform to natural language syntax within the preset window range, priority is given to searching for sentence-ending markers (such as periods, question marks, exclamation marks, and other symbols that represent complete sentences). If no sentence-ending marker is found, search for sentence-ending markers (such as commas and other symbols) within the sentence.

[0071] In practical applications, assume that l = max_chunk_size fine_slice , the preset window range is δ, and δ≈0.5·max_chunk_size is set, that is, it is necessary to find a segmentation mark that conforms to the natural language syntax in the range [l,u] and use the mark as the tangent point p.

[0072] 2) When the cut point of the sub-slice is located inside the content of a special structured format, the cut point is moved to the end of the special structured format.

[0073] The special structured format mentioned here refers to special content such as code blocks, tables, and lists, while other paragraphs and the like belong to the normal structured format. Which belongs to the normal structured format and which belongs to the special structured format can be agreed or set in advance.

[0074] 3) When the length of specially structured content in a slice exceeds the set threshold, it is split using row-level splitting; otherwise, it is not split. Row-level splitting means splitting special content such as code blocks, tables, and lists at the row level, without splitting in the middle of a row.

[0075] Optionally, to make the context more coherent, the end of the previous sub-slice can be concatenated as the prefix of the next sub-slice, forming adjacent sub-slices with partially overlapping content. This method can be expressed as:

[0076]

[0077] in, Indicates the next sub-slice after adding the prefix, tail(C i(j-1) ,k) represents the content of the last k characters of the previous sub-slice (for example, k=50), and || represents the concatenation operation.

[0078] Step 203: Map the sub-slice to the vector space, and save the sub-slice mapped vector in the vector database.

[0079] After determining the sub-slices using the above method, these sub-slices can be mapped to the vector space and stored in the vector database. Mapping the sub-slices to the vector space here refers to extracting features from the sub-slices and converting them into vectors to facilitate the subsequent vector recall method.

[0080] In the embodiment of the present application, the slice is further subdivided into sub-slices and the sub-slices are mapped to the vector space. In another embodiment, if the slice size is appropriate, the slice may not be subdivided and the slice may be directly mapped to the vector space.

[0081] The above are slices or sub-slices generated based on the input document itself. In another embodiment, the overall core content of the input document can be extracted to generate additional slices. In actual applications, slices usually represent local information of the input document. When faced with global semantic tasks such as processing full-text summaries or core ideas, there may be recall omission problems. Therefore, if another embodiment of the present application adds slices that reflect the overall core content of the input document, it can cope with this situation and improve recall capabilities. In order to distinguish it from other slices, the slice that reflects the overall core content of the input document can be called a global semantic slice, which is expressed as:

[0082] semantic_summarize(D)→C sum

[0083] Among them, D represents the input document, semantic_summarize() represents the operation of refining global semantics, and C sumRepresenting a global semantic slice. The operation of extracting global semantics can be implemented using existing methods, such as using a large language model or other methods. The global semantic slice can be stored in the Elasticsearch (ES) database like other slices (step 201), mapped to a vector space, or further refined into sub-slices (step 202), mapped to a vector space, and then stored in the vector database (step 203).

[0084] By applying the embodiment of the present application, the input document can be reasonably segmented and vectorized according to the natural language structure, the slices can be saved in the Elastic Search (ES) database, the mapping vectors can be saved in the vector database, and additional global semantic slices can be added to prepare for subsequent recall operations for various user queries.

[0085] The previous section describes how to segment and vectorize input documents and store them in the ElasticSearch (ES) database and vector database, respectively. These databases are typically stored on the backend. At this point, if a user sends a query through a browser on the frontend, the system searches the ElasticSearch (ES) database and vector database on the backend and returns the matching information, known as recall.

[0086] Regarding the method of recalling the first recall set from the ES database using the keyword index recall method in the above step 102, the embodiment of the present application also provides a specific implementation method. Figure 3 This is a flowchart of the keyword index recall method in Example 3 of this application. Figure 3 As shown, the method includes:

[0087] Step 301: Use the BM25 model recall method to recall from the ES database to obtain an initial recall set, and the keyword index recall method is the BM model recall method.

[0088] The BM25 model recall method is an information retrieval technology based on a probabilistic retrieval framework, which is mainly used to efficiently recall candidate documents related to the query in a large-scale document collection. In an embodiment of the present application, this step actually uses the BM25 model to compare the user query with the slices in the ES database, and places all slice-related information that meets the requirements in the recall set. Each element in the recall set is slice-related information that meets the query requirements, and the slice-related information usually includes a slice ID, a BM25 score value, and a BM25 ranking. Among them, the slice ID represents the unique identifier of the slice, the BM25 score value represents the value of the slice relative to the user query calculated using the BM25 scoring formula, and the BM25 ranking represents the descending sort position of the BM25 score value of the slice. Among them, the BM25 scoring formula is:

[0089]

[0090] Where Q represents the user query, t represents the term in the query, d represents the slice, tf(t,d) represents the term frequency in slice d, |d| represents the length of slice d, avgdl represents the average length of all slices in the set, DF(t) represents the number of slices containing term t, IDF(t) represents the inverse slice frequency (IDF(t) = log(df(t) + 0.5N - df(t) + 0.5)), N represents the total number of slices in the set, and k and b represent tuning parameters. Using this scoring formula, we can sort the elements of a set into descending order, and their descending order position is the BM25 ranking.

[0091] Step 302: Perform a weighted correction on the BM25 recall score of each element in the initial recall set using a pre-set weight. The weighted correction is expressed as log(1+W(D))×BM25 recall score. The corrected recall result is used as the first recall set, where W(D) is the pre-set weight.

[0092] In the embodiment of the present application, W(D)=γ1w uploader (D)+γ2w views (D)+γ3w feedback (D), where W(D) includes the document uploader’s credit weight w uploader (D) Document browsing frequency weight w views (D) The document is fed back with a comprehensive score weight w feedback (D), γ1, γ2, and γ3 are weighting coefficients.

[0093] The above step 301 is to use the existing BM25 model recall method to recall and obtain the initial recall set. In actual applications, the recall results with higher rankings in the recall set are usually returned to the front end first to be presented to the user. In actual applications, it may not be reliable to determine the priority of return based solely on BM25 scoring. For example, a document uploaded by an organizer with high credit is considered to be more credible, and its recall results should be recalled first. Based on this, the embodiment of the present application sets three weights for documents: document uploader credit weight w uploader (D) Document browsing frequency weight w views (D) The document is fed back with a comprehensive score weight w feedback (D). The higher the credit of the document uploader, the higher the credibility of the uploaded document. The weight of the document w uploader (D) is higher; the higher the frequency of document browsing, the higher the document exposure and the higher its credibility, and the weight of the document w views (D) is higher; the higher the comprehensive score of the document feedback, the higher the degree of recognition of the document and the higher its credibility. The weight of the comprehensive score of the document feedback is wfeedback Therefore, the embodiment of the present application does not directly use the recall result of the existing BM25 model as the final recall result, but adds weights to it for correction. The weighted correction operation is expressed as:

[0094] Score(D,Q)=log(1+W(D))×BM25(D,Q)

[0095] Among them, W(D) is the pre-set weight, including the document uploader’s credit weight w uploader (D) Document browsing frequency weight w views (D) The document is fed back with a comprehensive score weight w feedback (D), BM25(D, Q) represents the BM25 recall score of slice D for user query Q, and Score(D, Q) represents the corrected recall score. The corrected recall result is the first recall set. In addition, even if the actual application system directly uses the built-in BM25 model, it can also be externally incorporated with non-keyword factors such as weight correction, and smooth correction can be performed using the method of the embodiment of the present application, providing the system with higher customizability and intelligence, and improving the overall matching degree of search results to user needs and document value.

[0096] In the above step 102, it is also necessary to use the vector recall method to recall the second recall set from the vector database. In practical applications, the user query can be converted into a vector representation (i.e., a query vector), and then semantic recall is performed from the vector database. Specifically, by calculating the similarity between the query vector and the vectors stored in the vector database (such as cosine similarity), the text slice that is most semantically similar to the user query can be quickly located. The use of cosine similarity scoring to determine the relevance between the user query and the slice can be expressed as:

[0097]

[0098] Where q represents the query vector, and d represents the slice vector in the vector database. If the slice is further subdivided into subslices, d represents the subslice vector. In this embodiment, regardless of whether the slice is further subdivided into subslices, the vector can be associated with a slice ID. Thus, when a vector in the vector database is hit, the corresponding slice ID can be returned.

[0099] The above-mentioned vector recall method can break through the limitations of keyword indexing, accurately capture the semantic intent of user queries, and recall slices with different words but similar semantics, thereby improving the comprehensiveness and accuracy of the recall results.

[0100] It has been described above that for user queries, a keyword index recall method is used to recall from the ES database to obtain a first recall set, and a vector recall method is used to recall from the vector database to obtain a second recall set. Regardless of the recall method used, there may be duplicate slices in the recall set. For example: a user query contains three keywords a, b, and c, and a slice D contains two keywords a and b, so it may be recalled twice. In addition, in actual applications, user queries can also be expanded to multiple queries with similar meanings, and these multiple queries are used for recall at the same time, so there may also be duplicate slices in the recall. Therefore, the embodiment of the present application aggregates each recall set separately.

[0101] Specifically, all recall results in the first recall set are aggregated to obtain a first recall list, and all recall results in the second recall set are aggregated to obtain a second recall list. The aggregation process for the first recall set is the same as the aggregation process for the second recall set.

[0102] Figure 4 This is a flow chart of the aggregation process in Example 4 of this application. Figure 4 As shown, the aggregation process includes:

[0103] Step 401: For each recall result, determine the slice ID of the recall result and determine the score of the recall method to which the recall result belongs.

[0104] As mentioned above, the input document is divided into several slices or refined into sub-slices, each slice has a unique identifier slice ID, and each sub-slice also corresponds to the slice ID of the slice to which it belongs. For keyword index recall (such as BM25 model recall), each recall result includes at least the slice ID, the score of the recall method to which the recall result belongs, and the ranking of the recall result. Similarly, for vector recall (such as cosine similarity), each recall result also includes at least the slice ID, the score of the recall method to which the recall result belongs, and the ranking of the recall result. Then, whether it is the first recall set or the second recall set, the slice ID and the score of the recall method to which the recall result belongs can be determined based on the information in each recall result.

[0105] Specifically, the recall set of a certain recall method is recorded as Each recall result in is recorded as r Si , s is a recall source (such as ES database or vector database), the slice ID is recorded as gid, the score of the recall result in the corresponding recall method is recorded as score, and the ranking of the recall result is recorded as rank.

[0106] Step 402: Merge the recall results of the same slice ID in the recall results in the set, add up the scores of the corresponding recall methods in the merging to obtain the total slice recall score, and count the frequency of occurrence of the same slice ID to obtain the slice recall frequency.

[0107] As mentioned above, there may be duplicate slices in the recall set. The slice IDs of the same slices are also the same, so they can be merged. The merging process is also called aggregation, which is expressed as:

[0108]

[0109] in, Indicates that the scores of the recall methods of the same slice are accumulated, and δ is the Kronecker delta function, that is: only when r Si When the slice ID is the same as a gid, the corresponding score r is accumulated. Si_score ; Indicates the frequency of occurrence of the same slice ID, that is, only when r Si The slice ID is accumulated only when it is the same as a gid, otherwise it is not accumulated; N S Indicates the total number of slices. That is, the recall set before aggregation After polymerization N S The recall results are aggregated, each result is a triple (d j ,s j ,c j ), where d j Indicates the slice ID, d j =gid j ;s j represents the sum of slice recall scores, c j Indicates the slice recall frequency,

[0110] Step 403: rank the merged recall results in descending order according to the slice recall frequency, and when the slice recall frequencies are the same, rank them in descending order according to the total slice recall score.

[0111] The merged recall results have been aggregated and are different from the recall results before aggregation, so they need to be re-sorted. In this embodiment, there are two sorting criteria, one is the slice recall frequency c j , the other is the sum of slice recall scores s j. In practical applications, the slicing recall frequencies can be sorted in descending order first. When the slicing recall frequencies are the same, they can be sorted in descending order according to the total slicing recall scores. The re - sorted list can be expressed as:

[0112]

[0113] Among them, represents the list after re - sorting a certain recall list, (d j , s j , c j ) represents a triple in the list. There are M S triples in the list. Usually, M S ≤N S . In this list, the following rules are satisfied:

[0114] 1) For any j < j′, it satisfies c j > c j′ , that is, the slice with a higher slicing recall frequency c j has a more forward rank;

[0115] 2) For c j = c j′ , it satisfies s j ≥s j′ , that is, for slices with equal slicing recall frequencies c j , the slice with a higher total slicing recall score is more forward.

[0116] The embodiments of this application describe the specific manner of aggregation processing. Aggregation processing of the first recall set according to the above method can obtain the first recall list, and aggregation processing of the second recall set according to the above method can obtain the second recall list. Since there are still two different lists here, in order to better return the recall results to the user, it is also necessary to fuse the rankings of the recall results obtained by different recall methods.

[0117] In another embodiment, the fusion using the Reciprocal Rank Fusion (RRF) method is specifically expressed as:

[0118]

[0119] Among them, rank i (d) represents the rank of slice d in the i - th recall list, k rrf represents the smoothing parameter of the RRF algorithm, usually set to 60 to control the weight distribution, and Score(d) represents the fused rank of slice d.

[0120] The first recall list refers to the list obtained after aggregation through keyword index recall, and the second recall list is the list obtained after aggregation through vector recall. In actual applications, if the original user query is semantically expanded to obtain multiple similar user queries, then each similar user query can obtain the first recall list and the second recall list. That is to say, there can be several first recall lists and several second recall lists in the embodiment of the present application. All these lists are fused through the above-mentioned RRF algorithm to obtain the fused ranking. Through the RRF algorithm, the results of keyword index recall and vector recall can be effectively fused to improve the relevance and accuracy of the overall recall.

[0121] The solution of the embodiment of the present application not only comprehensively considers the advantages of different recall strategies, but also reduces the deviations and limitations that may be caused by a single recall strategy, and uses the RRF algorithm for fusion processing to ensure that the information subsequently returned to the user has high relevance and accuracy.

[0122] In addition, in another embodiment, the neural network re-ranking model of the Cross-Encoder architecture can be used to re-rank the fused list to further optimize the sorting. The neural network re-ranking model of the Cross-Encoder architecture is a deep learning model for information retrieval and natural language processing tasks, mainly used to re-rank the preliminary retrieval results to further improve accuracy and relevance. The neural network re-ranking model of the Cross-Encoder architecture can be expressed as:

[0123]

[0124] in, Indicates that the user query Q and slice d are taken as input, and are jointly encoded by the Cross-Encoder model and output the relevance score Score re-rank (d). Finally, according to Score re-rank (d) Reorder the recall results and put the slices that best meet user needs at the front, further improving the efficiency of users in obtaining effective information and enhancing user experience.

[0125] The fifth embodiment of the method of the present application provides a method for recalling intelligent natural language information. In the fifth embodiment of the method, it is assumed that a large number of documents are input, including a product technical specification D in Markdown format. The product technical specification includes a first-level title (i.e., the main title, marked with "#"), a second-level title (i.e., a sub-title, marked with "##"), a third-level title (i.e., a lower-level sub-title, marked with "###"), and may also include other levels of titles as well as paragraphs, tables, lists and other formats, which will not be repeated here.

[0126] Figure 5 This is a flow chart of the fifth embodiment of the method for recalling intelligent natural language information in this application. Figure 5 As shown, the method embodiment of the present application includes:

[0127] Step 501: The input document is represented in a structured format, and is segmented according to the structured format, and the obtained segments are stored in an ElasticSearch (ES) database.

[0128] Step 502: further refine the slice into sub-slices.

[0129] Step 503: Map the sub-slice to the vector space, and save the vector of the sub-slice mapping in the vector database.

[0130] The above steps 501 to 503 are the same as steps 201 to 203 in the second embodiment of the method. The Markdown in the fifth embodiment of the present application is a structured format. When segmenting, a recursive segmentation driven by the title can be used to segment the product manual D into m candidate slices S1, S2, ... S i ,…S m . Among them, it is assumed that each candidate slice corresponds to a logical block in the product manual, such as "Product Introduction", "How to Use", "Precautions", etc. The recursive segmentation of slices driven by titles can better ensure that the semantics of the slices are natural and the structure is clear and reasonable. During the segmentation process, the candidate slices can also be processed according to the first length control logic set in advance. Assuming that the text under a candidate slice "How to Use" exceeds the maximum slice length max_chunk_size, it can be split according to the natural sentence structure to generate slices of reasonable length. The specific method of segmentation can be found in steps a1 and a2 in method embodiment 2, which will not be repeated here.

[0131] For the method of thinning and vectorizing the slice, refer to step b1 in the second embodiment of the method, and process according to the second length control logic. i for:

[0132] How to use

[0133] 1. Connect the device to a power source.

[0134] 2. Open the Control Panel and click the "Start" button...

[0135] 3.……

[0136] Then, the slice can be further refined as follows:

[0137] C i1 : "1. Plug the device into a power source."

[0138] C i2 : "2. Open the Control Panel and click the "Start" button..."

[0139] …

[0140] Afterwards, the subslices are mapped into the vector space.

[0141] Step 504: extract global semantics from the input document to obtain global semantic slices, and save the global semantic slices in the Elastic Search (ES) database.

[0142] Step 505: Save the mapping vector obtained by vectorizing the global semantic slice in a vector database.

[0143] The above steps 504 and 505 are to extract the core content of the input document and generate additional global semantic slices and vectors. For example, the product instruction manual D can be summarized using a large language model to obtain an additional global semantic slice C. sum , its content can be "This article introduces the basic operation process of the product, covering the interaction of startup, configuration and user interface, suitable for novice users". Then, the global semantic slice C sum After further vectorization, it is saved in the vector database.

[0144] Step 506: For user queries, the BM25 model recall method is used to recall the initial recall set from the ES database, and the keyword index recall method is the BM model recall method.

[0145] Step 507: Perform weighted correction on the BM25 recall score of each element in the initial recall set using a pre-set weight. The weighted correction is expressed as log(1+W(D))×BM25 recall score. The corrected recall result is used as the first recall set, where W(D) is the pre-set weight.

[0146] Step 508: For the user query, a second recall set is obtained by recalling from the vector database using a vector recall method.

[0147] Step 509: Aggregate all the recall results in the first recall set to obtain a first recall list.

[0148] Step 510: Aggregate all the recall results in the second recall set to obtain a second recall list.

[0149] The above steps 506, 507 and 509 are keyword index recall channels, and steps 508 and 510 are vector recall channels. The two channels can be processed in parallel without any order.

[0150] That is, the slice {Si} and global semantic slice C sum Saved in the ES database, the initial recall set is obtained from the ES database using the keyword index recall method (BM25) for the user query Q After weighted correction and aggregation processing Get the first recall list. Accordingly, the sub-slice {C ij} and global semantic slice C sum Encoded as a vector and stored in the vector database, the second recall set is obtained from the vector database using the vector recall method for user query Q Aggregate it Get the second recall list. Among them, d j Indicates the slice ID, s j represents the sum of slice recall scores, c j The specific keyword index recall method can be found in Method Example 3, and the aggregation processing method can be found in Method Example 4, which will not be described in detail here.

[0151] Step 511: The first recall list and the second recall list are fused and sorted using a reciprocal ranking fusion (RRF) method to obtain a recall result of the user query.

[0152] This step uses the reciprocal ranking fusion (RRF) method for fusion, which is specifically expressed as follows:

[0153]

[0154] Among them, rank i (d) represents the ranking of slice d in the i-th recall list, k rrf It represents the smoothing parameter of the RRF algorithm, which is usually set to 60 to control the weight distribution. Score(d) represents the ranking of slice d after fusion.

[0155] Step 512: Optimize and sort the fused sorted list using a neural network-based rearrangement model, and use the optimized sorting result as the recall result of the user query.

[0156] This step uses the Cross-Encoder architecture neural network rearrangement model to express it as:

[0157]

[0158] in, Indicates that the user query Q and slice d are taken as input, and are jointly encoded by the Cross-Encoder model and output the relevance score Score re-rank (d).

[0159] By applying the solution of the embodiment of this application, product manual document D can be segmented and vectorized, and stored in an ES database and a vector database. When a user needs to query, the recall is performed using both keyword index recall and vector recall. After aggregation, the recall lists are obtained. The different recall lists are then merged and finally optimized using a neural network-based rearrangement model. This can avoid the shortcomings of a single recall method and increase the comprehensiveness and accuracy of the recall.

[0160] The various embodiments described above in this application can be applied in different scenarios, such as intelligent question-answering applications, intelligent writing assistance, and document retrieval and management.

[0161] In intelligent question-answering applications, this system can quickly understand natural language questions posed by users and, through efficient information recall technology, accurately retrieve relevant answers from the knowledge base. Whether it's a simple FAQ or a complex professional question, the system provides accurate and rapid responses, helping users obtain the information they need and improving the user experience. It is widely used in customer service robots, online education platforms, and other fields.

[0162] In intelligent writing assistance, authors often need to find relevant information, reference examples, or acquire professional knowledge to enrich content and enhance argumentation. This system can serve as an intelligent writing assistant, helping authors quickly recall information related to the writing topic. For example, when writing a scientific article, the author enters relevant topics or keywords, and the system can accurately retrieve appropriate cases, data, theoretical explanations, etc. from a large amount of literature and technical reports, providing the author with inspiration and materials, improving writing efficiency and quality. It is suitable for various writing scenarios, such as academic paper creation, news report writing, copywriting planning, etc.

[0163] This system provides efficient document retrieval and management for document management systems in organizations such as enterprises and research institutes. Users can quickly locate relevant documents through natural language queries. Whether searching for specific project materials, research reports, or technical documentation, the system accurately retrieves relevant documents, improving work efficiency and promoting knowledge sharing and innovation.

[0164] This application also provides an intelligent natural language information recall device. Figure 6 This is a schematic diagram of the structure of the device for recalling intelligent natural language information implemented in this application. Figure 6 As shown, the device includes: a pre-processing module 601, a recall module 602, an aggregation processing module 603 and a fusion module 604.

[0165] The pre-processing module 601 is used to segment and vectorize the input document, save the obtained slices in the Elastic Search (ES) database, and save the obtained mapping vectors in the vector database.

[0166] The recall module 602 is used to recall a first recall set from the ES database using a keyword index recall method and to recall a second recall set from the vector database using a vector recall method in response to a user query.

[0167] The aggregation processing module 603 is used to aggregate all the recall results in the first recall set to obtain a first recall list, and to aggregate all the recall results in the second recall set to obtain a second recall list. Each element in the first recall list and the second recall list is an aggregated recall result. The aggregated recall result is a triple, including the slice ID, the total slice recall score, and the slice recall frequency. The elements in the first recall list and the second recall list are ranked according to the slice recall frequency and the total slice recall score.

[0168] The fusion module 604 is configured to fuse and sort the first recall list and the second recall list using a reciprocal ranking fusion (RRF) method to obtain a recall result of the user query.

[0169] That is, the preprocessing module 601 segments and vectorizes the input document, saves the obtained slices in the Elastic Search (ES) database, and saves the obtained mapping vectors in the vector database; for user queries, the recall module 602 uses the keyword index recall method to recall from the ES database to obtain a first recall set, and uses the vector recall method to recall from the vector database to obtain a second recall set; the aggregation processing module 603 aggregates all the recall results in the first recall set to obtain a first recall list, and aggregates all the recall results in the second recall set to obtain a second recall list; the fusion module 604 uses the reciprocal ranking fusion (RRF) method to fuse and sort the first recall list and the second recall list to obtain the recall result of the user query.

[0170] In another embodiment, the device further includes a rearrangement module 605, which optimizes and sorts the fused sorted list using a neural network-based rearrangement model, and uses the optimized sorting result as the recall result of the user query.

[0171] The present application also provides a computer-readable medium, which stores instructions that, when executed by a processor, can perform the steps in the method for recalling intelligent natural language information as described above. In practical applications, the computer-readable medium can be included in the device / apparatus / system described in the above embodiments, or it can exist independently without being assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method for recalling intelligent natural language information described in the above embodiments can be implemented. According to the embodiments disclosed in the present application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, for example, it can include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, but is not intended to limit the scope of protection of the present application. In the embodiments disclosed in the present application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device.

[0172] An embodiment of the present application further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0173] The flowcharts and block diagrams in the accompanying drawings of the present application show the possible implementation architecture, functions and operations of the systems, methods and computer program products according to the various embodiments disclosed in the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0174] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of this disclosure may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope of this application.

[0175] The principles and implementation methods of the present invention are described herein using specific embodiments. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not intended to limit this application. For those skilled in the art, changes can be made in the specific implementation methods and application scope based on the ideas, spirit and principles of the present invention. Any modifications, equivalent replacements, improvements, etc. made therein should be included within the scope of protection of this application.

Claims

1. A method for recalling intelligent natural language information, characterized in that: The method includes: The input document is split and vectorized, the resulting slices are stored in the ElasticSearch (ES) database, and the resulting mapping vectors are stored in the vector database; For user queries, a keyword index recall method is used to recall the first recall set from the ES database, and a vector recall method is used to recall the second recall set from the vector database; Aggregating all recall results in the first recall set to obtain a first recall list, and aggregating all recall results in the second recall set to obtain a second recall list, wherein each element in the first recall list and the second recall list is an aggregated recall result, and the aggregated recall result is a triplet including a slice ID, a total slice recall score, and a slice recall frequency, and the elements in the first recall list and the second recall list are ranked according to the slice recall frequency and the total slice recall score; The first recall list and the second recall list are fused and sorted using a reciprocal ranking fusion (RRF) method to obtain a recall result of the user query.

2. The method according to claim 1, characterized in that The steps of slicing and vectorizing the input document, storing the obtained slices in an Elasticsearch (ES) database, and storing the obtained mapping vectors in a vector database include: Representing the input document in a structured format, segmenting it according to the structured format, and storing the resulting segments in the ES database; further refining the slice into sub-slices; The sub-slice is mapped to a vector space, and the vectors mapped to the sub-slice are stored in the vector database.

3. The method according to claim 2, characterized in that The step of segmenting according to the structured format includes: Traversing the input document according to a hierarchical traversal strategy starting from the title of the input document to determine candidate slices; the hierarchical traversal strategy is layered according to the title of the input document; The candidate slices are processed according to a pre-set first length control logic to determine the slices; the first length control logic includes: When the length of the candidate slice is less than the minimum slice length, it is merged with the adjacent candidate slice; When the length of the candidate slice is greater than the maximum slice length, the candidate slice is split according to the natural sentence structure; If the candidate slice only contains a title but lacks a valid body, the candidate slice is discarded; The candidate slice retains the complete structural format of the slice itself.

4. The method according to claim 2, characterized in that The step of further refining the slice into sub-slices comprises: Each of the slices is further refined and processed according to a pre-set second length control logic to determine the sub-slices; the second length control logic is: When the length of the slice is equal to or greater than the maximum sub-slice length, a segmentation mark that conforms to the natural language syntax is searched within a preset window range at the maximum slice length. If found, the segmentation mark that conforms to the natural language syntax is used as the tangent point of the sub-slice. If not found, the maximum slice length is directly used as the tangent point of the sub-slice. If the length of the slice is less than the maximum sub-slice length, the slice is directly used as the sub-slice. When the cut point of the sub-slice is located inside the content of the special structured format, the cut point is moved to the end of the special structured format; When the length of the special structured format content in the slice exceeds a set threshold, it is split using a row-level splitting method; otherwise, it is not split.

5. The method according to claim 1, characterized in that When the input document is segmented and vectorized, the method further includes: Extracting global semantics from the input document to obtain global semantic slices, and storing the global semantic slices in the ElasticSearch (ES) database; The mapping vector obtained by vectorizing the global semantic slice is stored in the vector database.

6. The method according to claim 1, characterized in that The step of using the keyword index recall method to recall from the ES database to obtain the first recall set includes: An initial recall set is obtained by recalling from the ES database using the BM25 model recall method, where the keyword index recall method is the BM25 model recall method; The BM25 recall score value of each element in the initial recall set is weighted and corrected using a pre-set weight, and the corrected recall result is used as the first recall set. The weighted correction is expressed as log(1+W(D))×BM25 recall score value, where W(D) is the pre-set weight.

7. The method according to claim 6, characterized in that The W(D) is expressed as W(D)=γ1w uploader (D)+γ2w views (D)+γ3w feedback (D), wherein W(D) includes the document uploader's credit weight w uploader (D) Document browsing frequency weight w views (D) The document is fed back with a comprehensive score weight w feedback (D), γ1, γ2, and γ3 are weighting coefficients.

8. The method according to claim 1, characterized in that The aggregation process includes: For each of the recall results, determine the slice ID of the recall result, and determine the score of the recall method to which the recall result belongs; Merge the recall results of the same slice ID in the recall results in the set, accumulate the scores of the corresponding recall methods in the merging to obtain the total slice recall score, and count the occurrence frequency of the same slice ID to obtain the slice recall frequency; The merged recall results are ranked in descending order according to the slice recall frequencies, and when the slice recall frequencies are the same, they are ranked in descending order according to the sum of the slice recall scores.

9. The method according to claim 1, characterized in that After the step of fusing and sorting the first recall list and the second recall list using a reciprocal ranking fusion (RRF) method, the method further includes: The fused sorted list is optimized and sorted using a neural network-based rearrangement model, and the optimized sorting result is used as the recall result of the user query.

10. An intelligent natural language information recall device, characterized in that: The device includes: The preprocessing module is used to segment and vectorize the input document, save the resulting slices in the ElasticSearch (ES) database, and save the resulting mapping vectors in the vector database; A recall module, for user queries, uses a keyword index recall method to recall a first recall set from the ES database, and uses a vector recall method to recall a second recall set from the vector database; an aggregation processing module, configured to aggregate all the recall results in the first recall set to obtain a first recall list, and aggregate all the recall results in the second recall set to obtain a second recall list, wherein each element in the first recall list and the second recall list is an aggregated recall result, and the aggregated recall result is a triplet including a slice ID, a total slice recall score, and a slice recall frequency, and the elements in the first recall list and the second recall list are ranked according to the slice recall frequency and the total slice recall score; The fusion module uses a reciprocal ranking fusion (RRF) method to fuse and sort the first recall list and the second recall list to obtain a recall result of the user query.

11. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the intelligent natural language information recall method described in any one of claims 1 to 9 can be implemented.

12. A computer program product comprising computer instructions, wherein when executed by a processor, the computer instructions implement the intelligent natural language information recall method according to any one of claims 1 to 9.

Images