AI-based information retrieval method and system

Abstract

The invention provides an AI-based information retrieval method and system, and the method comprises the steps: constructing an index database, obtaining initial question data, and then obtaining final question data; segmenting the system help document into text slices, and converting the text slices into vector identification data; obtaining retrieval vector data corresponding to the final question data to obtain vector similarity between the retrieval vector data and the vector identification data, and selecting a first target text slice through the vector similarity; and selecting a second target text fragment from the system help document through the initial question data, sorting and scoring the first target text fragment and the second target text fragment based on the interactive behavior characteristics, and further feeding back an index result. Initial question data are continuously improved through semantic analysis, and the high matching degree between search and query is ensured; by extracting the interactive behavior features, the evaluation dimensions of the first target text piece and the second target text piece are unified, and it is ensured that retrieval can be completed rapidly, accurately and strongly associated based on user preferences.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an AI-based information retrieval method and system. Background Art

[0002] Traditional software help systems usually refer to the documents, manuals or built-in help functions provided with the software. Users can obtain corresponding answers to software-related questions by searching for questions in the software help system.

[0003] Such software help systems are widely used, but in modern software development practices, some of their limitations have begun to emerge. The main problem is that traditional help systems are often in the form of static documents. After users enter question keywords, traditional help systems index all related content based on the question keywords, which can easily lead to users obtaining answers that are less relevant to the question or even irrelevant. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide an AI-based information retrieval method and system, aiming to solve the technical problem in the prior art that users retrieve answers by entering question keywords in a software help system, but due to the static document format of the help system, it is difficult for users to accurately obtain answers corresponding to the questions.

[0005] In order to achieve the above objectives, in a first aspect, an embodiment of the present application provides an AI-based information retrieval method, comprising the following steps: Constructing an index library corresponding to the system help document, obtaining initial question data, and obtaining final question data based on the initial question data and the index library; Dividing the system help document into a plurality of text pieces, performing vector conversion on the text pieces to obtain vector identification data corresponding to the text pieces, and combining the vector identification data and the text pieces into a database; Acquire retrieval vector data corresponding to the final question data, acquire vector similarity between the retrieval vector data and the vector identification data, and select a plurality of first target text segments from the plurality of text segments according to the vector similarity; A number of second target text pieces are selected from the system help document using the initial question data, interaction behavior features of the first target text piece and the second target text piece are extracted, the number of first target text pieces and the number of second target text pieces are ranked and scored based on the interaction behavior features, a candidate queue is formed through the ranking and scoring, and an index result is fed back.

[0006] Compared with the prior art, the beneficial effects of the present invention are: by obtaining the preset questions based on the initial question data, the initial question data is continuously improved through semantic analysis to form the final question data, thereby ensuring a high degree of match between the search results and the user's query intention; by converting both the final question data and the system help document into vector data, a faster retrieval speed can be provided, thereby improving the retrieval efficiency; by introducing the full-text retrieval results between the initial question data and the system help document, the problem of reduced retrieval accuracy caused by vector matching is compensated, thereby ensuring that more comprehensive retrieval results can be provided according to user needs; and then by extracting the interactive behavior features, the evaluation dimensions of the first target text piece and the second target text piece are unified, thereby ensuring that the retrieval can be completed quickly, accurately, and strongly associated based on the user's preferences.

[0007] Furthermore, the step of constructing an index library corresponding to the system help document includes: Generate a number of subordinate keywords based on the system help document, and set preset questions according to the subordinate keywords; Different subordinate keywords are associated with main keywords according to the meanings of the subordinate keywords, and an index library is generated based on the main keyword, the subordinate keywords and the preset questions.

[0008] Furthermore, the step of obtaining final question data based on the initial question data and the index library includes: Performing text cleaning and word segmentation processing on the initial question data to obtain initial question keywords; Comparing the initial question keyword with the main keyword to select a first related keyword from a plurality of the main keywords; Based on a preset question corresponding to the first related keyword, obtaining a related answer from the user, and selecting a plurality of second related keywords from the plurality of subordinate keywords based on the related answer; The first related keyword and a plurality of the second related keywords are combined into final question data.

[0009] Furthermore, the step of dividing the system help document into a plurality of text segments includes: Dividing the system help document into a plurality of text blocks using paragraphs as delimiters; The importance of different words in the text block is obtained, key words are selected from a plurality of the words based on the importance, and the text block is divided into a plurality of text segments with the key words as nodes.

[0010] Furthermore, the calculation formula of the importance is: , in, represents the importance of the i-th word in the j-th document, represents the number of occurrences of the i-th word in the j-th document, represents the total number of words in the jth document, Indicates the total number of documents. represents the number of documents containing the i-th word, Represents a logarithmic function.

[0011] Furthermore, the step of performing vector conversion on the text slice to obtain vector identification data corresponding to the text slice includes: Based on the semantic features of the words in the text piece, the text piece is divided into a plurality of word groups, and the words are converted into sub-codes based on the number of words in the word groups; A plurality of the sub-codes are combined into an initial vector corresponding to the sentence in the text piece, a phase vector corresponding to the text piece is obtained according to the initial vector, and the phase vector is converted into vector identification data.

[0012] Furthermore, the formula for obtaining the vector identification data is: , in, Represents vector identification data, represents the number of sub-codes in the phase vector, represents the number of dimensions, Represents the phase vector.

[0013] Furthermore, the calculation formula of the vector similarity is: , in, represents the vector similarity, Represents the component of the vector identification data in the i-th dimension, Represents the component of the ith dimension of the retrieval vector data. Indicates the number of dimensions.

[0014] Furthermore, the interactive behavior features include retrieval association features, user association features, historical browsing features and content quality features.

[0015] In a second aspect, an embodiment of the present application provides an AI-based information retrieval system, which is applied to the AI-based information retrieval method as described in the first aspect above, and the system includes: An optimization module, used to construct an index library corresponding to the system help document, obtain initial question data, and obtain final question data based on the initial question data and the index library; A conversion module, used for dividing the system help document into a plurality of text pieces, performing vector conversion on the text pieces to obtain vector identification data corresponding to the text pieces, and combining the vector identification data and the text pieces into a database; A linking module, used to obtain retrieval vector data corresponding to the final question data, obtain vector similarity between the retrieval vector data and the vector identification data, and select a plurality of first target text segments from the plurality of text segments according to the vector similarity; An execution module is used to select a number of second target text pieces from the system help document through the initial question data, extract the interactive behavior features of the first target text piece and the second target text piece, sort and score the number of first target text pieces and the number of second target text pieces based on the interactive behavior features, form a candidate queue through the sorting and scoring, and feed back the index result.

[0016] In a third aspect, an embodiment of the present application provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the AI-based information retrieval method as described in the first aspect above is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the AI-based information retrieval method as described in the first aspect above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the AI-based information retrieval method in the first embodiment of the present invention; Figure 2 It is a structural block diagram of an AI-based information retrieval system in a second embodiment of the present invention; The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0019] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0020] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0022] See also Figure 1 The first embodiment of the present invention provides an AI-based information retrieval method, comprising the following steps: S10: construct an index library corresponding to the system help document, obtain initial question data, and obtain final question data based on the initial question data and the index library; Specifically, the step S10 includes: S110: generating a plurality of subordinate keywords based on the system help document, and setting preset questions according to the subordinate keywords; S120: Associating different subordinate keywords with main keywords according to the meanings of the subordinate keywords, and generating an index library based on the main keyword, the subordinate keywords and the preset question.

[0023] The subordinate keywords characterize the content features of different sentences or paragraphs in the system help document, and the main keywords are uniformly generated based on the subordinate keywords with consistent semantic types. The index library includes several main keywords, each of which corresponds to several subordinate keywords. It can be understood that each main keyword also corresponds to several preset questions. Taking the system help document of the financial system as an example, the main keyword can be expenditure, and the subordinate keywords can be salary, water and electricity, etc. It should be noted that the main keywords and the subordinate keywords can be defined according to the characteristics of the system.

[0024] S130: Perform text cleaning and word segmentation processing on the initial question data to obtain initial question keywords; The text cleaning refers to removing irrelevant characters in the initial question data, and the word segmentation processing refers to decomposing the initial question data after text cleaning based on a dictionary to form a plurality of words.

[0025] S140: comparing the initial question keyword with the main keyword to select a first related keyword from a plurality of the main keywords; S150: obtaining a related answer from the user based on a preset question corresponding to the first related keyword, and selecting a plurality of second related keywords from the plurality of subordinate keywords based on the related answer; The preset questions correspond to the subordinate keywords one by one. After determining the first associated keyword, the specific search needs of the user can be determined by triggering the corresponding preset questions. In this process, the final selection of the second associated keyword can be completed through multiple iterative comparisons.

[0026] S160: combining the first associated keyword and a plurality of the second associated keywords into final question data; S20: dividing the system help document into a plurality of text segments, performing vector conversion on the text segments to obtain vector identification data corresponding to the text segments, and combining the vector identification data and the text segments into a database; The step S20 comprises: S210: Divide the system help document into a plurality of text blocks using paragraphs as separation points; In some embodiments, after obtaining the text block, the subsequent steps can be directly processed, but the text block is generated based on paragraphs. Although more context information is retained, too much content will greatly increase the difficulty of retrieval. Therefore, the text block still needs to be segmented twice.

[0027] S220: Obtaining importance of different words in the text block, selecting key words from a plurality of the words based on the importance, and dividing the text block into a plurality of text segments with the key words as nodes; The calculation formula of the importance is: , in, represents the importance of the i-th word in the j-th document, represents the number of occurrences of the i-th word in the j-th document, represents the total number of words in the jth document, Indicates the total number of documents. represents the number of documents containing the i-th word, Represents a logarithmic function.

[0028] In this embodiment, the word with the highest importance is selected as the key word, and the sentences between the two key words are all classified under the previous key word to form the text segment.

[0029] S230: dividing the text fragment into a plurality of word groups based on semantic features of the words in the text fragment, and converting the words into sub-codes based on the number of words in the word groups; Assuming that each of the text pieces contains gender features: male, female; sports features: football, basketball, and badminton, then male and female form a word group, and male is converted into (10) and female is converted into (01) respectively; football, basketball, and badminton form a word group, and football is converted into (100), basketball is converted into (010), and badminton is converted into (001) respectively. It can be understood that in this example, the semantic features are behavioral features and sports features.

[0030] S240: combining a plurality of the sub-codes into an initial vector corresponding to a sentence in the text segment, acquiring a phase vector corresponding to the text segment according to the initial vector, and converting the phase vector into vector identification data; Specifically, the number of word groups in the text piece is obtained. If the number of word groups is 1, the initial vector is directly determined as a phase vector; if the number of word groups is greater than 1, the sub-codes at the same position in different initial vectors are weighted averaged to form a phase vector.

[0031] Still taking the above content as an example, assuming that the content in the text piece is today's results of men's football matches, today's results of men's basketball matches, and today's results of women's badminton matches, the initial vectors corresponding to the sentences in the text piece are: (10100), (10010), (01001), and then the three groups of initial vectors are weighted averaged.

[0032] The formula for obtaining the vector identification data is: , in, Represents vector identification data, represents the number of sub-codes in the phase vector, represents the number of dimensions, It should be noted that, because different text pieces correspond to different numbers of sub-codes in the phase vector, they can all be converted to the same dimension through matrix conversion to facilitate subsequent comparison and retrieval.

[0033] S30: obtaining retrieval vector data corresponding to the final question data, obtaining vector similarity between the retrieval vector data and the vector identification data, and selecting a plurality of first target text segments from the plurality of text segments according to the vector similarity; The acquisition method of the retrieval vector data is the same as that of the vector identification data, which will not be described here. The calculation formula of the vector similarity is: , in, represents the vector similarity, Represents the component of the vector identification data in the i-th dimension, Represents the component of the ith dimension of the retrieval vector data. Indicates the number of dimensions.

[0034] After obtaining the vector similarity between the retrieval vector data and each of the vector identification data, the different vector similarities are compared with a similarity threshold, and the text piece corresponding to the vector similarity greater than the similarity threshold is selected as the first target text piece.

[0035] S40: selecting a plurality of second target text pieces from the system help document according to the initial question data, extracting interactive behavior features of the first target text piece and the second target text piece, ranking and scoring the plurality of first target text pieces and the plurality of second target text pieces based on the interactive behavior features, forming a candidate queue according to the ranking and scoring, and feeding back an index result; It can be understood that the second target text piece is the search result obtained after full-text search from the system help document through the initial question data. The interactive behavior characteristics include search correlation characteristics, user correlation characteristics, historical browsing characteristics and content quality characteristics. The search correlation characteristics refer to the strength of the correlation between the text piece and the question retrieved by the user, the user correlation characteristics refer to the strength of the correlation between the text piece and the user's previous search data, the historical browsing characteristics refer to the number of times the text piece has been browsed by all users, and the content quality characteristics refer to the content quality of the text piece itself.

[0036] After obtaining the first target text piece and the second target text piece, the interactive behavior features are obtained, and the sorting scores corresponding to the first target text piece and the second target text piece are generated based on the feature scores of the interactive behavior features and the feature weights set for different interactive behavior features. The first target text piece and the second target text piece obtained are then sorted according to the sorting scores to form an index result, and the priority of the index feedback to the user is determined according to the different sortings.

[0037] By obtaining the preset questions based on the initial question data, the initial question data is continuously improved through semantic analysis to form the final question data, thereby ensuring a high degree of match between the search results and the user's query intention; by converting both the final question data and the system help document into vector data, a faster retrieval speed can be provided, thereby improving the retrieval efficiency; by introducing the full-text retrieval results between the initial question data and the system help document, the problem of reduced retrieval accuracy caused by vector matching is compensated, thereby ensuring that more comprehensive retrieval results can be provided according to user needs; and then by extracting the interactive behavior features, the evaluation dimensions of the first target text piece and the second target text piece are unified, thereby ensuring that the retrieval can be completed quickly, accurately, and strongly associated based on the user's preferences.

[0038] See also Figure 2 The second embodiment of the present invention provides an AI-based information retrieval system, which is applied to the AI-based information retrieval method described in the above embodiment, and will not be repeated here. As used below, the terms "module", "unit", "sub-unit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0039] The system comprises: The optimization module 10 is used to construct an index library corresponding to the system help document, obtain initial question data, and obtain final question data based on the initial question data and the index library; The optimization module 10 comprises: The first unit is used to generate a number of subordinate keywords based on the system help document, and set preset questions according to the subordinate keywords; The second unit is used to associate different subordinate keywords with main keywords according to the meanings of the subordinate keywords, and generate an index library based on the main keyword, the subordinate keywords and the preset question; The third unit is used to perform text cleaning and word segmentation processing on the initial question data to obtain initial question keywords; A fourth unit is used to compare the initial question keyword with the main keyword to select a first related keyword from a plurality of the main keywords; A fifth unit, configured to obtain a related answer from a user based on a preset question corresponding to the first related keyword, and select a plurality of second related keywords from the plurality of subordinate keywords according to the related answer; A sixth unit, configured to combine the first associated keyword and a plurality of the second associated keywords into final question data; The conversion module 20 is used to divide the system help document into a plurality of text pieces, perform vector conversion on the text pieces to obtain vector identification data corresponding to the text pieces, and combine the vector identification data and the text pieces into a database; The conversion module 20 includes: The seventh unit is used to divide the system help document into a plurality of text blocks using paragraphs as separation points; An eighth unit is used to obtain the importance of different words in the text block, select key words from a plurality of the words based on the importance, and divide the text block into a plurality of text segments with the key words as nodes; A ninth unit, configured to divide the text fragment into a plurality of word groups based on semantic features of the words in the text fragment, and convert the words into sub-codes based on the number of words in the word groups; A tenth unit is used to combine a plurality of the sub-codes into an initial vector corresponding to the sentence in the text piece, obtain a phase vector corresponding to the text piece according to the initial vector, and convert the phase vector into vector identification data; A linking module 30 is used to obtain the search vector data corresponding to the final question data, obtain the vector similarity between the search vector data and the vector identification data, and select a plurality of first target text segments from the plurality of text segments according to the vector similarity; The execution module 40 is used to select a number of second target text pieces from the system help document through the initial question data, extract the interactive behavior features of the first target text pieces and the second target text pieces, sort and score the number of first target text pieces and the number of second target text pieces based on the interactive behavior features, form a candidate queue through the sorting and scoring, and feed back the index result.

[0040] The present invention also provides a computer, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the AI-based information retrieval method as described in the above technical solution is implemented.

[0041] The present invention also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the AI-based information retrieval method as described in the above technical solution is implemented.

[0042] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0043] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An AI-based information retrieval method, characterized in that: The following steps are involved: Constructing an index library corresponding to the system help document, obtaining initial question data, and obtaining final question data based on the initial question data and the index library; Dividing the system help document into a plurality of text pieces, performing vector conversion on the text pieces to obtain vector identification data corresponding to the text pieces, and combining the vector identification data and the text pieces into a database; Acquire retrieval vector data corresponding to the final question data, acquire vector similarity between the retrieval vector data and the vector identification data, and select a plurality of first target text segments from the plurality of text segments according to the vector similarity; A number of second target text pieces are selected from the system help document using the initial question data, interaction behavior features of the first target text piece and the second target text piece are extracted, the number of first target text pieces and the number of second target text pieces are ranked and scored based on the interaction behavior features, a candidate queue is formed through the ranking and scoring, and an index result is fed back.

2. The AI-based information retrieval method according to claim 1, characterized in that: The step of constructing an index library corresponding to the system help document comprises: Generate a number of subordinate keywords based on the system help document, and set preset questions according to the subordinate keywords; Different subordinate keywords are associated with main keywords according to the meanings of the subordinate keywords, and an index library is generated based on the main keyword, the subordinate keywords and the preset questions.

3. The AI-based information retrieval method according to claim 2, characterized in that: The step of obtaining final question data based on the initial question data and the index library comprises: Performing text cleaning and word segmentation processing on the initial question data to obtain initial question keywords; Comparing the initial question keyword with the main keyword to select a first related keyword from a plurality of the main keywords; Based on a preset question corresponding to the first related keyword, obtaining a related answer from the user, and selecting a plurality of second related keywords from the plurality of subordinate keywords based on the related answer; The first related keyword and a plurality of the second related keywords are combined into final question data.

4. The AI-based information retrieval method according to claim 1, characterized in that: The step of dividing the system help document into a plurality of text pieces comprises: Dividing the system help document into a plurality of text blocks using paragraphs as delimiters; The importance of different words in the text block is obtained, key words are selected from a plurality of the words based on the importance, and the text block is divided into a plurality of text segments with the key words as nodes.

5. The AI-based information retrieval method according to claim 4, characterized in that: The calculation formula of the importance is: ， in, represents the importance of the i-th word in the j-th document, represents the number of occurrences of the i-th word in the j-th document, represents the total number of words in the jth document, Indicates the total number of documents. represents the number of documents containing the i-th word, Represents a logarithmic function.

6. The AI-based information retrieval method according to claim 1, characterized in that: The step of performing vector conversion on the text slice to obtain vector identification data corresponding to the text slice comprises: Based on the semantic features of the words in the text piece, the text piece is divided into a plurality of word groups, and the words are converted into sub-codes based on the number of words in the word groups; A plurality of the sub-codes are combined into an initial vector corresponding to the sentence in the text piece, a phase vector corresponding to the text piece is obtained according to the initial vector, and the phase vector is converted into vector identification data.

7. The AI-based information retrieval method according to claim 6, characterized in that: The formula for obtaining the vector identification data is: ， in, Represents vector identification data, represents the number of sub-codes in the phase vector, represents the number of dimensions, Represents the phase vector.

8. The AI-based information retrieval method according to claim 1, characterized in that: The calculation formula of the vector similarity is: ， in, represents the vector similarity, Represents the component of the vector identification data in the i-th dimension, Represents the component of the ith dimension of the retrieval vector data. Indicates the number of dimensions.

9. The AI-based information retrieval method according to claim 1, characterized in that: The interactive behavior features include retrieval association features, user association features, historical browsing features and content quality features.

10. An AI-based information retrieval system, applied to the AI-based information retrieval method according to any one of claims 1 to 9, characterized in that: The system comprises: An optimization module, used to construct an index library corresponding to the system help document, obtain initial question data, and obtain final question data based on the initial question data and the index library; A conversion module, used for dividing the system help document into a plurality of text pieces, performing vector conversion on the text pieces to obtain vector identification data corresponding to the text pieces, and combining the vector identification data and the text pieces into a database; A linking module, used to obtain retrieval vector data corresponding to the final question data, obtain vector similarity between the retrieval vector data and the vector identification data, and select a plurality of first target text segments from the plurality of text segments according to the vector similarity; An execution module is used to select a number of second target text pieces from the system help document through the initial question data, extract the interactive behavior features of the first target text piece and the second target text piece, sort and score the number of first target text pieces and the number of second target text pieces based on the interactive behavior features, form a candidate queue through the sorting and scoring, and feed back the index result.

Images