Information retrieval and guidance method and device based on big data software system

Abstract

The invention discloses an information retrieval and guidance method and device based on a big data software system, relates to the technical field of data processing, and solves the technical problem that retrieval results often contain a large amount of irrelevant or redundant information due to lack of deep analysis of retrieval problems and effective integration and screening of the retrieval results. According to the method, the retrieval problem is finely split according to the subject content, the retrieval intention of the user can be more accurately understood, compared with a traditional retrieval mode based on overall keywords, the retrieval pertinence and accuracy are improved, the situation that the retrieval result is disordered and inaccurate due to general use of the keywords is avoided, and the retrieval efficiency is improved. According to the semantic-based analysis method, information with similar contents can be identified more accurately, the limitation of matching only based on vocabularies traditionally is overcome, redundant information is effectively reduced, the retrieval result is more refined and valuable, the efficiency of obtaining useful information by a user is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and device for information retrieval and guidance based on a big data software system. Background Art

[0002] In the field of information retrieval, with the explosive growth of data volume, traditional information retrieval methods face many challenges. When faced with massive amounts of information, users often find it difficult to quickly and accurately obtain highly relevant and comprehensive information to the retrieval question.

[0003] The patent application with publication number CN115510085A discloses a method and device for information retrieval and guidance based on a big data software system, which relates to the technical field of data processing, including: obtaining keywords input by a user; processing the keywords using a preset tag attribute model to obtain keyword retrieval results; wherein the retrieval results include: exact matching function field information or fuzzy matching function field information; the function field information includes: the label of the function field, the name of the function field, the meaning of the function field and the uniform resource locator URL of the function field; in response to the user's access operation to the URL, jumping to the management page corresponding to the function field, so as to guide the user to use the management function corresponding to the function field in the big data software system based on the management page.

[0004] However, most of the existing search systems are based on simple keyword matching, lacking in-depth analysis of search questions and effective integration and screening of search results, resulting in search results often containing a large amount of irrelevant or redundant information, and users need to spend a lot of time and energy to identify useful content from the complex results. Moreover, for complex search needs, it is impossible to effectively explore the relationship between different parts, and it is difficult to provide systematic and targeted information, which cannot well meet the growing needs of users for accurate information acquisition in academic research, business decision-making, professional and technical exploration, etc. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method and device for information retrieval and guidance based on a big data software system, which solves the problem that the retrieval results often contain a large amount of irrelevant or redundant information due to the lack of in-depth analysis of the retrieval questions and the effective integration and screening of the retrieval results.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for information retrieval and guidance based on a big data software system, the method comprising the following steps: Step 1: Acquire the user's search information, split the search question according to the search information to obtain split search content, match the obtained split search content with the database to obtain split matching results, and filter the split matching results to obtain the same matching results; Step 2: Obtain the remaining split matching results, perform correlation content analysis to obtain related content, and integrate the related content to obtain the results to be analyzed; Step 3: Comprehensively analyze the obtained identical matching results and the results to be analyzed to obtain comprehensive analysis results, integrate the comprehensive analysis results corresponding to the same semantics to obtain secondary classification results, and analyze the repeated information corresponding to the secondary classification results to generate repeated analysis results; Step 4: Analyze the obtained duplicate analysis results, filter them by calculating the relevance between the content of the duplicate analysis results and the search question, and sort them from large to small according to the calculated relevance to generate search information.

[0007] As a further solution of the present invention, the specific method of obtaining the same matching result in step 1 is: The search information of the searching user is obtained, and then the search question is split according to the subject content to obtain split search content. The split search content is obtained and labeled as n, and n=1, 2, ..., m, where m represents the number of split search content. Then, the split search content n is searched and matched with the corresponding database respectively, and the corresponding split matching results are obtained. The split matching results corresponding to different split search contents are obtained, and then the split matching results are classified to obtain the same matching results.

[0008] As a further solution of the present invention, the specific method of obtaining the result to be analyzed in step 2 is: Obtain all remaining split matching results, and distinguish and mark the remaining split matching results as split matching results a, where a=A, B, ..., where a represents the type of split search content corresponding to the remaining split matching results, then obtain any type of split matching results as the analysis object, and record the corresponding matching results in the analysis object as i, where i=1, 2, ..., j, where j represents the number of matching results, and at the same time identify the matching content of matching result i, then perform correlation matching on matching result i and the database to obtain the related content, and so on, analyze all split matching results to obtain the corresponding related content; Then, similar analysis is performed on the associated contents corresponding to the different split matching results, and the associated contents with similar contents are screened to generate the results to be analyzed.

[0009] As a further solution of the present invention, the specific method of obtaining the secondary classification result in step 3 is: Get all the identical matching results and the results to be analyzed, and combine them to get the comprehensive analysis result, which is labeled as c, and c=1, 2, ..., r, where r represents the number of comprehensive analysis results. Then, the content semantics corresponding to the comprehensive analysis result c is obtained, and the similarity of the content semantics is calculated. The comprehensive analysis results with the same semantics are integrated to get the secondary classification result. At the same time, the number of results corresponding to the secondary classification result is obtained, and the number proportion corresponding to the secondary classification result is calculated. Then, they are sorted from large to small according to the number proportion.

[0010] As a further solution of the present invention, the specific method of obtaining the repeated analysis results in step 3 is: Obtain all secondary classification results, and at the same time obtain any group of secondary classification results as the target object, and obtain the corresponding comprehensive analysis results in the target object, then extract the repeated information in the comprehensive analysis results, and obtain the number of repeated information corresponding to the comprehensive analysis results in the target object, and obtain the corresponding comprehensive analysis results based on the repeated information as the standard and record them as the same results, and so on, analyze all the repeated information and generate the corresponding same results; Then, the types of repeated information of the same result are obtained, and if there is only one type of repeated information corresponding to the same result, the same result is eliminated. If there are multiple types of repeated information corresponding to the same result, the comprehensive analysis results corresponding to the multiple types of repeated information are obtained and retained, and the remaining comprehensive analysis results of the same result are eliminated to generate repeated analysis results.

[0011] As a further solution of the present invention, the specific method of generating the search information in step 4 is: Obtain all repeated analysis results and label them as k, where k=1, 2, ..., h, where h represents the number of repeated analysis results. Then obtain the search question, and use the search question as a standard to obtain the same words in the repeated analysis results, and obtain the similarity value between the repeated analysis results and the search question. Then substitute the number of identical words and the similarity value obtained into the formula Q=(S+G)×u to calculate the relevance of the repeated analysis results, where u is the preset proportional coefficient, S is the number of identical words, and G is the similarity value. Similarly, the correlation Qk corresponding to all repeated analysis results is calculated, and the mean of all correlations and values ​​is calculated. At the same time, the correlation Qk is screened based on the mean, and the repeated analysis results with correlation Qk less than the correlation mean are eliminated, and the repeated analysis results with correlation Qk greater than the correlation mean are retained. Then, they are sorted from large to small according to the correlation Qk to generate retrieval information.

[0012] An information retrieval and guidance device based on a big data software system, comprising an information acquisition module, an information matching and analysis module, a comprehensive analysis and processing module, and a retrieval information output module; The information acquisition module is used to acquire the user's search information and transmit the search information to the information matching analysis module; An information matching analysis module, which splits the search question according to the search information to obtain split search contents, matches the split search contents with the database to obtain split matching results, and simultaneously screens the split matching results to obtain identical matching results, and simultaneously performs correlation content analysis to obtain related contents, and integrates the related contents to obtain the results to be analyzed, and then transmits the results to be analyzed to the comprehensive analysis processing module; A comprehensive analysis processing module, which performs comprehensive analysis on the obtained identical matching results and the results to be analyzed to obtain comprehensive analysis results, integrates the comprehensive analysis results corresponding to the same semantics to obtain secondary classification results, and analyzes the repeated information corresponding to the secondary classification results to generate repeated analysis results, and screens the repeated analysis results by calculating the relevance between the content and the search question, and sorts them from large to small according to the calculated relevance to generate search information, and transmits the search information to the search information output module; The search information output module is used to display the acquired search information to the corresponding operator.

[0013] The present invention provides a method and device for information retrieval and guidance based on a big data software system. Compared with the prior art, it has the following beneficial effects: The present invention can more accurately understand the user's search intention by finely splitting the search questions according to the subject content. Compared with the traditional search method based on overall keywords, it improves the pertinence and accuracy of the search and avoids the confusion and inaccuracy of the search results caused by the general use of keywords. The semantic-based analysis method can more accurately identify information with similar content, overcome the limitations of traditional methods based only on vocabulary matching, effectively reduce redundant information, and make the search results more refined and valuable. Through the correlation content analysis, potential related content is mined and integrated into the results to be analyzed. It can reintegrate and utilize relevant information that may have been ignored, broaden the scope of users' access to information, and provide users with a more comprehensive and in-depth knowledge system.

[0014] By calculating the relevance of the content of repeated analysis results to the search questions, and filtering and sorting them based on the relevance, we ensure that the search information presented to users is highly relevant to the original search questions in a scientific and quantitative way, thereby improving the efficiency of users in obtaining useful information and improving the user experience. Traditional technologies often find it difficult to effectively quantify and sort the relevance of search results to questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a step method diagram of the present invention; Figure 2 This is a block diagram of the system principle of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] For example, see Figure 1 The present application provides a method for information retrieval and guidance based on a big data software system, which specifically includes the following steps: Step 1: Acquire the user's search information, and split the search question according to the search information to obtain split search content, match the obtained split search content with the database to obtain split matching results, and filter according to the split matching results to obtain the same matching results.

[0018] First, the search information of the searching user is obtained, and the search information includes the search question corresponding to the user. Then, the search question is split according to the subject content to obtain the split search content.

[0019] For example, if the search information input by the user is "research materials on the application status and development trends of artificial intelligence in the medical field", it can be finely split according to the internal logic and semantic association of the subject content, and can be split into multiple split search contents such as "the application status of artificial intelligence in the medical field", "the development trend of artificial intelligence in the medical field" and "research materials on the medical application of artificial intelligence".

[0020] The split search content is obtained and labeled as n, and n=1, 2, ..., m, where m represents the number of split search content. Then, the split search content n is searched and matched with the corresponding database respectively, and the corresponding split matching results are obtained. The split matching results obtained here are the matching results corresponding to different split search contents separately. The split matching results corresponding to different split search contents are obtained, and then the split matching results are classified to obtain the same matching results. The classification processing here specifically classifies the split matching results with similar contents. The specific content similarity is to calculate the similarity value of the content, which is calculated by the cosine similarity calculation method.

[0021] Step 2: Obtain the remaining split matching results, perform correlation content analysis to obtain related content, and integrate the related content to obtain the results to be analyzed.

[0022] Obtain all remaining split matching results, and the remaining split matching results here are represented as the remaining split matching results after removing the same matching results, specifically including different split matching results corresponding to the split search content, for example, if there are two split search contents, then the corresponding different split matching results are the matching results corresponding to the two split search contents, and distinguish the remaining split matching results and mark them as split matching results a, and a=A, B, ..., the specific matching results here have the same meaning as the remaining split matching results, wherein a represents the type of split search content corresponding to the remaining split matching results, then obtain any type of split matching results as the analysis object, and record the corresponding matching results in the analysis object as i, and i=1, 2, ..., j, wherein j represents the number of matching results, and at the same time identify the matching content of matching result i, then perform correlation matching on matching result i and the database, and the correlation matching here is represented as performing correlation matching on the corresponding content to obtain the related content, and so on, analyze all split matching results to obtain the corresponding related content; Then, similar analysis is performed on the associated contents corresponding to the different split matching results, and the associated contents with similar contents are screened to generate the results to be analyzed.

[0023] Step 3: Perform a comprehensive analysis on the identical matching results and the results to be analyzed to obtain a comprehensive analysis result, integrate the comprehensive analysis results corresponding to the same semantics to obtain a secondary classification result, and analyze the repeated information corresponding to the secondary classification result to generate a repeated analysis result.

[0024] Obtain all identical matching results and results to be analyzed, and combine the two to obtain a comprehensive analysis result, which is labeled c, and c=1, 2, ..., r, where r represents the number of comprehensive analysis results. Then, the content semantics corresponding to the comprehensive analysis result c is analyzed, and the similarity of the content semantics is calculated. The comprehensive analysis results with the same semantics are integrated to obtain a secondary classification result, and the secondary classification result here is specifically expressed as a comprehensive analysis result with the same semantic expression content. At the same time, the number of results corresponding to the secondary classification result is obtained, and the number proportion corresponding to the secondary classification result is calculated, and then the results are sorted from large to small according to the number proportion; All secondary classification results are obtained, and any group of secondary classification results is obtained as a target object, and the corresponding comprehensive analysis results in the target object are obtained, and then the repeated information in the comprehensive analysis results is extracted, and the repeated information here is represented by the same information existing in different comprehensive analysis results, and specifically can be the same information existing in two groups of comprehensive analysis results, or three groups, etc., and the number of repeated information corresponding to the comprehensive analysis results in the target object is obtained, and the number of types of repeated information is obtained here, and the corresponding comprehensive analysis results obtained with the repeated information as the standard are recorded as the same results, and the same results generated here are represented as comprehensive analysis results with the same repeated information, and so on, all repeated information is analyzed, and the corresponding same results are generated; Then, the types of repeated information of the same result are obtained, and if there is only one type of repeated information corresponding to the same result, the same result is eliminated. If there are multiple types of repeated information corresponding to the same result, the comprehensive analysis results corresponding to the multiple types of repeated information are obtained and retained, and the remaining comprehensive analysis results of the same result are eliminated to generate repeated analysis results.

[0025] Specifically, for the comprehensive analysis results with multiple types of repeated information in the same result, for example, there are five groups, and each group corresponds to a different type of repeated information. In this case, the comprehensive analysis result with the most types of repeated information is selected as the standard for retention, and the rest are directly discarded.

[0026] Step 4: Analyze the obtained duplicate analysis results, filter them by calculating the relevance between the content of the duplicate analysis results and the search question, and sort them from large to small according to the calculated relevance to generate search information.

[0027] Obtain all repeated analysis results and label them as k, where k=1, 2, ..., h, where h represents the number of repeated analysis results. Then obtain the search question, and use the search question as a standard to obtain the same words in the repeated analysis results, where the same words here are represented as word types, and obtain the similarity between the repeated analysis results and the search question, and the similarity is calculated using the Euclidean distance formula. Then substitute the number of identical words and the similarity into the formula Q=(S+G)×u to calculate the relevance of the repeated analysis results, where u is a preset proportional coefficient, S is the number of identical words, and G is the similarity value. Similarly, the correlation Qk corresponding to all repeated analysis results is calculated, and the mean of all correlations and values ​​is calculated. At the same time, the correlation Qk is screened based on the mean, and the repeated analysis results with correlation Qk less than the correlation mean are eliminated, and the repeated analysis results with correlation Qk greater than the correlation mean are retained. Then, they are sorted from large to small according to the correlation Qk, and the search information is generated and displayed to the corresponding operator.

[0028] For example 2, please refer to Figure 2 The present application provides an information retrieval and guidance device based on a big data software system, which specifically includes an information acquisition module, an information matching analysis module, a comprehensive analysis and processing module, and a retrieval information output module.

[0029] An information collection module, which is used to obtain the user's search information and transmit the search information to the information matching analysis module; An information matching analysis module, which splits the search question according to the search information to obtain split search contents, matches the split search contents with the database to obtain split matching results, and simultaneously screens the split matching results to obtain identical matching results, and simultaneously performs correlation content analysis to obtain related contents, and integrates the related contents to obtain results to be analyzed, and then transmits the results to be analyzed to the comprehensive analysis processing module, and the processing method here is the same as the processing process of step 2 and step 3 in embodiment 1; A comprehensive analysis processing module, which performs comprehensive analysis on the obtained identical matching results and the results to be analyzed to obtain comprehensive analysis results, integrates the comprehensive analysis results corresponding to the same semantics to obtain secondary classification results, and analyzes the repeated information corresponding to the secondary classification results to generate repeated analysis results, and screens the repeated analysis results by calculating the relevance between the content and the search question, and sorts them from large to small according to the calculated relevance and generates search information, and transmits the search information to the search information output module, and the processing method here is the same as the processing process of step three and step four in embodiment one; The search information output module is used to display the acquired search information to the corresponding operator.

[0030] Some of the data in the above formulas are numerically calculated by removing their dimensions. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0031] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for information retrieval and guidance based on a big data software system, characterized in that: The method comprises the following steps: Step 1: Acquire the user's search information, split the search question according to the search information to obtain split search content, match the obtained split search content with the database to obtain split matching results, and filter the split matching results to obtain the same matching results; Step 2: Obtain the remaining split matching results, perform correlation content analysis to obtain related content, and integrate the related content to obtain the results to be analyzed; Step 3: Comprehensively analyze the obtained identical matching results and the results to be analyzed to obtain comprehensive analysis results, integrate the comprehensive analysis results corresponding to the same semantics to obtain secondary classification results, and analyze the repeated information corresponding to the secondary classification results to generate repeated analysis results; Step 4: Analyze the obtained duplicate analysis results, filter them by calculating the relevance between the content of the duplicate analysis results and the search question, and sort them from large to small according to the calculated relevance to generate search information.

2. The method for information retrieval and guidance based on a big data software system according to claim 1, characterized in that: The specific method of obtaining the same matching result in step 1 is: The search information of the searching user is obtained, and then the search question is split according to the subject content to obtain split search content. The split search content is obtained and labeled as n, and n=1, 2, ..., m, where m represents the number of split search content. Then, the split search content n is searched and matched with the corresponding database respectively, and the corresponding split matching results are obtained. The split matching results corresponding to different split search contents are obtained, and then the split matching results are classified to obtain the same matching results.

3. The method for information retrieval and guidance based on a big data software system according to claim 1, characterized in that: The specific method of obtaining the result to be analyzed in step 2 is: Obtain all remaining split matching results, and distinguish and mark the remaining split matching results as split matching results a, where a=A, B, ..., where a represents the type of split search content corresponding to the remaining split matching results, then obtain any type of split matching results as the analysis object, and record the corresponding matching results in the analysis object as i, where i=1, 2, ..., j, where j represents the number of matching results, and at the same time identify the matching content of matching result i, then perform correlation matching on matching result i and the database to obtain the related content, and so on, analyze all split matching results to obtain the corresponding related content; Then, similar analysis is performed on the associated contents corresponding to the different split matching results, and the associated contents with similar contents are screened to generate the results to be analyzed.

4. The method for information retrieval and guidance based on a big data software system according to claim 1, characterized in that: The specific method of obtaining the secondary classification result in step 3 is: Get all the identical matching results and the results to be analyzed, and combine them to get the comprehensive analysis result, which is labeled as c, and c=1, 2, ..., r, where r represents the number of comprehensive analysis results. Then, the content semantics corresponding to the comprehensive analysis result c is obtained, and the similarity of the content semantics is calculated. The comprehensive analysis results with the same semantics are integrated to get the secondary classification result. At the same time, the number of results corresponding to the secondary classification result is obtained, and the number proportion corresponding to the secondary classification result is calculated. Then, they are sorted from large to small according to the number proportion.

5. The method for information retrieval and guidance based on a big data software system according to claim 1, characterized in that: The specific method of obtaining the repeated analysis results in step 3 is: Obtain all secondary classification results, and at the same time obtain any group of secondary classification results as the target object, and obtain the corresponding comprehensive analysis results in the target object, then extract the repeated information in the comprehensive analysis results, and obtain the number of repeated information corresponding to the comprehensive analysis results in the target object, and obtain the corresponding comprehensive analysis results based on the repeated information as the standard and record them as the same results, and so on, analyze all the repeated information and generate the corresponding same results; Then, the types of repeated information of the same result are obtained, and if there is only one type of repeated information corresponding to the same result, the same result is eliminated. If there are multiple types of repeated information corresponding to the same result, the comprehensive analysis results corresponding to the multiple types of repeated information are obtained and retained, and the remaining comprehensive analysis results of the same result are eliminated to generate repeated analysis results.

6. The method for information retrieval and guidance based on a big data software system according to claim 1, characterized in that: The specific method of generating the search information in step 4 is: Obtain all repeated analysis results and label them as k, where k=1, 2, ..., h, where h represents the number of repeated analysis results. Then obtain the search question, and use the search question as a standard to obtain the same words in the repeated analysis results, and obtain the similarity value between the repeated analysis results and the search question. Then substitute the number of identical words and the similarity value obtained into the formula Q=(S+G)×u to calculate the relevance of the repeated analysis results, where u is the preset proportional coefficient, S is the number of identical words, and G is the similarity value. Similarly, the correlation Qk corresponding to all repeated analysis results is calculated, and the mean of all correlations and values ​​is calculated. At the same time, the correlation Qk is screened based on the mean, and the repeated analysis results with correlation Qk less than the correlation mean are eliminated, and the repeated analysis results with correlation Qk greater than the correlation mean are retained. Then, they are sorted from large to small according to the correlation Qk to generate retrieval information.

7. An information retrieval and guidance device based on a big data software system, used to execute the information retrieval and guidance method based on a big data software system according to any one of claims 1 to 6, characterized in that: It includes information collection module, information matching and analysis module, comprehensive analysis and processing module and retrieval information output module.

8. The device for information retrieval and guidance based on a big data software system according to claim 7, characterized in that: The information acquisition module is used to acquire the user's search information and transmit the search information to the information matching analysis module; An information matching analysis module, which splits the search question according to the search information to obtain split search contents, matches the split search contents with the database to obtain split matching results, and simultaneously screens the split matching results to obtain identical matching results, and simultaneously performs correlation content analysis to obtain related contents, and integrates the related contents to obtain the results to be analyzed, and then transmits the results to be analyzed to the comprehensive analysis processing module; A comprehensive analysis processing module, which performs comprehensive analysis on the obtained identical matching results and the results to be analyzed to obtain comprehensive analysis results, integrates the comprehensive analysis results corresponding to the same semantics to obtain secondary classification results, and analyzes the repeated information corresponding to the secondary classification results to generate repeated analysis results, and screens the repeated analysis results by calculating the relevance between the content and the search question, and sorts them from large to small according to the calculated relevance to generate search information, and transmits the search information to the search information output module; The search information output module is used to display the acquired search information to the corresponding operator.

Images