Article recall sorting method, intelligent question and answer processing method and computing equipment

Abstract

The embodiment of the invention discloses an article recall sorting method, which comprises the following steps of: obtaining a question statement, and extracting a keyword; classifying the keywords by adopting an intelligent model to obtain field-related and irrelevant keywords; the keywords related to the field are matched with the abstracts to obtain a plurality of articles, and the articles are segmented; and performing semantic similarity sorting through vector splicing and similarity calculation. The method has the advantages that 1, the abstract generation and tagging technology is creatively applied, the database content is intelligently simplified, the data processing amount in the retrieval stage is remarkably reduced, the accuracy of preliminary screening is greatly improved, the efficiency bottleneck caused by traditional full-amount data scanning is effectively avoided, and high efficiency and accuracy of information retrieval are ensured; and 2, a multi-condition sorting algorithm is introduced, advanced technologies such as vector splicing and comparative analysis are fused, deep optimization of information extraction is realized, a most relevant result can be quickly positioned in mass data, and the quality and correlation of a final retrieval result are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer application and software technology, and in particular to an article recall sorting method, an intelligent question and answer processing method and a computing device. Background Art

[0002] The intelligent question-answering system should be able to understand semantics and automatically tag data, and can find the corresponding data and sources for the keywords and process descriptions of various events. At the same time, the system should have the ability to conduct association analysis, and realize the association between events, knowledge, and cases based on the knowledge base and case library, so as to assist users in conducting event analysis and data mining. In addition, the system can combine information systems, information retrieval technology, and large language models, and can assist in generating question-answering results in data analysis based on some common analysis processes and solutions.

[0003] Question-answering processing involves the following steps: first, data is preprocessed, text segmentation, vectorization, and keyword extraction are performed, and then the processed data is entered into the database. When a user asks a question, the data in the database is recalled and sorted, and finally an answer is generated.

[0004] When recalling and sorting articles in a database, the original method used is roughly as follows: first segment the text, then recall the articles in the database through keyword matching and vectorization, and then integrate and sort the recalled documents according to keyword weight and similarity. This ensures that documents closely related to the query topic are ranked first, improving the retrieval effect.

[0005] However, this approach has certain limitations. First, in the scenario of private deployment, due to the small number of users and short query terms, it is impossible to obtain a reliable keyword weight calculation method through word frequency and other methods. Second, recalling from a large number of article paragraphs only through simple keywords and semantics will cause serious information interference and it is impossible to obtain accurate candidate content. For example, "What is the test temperature of GNSS test instruments?" The keywords include "GNSS" and "temperature". If the keywords "GNSS" and "temperature" have the same weight, during the recall process, a large amount of content recalled based on "temperature" will drown out the information covered by "GNSS", and cause the content related to the correct answer in the final candidate results not to be recalled. Summary of the invention

[0006] In view of the technical defects pointed out in the background technology, the purpose of the embodiments of the present invention is to provide a recall sorting method for database articles, an intelligent question and answer processing method and a computing device.

[0007] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides a method for recalling and sorting articles in a database, comprising:

[0008] Obtain a question sentence, and extract multiple keywords from the question sentence;

[0009] Use the trained intelligent model to classify multiple keywords to obtain domain-related keywords and domain-irrelevant keywords;

[0010] Matching field-related keywords with abstracts to obtain multiple articles; the abstracts correspond to each article in the database;

[0011] Segment multiple articles by semantic aggregation relationship or specifying paragraph length to obtain multiple article paragraphs;

[0012] Through vector concatenation and similarity calculation, the semantic similarity of multiple article paragraphs is sorted by comparing with the question statement.

[0013] As a preferred implementation of the present application, before obtaining the question statement, the method further includes:

[0014] Use the big model to summarize each article in the database and get a summary;

[0015] The summary is stored in the database.

[0016] As a specific implementation of the present application, multiple article paragraphs are sorted by semantic similarity, specifically:

[0017] Introducing sorting conditions, wherein the sorting conditions include paragraph semantics, article title and publishing time;

[0018] Performing vector extraction on each of the sorting conditions to obtain a plurality of sub-vectors;

[0019] Concatenate multiple sub-vectors to obtain a concatenated vector;

[0020] The cosine similarity between the mother vector and the question semantics is calculated, and the multiple article paragraphs are sorted by semantic similarity in comparison with the question sentence; the question semantics is the question sentence or the question sentence processed by the large model.

[0021] In a second aspect, an embodiment of the present application provides an intelligent question-answering processing method, including:

[0022] Use the big model to summarize each article in the database and get a summary;

[0023] Obtain a question statement, and obtain multiple articles according to the question statement and the abstract;

[0024] Segment multiple articles by semantic aggregation relationship or specifying paragraph length to obtain multiple article paragraphs;

[0025] By using vector concatenation and similarity calculation methods, multiple article paragraphs are sorted by semantic similarity in comparison with the question statements.

[0026] Multiple article paragraphs and question statements are input into the large model to obtain intelligent question-answering results.

[0027] Furthermore, as a preferred implementation of the present application, before inputting the plurality of article paragraphs and question statements into the large model, the method further includes:

[0028] Filter multiple article paragraphs.

[0029] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the first aspect above.

[0030] In a fourth aspect, an embodiment of the present application provides another computer device, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method of the above-mentioned second aspect.

[0031] The implementation of the embodiment of the present invention has the following advantages:

[0032] 1. The embodiment of the present invention creatively uses summary generation and labeling technology to intelligently simplify the database content, significantly reducing the amount of data processing in the retrieval stage, thereby greatly speeding up and enhancing the accuracy of preliminary screening. This strategy effectively avoids the efficiency bottleneck caused by traditional full data scanning and ensures efficient and accurate information retrieval.

[0033] 2. A multi-condition sorting algorithm was introduced, which integrated cutting-edge technologies such as vector concatenation and comparative analysis to achieve deep optimization of information extraction. This mechanism can quickly locate the most relevant results in massive data, greatly improving the quality and relevance of the final search results. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art will be briefly introduced below.

[0035] Figure 1 is a flow chart of an article recall ranking method provided by an embodiment of the present invention;

[0036] Figure 2 It is a flow chart of vector concatenation and similarity calculation;

[0037] Figure 3 is a flow chart of an intelligent question-answering processing method provided by an embodiment of the present invention;

[0038] Figure 4 It is a structural diagram of a computing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0041] The inventive concept of the present invention is:

[0042] Use the big model to first summarize each article in the database, and then use the big model to filter out articles with high relevance. The method is also keyword weight, similarity and vectorization, and then sort the selected article paragraphs.

[0043] Please refer to Figure 1 , the article recall ranking method provided by the first embodiment of the present invention includes:

[0044] S101, using the big model to summarize each article in the database, obtain a summary, and store it in the database.

[0045] The inventor of the present application found that the source of the problem described in the background technology is that the amount of data is too large, so a solution can be provided around how to reduce the amount of data. In this embodiment, a large model is used to summarize each article in the database into a summary of about 200 words.

[0046] This step can be understood as data preprocessing. By generating a summary, it is prepared for the subsequent calculation of semantic similarity between the question and the summary, reducing the size of the candidate set during the recall process, and ensuring the accurate extraction of correct reference information.

[0047] S102, obtaining a question statement, and extracting a plurality of keywords from the question statement.

[0048] S103, using the trained intelligent model to classify multiple keywords to obtain domain-related keywords and domain-irrelevant keywords.

[0049] S104, matching the field-related keywords with the abstracts to obtain multiple articles.

[0050] Steps S102-S104 can be understood as retrieval steps. Taking the question sentence "What is the test temperature of the GNSS test instrument?" as an example, first build a private application vocabulary and demand description, extract the keywords "GNSS" and "temperature", and use the trained intelligent model to classify the keywords into domain-related keywords (such as "GNSS") and domain-irrelevant keywords (such as "temperature"). The domain-related keywords are used as a reference for article summary matching, that is, the keyword "GNSS" is matched with the abstract in the database to obtain domain-related content to achieve the purpose of reducing the amount of subsequent information processing.

[0051] S105, segmenting the multiple articles according to semantic aggregation relationships or designated paragraph lengths to obtain multiple article paragraphs.

[0052] S106, using a vector concatenation method and a similarity calculation method, and comparing the question sentence, sorting the multiple article paragraphs by semantic similarity.

[0053] Step S105 specifically includes:

[0054] Introducing sorting conditions, wherein the sorting conditions include paragraph semantics, article title and publishing time;

[0055] Performing vector extraction on each of the sorting conditions to obtain a plurality of sub-vectors;

[0056] Concatenate multiple sub-vectors to obtain a concatenated vector;

[0057] The cosine similarity between the mother vector and the question semantics is calculated (such as Figure 2 As shown), compare the question sentence and sort the semantic similarity of multiple article paragraphs; the question semantics is the question sentence or the question sentence processed by the large model.

[0058] From the above description, it can be seen that the article recall and ranking method provided by the embodiment of the present invention has the following advantages:

[0059] 1. The embodiment of the present invention creatively uses summary generation and labeling technology to intelligently simplify the database content, significantly reducing the amount of data processing in the retrieval stage, thereby greatly speeding up and enhancing the accuracy of preliminary screening. This strategy effectively avoids the efficiency bottleneck caused by traditional full data scanning and ensures efficient and accurate information retrieval.

[0060] 2. A multi-condition sorting algorithm was introduced, which integrated cutting-edge technologies such as vector concatenation and comparative analysis to achieve deep optimization of information extraction. This mechanism can quickly locate the most relevant results in massive data, greatly improving the quality and relevance of the final search results.

[0061] Please refer to Figure 3 , an embodiment of the present invention provides an intelligent question-answering processing method, comprising:

[0062] S201, using the big model to summarize each article in the database to obtain a summary.

[0063] S202, obtaining a question statement, and obtaining multiple articles according to the question statement and the abstract.

[0064] Step S202 specifically includes:

[0065] Extracting multiple keywords from the question sentence;

[0066] Use the trained intelligent model to classify multiple keywords to obtain domain-related keywords and domain-irrelevant keywords;

[0067] Field-related keywords are matched with abstracts to obtain multiple articles.

[0068] S203, segmenting the multiple articles according to semantic aggregation relationships or designated paragraph lengths to obtain multiple article paragraphs.

[0069] S204, sorting the semantic similarity of multiple article paragraphs by comparing the question sentence through a vector concatenation method and a similarity calculation method.

[0070] Step S204 specifically includes:

[0071] Introducing sorting conditions, wherein the sorting conditions include paragraph semantics, article title and publishing time;

[0072] Performing vector extraction on each of the sorting conditions to obtain a plurality of sub-vectors;

[0073] Concatenate multiple sub-vectors to obtain a concatenated vector;

[0074] The cosine similarity between the mother vector and the question semantics is calculated, and the multiple article paragraphs are sorted by semantic phase velocity in comparison with the question sentence; the question semantics is the question sentence or the question sentence processed by the large model.

[0075] It should be noted that in order to meet the needs of subsequent large models, after obtaining multiple article paragraphs, they need to be screened, for example, selecting the top 20,000 as the input of the subsequent large model.

[0076] S205, multiple article paragraphs and question sentences are input into the large model to obtain intelligent question answering results.

[0077] Among them, the intelligent question and answer results include the original question statement and the answer obtained after processing by the large model.

[0078] Based on the same inventive concept, Figure 4 As shown, an embodiment of the present invention provides a computing device for recalling and sorting articles in a database, or performing intelligent question-answering processing. The computing device may include: one or more processors 101, one or more input devices 102, one or more output devices 103, and a memory 104. The processors 101, input devices 102, output devices 103, and memory 104 are interconnected via a bus 105. The memory 104 is used to store a computer program, the computer program includes program instructions, and the processor 101 is configured to call the program instructions to execute the above Figure 1 or Figure 3 Methods Methods of the Examples section.

[0079] It should be understood that in the embodiment of the present invention, the processor 101 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0080] The input device 102 may include a keyboard, etc., and the output device 103 may include a display (LCD, etc.), a speaker, etc.

[0081] The memory 104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include a non-volatile random access memory. For example, the memory 104 may also store information on the device type.

[0082] In a specific implementation, the processor 101, input device 102, and output device 103 described in the embodiments of the present invention can execute the implementation method described in the embodiments of the article recall sorting method or the intelligent question and answer processing method provided in the embodiments of the present invention, which will not be repeated here.

[0083] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for recalling and ranking articles, characterized in that: include: Obtain a question sentence, and extract multiple keywords from the question sentence; Use the trained intelligent model to classify multiple keywords to obtain domain-related keywords and domain-irrelevant keywords; Match field-related keywords with abstracts to obtain multiple articles; The abstract corresponds to each article in the database; Segment multiple articles by semantic aggregation relationship or specifying paragraph length to obtain multiple article paragraphs; Through vector concatenation and similarity calculation, the semantic similarity of multiple article paragraphs is sorted by comparing with the question statement.

2. The article recall ranking method as claimed in claim 1, characterized in that: Before obtaining the question statement, the method further includes: Use the big model to summarize each article in the database and get a summary; The summary is stored in the database.

3. The article recall ranking method as claimed in claim 1, characterized in that: Sort multiple article paragraphs by semantic similarity, specifically: Introducing sorting conditions, wherein the sorting conditions include paragraph semantics, article title and publishing time; Performing vector extraction on each of the sorting conditions to obtain a plurality of sub-vectors; Concatenate multiple sub-vectors to obtain a concatenated vector; The cosine similarity between the mother vector and the question semantics is calculated, and the multiple article paragraphs are sorted by semantic similarity in comparison with the question sentence; the question semantics is the question sentence or the question sentence processed by the large model.

4. An intelligent question-answering processing method, characterized in that: include: Use the big model to summarize each article in the database and get a summary; Obtain a question statement, and obtain multiple articles according to the question statement and the abstract; Segment multiple articles by semantic aggregation relationship or specifying paragraph length to obtain multiple article paragraphs; By using vector concatenation and similarity calculation methods, multiple article paragraphs are sorted by semantic similarity in comparison with the question statements. Multiple article paragraphs and question statements are input into the large model to obtain intelligent question-answering results.

5. The intelligent question-answering processing method according to claim 4, characterized in that: According to the problem statement and abstract, multiple articles are obtained, specifically: Extracting multiple keywords from the question sentence; Use the trained intelligent model to classify multiple keywords to obtain domain-related keywords and domain-irrelevant keywords; Field-related keywords are matched with abstracts to obtain multiple articles.

6. The intelligent question-answering processing method according to claim 4, characterized in that: Sort multiple article paragraphs by semantic similarity, specifically: Introducing sorting conditions, wherein the sorting conditions include paragraph semantics, article title and publishing time; Performing vector extraction on each of the sorting conditions to obtain a plurality of sub-vectors; Concatenate multiple sub-vectors to obtain a concatenated vector; The cosine similarity between the mother vector and the question semantics is calculated, and the multiple article paragraphs are sorted by semantic phase velocity in comparison with the question sentence; the question semantics is the question sentence or the question sentence processed by the large model.

7. The intelligent question-answering processing method according to claim 4, characterized in that: Before inputting the plurality of article paragraphs and question statements into the large model, the method further comprises: screening the plurality of article paragraphs.

8. A computing device for recalling and sorting articles in a database, characterized in that: The computing device includes a processor, an input device, an output device and a memory, which are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 3.

9. A computing device for performing intelligent question-answering processing, characterized in that: The computing device includes a processor, an input device, an output device and a memory, which are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 4 to 7.

Images