Method and device for evaluating novelty and credibility of novelty search point under multi-model cooperation

Abstract

The invention discloses a novelty evaluation method and device for novelty search points under semantic understanding of large and small models, and the method comprises the steps: extracting novelty search keywords according to the novelty search points, constructing a retrieval formula, and screening out related literatures in a novelty search database according to the retrieval formula; extracting related steps and summarizing related points from the screened literature abstracts by taking novelty search points as a reference template; the method comprises the following steps: extracting structured semantic information from novelty search points and related step summaries, and aligning similar expressions; according to the structured semantic information, calculating the similarity distance of each dimension between the novelty search point and the related point, and calculating the novelty score of the novelty search point; and generating a comparison result by adopting a large language model according to the score in combination with the novelty search points and the content of the similar fragments, dynamically constructing a prompt template according to the comparison result, generating a summarized novelty search conclusion, and automatically generating a readable result text. According to the method, the comprehensiveness, accuracy and readability of novelty evaluation of novelty search points can be improved, and the illusion risk of a large model is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer technology, and specifically relates to a method and device for evaluating the novelty of a query point under multi-model collaboration. Background Art

[0002] Scientific and technological novelty search is based on the search points that reflect the subject content of the search project, with computer search as the main means, and the acquisition of closely related literature as the search goal. It uses comprehensive analysis and comparison methods to evaluate the novelty of the search project. Scientific and technological novelty search plays a role in providing objective evidence in avoiding duplicate scientific research project establishment, project results acceptance, scientific and technological achievement evaluation, and has made important contributions to supporting my country's scientific and technological development and promoting scientific and technological progress.

[0003] With the gradual improvement of the scientific and technological novelty search system, the digital resources generated by the novelty search process are constantly increasing, which provides a sufficient data foundation for intelligence application scenarios assisted by big model technology. The use of big language models can help intelligence workers improve the efficiency of multiple links such as document retrieval, reading, analysis and comparison in the process of scientific and technological novelty search. Therefore, a reliable evaluation method for the novelty of novelty search points based on big model understanding and semantic relevance calculation is needed to assist the efficient implementation of scientific and technological novelty search. Summary of the invention

[0004] One purpose of the present invention is to address the shortcomings of the prior art and provide a credible evaluation method for the novelty of novelty search points under multi-model collaboration, which can improve the comprehensiveness, accuracy and readability of the novelty evaluation conclusions of the novelty search points and avoid the risk of large model hallucination.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A method for evaluating the credibility of novelty of a query point under multi-model collaboration includes the following steps:

[0007] S1, extract novelty search keywords according to the novelty search point and construct a search formula, and screen out relevant documents in the novelty search database according to the search formula;

[0008] S2, using the search point as a reference template, extract relevant steps and summarize relevant points from the literature abstracts screened in step S1;

[0009] S3, extracts structured semantic information from the search points and related step overviews, and aligns similar expressions;

[0010] S4, based on the structured semantic information, calculate the similarity distances of each dimension between the query point and the related points, and calculate the novelty score of the query point;

[0011] S5, using a large language model to generate a comparison result based on the score of step 4 combined with the content of the search point and similar fragments, and dynamically build a prompt template based on the comparison result, generate a summary of the search conclusion, and automatically generate a readable result text.

[0012] Furthermore, step S1 specifically includes:

[0013] S11, input the search point, use the large language model to expand the topic and keywords, and build a search formula based on the prompt template generated by the search conditions;

[0014] S12, in the novelty search database, using S11 to generate a search query to obtain a set of candidate documents;

[0015] S13, based on text similarity, compare the search point and the abstract part of the candidate document in S12 to obtain a set of related document abstracts with a similarity higher than a set threshold.

[0016] Furthermore, the specific implementation of step S2 is as follows:

[0017] S21, according to the content characteristics of the novelty search point, extract relevant steps from each step of the relevant literature abstract in S1 using a large language model;

[0018] S22, based on the expression characteristics of the novelty search point, use the large language model to outline the relevant steps in an approximate style and summarize the relevant points.

[0019] Furthermore, the implementation of S3 includes:

[0020] S31, using the text extraction capability of the large language model, extracting short-sentence-sized structured semantic information from the search points and the related points of the comparative literature in step S2, wherein the structured semantic information includes the research object, research method, and research question;

[0021] S32, based on the structured semantic dimension of the novelty query point, use the large language model to rewrite the structured sentences of the relevant literature in the style of the structured sentences of the novelty query point.

[0022] Furthermore, the multiple dimensions in step 4 include research methods, research questions, and research objects.

[0023] Furthermore, the method for calculating the novelty score of the query point in step 4 is:

[0024] Based on the aligned structured semantic information, the pre-trained BERT model is used to map the text into a high-dimensional vector representation, and the cosine similarity distance between the query point and the related point vectors in each dimension is calculated. When calculating, the fine-grained similarity of each dimension is first determined, and the most relevant vector of each dimension of the related document is found for each single vector of each dimension of the query point as the fine-grained similarity distance result, and then the similarity distance of each dimension vector is averaged as the overall similarity distance of a single dimension.

[0025] After obtaining the similarity distances in each dimension, the novelty score of the query point is calculated. The calculation formula for the novelty score of the query point is as follows:

[0026] D C =1-cosθ;

[0027] D C (rp i,j )=w m *D C,m +w p *D C,p +w o *D C,o ;

[0028] NovelTyScore(p)=max(D C (rp i,1 ),D C (rp i,2 ),…);

[0029] Among them, D C Similar distance between pointing quantities; w m 、w p 、w o They represent the weights of the similarity distances of the research method, research question, and research object dimensions respectively. The sum of the weights of all dimensions is 1. The average value is taken as the weight, or the weight size is adjusted as needed; NovelTyScore(p) is the final novelty score value.

[0030] Furthermore, the method for generating a readable result text in S5 includes:

[0031] S51, combining the similarity distance calculation results of the structured semantic information of each dimension in S4, dynamically constructing the prompt template of the method comparison result text, and using the large language model to automatically generate a readable result text of the method comparison;

[0032] S52, taking the comparison results of each search point with the methods of the related literature as input, dynamically constructing a prompt template for the result text of the related method review, and automatically generating a readable result text of the related method review using a large language model;

[0033] S53, taking the method comparison results and method review results of each novelty search point with relevant literature as input, dynamically constructs a prompt template for the text of the overall novelty evaluation results of the novelty search point, and uses a large language model to automatically generate a readable result text of the overall novelty evaluation results of the novelty search point.

[0034] Another object of the present invention is to provide a device for implementing the above-mentioned method for evaluating the novelty of a query point under multi-model collaboration, comprising:

[0035] The relevant literature acquisition module is used to extract novelty search keywords according to the novelty search point and construct a search formula, and screen out relevant literature in the novelty search database according to the search formula;

[0036] The relevant step acquisition module is used to extract relevant steps from the screened literature abstracts and summarize the relevant points using the novelty search points as reference templates;

[0037] The structured information extraction module is used to extract structured semantic information from the search points and related step overviews, and align similar expressions;

[0038] A novelty calculation module for novelty query points, used to calculate the similarity distances of various dimensions between the novelty query points and related points based on structured semantic information, and calculate the novelty scores of the novelty query points;

[0039] The readable result generation module is used to generate comparison results based on the scores combined with the novelty search points and the content of similar fragments using a large language model, and dynamically build a prompt template based on the comparison results to generate a summary of the novelty search conclusions and automatically generate a readable result text.

[0040] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0041] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows: based on a large language model, the present invention assists in text understanding, extraction, comparison, and summarization in scientific and technological novelty retrieval, thereby improving the comprehensiveness, accuracy, and readability of the novelty evaluation conclusions of the novelty retrieval points; based on structured semantic information, the present invention infers the similarity between the novelty retrieval points and the relevant documents in various dimensions, and then comprehensively evaluates the correlation between the novelty retrieval points and the relevant documents, thereby evaluating the novelty of the novelty retrieval points, improving the credibility of the results, and avoiding the risk of large model hallucinations. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the novelty credibility evaluation method of the novelty query point under the multi-model collaboration of the present invention;

[0044] Figure 2 This is a diagram showing the specific implementation process of the method for evaluating the novelty of new search points under multi-model collaboration of the present invention. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be described clearly and completely below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0047] The present invention will be further described below in conjunction with specific embodiments, but the present invention is not limited thereto.

[0048] The Large Language Model is a deep learning model that has accumulated deep language knowledge and world information through training with a large amount of data in the process of large-scale text pre-training. This model usually adopts the Transformer architecture, which can capture long-distance dependencies and is mainly used in natural language understanding and generation tasks. It has demonstrated strong generalization and language expression capabilities, and can cope with complex natural language expressions, even in professional language tasks with limited sample sizes. Through the two stages of pre-training and fine-tuning, the large language model can not only achieve high accuracy and adaptability for specific tasks, but also learn the complex structure and rich semantics of the language, thus possessing excellent language understanding and generation capabilities. In the pre-training stage, the model has mastered a wide range of language knowledge, and the fine-tuning stage enables it to be optimized for specific tasks and fields. This model can not only handle simple queries, but also understand and generate texts with strong logic and complex semantics, effectively capture the nuances in the language, and understand the semantic changes in different contexts, thus strongly supporting a variety of language tasks.

[0049] Prompt Engineering refers to the design of effective prompts or queries (prompt) when using a pre-trained large language model to stimulate the model to generate specific responses or behaviors. The core of this technology lies in carefully constructed input sentences, which aim to maximize the output effect of the model and adjust the application of the language model according to specific task requirements. The key to prompt engineering is to understand the working mechanism of the model and the types of data exposed during pre-training, so as to design input sentences that can guide the model to provide information in the desired direction. Effective prompt design can not only improve the accuracy of task completion, but also enhance the model's ability to respond to complex queries. For example, by adjusting the details, wording and even grammatical structure of the prompt, the quality and relevance of the text generated by the model can be greatly affected.

[0050] Contextual Learning refers to the use of in-depth learning of large amounts of text data by models in the process of large-scale pre-training of language models to capture and understand subtle changes and semantic associations in the context. This learning method enables the model to generate or infer more accurate and relevant outputs based on previous inputs or given contexts. For example, when dealing with dialogue or text understanding tasks, by considering previous dialogue history or text content, the model can better understand the current query intent and adapt to different contextual needs. Large model contextual learning not only improves the performance of the model on specific tasks, but also enhances the model's ability to understand complex semantics and metaphors. The core advantage of this technology is that it improves the model's adaptability and flexibility, enabling it to perform excellently in a wider range of application scenarios.

[0051] The embodiments of the present invention combine technologies such as large language models, prompt engineering, and contextual learning to realize the automation of some processes of scientific and technological novelty search, including novelty search point extraction, document retrieval, document comparison, and novelty search report generation, thereby improving the efficiency and accuracy of novelty search. Through the deep text understanding and generation capabilities of large language models such as GPT-4, combined with interactive prompts, novelty search points are accurately extracted, and BERT semantic representation and similarity distance calculation are used to accurately match documents to ensure the relevance and reliability of the comparison content. These technologies not only save time and resources, but also improve the depth and comprehensiveness of novelty search results, and promote the modernization of scientific and technological novelty search work.

[0052] The present invention is further described in detail below. It should be understood that the implementation examples described herein are only used to illustrate and explain the invention, and are not used to limit the present invention.

[0053] like Figure 1 and Figure 2 As shown, the embodiment of the present invention discloses a method for evaluating the novelty of a query point under multi-model collaboration, comprising the following steps:

[0054] Step S1, using the large language model GPT-4 to extract novelty search keywords and construct a search formula to screen out relevant documents in the novelty search database; the specific process of this step includes:

[0055] S11, input the search point of the search item into GPT-4, use GPT-4 to expand the subject and keywords, and combine the prompt template designed for generating search conditions to automatically generate a combined logical search formula. In this embodiment, the specific template and example are as follows:

[0056] #Template 1: I am a scientific and technological novelty search worker. The name of the project to be searched is {project name}, the search scope is {search scope}, and the search point is {search point}. Please extract keywords from the search point and expand related words. Please output in json format ({{Chinese word: English word},{},{}}).

[0057] #Example 1-1:

[0058] ##Input: I am a scientific and technological novelty searcher. The name of the project to be searched is "Research and Application of Key Technologies for Electricity Spot Market Applicable to Super-large-scale Urban Power Grids". The search scope is "domestic and foreign (including achievement database)". The search points are "1. Based on the effective connection between multi-day unit combination and day-ahead market, day-ahead electricity energy and real-time auxiliary service serialization and joint clearing, an overall solution for the electricity spot market of super-large-scale urban power grids is created to solve the contradiction between the high proportion of external electricity in super-large-scale urban power grids and limited local regulation resources; 2. Based on the domestically produced independent and controllable optimization solver, a mathematical optimization engine suitable for the calculation and solution of electricity spot clearing is constructed, realizing the localization of key links in the electricity spot market; 3. Based on the Stackelberg model, a solution for electricity spot network congestion management that effectively takes into account the uncertainty on both sides of the source and load is proposed". Please extract keywords from the search points and expand related words. The output is required to be in json format ({{Chinese word: English word},{},{}}).

[0059] ##Output:{{"multi-day UC unit combination":"multi-day UC unit combination"},{"day-ahead market":"day-ahead market"},{"electrical energy":"electrical energy"},{"real-time ancillary service":"real-time ancillary service"},{"joint clearing":"joint clearing"},{"super large scale urban power grid":"super large scale urban power grid"},{"energy spot market":"energy spot market"},{"external energy"},{"resource allocation":"resource allocation"},{"autonomous and controllable":"autonomous and controllable"},{"optimization solver":"optimization solver"},{"clearing evaluation":"clearing evaluation"},{"mathematical optimization engine":"mathematical optimization engine"},{"localization":"localization"},{"source-load dual side":" combination"},{"uncertainty":"uncertainty"},{"model":"model"},{"network congestion management":"network congestion management"},{"solution":"solution"},{"valid convergence":"valid convergence"}}

[0060] #Template 2: Please combine the above search keywords to construct a combined search formula. The search formula templates for each database are provided by the knowledge base. The database to be searched is {search database}. Each database constructs 4 search formulas and is required to be output in json format ({{search database: {search formula 1, search formula 2, search formula 3, ...}}).

[0061] #Example 2-1:

[0062] ##Input: Please combine the above search keywords to construct a combined search formula. The search formula templates for each database are provided by the knowledge base. The databases to be searched are "1. China National Knowledge Infrastructure (CNKI); 2. INSPEC", and 4 search formulas are constructed for each database. The output is required to be in json format ({{Search database: {Search formula 1, Search formula 2, Search formula 3, ...}}).

[0063] ##Output: {{“China National Knowledge Infrastructure”:{“(multi-day unit OR unit combination)AND day-ahead marketAND ancillary serviceAND clearingAND spot”,“day-ahead AND ancillary serviceAND clearingAND spot”,“solverAND power marketAND spotAND clearing calculationAND(independentOR domestic)”,“modelAND source and loadAND spotAND congestion”}},{“INSPEC”:{“(unit p / 0commitment)AND(day p / 0ahead)AND Ancillary p / 0ServiceAND clearing”,“day p / 0ahead AND Ancillary p / 0Service AND clear”,“solverANDspotAND clearAND localization”,“(source)AND(uncertainty)AND(power p / 0market)AND(congestion)”}}}

[0064] S12, searching for documents in the novelty search database using the above search formula to form a candidate document set;

[0065] S13, using the BERT model to vectorize the candidate document set, using the cosine similarity algorithm to compare the similarity between the search point and the candidate document abstract, and screening out the comparative documents according to the set threshold;

[0066] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on the Transformer architecture, focusing on the vectorization of semantic representation. Through the bidirectional encoder architecture, BERT is able to simultaneously consider the context of each word in the text to generate richer semantic representations. Specifically, BERT converts text into high-dimensional vectors of fixed dimensions, which not only capture the literal meaning of words, but also cover the semantic information in the context. This vectorized representation enables BERT to provide accurate semantic understanding and reasoning capabilities in various natural language processing tasks.

[0067] Similarity calculation is a technique used to evaluate the similarity between two objects. It is widely used in fields such as information retrieval, recommendation systems, and natural language processing. Common similarity calculation methods include cosine similarity, Euclidean distance, and Jaccard similarity coefficient. Cosine similarity measures the similarity between two vectors by calculating the cosine value of the angle between them. It is suitable for similarity analysis after text vectorization. The closer the BERT vectors of two texts are in high-dimensional space, the higher their semantic similarity distance is, which significantly improves the model's understanding and reasoning capabilities.

[0068] Step S2, using the search point as a reference template, using GPT-4 to extract relevant steps from the comparative literature summary of step S1, and summarize relevant points; in this embodiment, the specific template and example are as follows:

[0069] #Template 1: I am a scientific and technological novelty searcher. In order to compare the similarity between the novelty search point and the comparative literature, I need to extract the relevant steps of the comparative literature and summarize them into relevant points. The novelty search point is {novelty search point}, and the summary of the literature to be processed is {comparative literature summary}. The required output format is: relevant point 1; relevant point 2; relevant point 3; ...

[0070] #Example 1-1:

[0071] ##Input: I am a scientific and technological novelty searcher. In order to compare the similarity between the novelty search points and the comparative literature, I need to extract the relevant steps of the comparative literature and summarize them into relevant points. The new search point is "Based on the effective connection between multi-day unit combination and day-ahead market, the serialized joint clearing of day-ahead electric energy and real-time auxiliary services, etc., an overall solution for the spot market of super-large-scale urban power grid is created to solve the contradiction between the high proportion of external electricity in super-large-scale urban power grid and limited local regulation resources", and the abstract of the pending literature is "Shanxi power grid is rich in coal, wind, light, water, and heating demand is large. The peak-shaving capacity of the power grid is difficult to meet the needs of the rapid development of new energy. It is urgent to establish a market-oriented mechanism for the joint operation of "spot + auxiliary services". Combined with the characteristics and policy background of Shanxi power grid, a joint optimization mechanism for spot and deep peak-shaving with deep integration of multi-day rolling unit combination, day-ahead market and real-time market is designed, and a mathematical model for the joint optimization and clearing of power spot and deep peak-shaving market based on multi-day rolling unit combination is established, and it is actually applied in a 7-day settlement trial run of Shanxi power grid. The trial run verifies the rationality of the mechanism and the effectiveness of the model, which is a useful attempt to solve the contradiction between the insufficient peak-shaving capacity of Shanxi power grid and the limited consumption of new energy in a market-oriented way", and the output format is required to be: related point 1; related point 2; ...

[0072] ##Output: This document designs a joint optimization mechanism for spot and deep peak-shaving that deeply integrates multi-day rolling unit combinations, day-ahead markets, and real-time markets; This document establishes a mathematical model for the joint optimization and clearing of the electricity spot and deep peak-shaving markets based on multi-day rolling unit combinations.

[0073] Step 3: Use GPT-4 to extract and align structured semantic information from the search points and the relevant step summaries of step S2. The specific process of this step includes:

[0074] S31, using the text extraction capability of GPT-4, extract short-sentence structured semantic information from the search point and the related points of the comparative literature in step S2; for example, in the example of step S2, the search point is "Based on the effective connection between multi-day unit combination and day-ahead market, the serialized joint clearing of day-ahead electricity and real-time auxiliary services, an overall solution for the spot power market of super-large-scale urban power grid is created to solve the contradiction between the high proportion of external electricity in super-large-scale urban power grid and limited local regulation resources", and the related point is "This document designs a joint optimization mechanism for spot and deep peak regulation with deep integration of multi-day rolling unit combination, day-ahead market and real-time market; This document establishes a mathematical model for joint optimization and clearing of spot power and deep peak regulation market based on multi-day rolling unit combination." Then its structured semantic information includes: (search point - Research methods: effective connection between multi-day unit combinations and day-ahead markets), (new points - research methods: serialized joint clearing of day-ahead electricity and real-time ancillary services), (new points - research issues: creating an overall solution for the power spot market of ultra-large-scale urban power grids to resolve the contradiction between a high proportion of external electricity in ultra-large-scale urban power grids and limited local regulation resources); (related points - research methods: a joint optimization mechanism for spot and deep peak regulation with deep integration of multi-day rolling unit combinations, day-ahead markets and real-time markets), (related points - research methods: a mathematical model for joint optimization and clearing of power spot and deep peak regulation markets based on multi-day rolling unit combinations), (related points - research issues: creating a market-based mechanism for the joint operation of "spot + ancillary services" to resolve the problem that the peak regulation capacity of the power grid is difficult to meet the needs of the rapid development of new energy), etc.

[0075] S32, based on the structured semantic dimension of the query point, use GPT-4 to align similar expressions and rewrite the structured sentences of the relevant literature in the style of the structured sentences of the query point. For example, in step S32, the relevant point of the research method after alignment and rewriting is "the effective connection between the multi-day unit combination and the spot market and the real-time market".

[0076] Step S4, based on the aligned structured semantic information, the pre-trained BERT model is used to map the text into a high-dimensional vector representation, and then the cosine similarity distance between the "new search point" and "related point" vectors is calculated in turn by dimension. For example, the similarity distance between the method vectors in the S3 example can reach up to 0.85, and the similarity distance between the research question vectors is only 0.6. A single dimension may contain multiple vectors, so the fine-grained similarity of each dimension is first determined. For each single vector of each dimension of the new search point, the most relevant vector of each dimension of the related literature is found as the fine-grained similarity result, and then the similarity distance of each dimension vector is averaged as the overall similarity distance of a single dimension. For example, the research method dimension of the new search point in the S3 example contains two vectors, and the most relevant method vectors are found from the research method dimension vector of the related literature, and the calculated similarity distances are 0.85 and 0.8 respectively, so the average similarity distance of the new search point method dimension is 0.825. The novelty calculation index system can be adjusted as needed, involving the weighted calculation of similarity distances of different structural dimensions within the document. Usually, the maximum similarity distance among multiple related documents is selected as the novelty score of the search point. The index system formula is as follows:

[0077] D C =1-cosθ;

[0078] D C (rp i,j )=w m *D C,m +w p *D C,p +w o *D C,o ;

[0079] NovelTyScore(p)=max(D C (rp i,1 ),D C (rp i,2 ),…);

[0080] Among them, D C Similar distance between pointing quantities; w m 、w p 、w o They represent the weights of the similarity distances of the research method, research question, and research object dimensions respectively. The sum of the weights of all dimensions is 1. The average value can be taken as the weight, or the weight size can be adjusted as needed. NovelTyScore(p) is the final novelty score value.

[0081] Step S5, call the GPT-4 interface to generate comparison results based on the content of the search points and similar fragments one by one; then splice the comparison results of each fragment into a new summary summary prompt, and dynamically construct a prompt template for generating the result report; use the text summary and generation capabilities of GPT-4 to generate a summary of the search conclusions, and fill them into the specified report template as required to automatically generate a readable result text; the specific template and example of this step are as follows:

[0082] #Template 1: Please generate the comparison results based on the similarity distance score above, combined with the content of the search point and the similar segment. The output is required to be in the following format: Similar literature: [Literature citation format], this literature proposes [main content], is similar to [search point] in terms of [similar segment], but the difference is [dissimilar segment].

[0083] #Example 1-1: Similar literature: "Design and practice of joint optimization mechanism of Shanxi power spot and deep peak-shaving market [J]. Ding Qiang et al. Power System Technology, 2021.45(06):2219-2228.", this literature proposes "a market design mechanism that jointly optimizes multi-day rolling unit combination, spot market and deep peak-shaving, and establishes a joint optimization clearing mathematical model of power spot and deep peak-shaving market". It is similar to "New Point 1" in terms of "organic integration model of multi-day unit combination and day-ahead spot market", but the difference is that "the research problems are different, and there are no relevant reports on super-large-scale urban power grids with high proportion of external electricity".

[0084] #Example 1-2: Similar literature: "Transmission and distribution coordinated congestion relief model and solution under the electricity spot market [J]. Huang Ying et al. Taiyuan: Journal of Taiyuan University of Technology, 2022: 1-13 [2022-04-12]", this literature proposes a "two-layer optimization dispatching model with distribution network companies as the main body participating in the management of power spot market cold blocking. The ATC algorithm is used to solve the upper and lower models to obtain the optimal dispatching strategy and maximize the utilization of transmission and distribution resources." It is similar to "New Point 3" in "establishing a model to coordinate the dispatch of controllable resources in the transmission network to alleviate the problem of transmission cold blocking", but the difference is that "the research methods are different, and there is no relevant content report on market-driven power generators and users as strategic investment entities of the transmission network under source and load uncertainty."

[0085] #Template 2: Based on the above comparison results, please summarize the relevant research of each search point, and summarize the similar and dissimilar segments by point. Output in the following format: After analyzing the search results, the following conclusions are drawn: For [search point], there are [similar segment 1; similar segment 2; ...] at home and abroad, but [dissimilar segment 1; dissimilar segment 2; ...].

[0086] #Example 2-1: After analyzing the search results, the following conclusions were drawn: Regarding "New Point 1", there have been "research on the coordination mechanism of ancillary service market and spot electricity market" at home and abroad; "research on the organic integration model of multi-day unit combination and day-ahead spot market", but "they are different in research scope, research purpose and research method, and there are no relevant content reports on the creation of overall solutions by means of effective connection between multi-day unit combination and day-ahead market, day-ahead electricity and real-time ancillary service serialization and joint clearing". Regarding "New Point 2", there have been "research on the use of optimization solvers to build mathematical optimization engines suitable for electricity spot clearing calculations and solutions" at home and abroad, but "the research methods are different, and the optimization calculation engines used are all foreign commercial software, and there are no relevant content reports on the localization of key links in the electricity spot market". Regarding “New Point 3”, there have been studies at home and abroad on “establishing a model to coordinate the dispatch of controllable resources in the transmission network to alleviate transmission congestion”; “establishing a model to solve channel congestion from the perspective of long-term investment risk aversion”; and “research on congestion relief based on market-driven transmission network investment”, but “they are different in research methods and technical means, and there are no reports on the Stackelberg optimization model and the establishment of an equilibrium model considering source-load uncertainty”.

[0087] #Template 3: Please combine the above search results to generate a readable search report. It is required to briefly summarize the project content and search points, give the search conclusion, and the word count should be controlled within 300 words.

[0088] #Example 3-1: In summary, this commissioned project proposes to create an integrated solution for power spot declaration, clearing, release, and evaluation suitable for ultra-large-scale urban power grids based on the effective connection between multi-day unit combinations and the day-ahead market, and the serialized joint clearing of day-ahead electric energy and real-time auxiliary services, so as to solve the contradiction between the high proportion of external electricity in ultra-large-scale urban power grids and limited local regulation resources; based on the domestically produced independent and controllable optimization solver, a mathematical optimization engine suitable for the calculation and solution of power spot clearing is constructed to realize the localization of key links in the power spot market; from the perspective of long-term investment planning, considering the uncertainty of sources and loads, a market-driven Stackelberg equilibrium model is established with power generators and users as strategic investment entities in the transmission network, which effectively responds to the channel congestion problem in the mature stage of spot market development. There are no literature reports on exactly the same research and application at home and abroad.

[0089] The embodiment of the present invention provides a device for implementing the above-mentioned method for evaluating the novelty of a query point under multi-model collaboration, characterized by comprising:

[0090] The relevant literature acquisition module is used to extract novelty search keywords according to the novelty search point and construct a search formula, and screen out relevant literature in the novelty search database according to the search formula;

[0091] The relevant step acquisition module is used to extract relevant steps from the screened literature abstracts and summarize the relevant points using the novelty search points as reference templates;

[0092] The structured information extraction module is used to extract structured semantic information from the search points and related step overviews, and align similar expressions;

[0093] A novelty calculation module for novelty query points, used to calculate the similarity distances of various dimensions between the novelty query points and related points based on structured semantic information, and calculate the novelty scores of the novelty query points;

[0094] The readable result generation module is used to generate comparison results based on the scores combined with the novelty search points and the content of similar fragments using a large language model, and dynamically build a prompt template based on the comparison results to generate a summary of the novelty search conclusions and automatically generate a readable result text.

[0095] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0096] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0097] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the specification of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the novelty of a query point under multi-model collaboration, characterized in that: The steps include: S1, extract novelty search keywords according to the novelty search point and construct a search formula, and screen out relevant documents in the novelty search database according to the search formula; S2, using the search point as a reference template, extract relevant steps and summarize relevant points from the literature abstracts screened in step S1; S3, extracts structured semantic information from the search points and related step overviews, and aligns similar expressions; S4, based on the structured semantic information, calculate the similarity distances of each dimension between the query point and the related points, and calculate the novelty score of the query point; S5, using a large language model to generate a comparison result based on the score of step 4 combined with the content of the search point and similar fragments, and dynamically build a prompt template based on the comparison result, generate a summary of the search conclusion, and automatically generate a readable result text.

2. The method for evaluating the novelty of a query point under multi-model collaboration according to claim 1 is characterized in that: Step S1 specifically includes: S11, input the search point, use the large language model to expand the topic and keywords, and build a search formula based on the prompt template generated by the search conditions; S12, in the novelty search database, using S11 to generate a search query to obtain a set of candidate documents; S13, based on text similarity, compare the search point and the abstract part of the candidate document in S12 to obtain a set of related document abstracts with a similarity higher than a set threshold.

3. The method for evaluating the novelty of a query point under multi-model collaboration according to claim 1 is characterized in that: The specific implementation of step S2 is: S21, according to the content characteristics of the novelty search point, extract relevant steps from each step of the relevant literature abstract in S1 using a large language model; S22, based on the expression characteristics of the novelty search point, use the large language model to outline the relevant steps in an approximate style and summarize the relevant points.

4. The method for evaluating the novelty of a query point under multi-model collaboration according to claim 1 is characterized in that: S3 implementations include: S31, using the text extraction capability of the large language model, extracting short-sentence-sized structured semantic information from the search points and the related points of the comparative literature in step S2, wherein the structured semantic information includes the research object, research method, and research question; S32, based on the structured semantic dimension of the novelty query point, use the large language model to rewrite the structured sentences of the relevant literature in the style of the structured sentences of the novelty query point.

5. The method for evaluating the novelty of a query point under multi-model collaboration according to claim 1 is characterized in that: The multiple dimensions in step 4 include research methods, research questions, and research objects.

6. The method for evaluating the novelty of a query point under multi-model collaboration according to claim 5 is characterized in that: Step 4: The method for calculating the novelty score of the query point is: Based on the aligned structured semantic information, the pre-trained BERT model is used to map the text into a high-dimensional vector representation, and the cosine similarity distance between the query point and the related point vectors in each dimension is calculated. When calculating, the fine-grained similarity of each dimension is first determined, and the most relevant vector of each dimension of the related document is found for each single vector of each dimension of the query point as the fine-grained similarity distance result, and then the similarity distance of each dimension vector is averaged as the overall similarity distance of a single dimension. After obtaining the similarity distances in each dimension, the novelty score of the query point is calculated. The calculation formula for the novelty score of the query point is as follows: D C =1-cosθ; D C (rp i,j )=w m *D C,m +w p *D C,p +w o *D C,o ; NovelTyScore(p)=max(D C (rp i,1 ),D C (rp i,2 ),…); Among them, D C Similar distance between pointing quantities; w m 、w p 、w o They represent the weights of the similarity distances of the research method, research question, and research object dimensions respectively. The sum of the weights of all dimensions is 1. The average value is taken as the weight, or the weight size is adjusted as needed; NovelTyScore(p) is the final novelty score value.

7. The method for evaluating the novelty of a query point under multi-model collaboration according to claim 1, characterized in that: Methods for generating readable result text in S5 include: S51, combining the similarity distance calculation results of the structured semantic information of each dimension in S4, dynamically constructing the prompt template of the method comparison result text, and using the large language model to automatically generate a readable method comparison result text; S52, taking the comparison results of each search point with the methods of the related literature as input, dynamically constructing a prompt template for the result text of the related method review, and automatically generating a readable result text of the related method review using a large language model; S53, taking the method comparison results and method review results of each novelty search point with relevant literature as input, dynamically constructs a prompt template for the text of the overall novelty evaluation results of the novelty search point, and uses a large language model to automatically generate a readable result text of the overall novelty evaluation results of the novelty search point.

8. A device for implementing the method for evaluating the novelty of a query point under multi-model collaboration as claimed in any one of claims 1 to 7, characterized in that: include: The relevant literature acquisition module is used to extract novelty search keywords according to the novelty search point and construct a search formula, and screen out relevant literature in the novelty search database according to the search formula; The relevant step acquisition module is used to extract relevant steps from the screened literature abstracts and summarize the relevant points using the novelty search points as reference templates; The structured information extraction module is used to extract structured semantic information from the search points and related step overviews, and align similar expressions; A novelty calculation module for novelty query points, used to calculate the similarity distances of various dimensions between the novelty query points and related points based on structured semantic information, and calculate the novelty scores of the novelty query points; The readable result generation module is used to generate comparison results based on the scores combined with the novelty search points and the content of similar fragments using a large language model, and dynamically build a prompt template based on the comparison results to generate a summary of the novelty search conclusions and automatically generate a readable result text.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

Images