Technical scheme evaluation method and device, equipment, storage medium and product
Through pre-trained large language model, technical solution evaluation is carried out, semantic feature analysis and similarity calculation is used to solve the problem of inefficiency of traditional evaluation methods, and efficient and accurate technical solution evaluation is achieved, reducing manual analysis costs and reducing information The risk of omission.
Patent Information
- Application Number
- CN202411131072.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-11-22
AI Technical Summary
Traditional technical solution evaluation methods are inefficient, difficult to process massive data, risk of information omission, and lack of intelligent support, resulting in incomplete and accurate assessment.
The pre-trained large language model is used to analyze and extract the semantic feature of the technical solution information. By calculating and sorting the similarity with the technical solution database, relevant technical solutions are automatically retrieved and evaluated, and creative scoring and semantic comparison analysis are provided.
It significantly improves the efficiency and comprehensiveness of technical solution evaluation, provides specific and quantifiable reference standards, reduces the risk of repeated innovation and infringement of other people's technical rights and interests, and reduces the cost of manual analysis.
Smart Images

Figure CN119003702A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a technical solution evaluation method, device, equipment, storage medium and product. Background Art
[0002] In the traditional process of evaluating the creativity of patent technical solutions, staff are usually required to select relevant technical solutions from patent databases, literature databases, and online resources, and then manually compare and analyze multiple technical solutions. However, this traditional method of technical solution evaluation is inefficient, consumes a lot of time and energy, has a high risk of missing information, and manual analysis has subjective biases, which can easily miss key prior arts. With the surge in the number of patents, manual technical solution evaluation methods are difficult to efficiently process and analyze massive amounts of patent data and technical literature.
[0003] Therefore, how to improve the efficiency and comprehensiveness of the creative evaluation of technical solutions is an issue that needs to be urgently addressed. Summary of the invention
[0004] The main purpose of this application is to provide a technical solution evaluation method, device, equipment, storage medium and product, aiming to solve the technical problems of low efficiency in creative evaluation of technical solutions and incomplete evaluation results.
[0005] To achieve the above objectives, the present application provides a technical solution evaluation method, which includes:
[0006] In response to a technical solution evaluation request, input technical solution information of the technical solution to be evaluated into a pre-trained large language model, and retrieve relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model;
[0007] According to the relevant technical solutions, a creative evaluation is performed on the technical solution to be evaluated to obtain a technical solution evaluation result.
[0008] In one embodiment, the step of retrieving relevant technical solutions from a preset technical solution database according to the technical solution information by using the pre-trained large language model comprises:
[0009] Performing semantic feature analysis on the technical solution information through a pre-trained large language model to obtain semantic feature information of the technical solution to be evaluated;
[0010] The technical solutions in the technical solution database are searched according to the semantic feature information to obtain relevant technical solutions.
[0011] In one embodiment, the semantic feature information of the technical solution to be evaluated includes a first technical solution feature vector of the technical solution to be evaluated, and the step of performing semantic feature analysis on the technical solution information through a pre-trained large language model to obtain the semantic feature information of the technical solution to be evaluated includes:
[0012] Performing semantic analysis on the technical solution to be evaluated by using a pre-trained large language model to obtain semantic information of the technical solution to be evaluated;
[0013] Preprocessing the semantic information, and extracting features from the preprocessed semantic information according to a preset feature extraction method to obtain a feature vector of the first technical solution;
[0014] The step of retrieving the technical solutions in the technical solution database according to the semantic feature information to obtain relevant technical solutions includes:
[0015] Obtaining existing solutions through the technical solution database;
[0016] Calculating the similarity between the technical solution to be evaluated and the existing solution according to the first technical solution feature vector;
[0017] Sort the existing solutions according to the similarity, and select a preset number of existing solutions as relevant technical solutions;
[0018] The step of conducting a creativity evaluation on the technical solution to be evaluated based on the relevant technical solution to obtain a technical solution evaluation result includes:
[0019] According to the similarity, obtaining the rating item similarity corresponding to the preset rating item;
[0020] Calculating the average similarity of the scoring items of the related technical solutions according to the scoring item similarity and the number of preset scoring items;
[0021] According to the average similarity of the scoring items and the preset technical creativity scoring rules, the technical solution to be evaluated is scored for technical creativity to obtain a technical solution evaluation result.
[0022] In one embodiment, the step of performing a creativity evaluation on the technical solution to be evaluated based on the related technical solution to obtain a technical solution evaluation result includes:
[0023] Extracting technical key points of the technical solution to be evaluated and the related technical solutions through a pre-trained large language model to obtain the technical key points of the technical solution to be evaluated and the technical key points of the related technical solutions;
[0024] A semantic comparative analysis is performed on the technical points of the technical solution to be evaluated and the technical points of the related technical solutions to obtain a technical solution evaluation result.
[0025] In one embodiment, before the step of calculating the similarity between the technical solution to be evaluated and the existing solution according to the first technical solution feature vector, the following steps are included:
[0026] Extracting features from the existing solution to obtain a feature vector of the existing solution;
[0027] An index is constructed according to the existing solution and the feature vector of the existing solution, and the existing solution and the feature vector of the existing solution are stored in the technical solution database according to the index.
[0028] In one embodiment, the step of performing a creativity evaluation on the technical solution to be evaluated to obtain a technical solution evaluation result further includes:
[0029] Testing the technical solution to be evaluated to determine the solution type of the technical solution to be evaluated;
[0030] If the solution type is a technical creativity point, the technical solution to be evaluated is scored for technical creativity based on the relevant technical solutions to obtain a technical solution evaluation result;
[0031] If the solution type is a technical briefing document, a semantic comparative analysis is performed on the technical solution to be evaluated based on the relevant technical solutions to obtain a technical solution evaluation result.
[0032] In addition, to achieve the above-mentioned purpose, the present application also provides a technical solution evaluation device, which includes:
[0033] A retrieval module, configured to respond to a technical solution evaluation request, input technical solution information of the technical solution to be evaluated into a pre-trained large language model, and retrieve relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model;
[0034] The evaluation module is used to perform a creative evaluation on the technical solution to be evaluated based on the relevant technical solution to obtain a technical solution evaluation result.
[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a technical solution evaluation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the technical solution evaluation method described above.
[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and a program for implementing the technical solution evaluation method is stored on the computer-readable storage medium. The program for implementing the technical solution evaluation method is executed by a processor to implement the steps of the technical solution evaluation method as described above.
[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a program for implementing the technical solution evaluation method, and when the computer program is executed by a processor, it implements the steps of the technical solution evaluation method described above.
[0038] One or more technical solutions proposed in this application have at least the following technical effects:
[0039] By utilizing a large-scale language model, an intelligent technical solution evaluation system is provided, which can automatically evaluate technical solutions, improve the efficiency and accuracy of technical solution evaluation, reduce costs, and solve problems existing in existing technologies. By inputting technical solution information into a preset technical solution database and performing a quick search, existing technical solutions related to the technical solution to be evaluated can be quickly located, avoiding the time-consuming process of traditional manual search or large-scale literature research, and significantly improving the work efficiency and comprehensiveness of technical solution evaluation. The retrieval and comparison of related technical solutions provide specific and quantifiable reference standards for technical solution evaluation, which helps to more accurately evaluate the creativity and practicality of the technical solution to be evaluated, identify and avoid repeated innovation of technical solutions, and avoid the risk of infringing on the rights and interests of others' technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0042] Figure 1 A flowchart of the first embodiment of the evaluation method of the technical solution of this application is provided;
[0043] Figure 2 A schematic diagram of the evaluation results of the first technical solution of the technical solution evaluation method of this application;
[0044] Figure 3 A schematic diagram of the evaluation results of the second technical solution of the technical solution evaluation method of this application;
[0045] Figure 4 This is a schematic diagram of the evaluation process of the technical solution evaluation method of this application;
[0046] Figure 5 This is a schematic diagram of the module structure of the technical solution evaluation device of the embodiment of the present application;
[0047] Figure 6 A schematic diagram of the device structure of the hardware operating environment involved in the technical solution evaluation method in the embodiment of the present application.
[0048] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0049] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0050] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0051] The main solution of the present application is: in response to a technical solution evaluation request, the technical solution information of the technical solution to be evaluated is input into a pre-trained large language model, and through the pre-trained large language model, the relevant technical solutions in a preset technical solution database are retrieved according to the technical solution information; based on the relevant technical solutions, the technical solution to be evaluated is creatively evaluated to obtain a technical solution evaluation result.
[0052] With the rapid development of science and technology and the increasing number of innovative activities, the number of technical solutions for inventions and innovations is also increasing. In order to ensure the originality of the invention, it is essential to conduct a comprehensive innovation evaluation of the technical solution. Traditional patent novelty search methods usually rely on manual retrieval and analysis, which is not only time-consuming and labor-intensive, but also prone to missing important information. Depending on the professional and technical capabilities of the technical inventor, it is difficult for the technical inventor to conduct accurate retrieval work independently, and it is impossible to comprehensively record and analyze relevant technical literature and patent documents, nor to form a detailed novelty search report, and there is a lack of auxiliary tools for rapid technical solution evaluation. Specifically, in the existing technical solution evaluation process, there are the following technical problems:
[0053] 1. Low retrieval efficiency: Traditional manual retrieval methods are time-consuming and difficult to cover all relevant technical literature and patent documents.
[0054] 2. High risk of missing information: Manual analysis is subject to subjective bias and can easily miss key existing technologies, affecting the accuracy of the search results.
[0055] 3. Difficulty in processing massive amounts of data: With the surge in the number of patents, existing novelty search methods are unable to effectively process and analyze massive amounts of patent data and technical literature.
[0056] 4. Lack of intelligent support: Most existing novelty search tools rely on simple keyword matching and lack the ability to deeply understand and analyze technical content.
[0057] Therefore, how to improve the efficiency and comprehensiveness of patent technology solution evaluation is an issue that needs to be urgently addressed.
[0058] This application provides an intelligent technical solution evaluation system by utilizing a large-scale language model, which can automatically evaluate technical solutions, improve the efficiency and accuracy of technical solution evaluation, reduce costs, and solve problems existing in the prior art. By inputting the technical solution information into a preset technical solution database and performing a quick search, it is possible to quickly locate the existing technical solutions related to the technical solution to be evaluated, avoiding the time-consuming process of traditional manual search or large-scale literature research, and significantly improving the work efficiency and comprehensiveness of the technical solution evaluation. The retrieval and comparison of related technical solutions provide specific and quantifiable reference standards for the evaluation of technical solutions, which helps to more accurately evaluate the creativity and practicality of the technical solution to be evaluated, identify and avoid repeated innovation of technical solutions, and avoid the risk of infringing on the rights and interests of others' technical solutions.
[0059] It should be noted that the execution subject of this embodiment can be a technical solution evaluation system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a technical solution evaluation device capable of realizing the above functions, etc., and this embodiment does not specifically limit this. The following takes the technical solution evaluation system as an example of the execution subject to illustrate this embodiment and the following embodiments.
[0060] Based on this, this application proposes a technical solution evaluation method for the first embodiment, please refer to Figure 1 , the technical solution evaluation method includes steps S10 to S20:
[0061] Step S10, in response to a technical solution evaluation request, inputting the technical solution information of the technical solution to be evaluated into a pre-trained large language model, and searching for relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model;
[0062] It should be noted that the request for technical solution evaluation may contain detailed information on the technical solution to be evaluated, including but not limited to the name of the technical solution, implementation steps, classification number application field or key technical points, etc.
[0063] The preset technical solution database is a pre-established and maintained technical solution database used to store a large amount of technical solution information. The technical solution database can also connect to multiple patent databases or academic literature databases (such as IEEEXplore, ScienceDirect, Google Scholar, etc.), obtain patent documents and academic papers, and collect industry technical standards, technical reports, market research reports and other relevant technical documents.
[0064] The technical solution information of the technical solution to be evaluated is input into the preset technical solution database, which is usually completed through a data interface or a user interface. During the input process, the technical solution information of the technical solution to be evaluated may need to be cleaned and formatted to ensure the accuracy and consistency of the information.
[0065] Large language models (such as GPT-4, etc.) are used to understand and analyze the technical solution information of the technical solution to be evaluated, and retrieve relevant technical solutions from the technical solution database. Large language models need to be trained with a large amount of text data before they are put into use. These training data may include technical literature, academic papers or technical reports in various fields. Through training, large language models can learn the laws and characteristics of language, so that they have the ability to understand and analyze the text of technical solutions. During the retrieval process, the large language model will first parse the technical solution information of the technical solution to be evaluated, and extract key information such as keywords, topics or text feature vectors. Use this key information to search and match in the technical solution database to find technical solutions related to the technical solution to be evaluated. The search results are usually displayed to users in a certain order of relevance. After retrieval and matching by the large language model, a series of technical solutions related to the technical solution to be evaluated will be obtained. These technical solutions may involve similar technical literature, the same or similar technical fields, methods for solving the same or similar technical problems, etc. After obtaining the relevant technical solutions, these technical solutions can be further analyzed and compared to evaluate the creativity, practicality or infringement risks of the technical solutions to be evaluated.
[0066] Step S20: Based on the related technical solutions, a creativity evaluation is performed on the technical solution to be evaluated to obtain a technical solution evaluation result.
[0067] It should be noted that scoring items can be set according to the technical implementation steps, application scenarios, technical effects and other aspects of the technical solution to be evaluated and the related technical solutions. Each scoring item corresponds to a specific technical point or innovation point in the technical solution to be evaluated. For each scoring item, calculate the similarity between the technical solution to be evaluated and the related technical solution in this item. This can be achieved through semantic similarity calculation, keyword matching and other methods. The results of each scoring item are weighted and summarized to obtain the average similarity. Set creative scoring rules according to actual needs, score the technical solution to be evaluated in combination with the average similarity and creative scoring rules, and obtain the technical solution evaluation results based on the scoring.
[0068] Furthermore, based on the detailed information of the technical solution to be evaluated and the related technical solutions, including but not limited to technical background, technical problems, technical solution description, technical implementation steps, application scenarios, technical effects and core advantages, large language models (such as BERT, GPT, etc.) can be used to pre-process the text, including text segmentation, keyword extraction, sentence importance assessment, etc., to identify and extract key technical points, build a list of technical points, and establish a comparative analysis framework, covering multiple dimensions such as technical background, technical implementation steps, and core advantages. Natural language processing technology (such as word vectors, semantic similarity calculation, etc.) is used to deeply understand the semantic relationship between technical points and conduct comparative analysis. Compare the technical points of the technical solution to be evaluated and the related technical solutions in various dimensions, identify similarities and differences, and obtain the technical solution evaluation results based on the similarities and differences.
[0069] This embodiment provides a method for evaluating a technical solution, specifically by utilizing a large-scale language model, to provide an intelligent technical solution evaluation system, which can automatically evaluate the technical solution, improve the efficiency and accuracy of the technical solution evaluation, reduce costs, and solve problems existing in the prior art. By inputting the technical solution information into a preset technical solution database and performing a quick search, it is possible to quickly locate the existing technical solutions related to the technical solution to be evaluated, avoiding the time-consuming process of traditional manual search or large-scale literature research, and significantly improving the work efficiency and comprehensiveness of the technical solution evaluation. The retrieval and comparison of related technical solutions provide a specific and quantifiable reference standard for the evaluation of the technical solution, which helps to more accurately evaluate the creativity and practicality of the technical solution to be evaluated, identify and avoid repeated innovation of the technical solution, and avoid the risk of infringing on the rights and interests of others' technical solutions.
[0070] In a feasible implementation, step S10 may include steps S101 to S102:
[0071] Step S101, performing semantic feature analysis on the technical solution information through a pre-trained large language model to obtain semantic feature information of the technical solution to be evaluated;
[0072] Step S102: searching the technical solutions in the technical solution database according to the semantic feature information to obtain relevant technical solutions.
[0073] It should be noted that the technical solution information includes but is not limited to technical description, innovation, application scenario, technical effect and performance of the technical solution to be evaluated. Semantic relevance is a measure of the degree of similarity or relevance of the technical solutions in the technical solution database at the semantic level. It is necessary to use a suitable search strategy to find relevant technical solutions in the technical solution database. The search strategy can be implemented based on vector space model, similarity calculation or machine learning algorithm. For example, the statistical search strategy can use the statistical information (such as co-occurrence frequency, mutual information, etc.) in a large-scale corpus to evaluate the semantic relevance between texts; the search strategy based on the vector space model (VSM) represents the text as a vector and evaluates the semantic relevance by calculating the similarity between vectors (such as cosine similarity). It is usually combined with word embedding technology (such as Word2Vec, GloVe, BERT, etc.) to map words or sentences into a high-dimensional space so that semantically similar texts are closer in space; the search strategy based on graph algorithm can use the graph structure to represent the semantic relationship between texts, and use graph algorithms (such as PageRank, HITS, etc.) to evaluate the relevance between nodes (texts).
[0074] Furthermore, by calculating the similarity or distance between the feature vector of the technical solution to be evaluated and the feature vector of each technical solution in the database, and selecting the relevant technical solutions according to the ranking results of the similarity or distance. Similarity calculation is a key link in the retrieval process. Similarity calculation can use a preset metric (such as cosine similarity, Euclidean distance, etc.) to compare the similarity between two feature vectors. Technical solutions with higher similarity often have a higher correlation with the technical solution to be evaluated in terms of content, technical features or application scenarios. Similarity calculation methods can be used for database matching, such as Manhattan distance, Pearson correlation coefficient, Levenshtein distance, Jaro-Winkler distance, etc. Through similarity calculation and sorting, a series of technical solutions related to the technical solution to be evaluated can be retrieved from the technical solution database, which will serve as an important reference for the subsequent creative evaluation process.
[0075] In this embodiment, the large language model is trained with a large amount of data and can deeply understand the complex structure and semantic relationship in natural language. In the process of feature extraction, the large language model can more accurately capture the key features in the technical solution information, including technical details, innovations, application scenarios, etc., thereby generating a more comprehensive and accurate feature vector. The use of a large language model for feature extraction reduces human subjectivity. The model performs objective analysis based on data and algorithms, thereby improving the objectivity and reliability of the evaluation results. The retrieval method based on the feature vector of the technical solution can quickly locate the technical solution related to the technical solution to be evaluated. By calculating the similarity or distance between the feature vectors, the scheme can ensure that the retrieved technical solution has a high correlation with the technical solution to be evaluated in terms of content, technical characteristics or application scenarios. Feature extraction through a pre-trained large language model improves the efficiency of processing a large amount of technical solution information, improves the efficiency and comprehensiveness of the creative evaluation of the technical solution to be evaluated, and saves manpower and time costs.
[0076] In a feasible implementation manner, the semantic feature information of the technical solution to be evaluated includes a first technical solution feature vector of the technical solution to be evaluated, and step S101 may include steps S1011 to S1012:
[0077] Step S1011, performing semantic analysis on the technical solution to be evaluated by using a pre-trained large language model to obtain semantic information of the technical solution to be evaluated;
[0078] Step S1012, preprocessing the semantic information, and performing feature extraction on the preprocessed semantic information according to a preset feature extraction method to obtain a feature vector of the first technical solution.
[0079] It should be noted that through semantic analysis, the large language model will generate semantic information about the technical solution to be evaluated, which may include semantic descriptions of the core ideas, technical details, application scenarios, innovations, etc. of the technical solution, as well as the logical relationships between these descriptions. This information will serve as the basis for subsequent feature extraction.
[0080] Preprocess the semantic information to convert the original semantic information into a form more suitable for feature extraction. Preprocessing may include operations such as removing irrelevant information (such as stop words, punctuation marks, etc.), word form restoration (restoring words of different forms to the same basic form), and stemming (extracting the main part of a word). These operations help reduce the noise and redundancy of the data and improve the efficiency and accuracy of feature extraction. The feature extraction method refers to the specific algorithm or rule for converting the preprocessed semantic information into a feature vector. The preset feature extraction method may be based on a variety of technologies such as word embedding, TF-IDF (Term Frequency-Inverse Document Frequency, word frequency-the breadth of distribution of words in a document collection, a commonly used weighted technology for information retrieval and text mining), text summarization, and topic models. For example, word embedding technology can be used to convert each word in the text into a vector in a high-dimensional space, and these vectors can be combined into a feature vector of the entire technical solution in some way (such as averaging, weighted summation, etc.). After preprocessing and feature extraction, the first technical solution feature vector of the technical solution to be evaluated is obtained, which can reflect the key features and important information of the technical solution to be evaluated at the semantic level. It will be used as input for subsequent steps of technical solution retrieval or creative evaluation, and for comparison and analysis with other technical solutions.
[0081] In this embodiment, the preprocessing process helps to remove irrelevant information (such as stop words, punctuation marks, etc.) and redundant information in the semantic information, thereby reducing the interference of data noise on feature extraction, and helping to improve the quality and accuracy of the feature vector. By preprocessing and extracting features of semantic information, a technical solution feature vector can be obtained, which accurately reflects the core content and key information of the technical solution, and deeply understands the complex semantic information in the technical solution to be evaluated. The semantic analysis process is highly automated and can be completed quickly without human intervention, which improves the efficiency of processing a large amount of technical solution information, saves manpower and time costs, and improves the objectivity and reliability of the evaluation results.
[0082] Further, step S102 may include steps S1021 to S1023:
[0083] Step S1021, obtaining existing solutions through the technical solution database;
[0084] Step S1022, calculating the similarity between the technical solution to be evaluated and the existing solution according to the first technical solution feature vector;
[0085] Step S1023 sorts the existing solutions according to the similarities, and selects a preset number of existing solutions as relevant technical solutions.
[0086] Further, step S1022 may include steps S10221 to S10222:
[0087] Step S10221, extracting features from the existing solution to obtain a feature vector of the existing solution;
[0088] Step S10222: construct an index according to the existing solution and the feature vector of the existing solution, and store the existing solution and the feature vector of the existing solution in the technical solution database according to the index.
[0089] It should be noted that the technical solution database usually includes detailed information of each technical solution, such as the name, number, solution content, solution designer and solution creation time of the technical solution. The existing solutions in the database are preprocessed, including removing noise in the text (such as HTML tags, special characters, stop words, etc.), and performing natural language processing tasks such as word segmentation and part-of-speech tagging. The preprocessed existing solutions are vectorized, and the existing solutions are converted into points in a high-dimensional vector space. These points (i.e., text vectors) can capture the semantic features of the existing solutions and obtain the feature vectors of the existing solutions. The feature vectors of the existing solutions are stored in the database and an index is built for them. The construction of the index can greatly speed up the query process of similar vectors. When a large number of existing solution feature vectors are stored in the database, the existing solution feature vectors similar to the vector queried by the user (i.e., the first technical solution feature vector) can be quickly located through the index.
[0090] By calculating the similarity between the feature vector of the first technical solution to be evaluated and the feature vector of the existing technical solution in the technical solution database, a similarity value can be obtained by using a calculation method such as cosine similarity or Euclidean distance. Figure 2 , according to the calculated similarity values, the technical solutions in the technical solution database are sorted, and the first preset number of technical solutions with the highest similarity are selected as the technical solutions most relevant to the technical solution to be evaluated.
[0091] In this embodiment, the similarity of different technical solutions can be accurately measured by calculating the similarity of feature vectors. The method of similarity calculation can be selected and adjusted according to specific needs. For example, multiple metrics such as cosine similarity, Euclidean distance, Jaccard similarity, etc. can be used to calculate the similarity between feature vectors. This accuracy makes the evaluation result more reliable and reduces the influence of human subjective judgment. By sorting the technical solutions according to the similarity and selecting the first preset number of technical solutions as related technical solutions, it can be ensured that the selected solution has a high correlation with the technical solution to be evaluated, which helps to improve the efficiency and accuracy of the evaluation.
[0092] Further, step S20 may include steps S201 to S203:
[0093] Step S201, obtaining the rating item similarity corresponding to the preset rating item according to the similarity;
[0094] Step S202, calculating the average similarity of the scoring items of the related technical solutions according to the similarity of the scoring items and the number of preset scoring items;
[0095] Step S203, performing technical creativity scoring on the technical solution to be evaluated according to the average similarity of the scoring items and a preset technical creativity scoring rule to obtain a technical solution evaluation result.
[0096] It should be noted that an in-depth analysis of the similarity values between the technical solution to be evaluated and the related technical solutions can further determine the aspects of the similarity between the technical solution to be evaluated and the related technical solutions, such as technical implementation steps, application scenarios or technical effects.
[0097] According to the preset technical creativity scoring rules, the scoring of the technical solution to be evaluated needs to comprehensively consider multiple aspects such as the technical implementation steps, application scenarios or technical effects of the technical solution as different scoring items. For example, if the technical solution to be evaluated is highly similar to a known high-scoring technical solution in multiple key points, then a higher score can be given in the corresponding scoring item.
[0098] Furthermore, the creative scoring rules may be calculation rules preset by the system, or may be customized according to user needs or actual evaluation needs. For example, different weight factors may be set for different scoring items, and the scoring results of each item may be aggregated and weighted to obtain a comprehensive evaluation result, which may include specific scores, grades or rankings.
[0099] For example, refer to Figure 2 The following is an example of the evaluation results. The evaluation results include the publication number, similarity and patent title, and the creativity score is calculated. An example of the calculation method of the technical creativity score is as follows. The higher the score, the more innovative it is:
[0100] Average similarity = (sum of similarities of all scoring items) / total number of scoring items. The scoring rules for technical creativity are as follows:
[0101] The average similarity is 40% to 100%, and the score is 0 to 60, which is considered to be not innovative;
[0102] The average similarity is 20% to 40%, and the score is 60 to 80, which is considered to be average in terms of novelty;
[0103] The average similarity is 10% to 20%, and the score is 80 to 90, which is considered to be good for novelty;
[0104] The average similarity is 0% to 10%, and the score is 90 to 100, which is considered to be very original.
[0105] In this embodiment, similarity analysis can quickly screen out existing technical solutions that are similar or related to the technical solution to be evaluated, thereby providing valuable reference information for evaluators. On this basis, evaluators can more efficiently apply the technical creativity scoring rules for evaluation. The preset technical creativity scoring rules provide clear standards and basis for evaluation, reducing the subjectivity and arbitrariness in the evaluation process. At the same time, similarity analysis is based on objective data and algorithms, which further enhances the objectivity of the evaluation and helps to ensure the fairness and credibility of the evaluation results. By combining similarity analysis and technical creativity scoring rules, multiple dimensions of technical solutions can be comprehensively considered. Similarity analysis reveals the correlation and differences between technical solutions, making the evaluation results more comprehensive and reducing the deviations that may be caused by single-dimensional evaluation.
[0106] In a feasible implementation, step S20 may include steps S204 to S205:
[0107] Step S204, extracting technical key points of the technical solution to be evaluated and the related technical solutions through a pre-trained large language model to obtain the technical key points of the technical solution to be evaluated and the technical key points of the related technical solutions;
[0108] Step S205 , performing semantic comparison analysis on the technical points of the technical solution to be evaluated and the technical points of the related technical solutions to obtain a technical solution evaluation result.
[0109] It should be noted that identifying technical points in texts through pre-trained large language models (such as BERT, GPT series models, etc.) usually involves steps such as text segmentation, keyword extraction, and sentence importance evaluation. Ultimately, the large language model will output a list of technical points for the technical solutions to be evaluated and the related technical solutions.
[0110] According to the content of the technical points, a comparative analysis framework is established. The comparative analysis framework can be a preset framework of the system, or it can be customized or adjusted according to user needs or actual evaluation needs. Specifically, it can include multiple dimensions such as technical background, steps for technical implementation, core advantages, core invention points, comparison of claim scope, and overall suggestions for technical solution creativity. Use natural language processing technology (such as word vectors, semantic similarity calculations, etc.) to conduct in-depth semantic understanding of technical points. On the basis of semantic understanding, the technical points of the technical solution to be evaluated and the technical points of related technical solutions are compared and analyzed. This includes identifying similarities and differences. At the same time, it is also possible to evaluate in which aspects the technical solution to be evaluated has unique advantages or improvements. Based on the results of the comparative analysis, the technical solution evaluation results are generated.
[0111] For example, refer to Figure 3 As an example of the evaluation results, the background and technical problems of the technical scheme to be evaluated are recorded as {text1-1}, the technical scheme and the core invention point are recorded as {text1-2}, and the core advantage is recorded as {text1-3}; at the same time, the background and technical problems of the related technical schemes are recorded as {text2-1}, the technical scheme and the core invention point are recorded as {text2-2}, and the core advantage is recorded as {text2-3}. The corresponding contents of the technical scheme to be evaluated and the related technical schemes are compared one by one, that is, the semantic understanding and semantic comparison capabilities of the large model are used to compare {text1-1} and {text2-1}, {text1-2} and {text2-2}, {text3-1} and {text3-3}, and the similarities and differences of each part are output. The technical scheme evaluation results are generated based on the comparison results of the above similarities and differences.
[0112] In this embodiment, the large language model has been trained with a large amount of text data and has strong semantic understanding and information extraction capabilities. It can more accurately extract key technical points from the technical solution text, reduce the possibility of information omission or misjudgment, and improve the comprehensiveness and efficiency of technical solution evaluation. By comparing the similarities and differences between the technical solution to be evaluated and the related technical solutions through semantic comparative analysis, the standardization and consistency of the evaluation process can be achieved, the subjectivity and differences that may exist in manual evaluation can be reduced, and the comparability and credibility of the evaluation results can be improved.
[0113] In a feasible implementation manner, the performing of creativity evaluation on the technical solution to be evaluated to obtain a technical solution evaluation result may further include steps S206 to S208:
[0114] Step S206, testing the technical solution to be evaluated to determine the solution type of the technical solution to be evaluated;
[0115] Step S207, if the solution type is a technical creativity point, then the technical solution to be evaluated is scored for technical creativity according to the related technical solutions to obtain a technical solution evaluation result;
[0116] Step S208: If the solution type is a technical briefing document, a semantic comparison analysis is performed on the technical solution to be evaluated based on the related technical solutions to obtain a technical solution evaluation result.
[0117] It should be noted that the large language model can further confirm the type of the technical solution to be evaluated by identifying keywords and terms in the text, or by combining contextual information and logical inference. If the determined type of technical solution is a technical creative point, the evaluation work will focus on the innovation, practicality, feasibility and other aspects of the creative point. If the determined type of technical solution is a technical briefing document, the technical briefing document usually contains detailed steps, methods, materials and other information for technical implementation. Therefore, the evaluation can be based on the completeness, accuracy, innovation or difference from existing technologies of the solution to be evaluated. The method of semantic comparative analysis is adopted, that is, the content of the technical briefing document is compared and analyzed with the relevant technical solutions in the technical solution database to evaluate its uniqueness and innovation. In addition, referring to Figure 4 When the technical solution to be evaluated is a technical creative point or a technical briefing document, the relevant technical solutions are determined through semantic relevance analysis, and then scored according to the creative scoring rules of the corresponding solution type, and compared with the relevant technical solutions. The creative scoring rules can be calculation rules preset by the system, or can be customized according to user needs or actual evaluation needs, and can be set with reference to the methods in the embodiments of steps S201 to S203. Comparative analysis requires the establishment of a comparative analysis framework, which can be a system preset framework, or can be customized or adjusted according to user needs or actual evaluation needs, and can be set with reference to the methods in the embodiments of steps S204 to S205.
[0118] The present application also provides a technical solution evaluation device, please refer to Figure 5 , the technical solution evaluation device comprises:
[0119] A retrieval module 10 is used to respond to a technical solution evaluation request, input technical solution information of the technical solution to be evaluated into a pre-trained large language model, and retrieve relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model;
[0120] The evaluation module 20 is used to perform a creativity evaluation on the technical solution to be evaluated based on the relevant technical solutions to obtain a technical solution evaluation result.
[0121] Optionally, the retrieval module 10 further includes:
[0122] Performing semantic feature analysis on the technical solution information through a pre-trained large language model to obtain semantic feature information of the technical solution to be evaluated;
[0123] The technical solutions in the technical solution database are searched according to the semantic feature information to obtain relevant technical solutions.
[0124] Optionally, the retrieval module 10 further includes:
[0125] Performing semantic analysis on the technical solution to be evaluated by using a pre-trained large language model to obtain semantic information of the technical solution to be evaluated;
[0126] Preprocessing the semantic information, and extracting features from the preprocessed semantic information according to a preset feature extraction method to obtain a feature vector of the first technical solution;
[0127] Obtaining existing solutions through the technical solution database;
[0128] Calculating the similarity between the technical solution to be evaluated and the existing solution according to the first technical solution feature vector;
[0129] Sort the existing solutions according to the similarity, and select a preset number of existing solutions as relevant technical solutions;
[0130] Optionally, the evaluation module 20 further includes:
[0131] According to the similarity, obtaining the rating item similarity corresponding to the preset rating item;
[0132] Calculating the average similarity of the scoring items of the related technical solutions according to the scoring item similarity and the number of preset scoring items;
[0133] According to the average similarity of the scoring items and the preset technical creativity scoring rules, the technical solution to be evaluated is scored for technical creativity to obtain a technical solution evaluation result.
[0134] Optionally, the evaluation module 20 further includes:
[0135] Extracting technical key points of the technical solution to be evaluated and the related technical solutions through a pre-trained large language model to obtain the technical key points of the technical solution to be evaluated and the technical key points of the related technical solutions;
[0136] A semantic comparative analysis is performed on the technical points of the technical solution to be evaluated and the technical points of the related technical solutions to obtain a technical solution evaluation result.
[0137] Optionally, the retrieval module 10 further includes:
[0138] Extracting features from the existing solution to obtain a feature vector of the existing solution;
[0139] An index is constructed according to the existing solution and the feature vector of the existing solution, and the existing solution and the feature vector of the existing solution are stored in the technical solution database according to the index.
[0140] Optionally, the evaluation module 20 further includes:
[0141] Testing the technical solution to be evaluated to determine the solution type of the technical solution to be evaluated;
[0142] If the solution type is a technical creativity point, the technical solution to be evaluated is scored for technical creativity based on the relevant technical solutions to obtain a technical solution evaluation result;
[0143] If the solution type is a technical briefing document, a semantic comparative analysis is performed on the technical solution to be evaluated based on the relevant technical solutions to obtain a technical solution evaluation result.
[0144] The technical solution evaluation device provided in the embodiment of the present application adopts the technical solution evaluation method in the above embodiment to solve the technical problem of technical solution evaluation. Compared with the prior art, the beneficial effects of the technical solution evaluation device provided in the embodiment of the present application are the same as the beneficial effects of the technical solution evaluation method provided in the above embodiment, and other technical features in the technical solution evaluation device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0145] The present application provides a technical solution evaluation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the technical solution evaluation method in the above-mentioned embodiment one.
[0146] Reference below Figure 6, which shows a schematic diagram of the structure of a technical solution evaluation device suitable for implementing the embodiment of the present application. The technical solution evaluation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The technical solution evaluation device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0147] like Figure 6 As shown, the technical solution evaluation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the technical solution evaluation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the technical solution evaluation device to communicate with other devices wirelessly or wired to exchange data. Although the technical solution evaluation device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided alternatively.
[0148] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0149] The technical solution evaluation device provided by the present application adopts the technical solution evaluation method in the above embodiment to solve the technical problem of technical solution evaluation. Compared with the prior art, the beneficial effects of the technical solution evaluation device provided by the present application are the same as the beneficial effects of the technical solution evaluation method provided by the above embodiment, and the other technical features in the technical solution evaluation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0150] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0151] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0152] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the technical solution evaluation method in the above-mentioned embodiment.
[0153] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0154] The above-mentioned computer-readable storage medium may be included in the technical solution evaluation device; or it may exist independently without being assembled into the technical solution evaluation device.
[0155] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the technical solution evaluation device, the technical solution evaluation device: in response to a technical solution evaluation request, inputs the technical solution information of the technical solution to be evaluated into a pre-trained large language model, and retrieves the relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model; based on the relevant technical solutions, performs a creative evaluation on the technical solution to be evaluated to obtain a technical solution evaluation result.
[0156] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0157] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0158] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0159] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned technical solution evaluation method, and can solve the technical problems of technical solution evaluation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the technical solution evaluation method provided in the above-mentioned embodiment, and will not be repeated here.
[0160] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned detection method when executed by a processor.
[0161] The computer program product provided in this application can solve the technical problem of technical solution evaluation. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the technical solution evaluation method provided in the above embodiment, which will not be repeated here.
[0162] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.
Claims
1. A technical solution evaluation method, characterized in that: The technical solution evaluation method includes: In response to a technical solution evaluation request, input technical solution information of the technical solution to be evaluated into a pre-trained large language model, and retrieve relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model; According to the relevant technical solutions, a creative evaluation is performed on the technical solution to be evaluated to obtain a technical solution evaluation result.
2. The method according to claim 1, characterized in that The step of retrieving relevant technical solutions in a preset technical solution database according to the technical solution information by using the pre-trained large language model comprises: Performing semantic feature analysis on the technical solution information through a pre-trained large language model to obtain semantic feature information of the technical solution to be evaluated; The technical solutions in the technical solution database are searched according to the semantic feature information to obtain relevant technical solutions.
3. The method according to claim 2, characterized in that The semantic feature information of the technical solution to be evaluated includes a first technical solution feature vector of the technical solution to be evaluated, and the step of performing semantic feature analysis on the technical solution information through a pre-trained large language model to obtain the semantic feature information of the technical solution to be evaluated includes: Performing semantic analysis on the technical solution to be evaluated by using a pre-trained large language model to obtain semantic information of the technical solution to be evaluated; Preprocessing the semantic information, and extracting features from the preprocessed semantic information according to a preset feature extraction method to obtain a feature vector of the first technical solution; The step of retrieving the technical solutions in the technical solution database according to the semantic feature information to obtain relevant technical solutions includes: Obtaining existing solutions through the technical solution database; Calculating the similarity between the technical solution to be evaluated and the existing solution according to the first technical solution feature vector; Sort the existing solutions according to the similarity, and select a preset number of existing solutions as relevant technical solutions; The step of conducting a creativity evaluation on the technical solution to be evaluated based on the relevant technical solution to obtain a technical solution evaluation result includes: According to the similarity, obtaining the rating item similarity corresponding to the preset rating item; Calculating the average similarity of the scoring items of the related technical solutions according to the scoring item similarity and the number of preset scoring items; According to the average similarity of the scoring items and the preset technical creativity scoring rules, the technical solution to be evaluated is scored for technical creativity to obtain a technical solution evaluation result.
4. The method according to claim 1, characterized in that The step of conducting a creativity evaluation on the technical solution to be evaluated based on the relevant technical solution to obtain a technical solution evaluation result includes: Extracting technical key points of the technical solution to be evaluated and the related technical solutions through a pre-trained large language model to obtain the technical key points of the technical solution to be evaluated and the technical key points of the related technical solutions; A semantic comparative analysis is performed on the technical points of the technical solution to be evaluated and the technical points of the related technical solutions to obtain a technical solution evaluation result.
5. The method according to claim 3, characterized in that Before the step of calculating the similarity between the technical solution to be evaluated and the existing solution according to the first technical solution feature vector, the method further comprises: Extracting features from the existing solution to obtain a feature vector of the existing solution; An index is constructed according to the existing solution and the feature vector of the existing solution, and the existing solution and the feature vector of the existing solution are stored in the technical solution database according to the index.
6. The method according to claim 1, characterized in that The step of conducting a creativity evaluation on the technical solution to be evaluated according to the related technical solution to obtain a technical solution evaluation result also includes: Testing the technical solution to be evaluated to determine the solution type of the technical solution to be evaluated; If the solution type is a technical creativity point, the technical solution to be evaluated is scored for technical creativity based on the relevant technical solutions to obtain a technical solution evaluation result; If the solution type is a technical briefing document, a semantic comparative analysis is performed on the technical solution to be evaluated based on the relevant technical solutions to obtain a technical solution evaluation result.
7. A technical solution evaluation device, characterized in that: The technical solution evaluation device comprises: A retrieval module, configured to respond to a technical solution evaluation request, input technical solution information of the technical solution to be evaluated into a pre-trained large language model, and retrieve relevant technical solutions in a preset technical solution database according to the technical solution information through the pre-trained large language model; The evaluation module is used to perform a creative evaluation on the technical solution to be evaluated based on the relevant technical solution to obtain a technical solution evaluation result.
8. A technical solution evaluation device, characterized in that: The technical solution evaluation device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the technical solution evaluation method according to any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the technical solution evaluation method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the technical solution evaluation method as described in any one of claims 1 to 6 are implemented.
Citation Information
Cited By
Motor vehicle structural damage assessment method and system and electronic equipment
CN119599291A
Method for calculating patent relevancy based on large model
CN120470109A
A method for calculating patent relevance based on a large model
CN120470109B
Contrastive analysis method and device of technical scheme
CN121327365A
