Scientific and technological innovation power index evaluation method based on large language model and adaptive learning

Abstract

The invention discloses a scientific and technological innovation power index evaluation method based on a large language model and adaptive learning. The method comprises the steps of collecting texts and metadata of a scientific and technological project to be evaluated; inputting a pre-trained large language model to extract a semantic feature set; inputting the semantic features into an embedding model to obtain an embedding matrix; calculating semantic similarity based on the embedded matrix and constructing an entity relation graph; combining the text, the metadata and the relational graph to extract time sequence characteristics of academic and industrial propagation, and generating a propagation influence vector; mapping and normalizing the semantic features, the embedded matrix and the propagation influence vector, and then storing into a feature library; taking the feature library and the historical annotation sample as supervision signals, and adopting an adaptive learning algorithm to obtain a weight estimation value; and the innovation power index of the science and technology project is calculated by using the feature library and the weight estimation value, and scientific and objective evaluation of the innovation power is realized. According to the method, semantic features, relation propagation features and weight adaptive learning are fused, and the comprehensiveness and reliability of scientific and technological innovation force evaluation are improved.

Description

Technical Field

[0001] This invention belongs to the field of science and technology project evaluation technology, specifically involving a method for evaluating a science and technology innovation capability index based on a large language model and adaptive learning. Background Technology

[0002] Technological innovation capability is a crucial indicator for measuring the core competitiveness of a nation, enterprise, or research institution. Its scientific and accurate assessment is of great significance for formulating science and technology development strategies, optimizing resource allocation, and promoting technological progress. Traditional methods for assessing technological innovation capability primarily rely on quantitative indicators, such as the number of patents, the number of paper citations, research funding, and industrial output, supplemented by expert scoring for qualitative evaluation. These methods can, to a certain extent, reflect the macro-level of technological innovation. For example, in existing technology, Chinese patent CN120633641A discloses a method and device for evaluating the effectiveness of technological breakthroughs based on a large language model. This method first constructs an evaluation index system covering dimensions such as technological innovation, technological feasibility, economic benefits, and environmental impact, collecting expert text evaluations of each project under each index; extracts semantic features of the text using a BERT model, combines this with sentiment analysis using a large language model, quantifies the results into satisfaction scores, and constructs an evaluation decision matrix; then, it uses the CRITIC method to calculate the index weights, and finally combines the grey relational analysis-ideal point method to comprehensively score and rank the projects.

[0003] However, this method has the following limitations: 1. It mainly relies on a pre-defined evaluation index system and expert textual evaluation, lacking utilization of unstructured data such as technical documents, experimental logs, code repository descriptions, and market / industry reports, making it difficult to comprehensively reflect the multidimensional innovation characteristics of science and technology projects. 2. Although this method uses BERT and large language models for sentiment analysis, it only stays at the level of quantifying sentiment tendencies and cannot deeply understand the semantic information in the text of science and technology projects, such as technical points, innovation statements, key parameters, and methodological steps, nor can it quantitatively assess the technological novelty of the projects. 3. Existing technologies lack a systematic integration of semantic information at different granularities (such as terms, sentences, documents, and entity levels), and have not constructed a relationship network between projects or technical elements, resulting in an inability to fully capture the correlation and dissemination relationships of scientific and technological innovation. 4. This method relies on the standard deviation and correlation of indicators, with fixed weights that cannot be adaptively adjusted, and does not provide quantification of the uncertainty of weight estimation, which may lead to bias and instability in the comprehensive score when facing multi-source heterogeneous data.

[0004] Therefore, existing technologies have significant shortcomings when processing large-scale, multi-source, and multi-dimensional scientific and technological project data, making it difficult to achieve a comprehensive, dynamic, interpretable, and high-precision assessment of the project's innovativeness. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for evaluating the index of scientific and technological innovation capability based on large language models and adaptive learning. The objective of this invention can be achieved through the following technical solutions: This invention provides a method for evaluating a technological innovation capability index based on a large language model and adaptive learning, comprising the following steps: Collect text and metadata of the technology projects to be evaluated; Input text and metadata into a pre-trained large language model, automatically generate semantic units based on predefined prompt templates, and output a set of semantic features. Input the semantic feature set into the embedding model to generate a multi-granularity vector representation and obtain the embedding matrix; Semantic similarity calculations are performed based on the embedding matrix, and an entity relationship graph is constructed. Based on the text, metadata, and entity relationship graph, the temporal features of academic and industrial communication are extracted to generate a communication impact vector; The semantic feature set, embedding matrix, and propagation influence vector are mapped, normalized, and then incorporated into the feature library; Using the feature library and historical labeled samples as supervision signals, an adaptive learning algorithm is used to learn feature weights and estimate weight uncertainty, and output weight estimates. Using the aforementioned feature library and weight estimates, the science and technology innovation capability index of the science and technology project to be evaluated is calculated, thereby realizing the evaluation of the science and technology innovation capability of the project to be evaluated.

[0006] Furthermore, the text and metadata of the technology projects to be evaluated include project introduction, patent abstracts, paper abstracts, technical roadmap documents, experiment / prototype logs, code repository descriptions, team member introductions, and relevant market / industry reports.

[0007] Furthermore, the step of inputting text and metadata into a pre-trained large language model, automatically generating semantic units based on predefined prompt templates, and outputting a set of semantic features specifically includes: The text and metadata are segmented into document, paragraph, sentence and term units, and each smallest segmented text unit is used as a candidate semantic unit to input into a predefined prompt template; Based on several predefined prompt templates, the pre-trained large language model is called in parallel for each candidate semantic unit, and the following semantic information types are extracted and used as atomic features: technical points, innovation statement, technical difficulties / risks, methods or process steps, key parameters and quantitative results, differences in prior work, expected application scenarios, implementation requirements, institutional / author entities, and time / stage information. For the same candidate semantic unit output from different prompt templates or different models, the results are aggregated according to prior weights or template confidence, and a summary text and initial model confidence score for the semantic unit are generated using a voting or confidence-weighted method. ; Calculate the source reliability score for each semantic unit by combining source metadata. The source reliability score is determined based on the document source type, the reputation of the publishing organization, and the time of publication, and is fused with the initial model confidence score to produce the final confidence score. The formula is: in, , , To preset weights, Standardized confidence level for the model output; Score for source reliability; The consistency coefficient for multiple templates / multi-models is defined as the ratio of the number of consistent outputs to the total number of templates called. A novelty score is calculated for each semantic unit. The novelty score is based on calculating the similarity between the semantic vector of the semantic unit and a historical corpus or a set of semantic vectors from prior work. The formula is: in, The similarity score is calculated with the most similar unit in the historical corpus. For each semantic unit, the text fragment, semantic type label, and final confidence score are considered. Novelty score The source identifier and timestamp are encapsulated into semantic feature records, and several semantic feature records are aggregated to form the semantic feature set as output.

[0008] Furthermore, the step of inputting the semantic feature set into the embedding model to generate a multi-granularity vector representation and obtaining the embedding matrix specifically includes: For each semantic feature record in the semantic feature set k Including text fragments and final confidence levels Novelty score Source reliability score With timestamp Call the embedding model to obtain atomic vector representations. and to Normalization is performed to obtain a standardized vector. ; Based on timestamp Calculate the time decay factor : in, The time decay rate, For the assessment time, Indicates the first k Timestamps of semantic features; Calculate the overall importance score for each semantic feature. : in, For the first k The final confidence level of each semantic feature To score for novelty, To score the reliability of the source, The time decay factor, These are weighting coefficients; Importance scores for all features Perform softmax normalization to obtain attention weights. : in, The total number of semantic features. For the first k Attention weights for each semantic feature; Map atomic vectors to corresponding granularity vectors based on a predefined granularity set. And aggregate them at each granularity according to attention weights to obtain a granularity aggregation vector: in, Indicates in granularity g Vector representation below; The granularity includes term granularity, sentence / segment granularity, document granularity, and entity granularity. Aggregate vectors and atomic vectors at different granularities together constitute the embedding matrix. in, For the embedding matrix, including K A standardized atomic vector and G A granularity aggregation vector.

[0009] Furthermore, the step of performing semantic similarity calculation and constructing an entity relationship graph based on the embedding matrix specifically includes: Embedding matrix The vectors in the data are divided into a set of entity vectors. Each of them Corresponding to a semantic unit or granularity aggregation unit; Calculate any two entity vectors Cosine similarity: in, Representing entities respectively With entity The vector representation of , The Euclidean norm of a vector; Representing entities With entity Cosine similarity; obtain the similarity matrix. ; According to the similarity matrix Calculate similarity threshold The similarity threshold The adaptive percentile method is used for calculation: in, This means taking the nth sample from a given set of samples. p percentile, For hyperparameters; Based on similarity threshold Similarity matrix Construct a binary adjacency matrix : in, For nodes i and j A binary indicator indicating whether an initial edge has been established between nodes; a value of 1 indicates that the node... i and j There are edges between them, and a value of 0 indicates a node. i and j There is no edge between them; Considering the attention weights between nodes, the edge weights are weighted to obtain a weighted adjacency matrix. The formula is: in, Represents a node i and j The edge weight between them. , They represent the first i , j Attention weights for each semantic feature; By weighted adjacency matrix Constructing an entity relationship diagram , where the set of nodes Corresponding entity vector set edge set Edge weights are determined by specified.

[0010] Furthermore, the step of extracting temporal features of academic and industrial communication based on text, metadata, and entity relationship graphs to generate a communication impact vector specifically includes: Extract a set of time-series events from the text and metadata. Each event includes a timestamp. Event type identifier With event intensity The events are sorted by time and output as a time series event set; the event type identifiers include academic citation events, reprints, new collaborations, and news reports, and the event intensity includes the number of new citations, the number of reprints, and the number of collaboration links; For entity relationship diagrams Each entity node i and each event type Calculate the cumulative impact Compared with short-term growth rate The cumulative impact is defined as the amount of impact within the observation window. Event intensity accumulation within: in, Represents entity nodes i In event type v The cumulative impact within the observation window, Indicates a point in time Corresponding to entity i And the event type is v The intensity of the event; The formula for the short-term growth rate is: in, Represents entity nodes i In event type v The short-term relative growth rate It is a very small positive constant. This represents the time series of event intensity after smoothing or normalization. For time difference; Calculate the node-level propagation potential score As a comprehensive measure, a normalized linear combination is used: in, This represents the normalization operator. Represents a node iThe time-series freshness score of recent activities is calculated based on the latest event time. , These are non-negative weighting coefficients; Propagation potential score of all nodes As a vector of propagation impact.

[0011] Furthermore, the step of mapping and normalizing the semantic feature set, embedding matrix, and propagation influence vector before incorporating them into the feature library specifically includes: For each semantic feature record in the semantic feature set, a mapping function is called to extract the text vector, semantic type label, confidence score, novelty score, and source information, and then numerically represented. For the atomic vectors and granular aggregation vectors in the embedding matrix M, dimension mapping is performed according to a unified low-dimensional space projection function to obtain comparable standard vector representations; The propagation potential energy scores of each node in the propagation influence vector are normalized to ensure that they are consistent with the numerical range of semantic features and embedded features. The semantic feature mapping results, embedding matrix mapping results, and propagation influence vector normalization results are aligned using a unified key-value index and stored in a unified structured feature library. Each record includes a feature identifier, a numerical feature vector, source information, and a timestamp.

[0012] Furthermore, the feature library includes a semantic feature sub-library, an embedding feature sub-library, and a propagation feature sub-library, which respectively store numericalized semantic features, standardized embedding vectors, and normalized propagation potential scores; the feature library includes a cross-project index area, which supports the storage and management of feature records for multiple scientific and technological projects. Each scientific and technological project corresponds to a set of semantic features, embedding features, and propagation features, thereby forming a unified feature storage system with multiple projects and multiple dimensions.

[0013] Furthermore, the step of using the feature library and historical labeled samples as supervision signals, employing an adaptive learning algorithm to learn feature weights and estimate weight uncertainty, and outputting weight estimates specifically includes: For each science and technology project feature vector in the feature library, pair it with its corresponding historical labeled sample label to form a training dataset. : in, Indicates the first i Feature vectors of a technology project Indicates the first i The real innovation index corresponding to each science and technology project is manually scored by experts in the field according to a unified evaluation standard. The total number of samples; Initialize the weight parameter vector: in, Indicates the first j The weight of each feature in the science and technology innovation capability index d This represents the dimension of each feature vector; Define the prediction function: in, For the prediction function, Indicates the first i The eigenvector of the _th technology project j Values ​​in each dimension; Historically labeled sample tags To provide the monitoring signal, construct the loss function: in, For loss function, Let be the function that measures the difference between the predicted and actual values, and let be the mean squared error function. This is a regularization term used to suppress overfitting; An adaptive learning algorithm is used to iteratively update the weight vector, with the following update rule: in, Indicates the first t The weight vector of the next iteration For the first t The adaptive learning rate for each iteration This represents the gradient of the loss function with respect to the weight vector. In the weight learning process, Bayesian inference or approximate uncertainty estimation methods are introduced to model the weight distribution and obtain the mean of the weight estimates. ; The mean of the weighted estimates As the final weight estimate ; Furthermore, the step of using the feature library and weight estimates to calculate the science and technology innovation capability index of the project to be evaluated, thereby achieving the evaluation of the science and technology innovation capability of the project, specifically includes: Feature vectors extracted from the science and technology projects to be evaluated Compared with the final weight estimate Perform a linear combination to calculate the science and technology innovation index: in, This represents the overall technological innovation index of the technology projects to be evaluated.d This represents the dimension of each feature vector. Indicates the first j The final weight estimates of each feature; This represents the eigenvector of the technology project to be evaluated. j The values ​​of each dimension.

[0014] Compared with the prior art, the present invention has the following advantages: (1) In the existing technology, the evaluation methods for scientific and technological innovation capability mainly rely on a pre-set evaluation index system and expert scoring, which lacks the utilization of unstructured data, resulting in the inability to fully reflect the multidimensional innovation characteristics of scientific and technological projects. This invention collects the text and metadata of the scientific and technological projects to be evaluated, including project introduction, patent abstract, paper abstract, technical roadmap document, experimental / prototype log, code repository description and related market / industry reports, and inputs them into a pre-trained large language model and embedding model to generate a semantic feature set and multi-granular vector representation, which are then incorporated into a unified feature library. This achieves full utilization of unstructured multi-source data, enabling the evaluation of scientific and technological innovation capability to comprehensively cover the technological innovation, feasibility, market potential and application value of the project, thereby improving the accuracy and completeness of the evaluation.

[0015] (2) Although existing technologies use BERT and large language models for sentiment analysis, they only quantify sentiment tendencies and lack an understanding of the deep semantic information in the text of scientific and technological projects, such as technical points, innovation statements, key parameters, methods and processes, and differences in prior work. They also cannot quantify the novelty of the projects. This invention extracts atomic features such as technical points, innovation statements, methods and processes, key parameters, differences in prior work, and application scenarios from each semantic unit and calculates a novelty score. It maps text information into structured semantic feature vectors and combines source reliability and multi-template / multi-model consistency for confidence weighting. This achieves a deep understanding of the technical content of the projects and quantifies their novelty, thereby effectively solving the problems of insufficient semantic understanding and novelty assessment and enhancing the scientificity and reliability of the assessment.

[0016] (3) Existing technologies lack systematic integration of semantic information at different granularities and have not constructed relationship networks between projects or technical elements, making it difficult to capture the correlation and dissemination relationships of technological innovation. This invention aggregates atomic vectors at the granularity of terms, sentences, documents, entities, etc., to construct an embedding matrix, and constructs an entity relationship graph based on cosine similarity calculation and adaptive thresholding method. It weights the relationships between nodes to form a dissemination network, realizing the modeling of multi-granularity and multi-relationship features. This enables the assessment of technological innovation capabilities to reflect the correlation and knowledge network structure between technical elements, thereby improving the comprehensiveness and interpretability of the assessment.

[0017] (4) Existing technologies calculate index weights using the CRITIC method, but the weights are fixed and cannot be adaptively adjusted. Furthermore, the lack of quantification of the uncertainty in weight estimation leads to bias and instability in the comprehensive score under multi-source heterogeneous data conditions. This invention trains the feature vectors of scientific and technological projects in the feature library and historical labeled samples using an adaptive learning algorithm. It iteratively updates the weight vectors and introduces Bayesian inference or approximate uncertainty estimation methods during the weight learning process to model the weight distribution. The estimated weights and their mean are output as the final weights, achieving adaptive weight adjustment and quantifying uncertainty, thereby improving the stability, reliability, and interpretability of the evaluation results.

[0018] (5) Existing technologies lack direct and unified methods for applying features and weights in comprehensive evaluation calculations, resulting in complex evaluation processes and difficulty in rapidly evaluating new projects. This invention directly calculates the science and technology innovation index by linearly combining the feature vectors extracted from the science and technology projects to be evaluated with the final weight estimates. This achieves rapid, repeatable, and quantifiable evaluation of science and technology innovation capabilities based on a unified feature library and weight model, thereby solving the problems of complexity and low efficiency in existing comprehensive evaluation methods and making the evaluation process more efficient and scalable.

[0019] (6) Existing technologies do not consider the time-series dissemination characteristics of projects in terms of academic citations, collaborative networks, industrial transformation, and news reports, making it difficult to measure the real-time impact and diffusion trend of innovative achievements. This invention extracts event time series, including academic citations, reprints, new collaborations, and news reports, calculates the cumulative impact at the node level, short-term growth rate, and time-series freshness, forms a dissemination impact vector, and stores it in a normalized feature library. This enables the quantification and dynamic characterization of the dissemination impact of scientific and technological projects in the academic and industrial fields, thereby solving the problem of the lack of dissemination characteristic analysis in existing technologies and enabling innovation assessment to reflect the real-time diffusion and impact of projects. Attached Figure Description

[0020] Figure 1 This is a flowchart of the technological innovation capability index evaluation method according to an embodiment of the present invention; Figure 2 This is a model diagram of the technological innovation capability index evaluation system according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] Example 1: This embodiment provides a method for evaluating a technological innovation capability index based on a large language model and adaptive learning, such as... Figure 1 As shown, it includes the following steps: Step S1: Collect text and metadata of the technology projects to be evaluated; The first step is to define the scope of the technology projects to be evaluated, including ongoing R&D projects, completed research projects, and related technology incubation projects. The types of text and metadata collected include: Basic project information: project overview, research objectives, basis for project approval, funding, implementation period, etc. Research output documents: patent abstracts and full texts, academic paper abstracts and full texts, technical roadmap documents, experimental reports, prototype development logs, and technical specifications. Software and data resources: code repository descriptions, algorithm model documentation, dataset descriptions and version history; Team and Organization Information: Team member profiles, information on R&D units and collaborating institutions, and records of guidance from experts or mentors; Market and industry-related documents: industry application reports, market analysis reports, competitor analysis, technology promotion and industrialization evaluation materials; Other relevant materials: news reports, industry white papers, policy documents, or project review opinions, etc.

[0023] Step S2: Input the text and metadata into the pre-trained large language model, automatically generate semantic units based on predefined prompt templates, and output a set of semantic features, specifically including: The text and metadata are segmented into document, paragraph, sentence and term units, and each smallest text unit after segmentation is used as a candidate semantic unit to input a predefined prompt template; Based on several predefined prompt templates, the pre-trained large language model is called in parallel for each candidate semantic unit, and the following semantic information types are extracted and used as atomic features: technical points, innovation statement, technical difficulties / risks, methods or process steps, key parameters and quantitative results, differences in prior work, expected application scenarios, implementation requirements, institutional / author entities, and time / stage information. For the same candidate semantic unit output from different prompt templates or different models, the results are aggregated according to prior weights or template confidence, and a summary text and initial model confidence score for the semantic unit are generated using a voting or confidence-weighted method. ; Calculate the source reliability score for each semantic unit by combining source metadata. The source reliability score is determined based on the document source type, the reputation of the publishing organization, and the timeliness of the document, and is then fused with the initial model confidence score to produce the final confidence score. The formula is: in, , , To preset weights, Standardized confidence level for the model output; Score for source reliability; The consistency coefficient for multiple templates / multi-models is defined as the ratio of the number of consistent outputs to the total number of templates called. A novelty score is calculated for each semantic unit. The novelty score is based on calculating the similarity between the semantic vector of the semantic unit and the semantic vector set of historical corpora or prior work. The formula is: in, The similarity score is calculated with the most similar unit in the historical corpus. For each semantic unit, the text fragment, semantic type label, and final confidence score are considered. Novelty score Source identifier and timestamp are encapsulated into semantic feature records, and several semantic feature records are aggregated to form a semantic feature set as output.

[0024] Step S3: Input the semantic feature set into the embedding model to generate a multi-granularity vector representation and obtain the embedding matrix, specifically including: For each semantic feature record in the semantic feature set k Including text fragments and final confidence levels Novelty score Source reliability score With timestamp Call the embedding model to obtain atomic vector representations. and to Normalization is performed to obtain a standardized vector. ; Based on timestamp Calculate the time decay factor : in, The time decay rate, For the assessment time, Indicates the first k Timestamps of semantic features; Calculate the overall importance score for each semantic feature. : in, For the first k The final confidence level of each semantic feature To score for novelty, To score the reliability of the source, The time decay factor, These are weighting coefficients; Importance scores for all features Perform softmax normalization to obtain attention weights. : in, The total number of semantic features. For the first k Attention weights for each semantic feature; Map atomic vectors to corresponding granularity vectors based on a predefined granularity set. And aggregate them at each granularity according to attention weights to obtain a granularity aggregation vector: in, Indicates in granularity g Vector representation below; Granularity includes term granularity, sentence / segment granularity, document granularity, and entity granularity. Aggregated vectors and atomic vectors at different granularities together constitute the embedding matrix. in, For the embedding matrix, including K A standardized atomic vector and G A granularity aggregation vector.

[0025] Step S4: Perform semantic similarity calculation based on the embedding matrix and construct an entity relationship graph, specifically including: Embedding matrix The vectors in the data are divided into a set of entity vectors. Each of them Corresponding to a semantic unit or granularity aggregation unit; Calculate any two entity vectors Cosine similarity: in, Representing entities respectively With entity The vector representation of , The Euclidean norm of a vector; Representing entities With entity Cosine similarity; obtain the similarity matrix. ; Based on the similarity matrix Calculate similarity threshold Similarity threshold The adaptive percentile method is used for calculation: in, This means taking the nth sample from a given set of samples. p percentile, For hyperparameters; Based on similarity threshold Similarity matrix Construct a binary adjacency matrix : in, For nodes i and j A binary indicator indicating whether an initial edge has been established between nodes; a value of 1 indicates that the node... i and j There are edges between them, and a value of 0 indicates a node. i and j There is no edge between them; Considering the attention weights between nodes, the edge weights are weighted to obtain a weighted adjacency matrix. The formula is: in, Represents a node i and j The edge weight between them. , They represent the first i , j Attention weights for each semantic feature; By weighted adjacency matrix Constructing an entity relationship diagram , where the set of nodes Corresponding entity vector set edge set Edge weights are determined by specified.

[0026] Step S5: Extract the temporal features of academic and industrial communication based on the text, metadata, and entity relationship graph, and generate a communication impact vector, specifically including: Extracting time-series event sets from text and metadata Each event includes a timestamp. Event type identifier With event intensity The event is sorted by time and output as a time series event set. Event type identifiers include academic citation events, reprints, new collaborations, and news reports. Event intensity includes the number of new citations, the number of reprints, and the number of collaboration links. For entity relationship diagrams Each entity node i and each event type Calculate the cumulative impact Compared with short-term growth rate The cumulative impact is defined as the amount of impact within the observation window. Event intensity accumulation within: in, Represents entity nodes i In event type v The cumulative impact within the observation window, Indicates a point in time Corresponding to entity i And the event type is v The intensity of the event; The formula for the short-term growth rate is: in, Represents entity nodes i In event type v The short-term relative growth rate It is a very small positive constant. This represents the time series of event intensity after smoothing or normalization. For time difference; Calculate the node-level propagation potential score As a comprehensive measure, a normalized linear combination is used: in, This represents the normalization operator. Represents a node i The time-series freshness score of recent activities is calculated based on the latest event time. , These are non-negative weighting coefficients; Propagation potential score of all nodes As a vector of propagation impact.

[0027] Step S6: Map and normalize the semantic feature set, embedding matrix, and propagation influence vector, then import them into the feature library. Specifically, this includes: For each semantic feature record in the semantic feature set, the mapping function is called to extract the text vector, semantic type label, confidence score, novelty score and source information, and then the data is represented numerically. For the atomic vectors and granular aggregate vectors in the embedding matrix M, dimension mapping is performed based on a unified low-dimensional space projection function to obtain comparable standard vector representations; The propagation potential energy scores of each node in the propagation influence vector are normalized to ensure that they are consistent with the numerical range of semantic features and embedded features. The semantic feature mapping results, embedding matrix mapping results, and propagation influence vector normalization results are aligned using a unified key-value index and stored in a unified structured feature library. Each record includes a feature identifier, a numerical feature vector, source information, and a timestamp.

[0028] Step S7: Using the feature library and historical labeled samples as supervision signals, an adaptive learning algorithm is used to learn feature weights and estimate weight uncertainty, and output weight estimates. The feature library includes a semantic feature sub-library, an embedding feature sub-library, and a propagation feature sub-library, which respectively store numericalized semantic features, standardized embedding vectors, and normalized propagation potential scores. The feature library also includes a cross-project index area, which supports the storage and management of feature records for multiple science and technology projects. Each science and technology project corresponds to a set of semantic features, embedding features, and propagation features, thus forming a unified feature storage system with multiple projects and multiple dimensions.

[0029] Step S7 specifically includes: For each science and technology project feature vector in the feature library, pair it with its corresponding historical labeled sample labels to form a training dataset. : in, Indicates the first i Feature vectors of a technology project Indicates the first i The real innovation index corresponding to each science and technology project is manually scored by experts in the field according to a unified evaluation standard. The total number of samples; Initialize the weight parameter vector: in, Indicates the first j The weight of each feature in the science and technology innovation capability index d This represents the dimension of each feature vector; Define the prediction function: in, For the prediction function, Indicates the first i The eigenvector of the _th technology project jValues ​​in each dimension; Historically labeled sample tags To provide the monitoring signal, construct the loss function: in, For loss function, Let be the function that measures the difference between the predicted and actual values, and let be the mean squared error function. This is a regularization term used to suppress overfitting; An adaptive learning algorithm is used to iteratively update the weight vector, with the following update rule: in, Indicates the first t The weight vector of the next iteration For the first t The adaptive learning rate for each iteration This represents the gradient of the loss function with respect to the weight vector. In the weight learning process, Bayesian inference or approximate uncertainty estimation methods are introduced to model the weight distribution and obtain the mean of the weight estimates. ; The mean of the weighted estimates As the final weight estimate ; Step S8: Using the feature library and weight estimates, calculate the science and technology innovation capability index of the project to be evaluated, thereby achieving the evaluation of the science and technology innovation capability of the project. Specifically, this includes: Feature vectors extracted from the science and technology projects to be evaluated Compared with the final weight estimate Perform a linear combination to calculate the science and technology innovation index: in, This represents the overall technological innovation index of the technology projects to be evaluated. d This represents the dimension of each feature vector. Indicates the first j The final weight estimates of each feature; This represents the eigenvector of the technology project to be evaluated. j The values ​​of each dimension.

[0030] Example 2: This embodiment provides a technology innovation capability index evaluation system based on a large language model and adaptive learning, such as... Figure 2 As shown, it includes: The data acquisition module is used to collect text and metadata of the technology projects to be evaluated, including project introduction, patent abstract, paper abstract, technical roadmap document, experiment / prototype log, code repository description, team member introduction and relevant market / industry report, and to preprocess and standardize the storage of raw data; The semantic feature extraction module inputs the collected text and metadata into the pre-trained large language model, automatically generates semantic units based on predefined prompt templates, and outputs a set of semantic features, including technical points, innovation statements, technical difficulties / risks, methods or process steps, key parameters and quantitative results, differences in prior work, expected application scenarios, implementation requirements, institutional / author entities and time / stage information; The multi-granularity embedding module inputs the semantic feature set into the embedding model to generate a multi-granularity vector representation, obtains the embedding matrix composed of atomic vectors and granular aggregated vectors, and calculates the comprehensive importance score and attention weight of the features. The entity relationship graph construction module calculates the semantic similarity between entity vectors based on the embedding matrix, constructs a weighted adjacency matrix, and generates an entity relationship graph that represents each entity and its semantic associations. The dissemination feature extraction module extracts the temporal features of academic and industrial dissemination based on text, metadata, and entity relationship graphs, calculates node-level dissemination potential scores, and generates dissemination impact vectors. The feature library management module maps and normalizes semantic features, embedding matrices, and propagation influence vectors into a unified structured feature library, supporting the storage and management of multi-project and multi-dimensional features. The adaptive weight learning module uses the feature library and historical labeled samples as supervision signals, adopts an adaptive learning algorithm to learn the weights of each feature, estimates the weight uncertainty, and outputs the final weight estimate. The innovation index calculation module uses the feature vectors of the science and technology projects to be evaluated and the final weight estimates to perform a linear combination, calculates the scores of the stratified sub-indicators and the overall science and technology innovation index, and realizes a comprehensive evaluation of the innovation of science and technology projects.

[0031] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0032] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A technology innovation power index evaluation method based on a large language model and adaptive learning, characterized in that, The method comprises the following steps: Collecting text and metadata of a to-be-evaluated scientific and technological project; Inputting the text and metadata into a pre-trained large language model to automatically extract semantic units based on a pre-defined prompt template and output a semantic feature set; Inputting the semantic feature set into an embedding model to generate a multi-granularity vector representation and obtain an embedding matrix; Performing semantic similarity calculation based on the embedding matrix and constructing an entity relationship graph; Extracting time sequence features of academic and industrial communication from the text and metadata and the entity relationship graph to generate a communication influence vector; Mapping and normalizing the semantic feature set, the embedding matrix, and the communication influence vector and then importing them into a feature library; Using the feature library and historical labeled samples as a supervision signal, learning feature weights and estimating weight uncertainty using an adaptive learning algorithm, and outputting a weight estimation value; Using the feature library and the weight estimation value to calculate the scientific and technological innovation index of the to-be-evaluated scientific and technological project to realize scientific and technological innovation evaluation of the to-be-evaluated scientific and technological project.

2. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The text and metadata of the to-be-evaluated scientific and technological project include project introduction, patent abstract, paper abstract, technical route document, experiment / prototype log, code repository description, team member profile, and related market / industry report.

3. The technology innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The method of inputting the text and metadata into a pre-trained large language model to automatically extract semantic units based on a pre-defined prompt template and output a semantic feature set comprises the following steps: Segmenting the text and metadata by document, paragraph, sentence, and term unit, and inputting each segmented minimum text unit as a candidate semantic unit into a pre-defined prompt template; Parallelly calling the pre-trained large language model for each candidate semantic unit based on a plurality of pre-defined prompt templates to extract the following semantic information types as atomic features: technical points, innovation declaration, technical difficulties / risk, method or process steps, key parameters and quantitative results, differences from prior work, expected application scenarios, implementation requirements, institution / author entity, and time / phase information. For the same candidate semantic unit results output by different prompt templates or different models, the result aggregation is performed according to a prior weight or a template trust degree, and a voting or confidence weighting method is used to generate the summary text of the semantic unit and the initial model confidence score ; calculating a source reliability score for each semantic unit in conjunction with provenance metadata , the source reliability score is determined based on document source type, publisher reputation, and time recency, and is fused with the initial model confidence to produce a final confidence , the formula is: wherein, , , is a preset weight, is a normalized confidence of the model output; is a source reliability score; is a multi-template / multi-model consistency coefficient, defined as the ratio of the number of consistent outputs to the total number of invoked templates; calculating a novelty score for each semantic unit based on computing a similarity between the semantic vector of the semantic unit and a set of historical corpus or prior work semantic vectors resulting in the formula: wherein, is the similarity to the most similar unit in the historical corpus; For each semantic unit, the text fragment, semantic type label, and final confidence score are considered. Novelty score The source identifier and timestamp are encapsulated into semantic feature records, and several semantic feature records are aggregated to form the semantic feature set as output.

4. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The method of inputting the semantic feature set into an embedding model to generate a multi-granularity vector representation and obtain an embedding matrix comprises the following steps: For each semantic feature record in the semantic feature set k Including text fragments and final confidence levels Novelty score Source reliability score With timestamp Call the embedding model to obtain atomic vector representations. and to Normalization is performed to obtain a standardized vector. ; Based on timestamps Computing time decay factors : wherein, is a time decay rate, is an evaluation time, denotes a timestamp of the k semantic feature; computing a composite importance score for each semantic feature : wherein, is the final confidence score for the k th semantic feature, is the novelty score, is the source reliability score, is a time decay factor, is a weighting coefficient; importance scores for all features softmax normalization to get attention weights : wherein, is the total number of semantic features, is the attention weight of the k th semantic feature; mapping the atom vectors to corresponding granularity vectors according to a predefined set of granularities and aggregating over each granularity with attention weights to obtain a granularity aggregated vector: wherein represents a vector representation at a granularity g under a granularity The granularity includes term granularity, sentence / paragraph granularity, document granularity, and entity granularity, and the aggregated vectors and atomic vectors at different granularities jointly constitute the embedding matrix. wherein, is an embedding matrix comprising K standardized atom vectors and G granularity aggregated vectors.

5. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The method of performing semantic similarity calculation based on the embedding matrix and constructing an entity relationship graph comprises the following steps: Divide the vectors in the embedding matrix into a set of entity vectors where each corresponds to one semantic unit or granularity aggregation unit; Computing the cosine similarity of any two entity vectors : wherein, respectively represent entities and a vector representation of entity , represents the Euclidean norm of a vector; represents the cosine similarity between entities and entity ; a similarity matrix is obtained; According to the similarity matrix A similarity threshold is calculated , the similarity threshold is calculated using an adaptive percentile method: wherein, denotes the percentile of the given sample set taken at the p percentile, is a hyperparameter; Based on a similarity threshold With a similarity matrix Constructing a binary adjacency matrix : wherein, is a node i with j a binary indication of whether an initial edge is established between nodes i and j , 1 indicating that there is an edge between nodes i and j , 0 indicating that there is no edge between nodes weighting the edge weights to obtain a weighted adjacency matrix, taking into account the attention weights between nodes and nodes , the formula is: wherein, representing nodes i with j edge weights of edges between nodes, , representing attention weights of the first i , j semantic features, respectively; from a weighted adjacency matrix constructing an entity relationship graph wherein the set of nodes corresponding set of entity vectors the set of edges the edge weights are specified by ​ 6. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The method of extracting time sequence features of academic and industrial communication from the text and metadata and the entity relationship graph to generate a communication influence vector comprises the following steps: extracting a time series event set from the text and metadata wherein each event comprises a timestamp , an event type identifier and an event intensity ; sorting the events by time and outputting the time series event set; the event type identifier comprises an academic citation event, a reprint, a cooperative addition, a news report, and the event intensity comprises the number of added citations, the number of reprints, and the number of cooperative edges For the entity relationship graph each entity node i and each event type , compute a cumulative impact and a short-term growth rate , where the cumulative impact is defined as the accumulation of event intensity within an observation window : wherein, represents an entity node i at an event type v under a cumulative impact amount within an observation window, represents an event intensity at a time point corresponding to an entity i and an event type v is The short-term growth rate formula is: wherein, represents an entity node i at event type v a short-term relative growth rate, is a very small positive constant, represents a smoothed or normalized event intensity time series, is a time difference; Computing node-level propagated potential scores As a comprehensive measure, a normalized linear combination is employed: wherein, denotes a normalization operator, denotes a node i a recent activity temporal freshness score, computed based on a latest event time; , are non-negative weight coefficients; propagating the potential score for all nodes as a propagation influence vector.

7. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The method of mapping and normalizing the semantic feature set, the embedding matrix, and the communication influence vector and then importing them into a feature library comprises the following steps: For each semantic feature record in the semantic feature set, a mapping function is called to extract a text vector, a semantic type label, a confidence score, a novelty score, and source information, and to perform numerical representation; For the atomic vectors and granularity aggregated vectors in the embedding matrix M, a unified low-dimensional space projection function is used for dimension mapping to obtain comparable standard vector representations; Normalization is performed on the node communication potential scores in the communication influence vector to make them consistent with the numerical intervals of the semantic features and the embedding features. The semantic feature mapping result, embedding matrix mapping result and propagation influence vector normalization result are aligned according to a unified key-value index and stored in a unified structured feature library, wherein each record includes a feature identifier, a numerical feature vector, source information and a time stamp.

8. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The feature library includes a semantic feature sub-library, an embedding feature sub-library and a propagation feature sub-library, which respectively store the numerical semantic features, the standardized embedding vectors and the normalized propagation potential scores; the feature library includes a cross-project index area, which supports the storage and management of feature records of multiple science and technology projects, and each science and technology project corresponds to a set of semantic features, embedding features and propagation features, thereby forming a multi-project and multi-dimensional unified feature storage system.

9. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The feature library and historical labeled samples are used as a supervision signal to learn feature weights and estimate weight uncertainty by using an adaptive learning algorithm, and a weight estimation value is output, specifically including: The feature library is formed by pairing each scientific and technological project feature vector in the feature library with a corresponding historical labeled sample label to form a training data set : in, Indicates the first i Feature vectors of a technology project Indicates the first i The real innovation index corresponding to each science and technology project is manually scored by experts in the field according to a unified evaluation standard. The total number of samples; Initializing a weight parameter vector: wherein, represents the weight of the j th feature in the technology innovation power index, d represents the dimension of each feature vector; Defining a prediction function: wherein, is a prediction function, denotes the value of the i dimension in the feature vector of the j technology project. and the historical annotation sample label as a supervision signal, and construct a loss function: wherein, is a loss function, is a difference measure function between the predicted value and the true value, and is a mean square error function, is a regularization term for suppressing overfitting; The weight vector is iteratively updated by using the adaptive learning algorithm, and the update rule is: wherein, denotes the weight vector of the t th iteration, is the adaptive learning rate of the t th iteration, is the gradient of the loss function with respect to the weight vector; In the weight learning process, Bayesian inference or approximate uncertainty estimation method is introduced to model the weight distribution and obtain the mean value of weight estimation ; the mean of the weight estimates as the final weight estimate .

10. The scientific and technological innovation power index evaluation method based on a large language model and adaptive learning according to claim 1, characterized in that, The feature library and the weight estimation value are used to calculate the scientific and technological innovation force index of the to-be-evaluated science and technology project, and the scientific and technological innovation force of the to-be-evaluated science and technology project is evaluated, specifically including: Feature vectors extracted for science and technology projects to be evaluated with the final weight estimates Performing linear combination, calculate science and technology innovation index: wherein, represents the overall scientific and technological innovation index of the scientific and technological project to be evaluated, d represents the dimension of each feature vector, represents the estimated value of the maximum weight of the j th feature; represents the value of the j th dimension in the feature vector of the scientific and technological project to be evaluated.

Images