A method and device for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis

Abstract

The present invention provides a method and device for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis, and the method includes: obtaining a text feature vector, a structural feature vector and a metadata feature vector of a scientific and technological intelligence source to be identified, and connecting them into a unified multimodal feature vector; inputting the multimodal feature vector into a hierarchical fusion model to obtain an output value of the hierarchical fusion model; obtaining the initial weight of each evaluation index, adjusting the initial weight of the evaluation index based on the domain similarity between the scientific and technological intelligence source to be identified and the predefined domain and the time perception trend of the scientific and technological theme in the scientific and technological intelligence source to be identified, and obtaining the final weight of each evaluation index, and obtaining the final value score of the scientific and technological intelligence source to be identified based on the output value of the hierarchical fusion model and the final weight of each evaluation index. The present invention can improve the evaluation accuracy, adaptability and timeliness of scientific and technological intelligence sources.

Description

Technical Field

[0001] The present invention relates to the field of digital analysis technology, and in particular to a method and device for evaluating a scientific and technological intelligence source for scientific and technological intelligence analysis. Background Art

[0002] In the current era of technological development, global scientific and technological intelligence information has exploded, covering a wide range of fields from cutting-edge scientific research trends to various technological applications. This information is often scattered across multiple websites, subscription platforms, and different publications, and is presented in different languages ​​and formats. Traditional scientific and technological intelligence source evaluation methods usually rely on manual screening and evaluation, which not only consumes a lot of time and human resources, but also makes it difficult to ensure the comprehensiveness and accuracy of the identification results, especially when faced with massive amounts of data.

[0003] On this basis, with the rise and application of machine learning, the scientific and technological intelligence source evaluation model developed for massive data is trained by collecting some existing manually labeled scientific and technological intelligence sources, so that the trained scientific and technological intelligence source evaluation model can automatically perform value assessment on the scientific and technological intelligence sources to be identified. However, the existing scientific and technological intelligence source evaluation model still has problems of insufficient accuracy and poor adaptability. Summary of the invention

[0004] In order to solve the above problems in the prior art, the present invention provides a method and device for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis, which can improve the evaluation accuracy and adaptability of scientific and technological intelligence sources.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] In a first aspect, the present invention provides a method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis, comprising the steps of:

[0007] S1, obtaining a text feature vector, a structural feature vector and a metadata feature vector of a scientific and technological intelligence source to be identified, and splicing the text feature vector, the structural feature vector and the metadata feature vector into a unified multimodal feature vector;

[0008] S2, inputting the multimodal feature vector into a hierarchical fusion model, and obtaining an output value of each evaluation index from the output of the hierarchical fusion model;

[0009] S3. Obtain the initial weight of each evaluation indicator, adjust the initial weight of the evaluation indicator based on the domain similarity between the scientific and technological intelligence source to be identified and the predefined domain and the time perception trend of the scientific and technological themes in the scientific and technological intelligence source to be identified, so as to obtain the final weight of each evaluation indicator, and obtain the final value score of the scientific and technological intelligence source to be identified based on the output value of the hierarchical fusion model and the final weight of each evaluation indicator.

[0010] The beneficial effect of the present invention is that by simultaneously processing the text content, structural features and metadata of a web page, important information that may be overlooked by a traditional single-modal method can be captured, thereby more accurately evaluating the quality and reliability of scientific and technological intelligence sources. The multimodal feature vector is processed through a hierarchical fusion model, and finally a basic score, a domain adaptation score and a time-aware score are formed to obtain a final value score, so as to dynamically optimize according to the characteristics of different scientific and technological fields and the dynamically changing intelligence environment, thereby improving the evaluation accuracy, adaptability and timeliness of scientific and technological intelligence sources.

[0011] Optionally, the step S1 comprises the steps of:

[0012] S11, obtaining the text feature vector Doc of the scientific and technological intelligence source to be identified vec for:

[0013] Doc vec =∑(TDWF(t,d,D)*Word vec (t)) / N

[0014] TDWF(t,d,D)=TF(t,d)*IDF(t,D)*DSF(t)

[0015] DSF(t)=log(1+F domain (t) / F general (t))

[0016] Where t represents a term, d represents a document, D represents a document set, TDWF(t,d,D) represents the term domain weight frequency, Word vec (t) represents the word vector of term t, N represents the number of terms in the document, TF(t,d) represents the term frequency of term t in document d, IDF(t,D) represents the inverse document frequency of term t, DSF(t) represents the domain-specific factor of term t, and F domain (t) represents the frequency of term t in the corpus of a specified scientific and technological field, F general (t) represents the frequency of term t in the general corpus;

[0017] S12, obtaining the structural complexity index SCI of the scientific and technological intelligence source to be identified and the information density of each key information area to form a structural feature vector, wherein the structural complexity index SCI and the information density of each key information area are:

[0018] SCI=∑(ws i *de i ) / N

[0019] Info density =∑(TDWF(t,r,D)) / L r

[0020] Where: ws i represents the predefined weight of label i, de i represents the nesting depth of tag i, N represents the total number of tags, r represents the text in the key information area, and L r The length of the text indicating the key information area;

[0021] S13, obtaining the metadata integrity score of the scientific and technological intelligence source to be identified, and obtaining a metadata feature vector according to the metadata integrity score, wherein the metadata integrity score Metascore is:

[0022] Metascore=∑(wmj*Ij) / M

[0023] Where wmj represents the weight of the j-th category metadata, Ij represents the indicator of the existence and completeness of the j-th category metadata, and M is the total number of metadata categories;

[0024] S14, splicing the text feature vector, the structure feature vector and the metadata feature vector into a unified multimodal feature vector;

[0025] Optionally, the word vector of the term t in step S11 is a pre-trained word embedding model to generate a vector representation for each term;

[0026] The key information area in step S12 is obtained by using a machine learning algorithm to identify the web page of the scientific and technological intelligence source to be identified;

[0027] The metadata in step S13 is extracted using a sequence labeling model.

[0028] Optionally, the metadata of the scientific and technological information source to be identified includes time information, author information, institution information, keywords and topics.

[0029] Optionally, the hierarchical fusion model in step S2 includes:

[0030] The feature layer preliminarily fuses the features of each modality in the multimodal feature vector through a self-attention mechanism, wherein the feature layer is respectively designed with corresponding encoders for the text modality, the structure modality and the metadata modality;

[0031] The interaction layer captures the relationship between different modal features through the cross-modal attention module, and uses the multi-head attention mechanism to independently calculate the attention of each head and then splice them. The gating unit is used to control the degree of fusion of different modal information, and finally generates a comprehensive cross-modal representation vector. The graph attention network is used to aggregate the information of all modalities to generate a global context vector;

[0032] The decision layer is used to concatenate the cross-modal representation vector and the global context vector output by the interaction layer to form the final feature vector, and finally output the result through ensemble learning.

[0033] Optionally, the ensemble learning comprises the steps of:

[0034] Design a dedicated sub-network for each preset evaluation task;

[0035] Introduce an attention mechanism to dynamically adjust the weights of different evaluation tasks and obtain the overall loss function generated according to the dynamically adjusted weights;

[0036] The prediction results of multiple base learners for each evaluation task are converted into probability distribution softmax(pred) through the softmax function, and then weighted average is performed to obtain the output result of each evaluation task.

[0037] Optionally, the evaluation indicators include information reliability, timeliness, content depth, citation completeness, author / institution authority and language expression quality.

[0038] Optionally, adjusting the initial weight of the evaluation index based on the domain similarity between the scientific and technological intelligence source to be identified and the predefined domain comprises the following steps:

[0039] Calculate the domain similarity Sim(d,Fk) between the scientific and technological intelligence source to be identified and the predefined domain:

[0040] Sim(d, Fk) = cosine(θ d , θ Fk )

[0041] Among them, cosine represents the cosine function, θ d represents the topic distribution of document d, θ Fk Represents the average topic distribution of the predefined domain FK;

[0042] Use the softmax function to convert the domain similarity Sim(d,Fk) into a weight adjustment factor α k , according to the weight adjustment factor α k Adjust the initial weight of each evaluation indicator to obtain the adjusted weight ws p ':

[0043] ws p '=ws p *(1+∑(α k *δ pk ))

[0044] α k =softmax(Sim(d,Fk))

[0045] Among them, ws p represents the initial weight of the pth evaluation index, δ pk It is the impact factor of the predefined domain FK on the pth evaluation indicator.

[0046] Optionally, adjusting the initial weight of the evaluation index based on the time-perceived trend of the scientific and technological topics in the scientific and technological intelligence source to be identified comprises the following steps:

[0047] The corresponding weights of evaluation indicators related to timeliness are adjusted according to the time perception trend of the scientific and technological topics in the scientific and technological intelligence source to be identified.

[0048] In a second aspect, the present invention provides a scientific and technological intelligence source evaluation device for scientific and technological intelligence analysis, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a scientific and technological intelligence source evaluation method for scientific and technological intelligence analysis of the first aspect is implemented.

[0049] Among them, the technical effect corresponding to the scientific and technological intelligence source evaluation device for scientific and technological intelligence analysis provided by the second aspect refers to the relevant description of the scientific and technological intelligence source evaluation method for scientific and technological intelligence analysis provided by the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the main process of a scientific and technological intelligence source evaluation method for scientific and technological intelligence analysis according to an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the framework of a scientific and technological intelligence source evaluation device for scientific and technological intelligence analysis according to an embodiment of the present invention.

[0052] Description of reference numerals:

[0053] 1. A scientific and technological intelligence source evaluation device for scientific and technological intelligence analysis;

[0054] 2. Processor;

[0055] 3. Memory. DETAILED DESCRIPTION

[0056] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0057] Embodiment 1

[0058] Please refer to Figure 1 to Figure 2 , a scientific and technological intelligence source evaluation method for scientific and technological intelligence analysis, comprising the steps of:

[0059] S1. Obtain the text feature vector, structure feature vector and metadata feature vector of the scientific and technological intelligence source to be identified, and splice the text feature vector, structure feature vector and metadata feature vector into a unified multimodal feature vector.

[0060] In this embodiment, the main purpose of the multimodal feature extraction technology is to comprehensively and deeply analyze and understand all aspects of the scientific and technological intelligence source to improve the accuracy and comprehensiveness of the recognition. By simultaneously processing the text content, structural features and metadata of the web page, the technology can capture important information that may be overlooked by the traditional single-modal method, thereby more accurately evaluating the quality and reliability of the scientific and technological intelligence source.

[0061] In this embodiment, step S1 includes the steps of:

[0062] S11. Obtain the text feature vector Doc of the scientific and technological intelligence source to be identified vec for:

[0063] Doc vec =∑(TDWF(t,d,D)*Word vec (t)) / N

[0064] TDWF(t,d,D)=TF(t,d)*IDF(t,D)*DSF(t)

[0065] DSF(t)=log(1+F domain (t) / F general (t))

[0066] Where t represents a term, d represents a document, D represents a document set, TDWF(t,d,D) represents the term domain weight frequency, Word vec (t) represents the word vector of term t, N represents the number of terms in the document, TF(t,d) represents the term frequency of term t in document d, IDF(t,D) represents the inverse document frequency of term t, DSF(t) represents the domain-specific factor of term t, and F domain (t) represents the frequency of term t in the corpus of a specified scientific and technological field, F general (t) represents the frequency of term t in the general corpus.

[0067] Among them, we need to pre-build a professional dictionary containing multiple scientific and technological fields for calculating DSF (Domain Specificity Factor). In this embodiment, the general corpus refers to a general domain including non-scientific and technological fields.

[0068] The word vector of term t is a pre-trained word embedding model to generate a vector representation for each term. The word embedding model can be Word2Vec or BERT.

[0069] S12. Obtain the structural complexity index SCI of the scientific and technological intelligence source to be identified and the information density of each key information area to form a structural feature vector, wherein the structural complexity index SCI and the information density of each key information area are:

[0070] SCI=∑(ws i *de i ) / N

[0071] Info density =∑(TDWF(t,r,D)) / L r

[0072] Where: ws i represents the predefined weight of label i, de i represents the nesting depth of tag i, N represents the total number of tags, r represents the text in the key information area, and L r The length of the text representing the key information area.

[0073] Among them, step S12 mainly analyzes the HTML structure of the web page to capture the organization and presentation form of the scientific and technological information source.

[0074] Among them, the key information area is obtained by using a machine learning algorithm to identify the web page of the scientific and technological intelligence source to be identified, such as the title, abstract, text, references, etc. The machine learning algorithm can be a conditional random field (CRF) to identify the key information area. It can also be used to use heuristic rules to identify the key information area.

[0075] S13. Obtain the metadata integrity score of the scientific and technological intelligence source to be identified, and obtain a metadata feature vector according to the metadata integrity score, wherein the metadata integrity score Metascore is:

[0076] Metascore=∑(wmj*Ij) / M

[0077] where wmj represents the weight of the j-th category metadata, Ij represents the indicator of whether the j-th category metadata exists and is complete, and M is the total number of metadata categories.

[0078] The value of Ij is 0 or 1, which is 1 if the metadata exists and is complete, and 0 otherwise.

[0079] The metadata is extracted using a sequence labeling model.

[0080] Among them, the metadata of the scientific and technological information source to be identified includes time information, author information, institutional information, keywords and topics.

[0081] Specifically, we design an intelligent parser that can extract key information from non-standardized metadata. The parser uses a combination of rule-based methods and machine learning models:

[0082] a) Rule-based approach: predefine a series of common metadata patterns and corresponding extraction rules.

[0083] b) Machine learning model: Use a sequence labeling model to identify and extract key information from free text. The sequence labeling model can be BiLSTM-CRF.

[0084] c) Metadata extraction

[0085] c1) Time information extraction and standardization:

[0086] Pay special attention to time information such as release date, last update time, etc. Use natural language processing technology to identify various time formats and standardize them into a unified time representation.

[0087] c2) Author and institution information extraction:

[0088] Named Entity Recognition (NER) technology is used to extract author and institution information.

[0089] It is also possible to build a knowledge graph of scientific research institutions and authors to perform entity linking on the extracted information to obtain richer background information.

[0090] c3) Keyword and topic extraction:

[0091] In addition to directly extracting keywords from metadata, topic models are also used to extract potential topics from web page content. These topics are mapped to predefined science and technology field classifications to obtain the field distribution of web pages. Topic models such as LDA (Latent Dirichlet Allocation) are used.

[0092] S14, concatenate the text feature vector, the structure feature vector and the metadata feature vector into a unified multimodal feature vector.

[0093] Among them, the multimodal feature vector is used as the input of the subsequent layer fusion model.

[0094] Therefore, multimodal feature extraction technology can comprehensively capture all aspects of scientific and technological intelligence sources and provide rich and accurate feature representation for subsequent intelligence source identification and evaluation.

[0095] S2, inputting the multimodal feature vector into a hierarchical fusion model, and obtaining an output value of each evaluation index from the output of the hierarchical fusion model;

[0096] In this embodiment, the main purpose of the hierarchical fusion model is to effectively integrate the various features obtained by multimodal feature extraction technology to achieve a comprehensive and accurate evaluation of scientific and technological intelligence sources. This model aims to capture the complex interactions between different modal features and make more intelligent and reliable judgments on this basis. Through a hierarchical design, the model can gradually refine and fuse information, and ultimately obtain a comprehensive evaluation of the quality of scientific and technological intelligence sources.

[0097] Specifically, the hierarchical fusion model in step S2 includes:

[0098] 1. Feature layer: The self-attention mechanism is used to preliminarily integrate the features of each modality in the multimodal feature vector.

[0099] Specifically:

[0100] a) Feature Standardization:

[0101] First, the features of different modes are standardized to ensure that the scales of the features are consistent. We use the Z-score standardization method:

[0102] Z = (X-μ) / σ

[0103] Where X is the original feature value, μ is the mean of the feature, and σ is the standard deviation.

[0104] b) Feature Enhancement:

[0105] The self-attention mechanism is used to enhance the features in each modality. For each modality m, we calculate the enhanced feature A m :

[0106] A m =softmax(W q *F m *W k ^T*F m ^T)*F m *W v

[0107] Among them, T is the representation of matrix transpose, F m is the characteristic matrix of mode m, W q , W k and W v is the learnable parameter matrix.

[0108] c) Modality specific encoder:

[0109] Corresponding encoders are designed for text mode, structure mode, and metadata mode. These encoders adopt a structure similar to Transformer, but the details are adjusted according to the characteristics of each modality. For example, the text encoder uses position encoding to preserve word order information, while the structure encoder introduces tree position encoding to capture the hierarchical structure of HTML.

[0110] 2. The interaction layer captures the relationship between different modal features through the cross-modal attention module, and uses the multi-head attention mechanism to independently calculate the attention of each head and then splice them. The gating unit is used to control the degree of fusion of different modal information, and finally generates a comprehensive cross-modal representation vector. The graph attention network is used to aggregate the information of all modalities to generate a global context vector;

[0111] Among them, the main task of the interaction layer is to capture the relationship between different modal features and generate a comprehensive cross-modal representation.

[0112] The details are as follows:

[0113] a) Cross-modal attention mechanism:

[0114] We design an innovative cross-modal attention module. For any two modalities i and j, we calculate the attention between them:

[0115] A ij =softmax(W q *F i *W k ^T*F j ^T)*F j *W v

[0116] Among them, F i and F j are the feature matrices of mode i and j respectively.

[0117] b) Multi-head attention mechanism:

[0118] In order to capture richer cross-modal relationships, we use a multi-head attention mechanism. Each head calculates attention independently and then concatenates the results:

[0119] MultiHead(F i ,F j )=Concat(head1,...,head h )*W o

[0120] Where h is the number of heads, W o is the output projection matrix, and the head uses cross-modal attention A ij Specifically, head1=A ij *V^1, to h in sequence, several heads use several different V matrices to calculate several times, and then splice to get the final multi-head attention.

[0121] c) Gating mechanism:

[0122] The gating unit is introduced to control the degree of fusion of different modal information. For each pair of modalities i and j, we calculate a fusion gate:

[0123] G ij =sigmoid(W g *[F i ;F j ])

[0124] The final cross-modal representation is:

[0125] F ij =G ij *MultiHead(F i ;F j )+(1-G ij )*F i

[0126] d) Global context aggregation:

[0127] A graph attention network (GAT) is used to aggregate information from all modalities and generate a global context vector. Each modality is a node in the graph, and the weight of the edge is determined by the similarity between the modalities.

[0128] 3. The decision layer is used to concatenate the cross-modal representation vector and the global context vector output by the interaction layer to form the final feature vector, and finally output the result through ensemble learning.

[0129] Among them, ensemble learning includes the following steps:

[0130] a) Design a dedicated sub-network for each preset evaluation task;

[0131] b) Introduce an attention mechanism to dynamically adjust the weights of different evaluation tasks and obtain the overall loss function generated according to the dynamically adjusted weights;

[0132] c) The prediction results of multiple base learners for each evaluation task are converted into probability distribution softmax(pred) through the softmax function, and then weighted average is performed to obtain the output result of each evaluation task.

[0133] In this embodiment, ensemble learning is performed within each evaluation task, and multiple prediction results of the evaluation task are weighted averaged. Specifically, we use the improved XGBoost algorithm and introduce a soft voting mechanism. The prediction results of each base learner are converted into a probability distribution through the softmax function, and then weighted averaged:

[0134] Final pred =Σ(w i *softmax(pred i )) / Σ(w i )

[0135] where w i is the weight of different models (base learners), which can be determined by the performance of the validation set. Different model values ​​are different models used for this evaluation task, which may include: different parameter settings of the same algorithm, such as decision trees of different depths; different types of algorithms, such as decision trees, random forests, neural networks, etc.; the results of the same model trained on different data subsets.

[0136] For example, suppose that for the evaluation task of "information reliability", we have three base learners: Model A: decision tree; Model B: random forest; Model C: neural network. The weights of the three base learners are w A =0.3, w B = 0.4 and w C =0.3.

[0137] At this point, each model predicts the information reliability of the same scientific and technological intelligence source and obtains three prediction results: A =0.8,pred B =0.7,pred C = 0.75. Finally, we take a weighted average of these results according to the weight of each model and get the prediction result of "information reliability" Final pred =(0.30.8+0.40.7+0.3*0.75) / (0.3+0.4+0.3)=0.745.

[0138] Ensemble learning combines the advantages of multiple models to improve the stability and accuracy of predictions and reduce the deviation and variance that may be caused by a single model.

[0139] The model training and optimization process is as follows:

[0140] a) Phased training strategy:

[0141] First, we pre-train the encoders of each modality separately, then fix these encoders, train the interaction layer and decision layer, and finally perform end-to-end fine-tuning.

[0142] b) Adversarial Training:

[0143] Adversarial examples are introduced to improve the robustness of the model. During training, we add small perturbations to the input features:

[0144]

[0145] Among them, ε is the perturbation size, L is the loss function, represents the gradient of the loss function L with respect to the input x, θ is the parameter of the model, and y is the true label;

[0146] c) Knowledge Distillation:

[0147] We use a pre-trained large language model (BERT) as a teacher model to enhance our model’s ability to understand scientific texts through knowledge distillation.

[0148] d) Regularization Techniques:

[0149] Various regularization techniques are used to prevent overfitting, including dropout, L2 regularization, and label smoothing.

[0150] Therefore, the hierarchical fusion model makes full use of multimodal features and captures the complex interactions between different types of information, thereby making a comprehensive and accurate assessment of scientific and technological intelligence sources.

[0151] Among them, the evaluation tasks correspond to the following evaluation indicators. In this embodiment, the output value of the hierarchical fusion model is a set of values ​​obtained according to the outputs of different evaluation tasks.

[0152] S3. Obtain the initial weight of each evaluation indicator, adjust the initial weight of the evaluation indicator based on the domain similarity between the scientific and technological intelligence source to be identified and the predefined domain and the time perception trend of the scientific and technological themes in the scientific and technological intelligence source to be identified, and obtain the final weight of each evaluation indicator. Based on the output value of the hierarchical fusion model and the final weight of each evaluation indicator, the final value score of the scientific and technological intelligence source to be identified is obtained.

[0153] The evaluation indicators include information reliability, timeliness, content depth, citation completeness, author / institution authority, and language quality. The corresponding evaluation tasks include regression tasks for scoring information source reliability, multi-class classification tasks for domain classification, regression tasks for timeliness evaluation, and regression tasks for content depth evaluation.

[0154] Optionally, adjusting the initial weight of the evaluation index based on the domain similarity between the scientific and technological intelligence source to be identified and the predefined domain includes the following steps:

[0155] Calculate the domain similarity Sim(d,Fk) between the scientific and technological intelligence source to be identified and the predefined domain:

[0156] Sim(d, Fk) = cosine(θ d , θ Fk )

[0157] Among them, cosine represents the cosine function, θ d represents the topic distribution of document d, θ Fk Represents the average topic distribution of the predefined domain FK;

[0158] Use the softmax function to convert the domain similarity Sim(d,Fk) into a weight adjustment factor α k , according to the weight adjustment factor α k Adjust the initial weight of each evaluation indicator to obtain the adjusted weight ws p ':

[0159] ws p '=ws p *(1+∑(α k *δ pk ))

[0160] α k =softmax(Sim(d,Fk))

[0161] Among them, ws prepresents the initial weight of the pth evaluation index, δ pk is the impact factor of the predefined domain FK on the pth evaluation indicator;

[0162] Optionally, adjusting the initial weight of the evaluation index based on the time-perceived trend of the scientific and technological theme in the scientific and technological intelligence source to be identified includes the following steps:

[0163] According to the time perception trend of scientific and technological topics in the scientific and technological intelligence sources to be identified, the corresponding weights of evaluation indicators related to timeliness are adjusted.

[0164] Specifically, this embodiment uses online topic models such as online LDA to continuously track the evolution of scientific and technological topics. The topic model is updated regularly to capture emerging and outdated topics, and the weights of relevant indicators are dynamically adjusted according to the time evolution trend of the topic. For rapidly developing emerging topics, the weight of timeliness indicators is increased; for mature and stable topics, the weight of content depth indicators is increased.

[0165] In this embodiment, the influence on time perception also includes:

[0166] a) Time decay function:

[0167] Introduce a time decay function to adjust the weight of historical data:

[0168] f(t)=exp(-λ*(t_now-t))

[0169] Where t is the timestamp of the data and λ is the decay rate.

[0170] b) Time Series Forecasting:

[0171] Use time series models such as ARIMA or Prophet to predict the future trends of each evaluation indicator and adjust the scoring strategy in advance.

[0172] In summary, this embodiment can capture important information that may be overlooked by traditional single-modal methods by simultaneously processing the text content, structural features and metadata of web pages, thereby more accurately evaluating the quality and reliability of scientific and technological intelligence sources. The multimodal feature vector is processed through a hierarchical fusion model, and finally a basic score, a domain adaptation score and a time-aware score are formed to obtain the final value score, so as to dynamically optimize according to the characteristics of different scientific and technological fields and the dynamically changing intelligence environment, thereby improving the evaluation accuracy, adaptability and timeliness of scientific and technological intelligence sources.

[0173] Embodiment 2

[0174] Please refer to Figure 2A scientific and technological intelligence source evaluation device 1 for scientific and technological intelligence analysis includes a memory 3, a processor 2, and a computer program stored in the memory 3 and executable on the processor 2. When the processor 2 executes the computer program, the steps in the above-mentioned embodiment 1 or 2 are implemented.

[0175] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0176] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, apparatuses or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0178] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0179] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0180] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0181] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A scientific and technological intelligence source evaluation method for scientific and technological intelligence analysis, characterized in that: Includes steps: S1, obtaining a text feature vector, a structural feature vector and a metadata feature vector of a scientific and technological intelligence source to be identified, and splicing the text feature vector, the structural feature vector and the metadata feature vector into a unified multimodal feature vector; The structural feature vector is composed of the structural complexity index of the scientific and technological intelligence source to be identified and the information density of each key information area, the structure is the HTML structure of the web page, and the metadata feature vector is obtained according to the metadata integrity score; S2, inputting the multimodal feature vector into a hierarchical fusion model, and obtaining an output value of each evaluation index from the output of the hierarchical fusion model; S3. Obtain the initial weight of each evaluation indicator, adjust the initial weight of the evaluation indicator based on the domain similarity between the scientific and technological intelligence source to be identified and the predefined domain and the time perception trend of the scientific and technological themes in the scientific and technological intelligence source to be identified, so as to obtain the final weight of each evaluation indicator, and obtain the final value score of the scientific and technological intelligence source to be identified based on the output value of the hierarchical fusion model and the final weight of each evaluation indicator.

2. A scientific and technological intelligence source evaluation method for scientific and technological intelligence analysis according to claim 1, characterized in that: The step S1 comprises the steps of: S11, obtaining the text feature vector Doc of the scientific and technological intelligence source to be identified vec for: Doc vec =∑(TDWF(t,d,D)*Word vec (t)) / N TDWF(t,d,D)=TF(t,d)*IDF(t,D)*DSF(t) DSF(t)=log(1+F domain (t) / F general (t)) Where t represents a term, d represents a document, D represents a document set, TDWF(t,d,D) represents the term domain weight frequency, Word vec (t) represents the word vector of term t, N represents the number of terms in the document, TF(t,d) represents the term frequency of term t in document d, IDF(t,D) represents the inverse document frequency of term t, DSF(t) represents the domain-specific factor of term t, and F domain (t) represents the frequency of term t in the corpus of a specified scientific and technological field, F general (t) represents the frequency of term t in the general corpus; S12, obtaining the structural complexity index SCI of the scientific and technological intelligence source to be identified and the information density of each key information area to form a structural feature vector, wherein the structural complexity index SCI and the information density of each key information area are: SCI=∑(ws i *de i ) / N Info density =∑(TDWF(t,r,D)) / L r Where: ws i represents the predefined weight of label i, de i represents the nesting depth of tag i, N represents the total number of tags, r represents the text in the key information area, and L r The length of the text indicating the key information area; S13, obtaining the metadata integrity score of the scientific and technological intelligence source to be identified, and obtaining a metadata feature vector according to the metadata integrity score, wherein the metadata integrity score Metascore is: Metascore=∑(wmj*Ij) / M Where wmj represents the weight of the j-th category metadata, Ij represents the indicator of the existence and completeness of the j-th category metadata, and M is the total number of metadata categories; S14, concatenating the text feature vector, the structure feature vector and the metadata feature vector into a unified multimodal feature vector.

3. A method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 2, characterized in that: The word vector of the term t in step S11 is a pre-trained word embedding model to generate a vector representation for each term; The key information area in step S12 is obtained by using a machine learning algorithm to identify the web page of the scientific and technological intelligence source to be identified; The metadata in step S13 is extracted using a sequence labeling model.

4. A method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 2, characterized in that: The metadata of the scientific and technological information source to be identified includes time information, author information, institution information, keywords and topics.

5. The method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 1, characterized in that: The hierarchical fusion model in step S2 includes: The feature layer preliminarily fuses the features of each modality in the multimodal feature vector through a self-attention mechanism, wherein the feature layer is respectively designed with corresponding encoders for the text modality, the structure modality and the metadata modality; The interaction layer captures the relationship between different modal features through the cross-modal attention module, and uses the multi-head attention mechanism to independently calculate the attention of each head and then splice them. The gating unit is used to control the degree of fusion of different modal information, and finally generates a comprehensive cross-modal representation vector. The graph attention network is used to aggregate the information of all modalities to generate a global context vector; The decision layer is used to concatenate the cross-modal representation vector and the global context vector output by the interaction layer to form the final feature vector, and finally output the result through ensemble learning.

6. A method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 5, characterized in that: The ensemble learning comprises the steps of: Design a dedicated sub-network for each preset evaluation task; Introduce an attention mechanism to dynamically adjust the weights of different evaluation tasks and obtain the overall loss function generated according to the dynamically adjusted weights; The prediction results of multiple base learners for each evaluation task are converted into probability distribution softmax(pred) through the softmax function, and then weighted average is performed to obtain the output result of each evaluation task.

7. A method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 5, characterized in that: The evaluation indicators include information reliability, timeliness, content depth, citation completeness, author / institution authority and language expression quality.

8. The method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 5, characterized in that: Adjusting the initial weight of the evaluation index based on the domain similarity between the to-be-identified scientific and technological intelligence source and the predefined domain comprises the following steps: Calculate the domain similarity Sim(d,Fk) between the scientific and technological intelligence source to be identified and the predefined domain: Sim(d, Fk) = cosine(θ d , θ Fk ) Among them, cosine represents the cosine function, θ d represents the topic distribution of document d, θ Fk Represents the average topic distribution of the predefined domain FK; Use the softmax function to convert the domain similarity Sim(d,Fk) into a weight adjustment factor α k , according to the weight adjustment factor α k Adjust the initial weight of each evaluation indicator to obtain the adjusted weight ws p ': ws p '=ws p *(1+∑(a k *d pk )) a k =softmax(Sim(d,Fk)) Among them, ws p represents the initial weight of the pth evaluation index, δ pk It is the impact factor of the predefined domain FK on the pth evaluation indicator.

9. A method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis according to claim 3, characterized in that: Adjusting the initial weight of the evaluation index based on the time perception trend of the scientific and technological theme in the scientific and technological intelligence source to be identified includes the following steps: The corresponding weights of evaluation indicators related to timeliness are adjusted according to the time perception trend of the scientific and technological topics in the scientific and technological intelligence source to be identified.

10. A scientific and technological intelligence source evaluation device for scientific and technological intelligence analysis, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for evaluating scientific and technological intelligence sources for scientific and technological intelligence analysis as described in any one of claims 1 to 9 is implemented.

Images