Method and device for identifying weak signal of emerging technology

Abstract

The invention discloses a method for identifying weak signals of an emerging technology. The method comprises the following steps of: extracting and normalizing keywords; preliminarily filtering weak signals of the emerging technology; evaluating a weak signal of the emerging technology; and comprehensively studying and judging the emerging technology weak signals. According to the method, academic papers and technical patent scientific and technical literature data are adopted, policy and strategy data are introduced to evaluate the importance of emerging technology weak signals, and the strength of national strategy layout is reflected; a pre-training model is adopted to represent keywords and related data, the occurrence conditions of the keywords in various types of data are reflected from the semantic level, and evaluation of emerging technology weak signals is accurately carried out; the method comprises the following steps of: performing multi-round filtering on emerging technology weak signals through preliminary filtering of keywords with overhigh word frequency and calculation and analysis of three indexes, namely burstiness, concentration ratio and importance, and eliminating noise; according to the method, a complete and systematic analysis process is constructed, and systematic evaluation and identification of emerging technology weak signals can be realized.

Description

Technical Field

[0001] The present invention relates to an identification method, and in particular to a method and a device for identifying weak signals of emerging technologies, and belongs to the field of signal identification. Background Art

[0002] With the rapid development of science and technology, emerging technologies are emerging more and more frequently, which has a profound impact on national security, economic development and social progress. The identification of weak signals has become a key link in science and technology foresight and innovation decision-making, and has important practical significance for foreseeing future trends, maintaining competitive advantages, capturing innovation opportunities and reducing potential risks. By identifying weak signals of emerging technologies early, it can support innovation entities to make forward-looking strategic decisions, actively respond to technological changes, and enhance their learning ability and influence, so as to take the lead in the ever-changing scientific and technological environment and market.

[0003] Currently, there are three main methods for quantitatively identifying weak signals of emerging technologies: (1) Methods that focus on the laws of scientific and technological output, including machine learning methods of scientometrics and text mining. YOONJ (Expert Systems with Applications, 2012, 39(16):12543-12550) uses the frequency of occurrence of keywords and document frequency to measure perception and recognition respectively, and then combines text mining technology to monitor the potential weak signals related to solar cells in online news; (2) Methods that focus on the cycle of technological development laws, such as TR IZ theory and S-curve model, etc., Zhai Dongsheng et al. (Journal of Intelligence, 2015, 34(8):31-36) proposed a method for identifying weak signals of emerging technologies based on patent data and TRIZ theory, and carried out empirical research in the field of "contact lens disinfection"; (3) Methods based on mathematical or management models, including hierarchical analysis method, mutation theory and technology disruptive risk prediction model, Deng Shengli et al. (Library and Information Work, 2016, 60(10):67-75) combined with the seven-force model, and constructed a quantitative identification method for weak signals of enterprise competition by using hierarchical analysis method and membership function. In summary, the existing methods still have a series of problems, including insufficient diversity of relevant data sources, lack of deep understanding of semantics in the identification of weak signals of emerging technologies, difficulty in distinguishing weak signals from noise, lack of systematic evaluation and verification methods in the identification of weak signals of emerging technologies, and imperfect monitoring and early warning system of weak signals of emerging technologies. Summary of the invention

[0004] In order to solve the defects in the prior art, the present invention discloses a method for identifying weak signals of emerging technologies, and its technical solution is as follows:

[0005] A method for identifying weak signals of emerging technologies, characterized by comprising the following steps:

[0006] Step 1: Keyword extraction and normalization;

[0007] Step 2: Preliminary filtering of weak signals of emerging technologies;

[0008] Step 3: Weak signal evaluation of emerging technologies;

[0009] Step 4: Comprehensive analysis of weak signals of emerging technologies.

[0010] The invention also discloses a device for identifying weak signals of emerging technologies.

[0011] The present invention discloses a device for identifying weak signals of emerging technologies, characterized in that it comprises a processor and a memory; the memory stores computer-readable instructions, and the processor is used to run the computer-readable instructions, wherein the computer-readable instructions execute the above method when running.

[0012] The present invention also discloses a non-volatile storage medium, characterized in that the non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the above method.

[0013] Beneficial Effects

[0014] (1) Using scientific and technological literature data such as academic papers and technical patents, and introducing policy and strategic data to assess the importance of weak signals of emerging technologies and reflect the strength of their national strategic layout;

[0015] (2) Use the open source pre-trained model Sentence-BERT to represent keywords and various types of data, reflect the occurrence of keywords in various types of data from a semantic level, and accurately evaluate the weak signals of emerging technologies;

[0016] (3) Through the preliminary filtering of keywords with too high frequency, as well as the calculation and analysis of three indicators, namely, suddenness, concentration, and importance, multiple rounds of filtering are performed on the weak signals of emerging technologies to eliminate noise as much as possible;

[0017] (4) Systematic evaluation method for weak signals of emerging technologies: The present invention constructs a complete and systematic analysis process, including keyword extraction and normalization, preliminary filtering, evaluation and assessment, and comprehensive analysis, which can realize the systematic evaluation and identification of weak signals of emerging technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The figure is a flow chart of the method for identifying weak signals of emerging technologies according to the present invention. DETAILED DESCRIPTION

[0019] The present invention discloses a method for identifying weak signals of emerging technologies, which includes the following four parts:

[0020] 1. Keyword extraction and standardization

[0021] The present invention identifies weak signals of emerging technologies at the keyword level. Specifically, using scientific and technological literature such as academic papers and technical patents as the core data source, abbreviations, keywords, technical terms, technical phrases, etc. are extracted from titles and abstracts. At the same time, the extracted keywords are merged by root restoration to achieve keyword standardization. The implementation process is as follows.

[0022] 1. Extract technical term abbreviations and their full names

[0023] a) First, all candidate technical term abbreviations are extracted with the help of regular expressions. The extraction criteria are: search for brackets in the text. If there is only one word in the brackets, and the first letter of the word is capitalized, and the word is not an English word with clear semantics (checked with the help of the nltk.corpus.words dictionary), then the character is considered as a candidate technical term abbreviation;

[0024] b) Secondly, starting from the position of the term abbreviation, search for the corresponding candidate full term. The search method is: search forward for a corresponding number of words according to the number of letters and capital letters in each candidate term abbreviation, and add the found words to the candidate full term list. Among them, the reason why "the number of capital letters" is selected as one of the search criteria is that in some cases the lowercase letters and capital letters in the abbreviation correspond to the same word, such as the full name of CdS is CadmiumSulfide (cadmium sulfide);

[0025] c) Third, verify the list of candidate term full names. The verification standard is: whether the first letter of the first word of the candidate term full name corresponds to the first letter of the candidate term abbreviation. If they correspond, they will be retained; if not, they will be discarded;

[0026] d) Fourth, the full name corresponding to the term abbreviation is finally determined. If the same candidate term abbreviation corresponds to multiple different candidate term full names, the entire text library will be checked, and the candidate full name with the largest number of occurrences will be used as the final term full name;

[0027] e) Finally, all successfully confirmed term abbreviations and their full names are returned.

[0028] 2. Replace all summary text in the text library

[0029] According to the mapping relationship between abbreviations and full terms, the abstract text is processed as follows:

[0030] a) Delete bracketed abbreviations in the abstract text, such as '(NLP)', to reduce subsequent noise;

[0031] b) Replace the abbreviations of terms without brackets with their corresponding full names to improve the accuracy of subsequent term extraction;

[0032] c) Return the summary text base without any abbreviations.

[0033] 3. Extract technical noun phrases

[0034] a) Tag the summary text with the part-of-speech tagging tool - the pos_tag function of the NLTK library;

[0035] b) Locating technical nouns or noun phrases: Locating nouns or consecutive noun phrases with part-of-speech tags of NN (noun), NNS (plural noun), NNP (proper noun), or NNPS (plural proper noun) as candidate technical nouns or noun phrases;

[0036] c) Extract technical noun phrases: Starting from the candidate technical noun or noun phrase, traverse 2 to 7 words in sequence. If the previous words are all JJ (adjective) or JJR (comparative adjective), JJS (superlative adjective), combine them with the subsequent noun or noun phrase to form a new noun phrase as the final technical noun phrase; if not, but a continuous noun phrase is located, add them together to the final technical noun phrase;

[0037] 4. Restoration of word roots

[0038] With the help of the WordNetLemmatizer toolkit in NLTK, the word form restoration work is completed, and all the extracted technical noun phrases are merged to obtain the keyword extraction results.

[0039] This process has strong versatility and accuracy, and can efficiently and accurately extract technical terms and phrases contained in academic papers, patents and other scientific and technological literature in any field, and ensure the reproducibility of the results.

[0040] (II) Preliminary filtering of weak signals from emerging technologies

[0041] A large number of keywords are extracted, and preliminary filtering is required. The frequency of keywords appearing in all scientific and technological literature (i.e., keyword frequency) can reflect the research popularity of the keyword. From the definition of the weak signal of emerging technology itself, keywords with too high a frequency have become a hot spot in field research and are not suitable as weak signals of emerging technologies. Therefore, it is necessary to filter keywords with too high a frequency. The present invention selects the frequency g index method to calculate the high-frequency word threshold and filter out keywords with too high a frequency. The calculation method is as follows.

[0042] a) Frequency calculation: Count the frequency of keywords appearing in the scientific literature data set. The statistical rule is that if a keyword appears in a scientific literature, the keyword frequency increases by 1. After the statistics are completed, the keywords are arranged in descending order of frequency.

[0043] b) Determine the high frequency threshold: The g index is a commonly used indicator for measuring the influence of academic publications and scholars. The present invention refers to the calculation method of the g index and uses the g index method to determine the high frequency threshold. The specific calculation method is: Assuming that the number of keywords with the same frequency starting from the gth is n, if the cumulative frequency of g keywords is greater than or equal to g 2 , and the cumulative frequency of (g+n) keywords is less than (g+n+1) 2 Then the keywords up to (g+n) are all high-frequency words in the phrase set. The expression is as follows:

[0044] and

[0045] Among them, n is the number of keywords with the same frequency.

[0046] c) High-frequency keyword filtering: The keywords up to (g+n) in the above steps are filtered as high-frequency keywords and are not used as basic data for subsequent weak signal evaluation and judgment of emerging technologies.

[0047] 3. Evaluation of weak signals of emerging technologies

[0048] For the keywords after the preliminary filtering, the present invention defines three evaluation indicators based on the characteristics of the weak signals of emerging technologies, including suddenness, concentration and importance, to further evaluate the weak signals of emerging technologies (i.e. keywords). The specific calculation method is as follows.

[0049] 1. Calculation of sudden indicators

[0050] For the above keywords, the suddenness index is defined based on the standard of "low frequency of occurrence and fast growth rate", and the calculation method is as follows.

[0051] a) Frequency calculation: Calculate the frequency of occurrence of keywords in the scientific literature dataset in the past 20 years each year, and record the total frequency as F.

[0052] b) Calculate the percentage: Calculate the percentage of the keyword's appearance frequency in the past two years, three years, and five years to the total appearance frequency F in the previous 20 years, and record them as N_2, N_3, and N_5 respectively.

[0053] c) Calculate the burst index: Calculate the burst index T of the keyword using the following formula:

[0054] T=N_2+N_3+N_53

[0055] d) Burst analysis: The higher the technology's burstiness, the more popular the keyword has been in the recent past, and the stronger its attribute as a weak signal of emerging technology is; conversely, the lower its popularity is, the weaker its attribute as a weak signal of emerging technology is.

[0056] Based on this, we can more accurately identify low-frequency keywords whose frequency has suddenly increased in the recent period as weak signals of emerging technologies.

[0057] 2. Calculation of concentration index

[0058] When identifying weak signals of emerging technologies, in addition to considering the frequency and growth rate of keywords, it is also necessary to evaluate the concentration of their distribution in scientific literature data. If the papers containing keywords show a high degree of similarity, it means that the semantics of keywords in different scenarios are more consistent, and they are more likely to be weak signals of emerging technologies with clear meanings. The specific calculation method is as follows:

[0059] a) Text vectorization: Use the Sentence-BERT model to convert each document into a high-dimensional vector. Sentence-BERT is a pre-trained language model architecture based on BERT. By building a variant suitable for sentence-level embedding representation, the entire sentence or paragraph is encoded into a high-dimensional vector to capture its semantic information. The basic steps of text vectorization include: ① Input text, taking the title and abstract of the document as input; ② Model encoding, the Siamese network structure based on the Sentence-BERT model encodes the input text into a high-dimensional vector (for example, 768 dimensions or higher); ③ Result vector, the output high-dimensional vector represents the semantic information of the input text. After the text is vectorized by the Sentence-BERT model, the deep semantic information of the text can be captured and applied to various natural language processing tasks. The semantic similarity between different texts can be measured by calculating the cosine similarity between vectors.

[0060] b) Calculate similarity: For each keyword, use AC (Aho-Corasick) automaton to match keywords and select the documents containing the keywords. For the high-dimensional vectors converted from these documents, use the cosine similarity algorithm to calculate the similarity between the high-dimensional vectors of the documents.

[0061]

[0062] Among them, A and B are high-dimensional vectors of two documents respectively. Cosine similarity can effectively measure the angle between document vectors, thereby reflecting their semantic similarity. For documents with high similarity, the result is close to 1.

[0063] c) Similarity mean calculation: Take a keyword as an example, assuming that it appears in n papers, the pairwise similarities calculated by the above b) similarity calculation step are Then calculate this value The mean of the similarity values ​​is denoted as S_mean.

[0064] d) Calculate the concentration index: define the emerging technology concentration index D. For each keyword, its concentration index D is consistent with the mean similarity between papers S_mean, that is, D = S_mean.

[0065] e) Analysis: The higher the concentration index D, the more concentrated the distribution of the keyword among different papers and the more consistent the meaning, and the greater its potential as a weak signal of emerging technology. Conversely, if D is lower, it indicates that the keyword may belong to a topic of more widespread discussion and its potential as a signal of emerging technology is weaker.

[0066] 3. Importance index calculation

[0067] In addition to the above-mentioned suddenness and concentration, it is also necessary to consider the importance of weak signals of emerging technologies, that is, whether they are mentioned in the overall national layout and economic and social development, and whether they are strategic priorities. In addition to the above-mentioned keywords, this step also involves the policy and strategic data of major developed countries in the world. The specific calculation method is as follows:

[0068] a) Text vectorization: Sentence-BERT text vectors are used to represent keywords, and the policy and strategy texts of major developed countries in the world are represented as high-dimensional vectors.

[0069] b) Calculate similarity: Calculate the similarity between two vectors through cosine similarity, and use the similarity of the sentence in the current policy and strategy text that is most semantically similar to the current keyword as the activity of the current keyword in the current policy and strategy text.

[0070] c) Calculate the importance index: Arrange the policy strategy texts in descending order of activity, take the average activity of the top 25% of the policy strategy texts, and use this as the overall activity of the current keyword in the policy strategies of the world's major developed countries, i.e., its importance.

[0071] d) Analysis: The higher the importance index, the more frequently the keyword is mentioned in the policy strategies of the world's major developed countries, the higher the importance of the keyword, and the greater its potential as a weak signal of emerging technologies; conversely, it indicates that the keyword is not a focus of the national strategic layout and its potential as a signal of emerging technologies is relatively weak.

[0072] (IV) Comprehensive analysis of weak signals of emerging technologies

[0073] Based on the quantitative analysis results of weak signals of emerging technologies, by integrating the professional insights of science and technology managers, field scientists and intelligence analysis experts, a systematic analysis of the connotation and importance of weak signals of emerging technologies is carried out from multiple dimensions such as scientificity, innovation and application potential. The basis for the analysis includes the following aspects:

[0074] a) Scientific connotation analysis: Combined with the quantitative calculation of weak signals, analyze their significance in terms of technological theory and scientific basis, and use methods such as bibliometrics and technological evolution paths to explore the uniqueness and scientific value of the signal in the technological system.

[0075] b) Innovation potential assessment: Through technology maturity analysis and technology life cycle model, determine whether weak signals represent potential technological innovation points and evaluate the innovative breakthroughs they may trigger in the existing technology system.

[0076] c) Feasibility analysis of the implementation path: Combined with the technology and resource requirements, the implementation path of the weak signal is evaluated. Using tools such as technical economics models and technology maturity assessment, the difficulty, required resources and risk factors of its transition from the laboratory stage to industrialization are determined.

[0077] d) Forecast of development prospects: Relying on quantitative analysis and cutting-edge knowledge of experts in the field, using intelligence monitoring, trend analysis, scenario analysis and other means, we predict the possible evolution trajectory of the weak signal in future technology and the market and its impact, so as to judge its long-term development potential.

[0078] The present invention uses scientific and technological literature data such as academic papers and technical patents, and introduces policy and strategic data to evaluate the importance of weak signals of emerging technologies, reflecting the strength of its national strategic layout; uses a pre-trained model to characterize keywords and related data, reflects the appearance of keywords in various types of data from a semantic level, and accurately carries out the evaluation of weak signals of emerging technologies; through preliminary filtering of keywords with too high frequency / too low frequency, and calculation and analysis of three indicators: burstiness, concentration, and importance, multiple rounds of filtering are performed on weak signals of emerging technologies to eliminate noise as much as possible; a complete and systematic analysis process is constructed, including data collection, keyword extraction, preliminary filtering, evaluation and assessment, and comprehensive analysis, which can realize the systematic evaluation and identification of weak signals of emerging technologies.

[0079] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for identifying weak signals of emerging technologies, characterized by comprising the following steps: Step 1: Keyword extraction and normalization; Step 2: Preliminary filtering of weak signals of emerging technologies; Step 3: Weak signal evaluation of emerging technologies; Step 4: Comprehensive analysis of weak signals of emerging technologies.

2. The method for identifying weak signals of emerging technologies according to claim 1, characterized in that: The step 1 further includes the following contents: identifying weak signals of emerging technologies at the keyword level: taking scientific and technological literature as the core data source, extracting abbreviations, keywords, technical terms, and technical phrases from titles and abstracts, and at the same time, merging the extracted keywords by root restoration to achieve keyword standardization.

3. The method for identifying weak signals of emerging technologies according to claim 1, characterized in that: The step 2 further includes the following contents: using the word frequency g index method to calculate the high-frequency word and low-frequency word boundary threshold, filtering out keywords with too high word frequency, and the specific calculation method is: arranging the keywords in descending order of frequency, and assuming that the number of keywords with the same frequency starting from the gth is n, if the cumulative frequency of g keywords is greater than or equal to g 2 , and the cumulative frequency of (g+n) keywords is less than (g+n+1) 2 Then the keywords up to and including the g+nth are all high-frequency words in the phrase set, and the expression is as follows: and Among them, n is the number of keywords with the same frequency, and S is the cumulative frequency of occurrence of the keywords.

4. The method for identifying weak signals of emerging technologies according to claim 1, characterized in that: The step 3 further includes the following contents: defining three evaluation indicators, namely, suddenness, concentration and importance, to further evaluate the weak signals of emerging technologies.

5. The method for identifying weak signals of emerging technologies according to claim 4, characterized in that: The burst index is calculated as follows: a) Frequency calculation: Calculate the frequency of occurrence of keywords in the scientific literature dataset for each year over the past 20 years, and record the total frequency as F; b) Calculate the percentage: Calculate the percentage of the keyword's frequency of occurrence in the past two years, three years, and five years to the total frequency of occurrence F in the previous 20 years, recorded as N_2, N_3, and N_5 respectively; c) Calculate the burst index: Calculate the burst index T of the keyword using the following formula: T=(N_2+N_3+N_5) / 3 d) Analysis: The higher the suddenness of a technology, the more popular the keyword has been in the recent past, and the stronger its attribute as a weak signal of an emerging technology is; conversely, the lower its popularity is, the weaker its attribute as a weak signal of an emerging technology is.

6. The method for identifying weak signals of emerging technologies according to claim 4, characterized in that: The concentration index is calculated as follows: a) Text vectorization: Use Sentence-BERT text vectors to represent keywords and convert the title and abstract of each paper into high-dimensional vectors; b) Calculate similarity: Evaluate the similarity between papers by calculating the cosine similarity between these vectors; for each keyword, find all papers containing the keyword and calculate the similarity between these papers; c) Similarity mean calculation: If a keyword appears in n papers, the pairwise similarities calculated by the above b) similarity calculation step are Then calculate this value The mean of the similarity values ​​is denoted as S_mean; d) Calculate the concentration index: define the emerging technology concentration index D. For each keyword, its concentration index D is consistent with the mean similarity between papers S_mean, that is, D = S_mean; e) Analysis: The higher the concentration index D, the more concentrated the distribution of the keyword among different papers and the more consistent the meaning, and the greater its potential as a weak signal of emerging technology. Conversely, if D is lower, it indicates that the keyword may belong to a topic of more widespread discussion and its potential as a signal of emerging technology is weaker.

7. The method for identifying weak signals of emerging technologies according to claim 4, characterized in that: The importance index is calculated as follows: a) Text vectorization: Sentence-BERT text vectors are used to represent keywords, and the policy and strategy texts of developed countries in the world are represented as high-dimensional vectors; b) Calculate similarity: Calculate the similarity between two vectors through cosine similarity, and take the similarity of the sentence in the current policy and strategy text that is most similar to the current keyword in semantics as the activity of the current keyword in the current policy and strategy text; c) Calculate the importance index: Arrange the policy strategy texts in descending order of activity, take the average activity of the top 25% of the policy strategy texts, and use this as the overall activity of the current keyword in the policy strategies of developed countries in the world, i.e., importance; d) Analysis: The higher the importance index, the more frequently the keyword is mentioned in the policy strategies of developed countries in the world, the higher the importance of the keyword, and the greater its potential as a weak signal of emerging technologies. Conversely, it indicates that the keyword is not a focus of national strategic layout and its potential as a signal of emerging technologies is relatively weak.

8. A device for identifying weak signals of emerging technologies, characterized in that: It comprises a processor and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein the computer-readable instructions execute the method described in any one of claims 1 to 7 when executed.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images