A method and device for identifying weak signals of emerging technologies

Abstract

A method for identifying weak signals of emerging technologies, comprising: keyword extraction and normalization; preliminary filtering of weak signals of emerging technologies; evaluation of weak signals of emerging technologies; and comprehensive analysis and judgment of weak signals of emerging technologies. The present invention uses academic papers and technical patents, scientific and technological literature data, and introduces policy and strategic data to evaluate the importance of weak signals of emerging technologies, reflecting the strength of its national strategic layout; uses a pre-trained model to represent keywords and related data, reflects the appearance of keywords in various types of data from a semantic level, and accurately conducts the evaluation of weak signals of emerging technologies; performs multiple rounds of filtering on weak signals of emerging technologies to eliminate noise through preliminary filtering of keywords with excessively high word frequencies, and calculation and analysis of three indicators: suddenness, concentration, and importance; the present invention constructs a complete and systematic analysis process, which can achieve systematic evaluation and identification of weak signals of emerging technologies.

Description

Technical Field

[0001] The present invention relates to an identification method, in particular to a method and device for identifying weak signals of emerging technologies, belonging to the field of signal identification. Background Art

[0002] With the rapid development of science and technology, emerging technologies are emerging more and more frequently, having a profound impact on national security, economic development, and social progress. Identifying weak signals in emerging technologies has become a key component of scientific and technological foresight and innovation decision-making, and is of great practical significance for anticipating future trends, maintaining competitive advantages, capturing innovation opportunities, and mitigating potential risks. Early identification of weak signals in emerging technologies can support innovative entities in making forward-looking strategic decisions and proactively responding to technological change. It also enhances their learning capabilities and influence, enabling them to maintain a leading position in the ever-changing scientific and technological environment and market.

[0003] Currently, there are three main methods for quantitatively identifying weak signals of emerging technologies: (1) Methods that focus on the laws of scientific and technological output, including machine learning methods of scientometrics and text mining. YOONJ (Expert Systems with Applications, 2012, 39(16):12543-12550) uses the frequency of occurrence of keywords and document frequency to measure perception and recognition respectively, and then combines text mining technology to monitor the potential weak signals related to solar cells in online news; (2) Methods that focus on the cycle of technological development laws, such as TR IZ theory and S-curve model, etc., Zhai Dongsheng et al. (Journal of Intelligence, 2015, 34(8):31-36) proposed a method for identifying weak signals of emerging technologies based on patent data and TRIZ theory, and carried out empirical research in the field of "contact lens disinfection"; (3) Methods based on mathematical or management models, including hierarchical analysis method, catastrophe theory and technology disruptive risk prediction model, Deng Shengli et al. (Library and Information Work, 2016, 60(10):67-75) combined with the seven-force model, constructed a quantitative identification method for weak signals of enterprise competition by using the hierarchical analysis method and membership function. In summary, the existing methods still have a series of problems, including insufficient diversity of relevant data sources, lack of deep understanding of semantics in the identification of weak signals of emerging technologies, difficulty in distinguishing weak signals from noise, lack of systematic evaluation and verification methods in the research on weak signal identification of emerging technologies, and imperfect monitoring and early warning system for weak signals of emerging technologies. Summary of the Invention

[0004] In order to solve the defects in the prior art, the present invention discloses a method for identifying weak signals of emerging technologies, and its technical solution is as follows:

[0005] A method for identifying weak signals of emerging technologies, characterized by comprising the following steps:

[0006] Step 1: Keyword extraction and normalization;

[0007] Step 2: Preliminary filtering of weak signals of emerging technologies;

[0008] Step 3: Weak signal evaluation of emerging technologies;

[0009] Step 4: Comprehensive analysis of weak signals of emerging technologies.

[0010] The invention also discloses a device for identifying weak signals of emerging technologies.

[0011] The present invention discloses a device for identifying weak signals of emerging technologies, characterized in that it includes a processor and a memory; the memory stores computer-readable instructions, and the processor is used to run the computer-readable instructions, wherein the computer-readable instructions execute the above-mentioned method when running.

[0012] The present invention also discloses a non-volatile storage medium, characterized in that the non-volatile storage medium includes a stored program, wherein when the program is run, it controls the device where the non-volatile storage medium is located to execute the above method.

[0013] Beneficial effects

[0014] (1) Using scientific literature data such as academic papers and technology patents, and introducing policy and strategic data to assess the importance of weak signals of emerging technologies and reflect the strength of their national strategic layout;

[0015] (2) Using the open-source pre-trained model Sentence-BERT to represent keywords and various data, it reflects the occurrence of keywords in various data at the semantic level and accurately evaluates the weak signals of emerging technologies;

[0016] (3) Through the preliminary filtering of keywords with too high a frequency, and the calculation and analysis of three indicators: suddenness, concentration, and importance, multiple rounds of filtering are performed on the weak signals of emerging technologies to eliminate noise as much as possible;

[0017] (4) Systematic evaluation method for weak signals of emerging technologies: The present invention constructs a complete and systematic analysis process, including keyword extraction and normalization, preliminary filtering, evaluation and assessment, and comprehensive analysis, which can realize the systematic evaluation and identification of weak signals of emerging technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the method for identifying weak signals of emerging technologies according to the present invention. DETAILED DESCRIPTION

[0019] The present invention discloses a method for identifying weak signals of emerging technologies, which includes the following four parts:

[0020] (1) Keyword extraction and standardization

[0021] This invention identifies weak signals of emerging technologies at the keyword level. Specifically, using scientific literature such as academic papers and technical patents as the core data source, abbreviations, keywords, technical terms, and technical phrases are extracted from titles and abstracts. Furthermore, root word reduction is used to merge the extracted keywords and achieve keyword standardization. The implementation process is as follows.

[0022] 1. Extract technical term abbreviations and their full names

[0023] a) First, use regular expressions to extract all candidate technical term abbreviations. The extraction criteria are: search for brackets in the text. If the brackets contain only one word, the first letter of the word is capitalized, and the word is not an English word with clear semantics (checked using the nltk.corpus.words dictionary), then the character is considered a candidate technical term abbreviation;

[0024] b) Secondly, starting from the position of the term abbreviation, search for the corresponding candidate full term. The search method is: according to the number of letters and capital letters in each candidate term abbreviation, search forward for a corresponding number of words, and add the found words to the candidate full term list. The reason why "the number of capital letters" is selected as one of the search criteria is that in some cases the lowercase letters and capital letters in the abbreviation correspond to the same word, such as the full name of CdS is Cadmium Sulfide.

[0025] c) Third, verify the list of candidate full names. The verification criterion is whether the first letter of the first word of the candidate full name corresponds to the first letter of the candidate abbreviation. If they do, they are retained; otherwise, they are discarded.

[0026] d) Fourth, the full name corresponding to the term abbreviation is finally determined. If the same candidate term abbreviation corresponds to multiple different candidate term full names, the entire text library will be checked and the candidate full name with the largest number of occurrences will be used as the final term full name;

[0027] e) Finally, all successfully confirmed term abbreviations and their full names are returned.

[0028] 2. Replace all summary text in the text library

[0029] Based on the mapping relationship between abbreviations and full terms, the abstract text is processed as follows:

[0030] a) Delete bracketed abbreviations in the abstract text, such as '(NLP)', to reduce subsequent noise;

[0031] b) Replace the abbreviations of terms without brackets with their corresponding full names to improve the accuracy of subsequent term extraction;

[0032] c) Return the summary text base without any abbreviations.

[0033] 3. Extract technical noun phrases

[0034] a) Tag the abstract text with the part-of-speech tagging tool - the pos_tag function of the NLTK library;

[0035] b) Locating technical nouns or noun phrases: Locate nouns or consecutive noun phrases with the part-of-speech tag NN (noun), NNS (plural noun), NNP (proper noun), or NNPS (plural proper noun) as candidate technical nouns or noun phrases;

[0036] c) Extracting technical noun phrases: Starting with a candidate technical noun or noun phrase, traverse forward 2 to 7 words in sequence. If the preceding words are all JJ (adjective), JJR (comparative adjective), or JJS (superlative adjective), combine them with the subsequent noun or noun phrase to form a new noun phrase as the final technical noun phrase. If there are no such words, but a continuous noun phrase is located, add them together to the final technical noun phrase.

[0037] 4. Restoring word roots

[0038] With the help of the WordNetLemmatizer toolkit in NLTK, the word form restoration work is completed, and all the extracted technical noun phrases are merged to obtain the keyword extraction results.

[0039] This process is highly versatile and accurate, and can efficiently and accurately extract technical terminology phrases contained in academic papers, patents, and other scientific literature in any field, while ensuring the reproducibility of the results.

[0040] (2) Preliminary filtering of weak signals from emerging technologies

[0041] The number of keywords extracted is large, and preliminary filtering is required. The frequency of keywords appearing in all scientific and technological documents (i.e., keyword frequency) can reflect the research enthusiasm of the keyword. From the definition of the weak signal of emerging technology itself, keywords with too high a frequency have become a hot spot in field research and are not suitable as weak signals of emerging technology. Therefore, it is necessary to filter keywords with too high a frequency. The present invention selects the word frequency g index method to calculate the high-frequency word threshold and filter out keywords with too high a frequency. The calculation method is as follows.

[0042] a) Frequency calculation: Count the frequency of keywords appearing in the scientific literature dataset. The statistical rule is that if a keyword appears in a scientific literature, the keyword frequency increases by 1. After the statistics are completed, the keywords are sorted in descending order of frequency.

[0043] b) Defining high frequency threshold: The g index is a commonly used indicator to measure the influence of academic publications and scholars. The present invention refers to the calculation method of the g index and uses the word frequency g index method to determine the high frequency word threshold. The specific calculation method is: Assume that the number of keywords with the same frequency starting from the gth is n, if the cumulative frequency of g keywords is greater than or equal to g 2 , and the cumulative frequency of (g+n) keywords is less than (g+n+1) 2 Then the keywords up to (g+n) are all high-frequency words in the phrase set. The expression is as follows:

[0044] and

[0045] Where n is the number of keywords with the same frequency.

[0046] c) High-frequency keyword filtering: The keywords up to (g+n) in the above steps are filtered as high-frequency keywords and are not used as basic data for subsequent weak signal evaluation and analysis of emerging technologies.

[0047] (3) Weak Signal Evaluation of Emerging Technologies

[0048] For the keywords after the initial filtering, this paper defines three evaluation indicators based on the characteristics of the weak signals of emerging technologies: suddenness, concentration, and importance, to further evaluate the weak signals of emerging technologies (i.e., keywords). The specific calculation method is as follows.

[0049] 1. Calculation of sudden indicators

[0050] For the above keywords, the suddenness index is defined based on the standard of "low frequency of occurrence and fast growth rate", and the calculation method is as follows.

[0051] a) Frequency calculation: Calculate the frequency of occurrence of keywords in the scientific literature dataset in each year over the past 20 years, and record the total frequency as F.

[0052] b) Calculate the percentage: Calculate the percentage of the keyword's frequency of occurrence in the past two years, three years, and five years to the total frequency F of occurrence in the previous 20 years, and record them as N_2, N_3, and N_5 respectively.

[0053] c) Calculate the burst index: Calculate the burst index T of the keyword using the following formula:

[0054] T=N_2+N_3+N_53

[0055] d) Burstiness analysis: The higher the technology's burstiness, the more popular the keyword has been in recent times, and the stronger its weak signal attribute of emerging technology is; conversely, the lower the popularity, the weaker its weak signal attribute of emerging technology is.

[0056] Based on this, we can more accurately identify low-frequency keywords that have suddenly increased in frequency in the recent period as weak signals of emerging technologies.

[0057] 2. Calculation of concentration index

[0058] When identifying weak signals of emerging technologies, in addition to considering the frequency and growth rate of keywords, it is also necessary to assess their concentration in scientific literature data. A high degree of similarity between papers containing keywords indicates that the semantics of the keywords are more consistent across different scenarios, making them more likely to be weak signals of emerging technologies with clear meanings. The specific calculation method is as follows:

[0059] a) Text vectorization: Use the Sentence-BERT model to convert each document into a high-dimensional vector. Sentence-BERT is a pre-trained language model architecture based on BERT. By constructing a variant suitable for sentence-level embedding representation, it encodes the entire sentence or paragraph into a high-dimensional vector to capture its semantic information. The basic steps of text vectorization include: ① Input text, taking the title and abstract of the document as input; ② Model encoding, based on the Siamese network structure of the Sentence-BERT model, the input text is encoded into a high-dimensional vector (for example, 768 dimensions or higher); ③ Result vector, the output high-dimensional vector represents the semantic information of the input text. After the text is vectorized by the Sentence-BERT model, the deep semantic information of the text can be captured and applied to various natural language processing tasks. The semantic similarity between different texts can be measured by calculating the cosine similarity between vectors.

[0060] b) Calculating Similarity: For each keyword, we use an AC (Aho-Corasick) automaton to perform keyword matching and select documents containing the keyword. Using the cosine similarity algorithm, we calculate the similarity between the high-dimensional vectors of these documents.

[0061]

[0062] Where A and B are high-dimensional vectors of two documents. Cosine similarity effectively measures the angle between document vectors, reflecting their semantic similarity. For documents with high similarity, the result is close to 1.

[0063] c) Similarity mean calculation: Take a keyword as an example, assuming it appears in n papers, the pairwise similarities calculated by the above b) similarity calculation step are Then calculate this value The mean of the similarity values ​​is recorded as S_mean.

[0064] d) Calculate the concentration index: Define the emerging technology concentration index D. For each keyword, its concentration index D is consistent with the mean similarity between papers S_mean, that is, D = S_mean.

[0065] e) Analysis: The higher the concentration index D, the more concentrated the distribution of the keyword among different papers and the more consistent the meaning, and the greater its potential as a weak signal of emerging technologies. Conversely, if D is low, it indicates that the keyword may belong to a topic that is more widely discussed and has a weaker potential as a signal of emerging technologies.

[0066] 3. Importance Index Calculation

[0067] In addition to the aforementioned suddenness and concentration, we also need to consider the importance of weak signals of emerging technologies, that is, whether they are mentioned in the overall national layout and economic and social development, and whether they are strategic priorities. In addition to the above keywords, this step also involves the policy and strategic data of major developed countries in the world. The specific calculation method is as follows:

[0068] a) Text vectorization: Sentence-BERT text vectors are used to represent keywords and the policy and strategic texts of major developed countries in the world are represented as high-dimensional vectors.

[0069] b) Calculate similarity: Calculate the similarity between two vectors using cosine similarity. The similarity of the sentence in the current policy and strategy text that is most semantically similar to the current keyword is used as the activity of the current keyword in the current policy and strategy text.

[0070] c) Calculate the importance index: Arrange the policy and strategy texts in descending order of activity, take the average activity of the top 25% of the policy and strategy texts, and use this as the overall activity of the current keyword in the policy strategies of the world's major developed countries, i.e., its importance.

[0071] d) Analysis: The higher the importance index, the more frequently the keyword is mentioned in the policies and strategies of major developed countries in the world. The higher the importance of the keyword, the greater its potential as a weak signal of emerging technologies. Conversely, it indicates that the keyword is not a focus of national strategic layout and its potential as a signal of emerging technologies is relatively weak.

[0072] (IV) Comprehensive analysis of weak signals from emerging technologies

[0073] Based on the quantitative analysis of weak signals of emerging technologies, and by integrating the professional insights of science and technology managers, field scientists, and intelligence analysis experts, we systematically assess the connotation and importance of weak signals of emerging technologies from multiple dimensions, including scientificity, innovation, and application potential. The basis for this assessment includes the following aspects:

[0074] a) Scientific Connotation Analysis: Combined with the quantitative calculation of weak signals, analyze their significance in terms of technological theory and scientific foundations, and use methods such as bibliometrics and technological evolution paths to explore the uniqueness and scientific value of the signal in the technological system.

[0075] b) Innovation potential assessment: Through technology maturity analysis and technology life cycle models, determine whether weak signals represent potential technological innovations and evaluate the innovative breakthroughs they may trigger in the existing technology system.

[0076] c) Feasibility Analysis of Implementation Paths: Evaluate the implementation paths for the weak signal based on the technology and resource requirements. Utilize tools such as techno-economic models and technology maturity assessments to determine the difficulty, resources required, and risk factors involved in transitioning from the laboratory stage to industrialization.

[0077] d) Development Prospect Forecast: Relying on quantitative analysis and cutting-edge knowledge from domain experts, and using intelligence monitoring, trend analysis, scenario analysis, and other means, we predict the possible evolution trajectory of this weak signal in future technology and markets and its impact, thereby assessing its long-term development potential.

[0078] The present invention uses scientific literature data such as academic papers and technical patents, and introduces policy and strategy data to evaluate the importance of weak signals of emerging technologies, reflecting the strength of its national strategic layout; uses a pre-trained model to characterize keywords and related data, reflects the appearance of keywords in various types of data from a semantic level, and accurately carries out the evaluation of weak signals of emerging technologies; through preliminary filtering of keywords with too high / too low frequency, and calculation and analysis of three indicators of suddenness, concentration and importance, multiple rounds of filtering are performed on weak signals of emerging technologies to eliminate noise as much as possible; a complete and systematic analysis process is constructed, including data collection, keyword extraction, preliminary filtering, evaluation and assessment, and comprehensive research and judgment, which can realize the systematic evaluation and identification of weak signals of emerging technologies.

[0079] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying weak signals of emerging technologies, characterized by: The steps include: Step 1: Keyword extraction and normalization; Step 2: Preliminary filtering of weak signals of emerging technologies: Use the word frequency g index method to calculate the threshold for dividing high-frequency and low-frequency words, and filter out keywords with too high a frequency. The specific calculation method is: sort the keywords in descending order of frequency, and assume that the number of keywords with the same frequency starting from the gth is n. If the cumulative frequency of g keywords is greater than or equal to g, then the keyword will be filtered out. 2 , and the cumulative frequency of (g+n) keywords is less than (g+n+1) 2 Then the keywords up to and including the g+nth word are all high-frequency words in the phrase set, and the expression is as follows: ; Where n is the number of keywords with the same frequency, and S is the cumulative frequency of keywords; Step 3: Evaluation of weak signals of emerging technologies: Define three evaluation indicators: suddenness, concentration, and importance to evaluate weak signals of emerging technologies; The suddenness evaluation index is: calculate the percentage of the keyword's appearance frequency in the past two years, the past three years, and the past five years to the total appearance frequency in the previous 20 years, and use the average of the percentages as the suddenness evaluation index; The concentration evaluation index is: for each keyword, find all papers containing the keyword, calculate the similarity between these papers, and calculate the mean of the similarities as the concentration evaluation index; The importance evaluation index is as follows: the similarity of the sentence with the most similar semantics to the current keyword in the current policy and strategy text is used as the activity of the current keyword in the current policy and strategy text; the policy and strategy texts are sorted in descending order of activity, and the average activity of the top 25% of the policy and strategy texts is taken as the importance evaluation index; Step 4: Comprehensive analysis of weak signals of emerging technologies.

2. The method for identifying weak signals of emerging technologies according to claim 1, characterized in that: The step 1 further includes the following contents: identifying weak signals of emerging technologies at the keyword level: using scientific and technological literature as the core data source, extracting abbreviations, keywords, technical terms, and technical phrases from titles and abstracts; at the same time, merging the extracted keywords using root restoration to achieve keyword standardization.

3. The method for identifying weak signals of emerging technologies according to claim 1, wherein: The concentration evaluation index is calculated as follows: a) Text vectorization: Use Sentence-BERT text vectors to represent keywords and convert the title and abstract of each paper into high-dimensional vectors; b) Calculate similarity: Evaluate the similarity between papers by calculating the cosine similarity between these vectors; for each keyword, find all papers containing the keyword and calculate the similarity between these papers; c) Similarity mean calculation: If a keyword appears in n papers, the pairwise similarities calculated by the above b) similarity calculation step are Then calculate this value The mean of the similarity values ​​is denoted as S_mean; d) Calculate the concentration index: define the emerging technology concentration index D. For each keyword, its concentration index D is consistent with the mean similarity between papers S_mean, that is, D = S_mean; e) Analysis: A higher concentration index, D, indicates that the keyword is more concentrated and consistent across papers, and has a greater potential to serve as a weak signal of emerging technologies. Conversely, a lower D indicates that the keyword may belong to a more widely discussed topic and has a weaker potential to serve as a signal of emerging technologies. The evaluation index of suddenness is calculated as follows: a) Frequency calculation: Calculate the frequency of occurrence of keywords in the scientific literature dataset for each year over the past 20 years, and record the total frequency as F; b) Calculate the percentage: Calculate the percentage of the keyword's frequency of occurrence in the past two years, three years, and five years relative to the total frequency of occurrence in the previous 20 years (F), and record them as N_2, N_3, and N_5 respectively; c) Calculate the burstiness index: Calculate the burstiness index T of the keyword using the following formula: T=(N_2+N_3+N_5) / 3 d) Analysis: The higher the technology's suddenness, the more popular the keyword has been in recent times, and the stronger its weak signal of emerging technology is. Conversely, the lower the popularity, the weaker its weak signal of emerging technology is. Step 4: Comprehensive analysis of weak signals of emerging technologies.

4. The method for identifying weak signals of emerging technologies according to claim 1, wherein: The evaluation index of the importance is calculated as follows: a) Text vectorization: Using Sentence-BERT text vectors to represent keywords, the policy and strategy texts of developed countries around the world are represented as high-dimensional vectors; b) Calculate similarity: Calculate the similarity between two vectors using cosine similarity. The similarity of the sentence in the current policy and strategy text that is most semantically similar to the current keyword is used as the activity level of the current keyword in the current policy and strategy text. c) Calculate the importance index: Arrange the policy and strategy texts in descending order of activity, take the average activity of the top 25% of the policy and strategy texts, and use this as the overall activity of the keyword in the policies and strategies of developed countries around the world, i.e., its importance; d) Analysis: The higher the importance index, the more frequently the keyword is mentioned in the policies and strategies of developed countries around the world. The higher the importance of the keyword, the greater its potential as a weak signal of emerging technologies. Conversely, it indicates that the keyword is not a focus of national strategic layout and its potential as a signal of emerging technologies is relatively weak.

5. A device for identifying weak signals of emerging technologies, characterized in that: The method comprises a processor and a memory; the memory stores computer-readable instructions, and the processor is used to execute the computer-readable instructions, wherein the computer-readable instructions execute the method according to any one of claims 1 to 4 when executed.

6. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the method according to any one of claims 1 to 4.

Images