A method and system for identifying emerging technologies based on an emerging technology tag library

Abstract

The present invention belongs to the field of technology classification and identification, and specifically discloses a method and system for identifying emerging technologies based on an emerging technology label library, which includes: obtaining emerging technology documents from different years, and then extracting the associated text of time-technology keywords-technology description-representation vector, and storing it in the emerging technology label library; extracting the abstract of the patent to be identified as the technology description text, and characterizing the technology description text through a text representation model to obtain a patent representation vector; matching the patent representation vector with the associated text of the same year in the emerging technology label library; respectively calculating the similarity between the patent representation vector and the representation vector in the associated text to obtain the maximum similarity; constructing similarity statistical curves under different years, and obtaining the part with sufficient confidence as the emerging technology according to the corresponding statistical distribution. The present invention solves the problem of existing emerging technology classification methods that use expert knowledge to label emerging technologies, and improves the reliability of the method.

Description

Technical Field

[0001] The present invention belongs to the field of technology classification and identification, and more specifically, relates to an emerging technology identification method and system based on an emerging technology label library. Background Art

[0002] The most widely accepted definition of emerging technologies is that proposed by the Wharton School of the University of Pennsylvania: emerging technologies are innovative, scientifically grounded technologies that have the potential to create a new industry or transform an existing one. Finding an effective emerging technology identification method that can accurately identify emerging technologies is of great strategic value.

[0003] With the rapid development of technology and the constant push of technological revolutions, the process of forming emerging technologies has become more complex than before, making the identification of emerging technologies more difficult. Emerging technology discovery is primarily categorized into two methods: qualitative and quantitative. Qualitative methods incorporate expert knowledge through Newman clustering, while quantitative methods primarily focus on patent information, identifying emerging technologies through literature analysis or mining techniques using patent text or through machine learning using patent indicators. Patent information, which encompasses both the timeliness and comprehensiveness of technology, two core characteristics that distinguish emerging technologies, is a crucial source of technological information, and therefore, most emerging technology discovery relies on patent data.

[0004] With the continuous improvement of computing power, the development of big data, machine learning, and deep learning methods has continued to accelerate, bringing new developments to emerging technology discovery. Network analysis focuses on the relationships between nodes. With the development of technology, the foundation of network analysis has gradually evolved from single-node, single-connection networks to multi-node, multi-connection, multi-layer networks (heterogeneous networks). Network analysis also provides many parameters for characterizing network characteristics for intelligence analysis. However, as new methods emerge, emerging technology discovery methods also face the problem of insufficient labeled emerging technology samples to support complex models.

[0005] However, existing labeling methods are relatively dependent on expert knowledge, and the process of judging emerging technologies is complex and relies on expert judgment, which affects the reliability of the method. At the same time, labeling problems for large-scale data from different fields is relatively cumbersome and time-consuming to solve. Summary of the Invention

[0006] In response to the above-mentioned defects or improvement needs of the existing technology, the present invention provides an emerging technology identification method and system based on an emerging technology label library, which aims to solve the problems of low efficiency in labeling patent data, reliance on expert knowledge, and low reliability.

[0007] To achieve the above objectives, according to a first aspect of the present invention, a method for identifying emerging technologies based on an emerging technology tag library is proposed, comprising the following steps:

[0008] Obtain emerging technology documents from different years in the research field; extract technical keywords and time from each emerging technology document, obtain technical descriptions based on technical keywords, and characterize the technical descriptions using a text representation model to obtain representation vectors, thereby forming multiple associated texts of time-technical keyword-technical description-representation vectors, which are stored in the emerging technology tag library;

[0009] Extract the abstract of the patent to be identified as the technical description text, characterize the technical description text through the text representation model, and obtain the corresponding patent representation vector;

[0010] All patent representation vectors are classified by year, and the patent representation vectors are matched with the associated text of the same year in the emerging technology label library; the similarity between the patent representation vector and all the representation vectors in the matched associated text is calculated respectively, and the maximum similarity is taken as the maximum similarity of the patent representation vector; based on the obtained maximum similarity, a similarity statistical curve is constructed under different years, and then the patents to be identified that meet the preset confidence level are selected as emerging technologies.

[0011] As a further preference, the text representation model is a pre-trained BERT model.

[0012] As a further preference, the training process of the BERT model is as follows:

[0013] Preprocess the technical description text in the emerging technology tag library; use the BERT model to process the preprocessed technical description text, and at the same time obtain the loss of the BERT model through the cross entropy function and perform backpropagation based on the loss to train the BERT model; when the loss of the BERT model meets the requirements, stop training and obtain the initially trained BERT model;

[0014] Obtain the patent abstract text of the field to be studied, and use the same method as the above technical description text to retrain the initially trained BERT model with the patent abstract text to obtain the trained BERT model.

[0015] As a further preferred embodiment, the technical description text is preprocessed, specifically:

[0016] Mask the technical description text, that is, convert some words in the text into masks; for the masked data, replace part of it with the mask label, replace part with other words randomly, and keep the rest unchanged; finally, fill the text with zeros to meet the dimensionality requirements of the BERT model.

[0017] As a further optimization, the BERT model processes the input text as follows:

[0018] The BERT model extracts word information, the position of the word in the sentence, and whether the sentence to which the word belongs is the first sentence from the input text, and processes it into three-dimensional data; then, the information in the three-dimensional data is combined together through the multi-head attention mechanism to obtain the word representation vector; finally, the word representation vector is mapped to the word's word vector information through the GELUT activation function, and the word vector information is combined into the text representation vector, which is the output information of the BERT model.

[0019] As a further preference, the emerging technology tag library is a hive distributed database, which stores text data according to the time dimension.

[0020] As a further preferred method, when calculating the similarity, Spark is used to perform distributed computing according to different years to obtain the similarity between the patent representation vector and all representation vectors in the matching associated text under the corresponding time slice.

[0021] As a further preferred embodiment, the emerging technology documents include Gartner's Emerging Technology Hype Cycle, the Global Engineering Focus released by the Academy of Engineering, and the emerging technology list in Wikipedia; obtaining the technology description through technology keywords, specifically obtaining the corresponding technology description through searching Wikipedia through technology keywords.

[0022] According to a second aspect of the present invention, an emerging technology identification system based on an emerging technology tag library is provided, which includes an emerging technology tag library construction module, a patent representation vector extraction module, and an emerging technology identification module, wherein:

[0023] The emerging technology tag library construction module is used to obtain emerging technology documents from different years; extract technical keywords and time from each emerging technology document, obtain technical descriptions based on the technical keywords, and characterize the technical descriptions using a text characterization model to obtain characterization vectors, thereby forming multiple associated texts of time-technical keywords-technical descriptions-characterization vectors, and storing them in the emerging technology tag library;

[0024] The patent representation vector extraction module is used to extract the abstract of the patent to be identified as a technical description text, and to represent the technical description text through a text representation model to obtain a corresponding patent representation vector;

[0025] The emerging technology identification module is used to classify all patent representation vectors according to year, match the patent representation vectors with the associated text of the same year in the emerging technology label library; obtain the similarity between the patent representation vector and all representation vectors in the matched associated text, and take the maximum similarity as the maximum similarity of the patent representation vector; based on the obtained maximum similarity, construct a similarity statistical curve under different years, and then select the patent to be identified that meets the preset confidence level as an emerging technology.

[0026] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying emerging technologies based on the emerging technology tag library is implemented.

[0027] In general, the above technical solutions conceived by the present invention have the following technical advantages compared with the existing technology:

[0028] 1. The present invention constructs an emerging technology label library and then solves the similarity of text representation vectors to identify emerging technologies in different years. This can solve the problem that existing emerging technology classification methods partially rely on qualitative methods and label emerging technologies through expert knowledge, thereby improving the reliability of the method. At the same time, it provides a unified solution for determining whether a technology patent is an emerging technology. It can label patent technologies in different fields and reduce the time spent on labeling emerging technologies.

[0029] 2. The present invention introduces the BERT model trained in the corresponding patent field for text representation, improves the accuracy of text representation after training on a specified text corpus, and provides a data basis for subsequent similarity calculation.

[0030] 3. The present invention utilizes the hive distributed database to store data according to the time dimension to provide an environmental foundation for the subsequent distributed calculation of similarity. At the same time, through the distributed computing method, similarity calculation is performed on technical patents of different years and corresponding matching emerging technologies. The distributed method greatly reduces the time required for calculation, and the statistical distribution method is used to determine whether it is an emerging technology, thereby improving the reliability of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a method for identifying emerging technologies based on an emerging technology tag library according to an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of data storage of an emerging technology tag library according to an embodiment of the present invention;

[0033] Figure 3 Schematic diagram of a similarity curve according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0035] An embodiment of the present invention provides an emerging technology identification method based on an emerging technology tag library, such as Figure 1 As shown, the following steps are included:

[0036] (1) For the research field, obtain documents related to emerging technologies in different years and organize documents on emerging technologies from multiple sources.

[0037] Specifically, we introduced data from multiple different sources as data sources, including Gartner's Emerging Technology Hype Cycle, the Global Engineering Focus released by the Chinese Academy of Engineering, and the emerging technology list provided by Wikipedia, to form an emerging technology document. Through multi-source data, we eliminated possible biases in different types of data.

[0038] (2) Process the emerging technology documents, obtain data of three fields: time, technology keywords, and technology description, and store them in the emerging technology tag library.

[0039] Specifically, different organization schemes are adopted for different documents: for the Gartner Emerging Technology Hype Cycle, technical texts from the emerging technology hype curve at different time slices are obtained as technical keywords; for documents such as the Global Engineering Focus published by the Academy of Engineering, technical keywords from different years are obtained;

[0040] Then, based on the obtained technical keywords, the corresponding technical description is searched through Wikipedia, and the three fields of technical keywords, technical description, and year are stored in the emerging technology tag library.

[0041] Furthermore, the emerging technology tag library stores data in the distributed data store Hive. Hive is a data warehouse tool based on Hadoop that is used for data extraction, transformation, and loading. It is a mechanism for storing, querying, and analyzing large amounts of data stored in Hadoop. The Hive data warehouse tool maps structured data files into database tables and provides SQL query capabilities, converting SQL statements into MapReduce tasks for execution.

[0042] (3) The acquired technical description is represented by a text representation model, and the representation vector is obtained as the fourth field and stored in the emerging technology tag library.

[0043] Furthermore, the text representation model is a pre-trained BERT model. This embodiment uses the BERT-Patent model trained with patent corpus to obtain text representation. After obtaining the representation vector, it is stored in the distributed database Hive according to year, completing the entire emerging technology tag library construction process.

[0044] The emerging technology tag library constructed, such as Figure 2 As shown, distributed storage is performed according to different years to facilitate distributed calculation of similarity according to different times. It mainly stores four fields related to emerging technologies: year, technology keywords, technology description and text representation.

[0045] Furthermore, the BERT model is trained based on the emerging technology tag library, which includes the following steps:

[0046] (3-1) Obtain the patent abstract text in the research field (it can be in the research field), and preprocess the patent abstract text and the technical description text in the emerging technology tag library to meet the requirements of the BERT model; specifically, the method of preprocessing the data for input to the model is as follows:

[0047] (3-1-1) For the input text data, mask processing is performed first. For the sentences in the long text, 15% of the sentences are converted into masks. To speed up the subsequent training, the text data is used as the input of different epochs of the BERT model. The data of different epochs are segmented, and the processing steps (3-1-2) are performed separately for the data of each epoch.

[0048] (3-1-2) For the masked data, 80% of them are replaced with mask labels, 10% are randomly replaced with other words, and 10% remain unchanged;

[0049] (3-1-3) According to the required dimension of the BERT model, the input data that does not meet the requirements is padded with zeros to meet the dimensional requirements of the model.

[0050] (3-2) Input the preprocessed technical description text in the emerging technology tag library into the BERT model; use the BERT model to process the preprocessed technical description text, and at the same time obtain the loss of the BERT model through the cross entropy function and perform backpropagation based on the loss to train the BERT model; when the loss of the BERT model meets the requirements, stop training and obtain the preliminarily trained BERT model.

[0051] Specifically, the BERT model extracts word information, its position within a sentence, and whether the sentence it belongs to is the first sentence from the input text, and processes it into three-dimensional data. This information is then combined using a multi-head attention mechanism to obtain a word representation vector. Finally, the GELUT activation function maps the word representation vector to its own word vector information, which is then combined into the text representation vector, the output of the BERT model. The multi-head attention mechanism captures information about different words at different positions, thereby obtaining richer features. The GELUT activation function maps input to output, mitigating the vanishing gradient problem and accelerating training convergence. The activation function can be normalized to ensure that the accumulated gradient after combination is not excessive.

[0052] (3-3) Using the same method as the technical description text in the emerging technology tag library, the preprocessed patent abstract text is input into the preliminarily trained BERT model, and it is further trained to obtain the trained BERT model, that is, the text representation model.

[0053] (4) For the patents to be labeled, the abstract of the patent to be identified is extracted as the technical description text, and the technical description text is represented by the text representation model to obtain the corresponding patent representation vector.

[0054] (5) Obtain the similarity between the patent representation vector obtained in step (4) and the representation vector in the emerging technology label library, and obtain the data with sufficient confidence as the emerging technology according to the statistical distribution of the similarity data.

[0055] Specifically, step (5) includes the following steps:

[0056] (5-1) Classify patents according to their different years and match them with the emerging technologies of the corresponding years, thereby matching the patent representation vector with the associated text of the same year in the emerging technology label library;

[0057] (5-2) Using Spark to perform distributed computing according to different years, obtain the similarity between patent technologies and emerging technologies in the corresponding time slices; that is, for a patent to be labeled in a certain year, calculate the similarity between the patent representation vector and all representation vectors of that year, and take the maximum similarity as the maximum similarity of the patent representation vector;

[0058] (5-3) Based on the obtained series of maximum similarities, construct similarity statistical curves for different years;

[0059] (5-4) Based on the similarity statistical curve, the part with sufficient confidence is obtained according to the corresponding statistical distribution as the emerging technology.

[0060] Specifically, Spark is a fast and general computing engine designed for large-scale data processing. Spark has the advantages of Hadoop MapReduce, but unlike MapReduce, its intermediate job output results can be stored in memory. Through memory operations, Spark has faster speed and better compatibility, which improves the speed of the entire process.

[0061] The following are specific embodiments:

[0062] For the CNC field, 10,231 patent data in the corresponding field in 2022 were obtained from the DI database, and matched with the emerging technology label library data of the corresponding year to obtain the normal distribution curve of their similarity, such as Figure 3 As shown in the figure, patents with similarity distribution in a high confidence interval (confidence greater than 99%) are selected as emerging technologies, so as to use quantitative methods to determine whether patents in the field of CNC belong to emerging technologies.

[0063] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying emerging technologies based on an emerging technology tag library, characterized in that: The steps include: Obtain emerging technology documents from different years in the research field; extract technical keywords and time from each emerging technology document, obtain technical descriptions based on technical keywords, and characterize the technical descriptions using a text representation model to obtain representation vectors, thereby forming multiple associated texts of time-technical keyword-technical description-representation vectors, which are stored in the emerging technology tag library; Extract the abstract of the patent to be identified as the technical description text, characterize the technical description text through the text representation model, and obtain the corresponding patent representation vector; All patent representation vectors are classified by year and matched with the associated text of the same year in the emerging technology label library. The similarity between the patent representation vector and all the representation vectors in the matched associated text is calculated, and the maximum similarity is taken as the maximum similarity of the patent representation vector. Based on the obtained maximum similarity, similarity statistical curves for different years are constructed, and then the patents to be identified that meet the preset confidence level are selected as emerging technologies. The text representation model is a pre-trained BERT model. The training process of the BERT model is as follows: Preprocess the technical description text in the emerging technology tag library. Specifically, mask the technical description text, that is, convert some words in the text into masks. For the masked data, replace part of it with the mask label, replace part with other words randomly, and keep the rest unchanged. Finally, pad the text with zeros to meet the dimensionality requirements of the BERT model. The BERT model is used to process the preprocessed technical description text. At the same time, the loss of the BERT model is obtained through the cross entropy function and backpropagation is performed according to the loss to train the BERT model. When the loss of the BERT model meets the requirements, the training is stopped and the preliminary trained BERT model is obtained. Obtain the patent abstract text of the field to be studied, and use the same method as the above technical description text to retrain the initially trained BERT model with the patent abstract text to obtain the trained BERT model; The BERT model processes the input text as follows: The BERT model extracts word information, the position of the word in the sentence, and whether the sentence to which the word belongs is the first sentence from the input text, and processes it into three-dimensional data; then, the information in the three-dimensional data is combined together through the multi-head attention mechanism to obtain the word representation vector; finally, the word representation vector is mapped to the word's word vector information through the GELUT activation function, and the word vector information is combined into the text representation vector, which is the output information of the BERT model.

2. The emerging technology identification method based on the emerging technology tag library according to claim 1, characterized in that: The emerging technology tag library is a hive distributed database, which stores text data according to the time dimension.

3. The emerging technology identification method based on the emerging technology tag library according to claim 2, characterized in that: When calculating the similarity, Spark is used to perform distributed computing according to different years to obtain the similarity between the patent representation vector in the corresponding time slice and all the representation vectors in the matching associated text.

4. The emerging technology identification method based on the emerging technology tag library according to any one of claims 1 to 3, characterized in that: The emerging technology documents include Gartner's Hype Cycle for Emerging Technologies, the Global Engineering Focus released by the Academy of Engineering, and the emerging technology list on Wikipedia; technology descriptions are obtained through technology keywords, specifically by searching Wikipedia for corresponding technology descriptions through technology keywords.

5. A system for implementing the emerging technology identification method based on the emerging technology tag library according to any one of claims 1 to 4, characterized in that: It includes an emerging technology label library construction module, a patent representation vector extraction module, and an emerging technology identification module, among which: The emerging technology tag library construction module is used to obtain emerging technology documents from different years; extract technical keywords and time from each emerging technology document, obtain technical descriptions based on the technical keywords, and characterize the technical descriptions using a text characterization model to obtain characterization vectors, thereby forming multiple associated texts of time-technical keywords-technical descriptions-characterization vectors, and storing them in the emerging technology tag library; The patent representation vector extraction module is used to extract the abstract of the patent to be identified as a technical description text, and to represent the technical description text through a text representation model to obtain a corresponding patent representation vector; The emerging technology identification module is used to classify all patent representation vectors according to year, match the patent representation vectors with the associated text of the same year in the emerging technology label library; obtain the similarity between the patent representation vector and all representation vectors in the matched associated text, and take the maximum similarity as the maximum similarity of the patent representation vector; based on the obtained maximum similarity, construct a similarity statistical curve under different years, and then select the patent to be identified that meets the preset confidence level as an emerging technology.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying emerging technologies based on an emerging technology tag library according to any one of claims 1 to 4 is implemented.

Images