Computer-implemented method, device and computer program product

Abstract

Embodiments of the present disclosure relate to a computer-implemented method, a device and a computer program product. The method comprises extracting a respective topic of a set of documents published at a first time period; determining a respective semantic information of the topic and a frequency of occurrence in the set of documents; and determining, based on the semantic information of the topic and the frequency, a number of documents associated with the topic in a second time period according to a prediction model. The second time period is after the first time period. Embodiments of the present disclosure can better predict a trend of the topic to occur in the future based on the semantic information of the topic and the frequency.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of computers, and more specifically to computer-implemented methods, apparatus, and computer program products. Background Technology

[0002] The emergence and widespread application of technologies such as big data, the Internet of Things, and artificial intelligence have generated massive amounts of data across various fields. From this massive data, various kinds of knowledge can be efficiently and transparently extracted and integrated, and predictions can be made about future technological developments. For example, massive data can be used to predict the future popularity of various topics, such as trending technologies. However, the accuracy of predicting future trends in popularity for different topics needs further improvement. Summary of the Invention

[0003] Embodiments of this disclosure provide computer-implemented methods, apparatus, and computer program products.

[0004] In a first aspect of this disclosure, a computer-implemented method is provided. The method includes extracting the topics of a set of documents whose publication times fall within a first time period; determining the semantic information of each topic and its frequency of occurrence within the set of documents; and, based on the semantic information and frequency of the topics, determining, according to a prediction model, the number of documents associated with the topics in a second time period following the first time period.

[0005] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit and at least one memory. The at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform an action including: extracting the topics of a set of documents whose publication times are within a first time period; determining the semantic information of each topic and its frequency of occurrence in the set of documents; and, based on the semantic information and frequency of the topics, determining, according to a prediction model, the number of documents associated with the topics within a second time period, the second time period following the first time period.

[0006] In a third aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. When executed by a device, the machine-executable instructions cause the device to perform any step of the method described in the first aspect of this disclosure.

[0007] The summary section is provided to present the chosen concepts in a simplified form, which will be further described in the detailed description below. The summary section is not intended to identify key or essential features of this disclosure, nor is it intended to limit the scope of this disclosure. Attached Figure Description

[0008] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0009] Figure 1 A schematic block diagram of an example environment that can be implemented therein according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A schematic diagram of a prediction model according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A flowchart is shown illustrating an example method for predicting topic trends according to some embodiments of this disclosure;

[0012] Figure 4 Example block diagrams of subject matter extracted according to some embodiments of this disclosure are shown;

[0013] Figure 5 A schematic diagram illustrating the subject extraction results according to some embodiments of the present disclosure is shown;

[0014] Figure 6 A schematic diagram illustrating example output results of topic trend prediction according to some embodiments of the present disclosure and output results of conventional schemes is shown;

[0015] Figure 7 A schematic diagram illustrating yet another output result of topic trend prediction according to some embodiments of the present disclosure; and

[0016] Figure 8 A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0017] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0018] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0019] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0020] In embodiments of this disclosure, the term "model" refers to a system capable of processing inputs and providing corresponding outputs. Taking a neural network model as an example, it typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Models used in deep learning applications (also referred to as "deep learning models") often include many hidden layers, thereby extending the depth of the network. The layers of a neural network model are sequentially connected such that the output of the previous layer is used as the input of the next layer, where the input layer receives the inputs to the neural network model, and the output of the output layer serves as the final output of the neural network model. Each layer of a neural network model includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above. In this document, the terms "neural network," "model," "network," and "neural network model" are used interchangeably.

[0021] As mentioned above, several schemes have been proposed to predict the future attention given to various topics, such as popular technical topics, based on large amounts of data. In some conventional schemes, the frequency of attention given to various topics in the past period is often used to predict the frequency of attention given to those topics in the future. For example, the frequency of a particular topic appearing in papers published in the past period can be used to predict the number of papers published on that topic in the future.

[0022] However, this conventional method of predicting theme trends only considers the frequency of attention a theme receives over a past period, without taking into account other factors that influence theme trends. For example, some themes are correlated with each other. The future trend of one theme may affect the future trends of other related themes. This conventional method does not account for such inter-theme influence. Therefore, the accuracy of this conventional approach to theme trend prediction needs further improvement.

[0023] Embodiments of this disclosure propose a computer-implemented method to address one or more of the problems described above and / or other potential problems. In this approach, the topics of a set of documents published over a past period are extracted. The approach also includes determining semantic information for each of the different topics and their frequency of occurrence within the set of documents. Furthermore, based on the semantic information and frequency of the topics, the approach further includes determining, according to a predictive model, the number of documents associated with a particular topic in a future period.

[0024] This approach considers not only the frequency of a topic's appearance over a period of time but also its semantics. By taking both frequency and semantics into account, it becomes possible to better predict the future trends of a topic's popularity. This is particularly true for multiple topics with semantic connections, making future trend predictions more meaningful.

[0025] The basic principles and several exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0026] Figure 1 A schematic diagram of an environment 100 for predicting future trends of a subject matter, according to some embodiments of this disclosure, is shown. It should be understood that... Figure 1 The number and arrangement of entities, elements and modules shown are merely examples; environment 100 may include different numbers and arrangements of entities, elements and modules.

[0027] exist Figure 1 In environment 100, computing system 120 extracts the topics of a set of documents 110. Each document in this set of documents 110 has a publication time or posting time within a first time period. For example, the documents in this set of documents 110 can be various types of documents such as journal articles, news reports, and professional books. Computing system 120 can use a preset algorithm or a pre-trained topic extraction model to extract the topics of the set of documents 110. For example, a model based on Computer Science Ontology (CSO) can be used to extract the topics of the set of documents 110. An example topic extraction process will be described in more detail below.

[0028] The computing device 120 determines the semantic information 140 of each topic 130 and its frequency of occurrence in a set of documents 110. In this document, the frequency of occurrence of topic 130 in a set of documents 110 represents the number of documents in that set of documents related to topic 130. The computing system 120 may use a pre-defined algorithm or a pre-trained semantic determination model to determine the semantic information of the topics. Similarly, the computing system 120 may use a pre-defined algorithm or a pre-trained frequency determination model to determine the frequency of the topics. An example of the semantic information determination and frequency determination process will be described in more detail below.

[0029] like Figure 1 As shown, computing device 120 determines the number 170 of documents associated with topic 130 within a second time period based on semantic information 140 and frequency 150 of topic 130, according to prediction model 160. The second time period is a period following the first time period. For example, the second time period is a future time period. The number 170 can reflect the future attention trend of topic 130.

[0030] In some embodiments, the prediction model 160 may be a pre-trained neural network model. Figure 2 A schematic diagram of an example prediction model 160 according to some embodiments of the present disclosure is shown. Figure 2 As shown, prediction model 160 can be a Long Short-Term Memory (LSTM) network. Figure 2 In the example, prediction model 160 includes an input gate 230, an output gate 240, and a forget gate 250. The input x received by prediction model 160 is... t 210 is input to input gate 230, output gate 240, and forget gate 250. After, for example... Figure 2 The internal computational logic shown (i.e., including multiple multiplication gates, addition gates, etc.) can derive the result corresponding to the input x. t The result of 210 is the output 220. Output 220 can correspond to the input x. t The number or frequency of predictions for 210.

[0031] By employing prediction models such as LSTM160, the vanishing gradient problem encountered by recurrent neural networks (RNNs) can be overcome. This is achieved by enforcing a constant error flow within the internal states of special units called "memory cells" through multiple multiplication gates within the LSTM. Using methods such as... Figure 2 The example prediction model 160 shown, through input gate 230, output gate 240, and forget gate 250, prevents the memory content from being disturbed by irrelevant inputs and outputs, thereby achieving long-term memory storage. Furthermore, due to LSTM's ability to learn long-term correlations in sequences, LSTM does not require a pre-specified time window and does not need to accurately model complex multivariate sequences. Therefore, using a prediction model 160 such as LSTM can better predict topic trends.

[0032] It should be understood that prediction model 160 can also use other machine learning models. Figure 2 The example prediction model 160 shown is for illustrative purposes only, and the scope of this disclosure is not limited in this respect.

[0033] The above combination Figure 1An example environment 100 according to some embodiments of the present disclosure is described below. Reference will be made to... Figure 3 This document describes a flowchart of a method 300 for predicting topic trends according to some embodiments of the present disclosure. Method 300 may be... Figure 1 The computing system 120 is implemented. It should be understood that method 300 can also be performed by other suitable devices or apparatuses. Method 300 may include additional actions not shown and / or the actions shown may be omitted; the scope of this disclosure is not limited in this respect. For ease of explanation, reference will be made to... Figure 1 Let's describe method 300.

[0034] like Figure 3 As shown, at 310, the computing system 120 extracts the topics of a group of documents 110 whose publication time falls within a first time period. For example, the computing system 120 may extract documents published within the first time period as a group of documents 110 from a database (e.g., Springer, CNKI, etc.) or from other online resources (e.g., Google, etc.). The first time period may be a period of time from the current time, such as a week, a month, a quarter, or other time lengths.

[0035] In some embodiments, the computing system 120 may extract all documents whose publication time falls within a first time period as a set of documents 110. Additionally or alternatively, the computing system 120 may extract a predetermined number of documents with the highest number of visits from all documents whose publication time falls within the first time period as a set of documents 110. The predetermined number may be, for example, 1000, 10000, or any other suitable number. It should be understood that other methods may also be used to obtain a set of documents 110.

[0036] The computing system 120 can use a topic classification model to extract the topics of each document 110. For example, the topic classification model could be a pre-trained CSO topic classification model. Figure 4 An example block diagram 400 is shown illustrating topic extraction using a topic classification model 410 according to some embodiments of the present disclosure.

[0037] like Figure 4 As shown, the topic classification model 410 may include a syntax module 420, a semantic module 430, and a post-processing module 440. The topic classification model 410 can receive a set of documents 110 and extract the topic of each document in the set of documents 110. The topic extraction process of the topic classification model 410 will be described below using document 401 in the set of documents 110 as an example.

[0038] In some embodiments, the syntax module 420 may divide the content of document 401 into multiple N-grams. In some embodiments, the syntax module 420 may divide the title, summary, and keywords of document 401 into multiple N-grams. Alternatively, the syntax module 420 may also divide other content of document 401, such as the content of the concluding summary, into multiple N-grams. This approach is not limited in this respect. The syntax module 420 further determines the similarity of each N-gram to different topics (or concepts).

[0039] In some embodiments, semantic module 430 may be a module based on CSO and word tagging or embedding. Semantic module 430 may extract entities from multiple segments divided by document 401. In this document, an entity may refer to a word or phrase associated with a topic. For example, an entity may be a phrase associated with the topic "neural network" such as "image recognition," "3D reconstruction," etc.

[0040] Additionally or alternatively, the semantic module 430 can also perform topic identification on the extracted entities. For example, if the extracted entity is "3D reconstruction", the semantic module 430 can identify it as associated with "neural network".

[0041] In some embodiments, the semantic module 430 may sort the topics identified in document 401 according to the number or frequency of the topics identified. The semantic module 430 may select the topics with the highest sorted number (e.g., 5) as the topics of document 401. It should be understood that the number 5 listed herein is merely illustrative, and the predetermined number may be any suitable number.

[0042] like Figure 4 As shown, the topic classification model 410 may further include a post-processing module 440. The post-processing module 440 can combine the word segments output by the syntax module 420 and the topics output by the semantic module 430. For example, the post-processing module 440 can combine word segments and topics based on the similarity between each word segment and the topic determined by the syntax module 420. In some embodiments, the post-processing module 440 uses, for example, a knowledge graph to combine the word segments and topics in its output.

[0043] Additionally or alternatively, in some embodiments, the post-processing module 440 also filters the output. For example, if the post-processing module 440 can determine the similarity between a predetermined number of topics output by the semantic module 430 for document 401, and if it determines that a topic among the predetermined number of topics has a similarity lower than a similarity threshold (e.g., the similarity could be a value between 0 and 1), that topic can be removed.

[0044] In some embodiments, the post-processing module 440 may further enhance the output. For example, if the post-processing module 440 determines that the output topic contains a first topic and a more refined second topic that can be included within the first topic, the post-processing module 440 may remove the first topic from the output. For example, if the post-processing module 440 determines that the output includes the topics "artificial intelligence" and "recurrent neural network," the post-processing module 440 may retain only the topic "recurrent neural network" and remove the topic "artificial intelligence."

[0045] The above combination Figure 4 The process of extracting a topic from document 401 in a set of documents 110 is described. It should be understood that other documents in the set of documents 110 can also be extracted in a similar manner.

[0046] By using the above combination Figure 4 The described topic classification model 410 is able to more accurately extract the topics of a group of documents. It should be understood that... Figure 4 The topic classification model 410 described is for illustrative purposes only and is not restrictive. The computing system 120 may employ other appropriate algorithms or pre-trained machine learning models to extract the topics of each set of documents 110.

[0047] Figure 5 A schematic diagram of a subject extraction result 500 according to some embodiments of the present disclosure is shown. Figure 5 The topic extraction result 500 shown can be obtained through reference. Figure 4 The topic classification model 410 described herein. Figure 5 The topic extraction result 500 in the example can also be obtained using other topic extraction methods. For example... Figure 5 As shown, document information 510 illustrates document information 510 for an example document in a set of documents 110. Document information 510 includes the document's title, summary, and keywords. It should be understood that the computing system 120 can also receive document information for other documents in the set of documents 110.

[0048] Figure 5 The example results include over 520 output results. For instance, the results included in the "union" field 530 include topics extracted from document information 510. The results included in the "enhanced" field 540 include filtered and enhanced topics obtained by post-processing module 440 from the results included in the "union" field 530.

[0049] exist Figure 5The example also shows other additional output results. For example, the “syntactic” field may contain segments obtained by the syntactic module 420. The “semantic” field may contain topics identified by the semantic module 430. The “explanation” field may contain information combining topics with corresponding segments. Furthermore, Figure 5 Output 550, corresponding to another document, is also shown. Output 550 is similar to output 520.

[0050] It should be understood that Figure 5 The illustrated result diagram 500 is merely illustrative and not limiting. In some embodiments, the output may include only the subject matter and not other relevant information. Figure 5 Additional information may be included in the output. In some embodiments, the output may include more additional information. By including information beyond the subject in the output, more information can be provided for analysis during subsequent presentation of the results to the user.

[0051] Continue to refer to Figure 3 At 320, the computing system 120 determines the semantic information 140 of each topic and its frequency of occurrence in a set of documents 110. In some embodiments, the computing system 120 may determine the semantic information and frequency of each topic extracted from each document in the set of documents 110. In some embodiments, the computing system 120 may also select some topics from the topics extracted from each document in the set of documents 110, such as one or more topics of interest, to determine their respective semantic information and frequency.

[0052] In some embodiments, the computing system 120 can determine the semantic information of a topic by determining a semantic representation time series within a first time period. For example, the topic at each time interval within the first time period can be encoded as a semantic representation time series. The semantic representation at each time interval in the semantic representation time series is used to represent the semantic meaning of the topic, and it can also be regarded as a word embedding.

[0053] Taking the simple statements "Have a good day" and "Have a great day" as examples, the set of words {"have", "a", "good", "great", "day"} contained in these two statements has a dimension of 5. Therefore, a 5-dimensional vector can be used to encode each word separately. For example, the word "have" can be encoded as [1,0,0,0,0]. The word "a" can be encoded as [0,1,0,0,0]. The word "good" can be encoded as [0,0,1,0,0]. The word "great" can be encoded as [0,0,0,1,0]. The word "day" can be encoded as [0,0,0,0,1]. A similar approach can be used to encode each topic at each time interval within the first time period, thereby obtaining a semantically represented time series.

[0054] In some embodiments, a pre-trained semantic coding model can be used to determine the semantic representation time series. The pre-trained semantic coding model can be implemented using any suitable type of neural network. For example, a Transformer or a Transformer-based Bidirectional Encoder (BERT) can be used to implement the pre-trained semantic coding model. It should be understood that the semantic coding model used in this paper is applicable to various languages, such as English, Chinese, etc.

[0055] In some embodiments, the pre-trained semantic encoding model determines the semantic representation of the time series at a given time interval based on words or phrases in a set of documents 110 whose publication time is no later than that time interval.

[0056] In some embodiments, the classification tokens ([cls]tokens) corresponding to a topic determined by a pre-trained semantic encoding model can be used as the semantic representation of that topic. This allows topics with different numbers of words to be represented using a single classification token with the same dimension. This facilitates subsequent tasks such as topic trend prediction.

[0057] The time intervals within the first time period can be the same or different. For example, if the first time period is one week, it can have seven time intervals, with each day as a separate interval. A pre-trained semantic encoding model can determine the semantic information of the topic at each time interval within a time period. It should be understood that the length of the first time period and the intervals between the time intervals listed in this disclosure are illustrative only and not restrictive.

[0058] By employing pre-trained semantic encoding models such as BERT to obtain semantic representations of various topics, the semantic information of the topics can be accurately extracted. Furthermore, by appropriately encoding the semantic information, subsequent tasks such as topic trend prediction can be facilitated.

[0059] In some embodiments, the computing system 120 can determine the frequency of a topic by determining the frequency representation time series of the topic within a first time period. For example, the computing system 120 can determine the frequency representation of the topic at a time interval point within the first time period based on the number of documents in a set of documents 110 whose publication time is no later than that time interval point that correspond to the topic.

[0060] Alternatively or additionally, the computing system 120 may also use position-extended coding to determine the frequency representation of the topic's frequency representation time series at various time intervals. Position-extended coding can, for example, be an extended coding of a cosine probability function. An example position code is shown below:

[0061]

[0062] In equation (1), t represents the frequency of the topic, and D it Let t represent the frequency representation. k represents the k-th component of the frequency representation, which has dimension d, and k ≤ d.

[0063] In some embodiments, ω in equation (1) n It can be represented as follows:

[0064]

[0065] By employing the position-extended coding methods listed above, the one-dimensional frequency of a topic can be extended into a multi-dimensional frequency representation. Equation (3) below shows one such frequency representation obtained through encoding. Example:

[0066]

[0067] In some embodiments, the frequency representation dimension d can be a pre-set number greater than 1. For example, if the semantic representation of the time series at each time interval is 728 dimensions, then d can be set to 728 dimensions. It should be understood that the number of dimensions listed above is merely illustrative. It should be understood that d can be set to other dimensions, the same as, more than, or fewer than the semantic representation dimension. The frequency representation can take the form of a vector, as shown in (3) above. The frequency representation can also take other data forms. This scheme is not limited in this respect.

[0068] It should be understood that the position extension coding methods listed above are merely illustrative and not restrictive. Other extension coding methods can be used to extend one-dimensional frequencies into multi-dimensional frequency representations.

[0069] By representing the frequency of a topic as a multidimensional frequency representation, we can avoid ignoring the frequency of the topic when predicting its trend due to the difference between multidimensional semantic representation and single-dimensional frequency representation. In this way, we can more fully consider the semantic information and frequency of the topic, without missing the influence of frequency in the prediction due to the difference in dimensions.

[0070] Continue to refer to Figure 3 At point 330, the computing system 120, based on the semantic information 140 and frequency 150 of the topic, determines the number of documents associated with the topic within a second time period according to the prediction model 160. This second time period follows the first time period. For example, the second time period could be a future time period after the current time. For example, the second time period could be a future week, month, year, etc.

[0071] In some embodiments, the prediction model 160 may employ a pre-trained LSTM model, such as the referenced model. Figure 2 As described. In some embodiments, the prediction model 160 may also use other suitable neural network models.

[0072] In some embodiments, the computing system 120 may determine a number time series of a topic within a second time period. The number time series includes the number of documents associated with the topic at various time intervals within the second time period. For example, the computing system 120 may determine the number time series of a topic within the second time period based on a semantic representation time series and a frequency representation time series of the topic, according to a prediction model 160.

[0073] In some embodiments, the computing system 120 may determine the number of documents associated with each topic extracted from a set of documents 110 during a second time period. In some embodiments, the computing system 120 may analyze only some of these topics, such as topics of interest to certain users. That is, the computing system 120 may determine only the number of documents associated with one or more topics extracted from a set of documents 110 during the second time period.

[0074] This approach allows for the prediction of future trends in topics by considering not only the frequency of their past occurrences but also their semantic information. This leads to more accurate trend predictions. Particularly for related topics, analyzing their semantics allows for consideration of their interrelationships, resulting in more accurate predictions. For example, the topics "artificial intelligence" and "machine learning" have similar semantics. Using this method, we can predict similar trends for the topics "artificial intelligence" and "machine learning." Furthermore, by using this method to predict the number of documents associated with a topic in the future, we can analyze and predict future technological trends.

[0075] Figure 6 A schematic diagram is shown of an example output 600 of a topic trend prediction according to some embodiments of the present disclosure and an output 650 of a conventional scheme. Figure 6 Example output 600 represents a prediction of the number of topic-related documents at a given time using the scheme disclosed herein, based on information from July 24, July 25, and July 26. Example output 650 represents a prediction of the number of topic-related documents at a given time using a conventional scheme, based on information from July 24, July 25, and July 26.

[0076] Figure 6 Result 610 represents the prediction result of the subject "3D face reconstruction" using the scheme of this disclosure. Result 620 represents the prediction result of the subject "point cloud" using the scheme of this disclosure. Result 630 represents the prediction result of the subject "3D face reconstruction" using a conventional scheme. Result 640 represents the prediction result of the subject "point cloud" using a conventional scheme.

[0077] Point clouds are a common method for processing 3D images. In some documents on the topic of "3D face reconstruction," the term "point cloud" may not be mentioned in key sections such as the title, abstract, and keywords. However, these documents are likely actually related to "point clouds." Figure 6 As shown in results 660 and 670, using the conventional approach, only an increase in the number of documents associated with the topic "3D face reconstruction" can be predicted. However, prediction result 660 from the conventional approach indicates that the number of documents associated with the topic "point cloud" has not increased.

[0078] In comparison, the scheme disclosed herein takes into account the semantic relationship between the topic "3D face reconstruction" and the topic "point cloud". Therefore, results 620 and 610 both predict an increase in the number of documents associated with both the topics "3D face reconstruction" and "point cloud".

[0079] By comparing results 600 and 650, it can be seen that the topic prediction results obtained by using the scheme of this disclosure can take into account the semantic information of the topic, thereby better predicting the future trend of the topic.

[0080] It should be understood that Figure 6 The results shown are merely illustrative. Figure 6 The example only shows future trend forecasts for two topics. In some embodiments, forecasts can be made for fewer or more topics. Furthermore, more or better information from past documents can be used to forecast topic trends for shorter or longer future timeframes.

[0081] Through testing, the conventional approach to theme trend prediction exhibits a loss law of approximately 0.1% over 2000 rounds. In contrast, the proposed approach demonstrates a loss law of approximately 0.0625% over the same period. Therefore, the proposed approach outperforms the conventional approach in theme trend prediction and provides better forecasting.

[0082] Figure 7 A schematic diagram illustrating another output 700 of topic trend prediction according to some embodiments of the present disclosure is shown. Using the methods of the present disclosure, it is also possible to predict the level of attention given to various different topics at a given point in time. For example, as Figure 7 As shown, different grayscale blocks can represent different topics. The area of ​​each block can represent the number of published documents associated with that topic. For example, block 710 indicates that the topic corresponding to this block received the most attention at that point in time.

[0083] It should be understood that, according to the topic trend prediction method of the embodiments of this disclosure, other than... Figure 6 and Figure 7 In addition to other predictive information, this solution allows for the prediction and analysis of future technological developments.

[0084] Figure 8 A schematic block diagram of an example device 800 that can be used to implement embodiments of the present disclosure is shown. For example, such as Figure 1 The computing system 120 shown can be implemented by device 800. For example... Figure 6As shown, device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 802 or loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. CPU 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0085] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as a keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as a disk, optical disk, etc.; and communication unit 809, such as a network card, modem, wireless transceiver, etc. In some embodiments, input samples can be input to device 800 via input unit 806. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0086] The various processes and handling described above, such as method 300, can be executed by processing unit 801. For example, in some embodiments, method 300 can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU 801, one or more actions of method 300 described above can be performed.

[0087] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0088] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0089] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0090] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0091] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0092] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0093] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method, comprising: Extract the themes of a group of documents whose publication time falls within the first time period; Using at least a first machine learning model, the semantic information of each of the topics and the frequency of the topics appearing in the set of documents are determined, wherein the first machine learning model includes a transformer-based model; as well as Based on the semantic information and frequency of the topic, the number of documents associated with the topic in a second time period is determined according to a second machine learning model, the second machine learning model being different from the first machine learning model, the second machine learning model including a time series prediction model, wherein the second time period is after the first time period; The first machine learning model is configured to encode the semantic information of the topic and the frequency of the topic appearing in the set of documents; The result of encoding the semantic information of the topic and the frequency of the topic includes: for each topic in at least a subset of the topics, (i) a semantic encoding component, the semantic encoding component including at least a classification label for the semantic information of the topic; and (ii) a frequency encoding component, the frequency encoding component including a plurality of frequency-related values ​​generated based on a specified probability function for the frequency of occurrence of the topic; The second machine learning model is configured to process the semantic coding component and the frequency coding component to determine the number of documents associated with the topic during the second time period; Determining the frequency includes determining the frequency representation time series of the topic within the first time period; Determining the frequency representation time series includes: For each time interval within the first time period, based on the number of documents corresponding to the topic, the frequency representation of the time series at that time interval is determined; and The frequency representation at the time interval point includes: The frequency representation is determined by using position-extended encoding based on the number of documents corresponding to the topic.

2. The method according to claim 1, wherein determining the semantic information includes: Determine the semantic representation time series of the topic within the first time period.

3. The method according to claim 2, wherein determining the semantic representation time series comprises: For each time interval point within the first time period, based on words or phrases in the document group whose publication time is no later than the time interval point that correspond to the topic, the semantic representation of the semantic representation time series at the time interval point is determined according to the semantic encoding model.

4. The method of claim 1, wherein determining the frequency represents the frequency of the time series at the time interval point, the frequency representing the number of documents corresponding to the topic among the documents in the set of documents whose publication time is no later than the time interval point.

5. The method of claim 1, wherein the frequency represents having a predetermined dimension, the predetermined dimension being greater than one dimension.

6. The method of claim 1, wherein extracting the topic of each of the set of documents comprises: By using a topic classification model, a predetermined number of topics are extracted from each of the set of documents.

7. The method of claim 1, wherein determining the number of documents associated with the topic during the second time period comprises: Determine the number of time series of the topic within the second time period, wherein the number of time series includes the number of documents associated with the topic at each time interval within the second time period.

8. An electronic device, comprising: At least one processor; as well as At least one memory storing computer program instructions, wherein the computer program instructions, when executed by the at least one processor, cause the electronic device to perform an action, the action including: Extract the themes of a group of documents whose publication time falls within the first time period; Using at least a first machine learning model, the semantic information of each of the topics and the frequency of the topics appearing in the set of documents are determined; the first machine learning model includes a transformer-based model; and Based on the semantic information and frequency of the topic, the number of documents associated with the topic in a second time period is determined according to a second machine learning model, the second machine learning model being different from the first machine learning model, the second machine learning model including a time series prediction model, wherein the second time period is after the first time period; The first machine learning model is configured to encode the semantic information of the topic and the frequency of the topic appearing in the set of documents; The result of encoding the semantic information of the topic and the frequency of the topic includes: for each topic in at least a subset of the topics, (i) a semantic encoding component, the semantic encoding component including at least a classification label for the semantic information of the topic; and (ii) a frequency encoding component, the frequency encoding component including a plurality of frequency-related values ​​generated based on a specified probability function for the frequency of occurrence of the topic; The second machine learning model is configured to process the semantic coding component and the frequency coding component to determine the number of documents associated with the topic during the second time period; Determining the frequency includes determining the frequency representation time series of the topic within the first time period; Determining the frequency representation time series includes: For each time interval within the first time period, based on the number of documents corresponding to the topic, the frequency representation of the time series at that time interval is determined; and The frequency representation at the time interval point includes: The frequency representation is determined by using position-extended encoding based on the number of documents corresponding to the topic.

9. The electronic device of claim 8, wherein determining the semantic information includes: Determine the semantic representation time series of the topic within the first time period.

10. The electronic device of claim 9, wherein determining the semantic representation time series comprises: For each time interval point within the first time period, based on words or phrases in the document group whose publication time is no later than the time interval point that correspond to the topic, the semantic representation of the semantic representation time series at the time interval point is determined according to the semantic encoding model.

11. The electronic device of claim 8, wherein determining the frequency represents the frequency of the time series at the time interval point, and the frequency represents the number of documents corresponding to the topic among the documents in the set of documents whose publication time is no later than the time interval point.

12. The electronic device of claim 8, wherein the frequency represents a predetermined dimension, the predetermined dimension being greater than one dimension.

13. The electronic device of claim 8, wherein extracting the topic of each of the set of documents comprises: By using a topic classification model, a predetermined number of topics are extracted from each of the set of documents.

14. The electronic device of claim 8, wherein determining the number of documents associated with the topic during the second time period comprises: Determine the number of time series of the topic within the second time period, wherein the number of time series includes the number of documents associated with the topic at each time interval within the second time period.

15. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a device to perform the method according to any one of claims 1-7.

Images