Method and system for predicting housing price index based on real estate online news emotion

Abstract

The invention discloses a method and a system for predicting a housing price index based on real estate online news emotion. The method comprises the following steps: collecting news data at a current time point and a next time point and the housing price index of a target city; inputting the news data of the next time point into the fine-tuned BERT model, predicting to obtain a news emotion value of the next time point, and calculating to obtain a news emotion index of the next time point; performing sentiment analysis on the real estate online news data about the target city at the current time point to obtain a news sentiment index about the target city at the current time point; according to a preset time sequence prediction model, obtaining a difference value between the news emotion index and the housing price index of the target city at the next time point; and calculating to obtain a housing price index about the target city at the next time point. According to the method, a new way is provided for accurate prediction of the housing price index, and a more reliable decision basis is provided for investors and decision makers.

Description

Technical Field

[0001] The present invention relates to the technical field of real estate data mining, and in particular to a method and system for predicting a housing price index based on real estate online news sentiment. Background Art

[0002] The real estate market has always been a focus of attention and is one of the focal points in the economic field. The development of the real estate industry not only reflects the state of the national economy, but also attracts the attention of housing traders. Generally, the state of the real estate market is understood through the real estate price index released by the National Bureau of Statistics. Investors, government decision-makers and the general public are eager to understand the future market trend, but the real estate price index released by the state has a long lag period. For example, the monthly real estate price index will not be released until around the 15th of the next month, which cannot express the development dynamics of the real estate market in a timely manner, which is not conducive to real estate practitioners to judge the market situation in real time. Traditional house price prediction methods mainly rely on macro factors, including economic indicators and policy changes. However, these factors are often difficult to fully and accurately reflect the actual market situation. Therefore, with the continuous development of science and technology, emerging data analysis technology and artificial intelligence tools have gradually become an important part of real estate market research, providing more accurate and comprehensive information for prediction models, thereby better guiding investment decisions and policy making. This comprehensive analysis that combines traditional methods and advanced technologies is expected to provide more accurate and reliable house price forecasts for all parties and promote more scientific and flexible market operations.

[0003] With the continuous advancement of the Internet and big data technology, the widespread use of sensors, social media and other data sources has provided a richer information basis for housing price forecasting. Past research has mainly relied on statistical analysis of macroeconomic factors, which is not accurate and cannot complete housing price index forecasts in a timely manner. Summary of the invention

[0004] In order to at least partially solve the problem of inaccurate and untimely prediction of housing price index based on macroeconomic analysis, the present invention provides a method and system for predicting housing price index based on real estate online news sentiment, wherein the forecast news sentiment value for next month is obtained by a fine-tuned BERT model, the forecast difference between the news sentiment index at the next time point and the housing price index at the next time point is obtained by a time series prediction model, and then the housing price index at the next time point is obtained according to the forecast news sentiment value at the next time point and the forecast difference between the news sentiment index at the next time point and the housing price index at the next time point, and accurate and timely prediction of housing price index is achieved by comprehensive use of Internet data, natural language processing, sentiment analysis, time series prediction and other technical means.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] The first aspect of the present invention proposes a method for predicting a housing price index based on real estate online news sentiment, comprising:

[0007] Step 1: Collect online real estate news data about the target city at the current time point and the next time point, as well as the housing price index of the target city at the current time point, to facilitate the prediction of the housing price index;

[0008] Step 2: Input the real estate online news data about the target city at the next time point into the fine-tuned BERT model to predict the news sentiment value about the target city at the next time point. According to the news sentiment value about the target city at the next time point, the news sentiment index about the target city at the next time point is obtained, which is convenient for the subsequent prediction of the housing price index.

[0009] Step 3: Perform sentiment analysis on the online real estate news data about the target city at the current time point to obtain the news sentiment index about the target city at the current time point;

[0010] Step 4: Calculate the difference between the news sentiment index and the housing price index of the target city at the current time point, input the difference into a preset time series prediction model, and predict the difference between the news sentiment index and the housing price index of the target city at the next time point, so as to predict the housing price index based on the difference;

[0011] Step 5: Calculate the housing price index of the target city at the next time point based on the difference between the news sentiment index and the housing price index of the target city at the next time point and the news sentiment index of the target city at the next time point.

[0012] Furthermore, the fine-tuned BERT model includes a BERT model and a fully connected layer, and the fully connected layer is arranged behind the BERT model.

[0013] Furthermore, the method further includes: obtaining the news sentiment index of the target city at time point t according to the news sentiment value of the target city at time point t according to the following formula:

[0014]

[0015] in, is the average news sentiment value at time point t, N t is the number of news data recorded at time point t, A is the news sentiment value at the current time point t, A(t,i) is the news sentiment value recorded for the ith time point t, S is the news sentiment index at the current time point t, and A min is the minimum news sentiment value at time point t, A max is the maximum news sentiment value at time point t.

[0016] Furthermore, the time series prediction model includes a ReLU layer, a maximum pooling layer, a random dropout layer, a first long short-term memory network layer, a second long short-term memory network layer and a fully connected layer;

[0017] The ReLU layer, the maximum pooling layer, the random inactivation layer, the first long short-term memory network layer, the second long short-term memory network layer and the fully connected layer are connected in sequence.

[0018] Furthermore, the calculation of the difference between the news sentiment index and the housing price index of the target city at the current time point specifically includes:

[0019] D t =S t -P t

[0020] Among them, D t is the difference between the news sentiment index at time point t and the house price index at time point t, S t is the news sentiment index at time point t, P t is the house price index at time point t.

[0021] The second aspect of the present invention proposes a system for predicting housing price index based on real estate online news sentiment, comprising:

[0022] The data collection module is used to collect online real estate news data about the target city at the current time point and the next time point, as well as the housing price index of the target city at the current time point, so as to facilitate the prediction of the housing price index;

[0023] The news sentiment index module is used to input the real estate online news data about the target city at the next time point into the fine-tuned BERT model, predict the news sentiment value about the target city at the next time point, and obtain the news sentiment index about the target city at the next time point based on the news sentiment value about the target city at the next time point, so as to facilitate the subsequent prediction of the housing price index;

[0024] A processing module, used to perform sentiment analysis on online real estate news data about the target city at the current time point, and obtain a news sentiment index about the target city at the current time point;

[0025] A difference module is used to calculate the difference between the news sentiment index and the housing price index of the target city at the current time point, input the difference into a preset time series prediction model, and predict the difference between the news sentiment index and the housing price index of the target city at the next time point, so as to predict the housing price index according to the difference;

[0026] The prediction module is used to calculate the housing price index of the target city at the next time point based on the difference between the news sentiment index and the housing price index of the target city at the next time point and the news sentiment index of the target city at the next time point.

[0027] The third aspect of the present invention proposes an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for predicting a house price index based on real estate online news sentiment as described in the first aspect above is implemented.

[0028] The fourth aspect of the present invention proposes a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a method for predicting a housing price index based on real estate online news sentiment as described in the first aspect above.

[0029] Beneficial effects of the present invention:

[0030] (1) The present invention constructs a news sentiment index. By solving the difference between the news sentiment index and the housing price index, the housing price index at the next time point can be predicted in a timely manner based on the news sentiment index and the difference between the news sentiment index and the housing price index.

[0031] (2) The present invention constructs a fine-tuned BERT model, adding a fully connected layer after the standard BERT model for predicting sentiment values, which facilitates the subsequent solution of the news sentiment index. By constructing a time series prediction model, the prediction difference between the news sentiment index at the next time point and the housing price index at the next time point is calculated, thereby improving the accuracy of the housing price index prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of a method for predicting a housing price index based on real estate online news sentiment provided by an embodiment of the present invention.

[0033] Figure 2 The present invention provides a second flowchart of a method for predicting housing price index based on real estate online news sentiment.

[0034] Figure 3 An architectural diagram of a system for predicting housing price indices based on real estate online news sentiment provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Example 1

[0037] like Figure 1 As shown, a method for predicting the house price index based on the sentiment of real estate online news includes:

[0038] S101: Collecting real estate online news data about the target city at the current time point and the next time point, and the housing price index of the target city at the current time point.

[0039] S102: Input the real estate online news data about the target city at the next time point into the fine-tuned BERT model, predict the news sentiment value about the target city at the next time point, and obtain the news sentiment index about the target city at the next time point based on the news sentiment value about the target city at the next time point.

[0040] Specifically, the present invention needs to use the BERT model to predict the news sentiment value, so it is necessary to construct an appropriate training data set to fine-tune the original BERT model so that it has the function of predicting the news sentiment value. Specifically, the process of constructing the training data set is as follows: crawling online real estate news data within a certain period of time from the search engine website using crawler technology with a specific keyword combination (city name + time + house price, city name + time + real estate price), where the data collected for each record includes information such as release time, news title, and news summary.

[0041] Then the collected data is cleaned and deduplicated. Specifically, data containing repeated information, information unrelated to housing prices, and non-Chinese symbols are deleted. In order to increase attention, news headlines often have exaggerated emotional expressions, which are inconsistent with the emotions of the news content. Therefore, the news summary and news content are separated, and manual emotional annotation is performed to construct an online real estate news dataset for a specific city as a training dataset. Among them, the annotation criteria are based on the rise and fall of housing prices reflected in the text description, which is labeled as 1, -1, 0 (i.e., news emotional value label), where 1 is optimistic, -1 is pessimistic, and 0 is neutral.

[0042] S103: Perform sentiment analysis on the real estate online news data about the target city at the current time point to obtain the news sentiment index about the target city at the current time point.

[0043] S104: Calculate the difference between the news sentiment index and the housing price index of the target city at the current time point, input the difference into a preset time series prediction model, and predict the difference between the news sentiment index and the housing price index of the target city at the next time point.

[0044] S105: Calculate the housing price index of the target city at the next time point according to the difference between the news sentiment index and the housing price index of the target city at the next time point and the news sentiment index of the target city at the next time point.

[0045] like Figure 2 As shown, the present invention can quickly obtain a large amount of news information related to housing prices by using crawler technology, and then clean the data and annotate the sentiment to build a large and timely data set. The fine-tuned BERT model is constructed using natural language sentiment analysis technology. In addition, the time series prediction model is introduced to predict the difference between the news sentiment index and the housing price index. In this way, the housing price news data of the current month can be used to quickly predict the sentiment value and obtain the news sentiment index using the constructed fine-tuned BERT model. Finally, the predicted difference and the news sentiment index can be used to obtain the housing price index at the next time point.

[0046] Example 2

[0047] Based on the above embodiment, the embodiment of the present invention provides a method for constructing a news sentiment index, which specifically includes:

[0048] First, a fine-tuned BERT model is constructed, and the real estate news data is input into the fine-tuned BERT model to obtain the news sentiment value. Then, the news sentiment index is calculated based on the news sentiment value.

[0049] Specifically, the bidirectional transformer (BERT) model is customized to adapt to the sentiment analysis task in the real estate field by fine-tuning. The present invention uses an open source pre-trained BERT model as a basic model. First, the input format of the news data input into the BERT model is adjusted to convert the input news text into a form that the model can process, including word segmentation and adding special tags such as CLS and SEP. Then, a classification task layer, i.e., a fully connected layer, is added to the BERT model for sentiment classification. Cross entropy is selected as the loss function to measure the difference between the model output and the actual sentiment label. Subsequently, the model parameters are adjusted to minimize the loss function by optimization algorithms such as back propagation and gradient descent. In the training stage, the real estate news data of the historical t months is divided into a training set and a validation set according to 8:2. The fine-tuned model is evaluated by using an independent validation set, and hyperparameters such as learning rate and number of fine-tuning layers are adjusted according to the performance to improve the generalization ability of the model, so that the BERT model can perform well on specific sentiment analysis tasks and improve the performance and adaptability of the model.

[0050] The news sentiment values ​​predicted by the fine-tuned BERT model within a certain period are calculated to get the average value, and the average value is normalized to the maximum and minimum values, and the data is mapped to the range of 0 to 1. The obtained data is the news sentiment index, which is specifically expressed by the following formula:

[0051]

[0052] in, is the average news sentiment value at time point t, N t is the number of news data recorded at time point t, A is the news sentiment value at the current time point t, A(t,i) is the news sentiment value recorded for the ith time point t, S is the news sentiment index at the current time point t, and A min is the minimum news sentiment value at time point t, A max is the maximum news sentiment value at time point t.

[0053] Specifically, the time point t can be customized according to user needs, for example, it can be a month. According to the historical real estate news data of t months used when building the fine-tuned BERT model, the news sentiment index of t months can be obtained.

[0054] Example 3

[0055] Based on the above embodiments, an embodiment of the present invention provides a method for constructing a time series prediction model, which specifically includes:

[0056] A time series prediction model was built based on the convolutional neural network-long short-term memory (CNN_LSTM) architecture. The time series prediction model includes a ReLU layer, a maximum pooling layer, a random dropout layer, a first long short-term memory network layer, a second long short-term memory network layer, and a fully connected layer. ReLU is used as the activation function to introduce nonlinear relationships. The feature data is then downsampled through the maximum pooling layer to reduce the dimension and retain key information. To prevent overfitting, a random dropout layer is added to randomly discard some neurons. Next, the first long short-term memory network layer and the second long short-term memory network layer are connected in sequence to capture the timing information and long-term dependencies in the sequence data by modeling the time steps. Finally, a fully connected layer is used to output and predict the sentiment value of the next month.

[0057] When training and tuning the time series prediction model, we first obtain the original monthly housing price index data from the National Bureau of Statistics. In order to ensure that it has a similar scale to the news sentiment index, it is normalized. The converted data value is the housing price index of each month, and the housing price index of each month is used to construct the housing price index sequence P. The news sentiment index of each month is constructed as the news sentiment index sequence S′, and the difference sequence D between the news sentiment index sequence S′ and the housing price index sequence P is calculated, and then the difference between the news sentiment index of each month and the housing price index of each month is obtained. The specific process is expressed by the following formula:

[0058] D=S′-P

[0059] Among them, D is the difference sequence between the news sentiment index sequence and the housing price index sequence P, S′ is the news sentiment index sequence, and P is the housing price index sequence.

[0060] The data in the difference sequence D between the news sentiment index sequence and the housing price index sequence P are divided into training set, test set and validation set according to the ratio of 60%, 30% and 10%. The k-Fold 5-fold cross-validation method is used in the training process, and the mean absolute percentage error (MAPE) is selected as the training target parameter. The trained model is iteratively optimized and tested until the MAPE is minimized. In order to optimize the training process of the model, the Adam optimizer is used, which combines adaptive learning rate adjustment and momentum method to optimize model parameters quickly and stably. The model is then verified using the validation set, and the model effect is evaluated by the verification results. When the model accuracy is not ideal, that is, MAPE ≥ 20%, the model is hyper-tuned again until a complete time series prediction model is built.

[0061] Example 4

[0062] Based on the above embodiment, the embodiment of the present invention provides a method for calculating the housing price index, which specifically includes:

[0063] The real estate online news data about the target city in month t+1 is input into the fine-tuned BERT model to obtain the news sentiment value about the target city in month t+1, and the news sentiment index about the target city in month t+1 is obtained by calculation.

[0064] Calculate the difference between the news sentiment index and the housing price index of the target city in month t. The specific process is expressed by the following formula:

[0065] D t =S t -P t

[0066] Among them, D t is the difference between the news sentiment index at time point t and the house price index at time point t, S t is the news sentiment index at time point t, P t is the house price index at time point t. Specifically, time point t is month t.

[0067] The difference between the news sentiment index and the housing price index of the target city in month t is input into the preset time series prediction model, and the difference between the news sentiment index and the housing price index of the target city in month t+1 is predicted. Then, the housing price index of month t+1 is calculated. The specific process is expressed by the following formula:

[0068] P t+1 =S t+1 -D t+1

[0069] Among them, D t+1 is the difference between the news sentiment index at time point t+1 and the house price index at time point t+1, S t+1 is the news sentiment index at time point t+1, P t+1 is the house price index at time point t+1. Specifically, time point t+1 is month t+1.

[0070] Example 5

[0071] Corresponding to the above method, such as Figure 3 As shown, a system for predicting housing price index based on real estate online news sentiment includes:

[0072] A data collection module is used to collect real estate online news data about the target city at the current time point and the next time point, as well as the housing price index of the target city at the current time point;

[0073] The sentiment index module is used to input the real estate online news data about the target city at the next time point into the fine-tuned BERT model, predict the news sentiment value about the target city at the next time point, and obtain the news sentiment index about the target city at the next time point according to the news sentiment value about the target city at the next time point;

[0074] A processing module, used to perform sentiment analysis on online real estate news data about the target city at the current time point, and obtain a news sentiment index about the target city at the current time point;

[0075] A difference module is used to calculate the difference between the news sentiment index and the housing price index of the target city at the current time point, input the difference into a preset time series prediction model, and predict the difference between the news sentiment index and the housing price index of the target city at the next time point;

[0076] The prediction module is used to calculate the housing price index of the target city at the next time point based on the difference between the news sentiment index and the housing price index of the target city at the next time point and the news sentiment index of the target city at the next time point.

[0077] It should be noted that the system for predicting the housing price index based on the sentiment of real estate online news provided in the embodiment of the present invention is to realize the method for predicting the housing price index based on the sentiment of real estate online news in the above embodiment. Its specific functions can be referred to the above method embodiments and will not be repeated here.

[0078] Example 6

[0079] Based on the above embodiments, an embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, a method for predicting a house price index based on real estate online news sentiment as described in the above embodiments is implemented.

[0080] Preferably, an embodiment of the present invention further provides a computer-readable storage medium, the storage medium comprising a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a method for predicting a housing price index based on real estate online news sentiment as described in the above embodiment.

[0081] In summary, the present invention constructs a news sentiment index. By solving the difference between the news sentiment index and the housing price index, the housing price index at the next time point can be predicted in time according to the news sentiment index and the difference between the news sentiment index and the housing price index. The present invention constructs a fine-tuned BERT model, and adds a fully connected layer after the standard BERT model for predicting sentiment values, which facilitates the subsequent solution of the news sentiment index. By constructing a time series prediction model, the calculation of the predicted difference between the news sentiment index at the next time point and the housing price index at the next time point is realized, thereby improving the accuracy of the housing price index prediction.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting housing price index based on real estate online news sentiment, characterized in that: include: Step 1: Collect online real estate news data about the target city at the current time point and the next time point, as well as the housing price index of the target city at the current time point; Step 2: Input the real estate online news data about the target city at the next time point into the fine-tuned BERT model, predict the news sentiment value about the target city at the next time point, and obtain the news sentiment index about the target city at the next time point based on the news sentiment value about the target city at the next time point; Step 3: Perform sentiment analysis on the online real estate news data about the target city at the current time point to obtain the news sentiment index about the target city at the current time point; Step 4: Calculate the difference between the news sentiment index and the housing price index of the target city at the current time point, input the difference into a preset time series prediction model, and predict the difference between the news sentiment index and the housing price index of the target city at the next time point; Step 5: Calculate the housing price index of the target city at the next time point based on the difference between the news sentiment index and the housing price index of the target city at the next time point and the news sentiment index of the target city at the next time point.

2. The method for predicting house price index based on real estate online news sentiment according to claim 1, characterized in that: The fine-tuned BERT model includes a BERT model and a fully connected layer, and the fully connected layer is set after the BERT model.

3. The method for predicting house price index based on real estate online news sentiment according to claim 1, characterized in that: Also includes: According to the news sentiment value about the target city at time point t, the news sentiment index about the target city at time point t is obtained according to the following formula: in, is the average news sentiment value at time point t, N t is the number of news data recorded at time point t, A is the news sentiment value at the current time point t, A(t,i) is the news sentiment value recorded for the ith time point t, S is the news sentiment index at the current time point t, and A min is the minimum news sentiment value at time point t, A max is the maximum news sentiment value at time point t.

4. The method for predicting house price index based on real estate online news sentiment according to claim 1, characterized in that: The time series prediction model includes a ReLU layer, a maximum pooling layer, a random dropout layer, a first long short-term memory network layer, a second long short-term memory network layer and a fully connected layer; The ReLU layer, the maximum pooling layer, the random inactivation layer, the first long short-term memory network layer, the second long short-term memory network layer and the fully connected layer are connected in sequence.

5. The method for predicting house price index based on real estate online news sentiment according to claim 1, characterized in that: The calculating of the difference between the news sentiment index and the housing price index of the target city at the current time point specifically includes: D t =S t -P t Among them, D t is the difference between the news sentiment index at time point t and the house price index at time point t, S t is the news sentiment index at time point t, P t is the house price index at time point t.

6. A system for predicting housing price index based on real estate online news sentiment, characterized in that: include: A data collection module is used to collect real estate online news data about the target city at the current time point and the next time point, as well as the housing price index of the target city at the current time point; The news sentiment index module is used to input the real estate online news data about the target city at the next time point into the fine-tuned BERT model, predict the news sentiment value about the target city at the next time point, and obtain the news sentiment index about the target city at the next time point based on the news sentiment value about the target city at the next time point; A processing module, used to perform sentiment analysis on online real estate news data about the target city at the current time point, and obtain a news sentiment index about the target city at the current time point; A difference module is used to calculate the difference between the news sentiment index and the housing price index of the target city at the current time point, input the difference into a preset time series prediction model, and predict the difference between the news sentiment index and the housing price index of the target city at the next time point; The prediction module is used to calculate the housing price index of the target city at the next time point based on the difference between the news sentiment index and the housing price index of the target city at the next time point and the news sentiment index of the target city at the next time point.

7. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for predicting a house price index based on real estate online news sentiment as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a method for predicting a housing price index based on real estate online news sentiment as described in any one of claims 1 to 5.

Images