Regional industry trend prediction method and system based on big data analysis

Abstract

The invention discloses a regional industry trend prediction method and system based on big data analysis, and relates to the technical field of data processing, and the method comprises the steps: S1, collecting the development data of each historical industry of a city, and carrying out the preprocessing of the historical data; s2, constructing a self-attention mechanism model containing a multi-layer Transform architecture, and training the model by using the preprocessed industry historical data to obtain a basic model for predicting the industry development trend; and S3, collecting specific data of the target industry, preprocessing the specific data of the target industry, extracting special trend data, and taking the trained basic model as a transfer learning model. The industry development is predicted by using the special prediction model, and through combination of deep learning and transfer learning, the development of the industry is predicted. High efficiency and precision of industry development prediction are realized, and the generalization ability and scene adaptability of the model are improved by collecting historical industry development data.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for regional industry trend prediction based on big data analysis. Background Art

[0002] With the rapid development of the economy and the continuous advancement of information technology, the development trends of regional industries are of great significance to government decision-making, corporate strategic planning, and investor decision-making. Traditional industry trend forecasting methods mainly rely on historical data and expert experience. Although these methods are effective to a certain extent, they are powerless in the face of complex practical application scenarios. The development trends of different industries are different. A single prediction for a certain industry cannot be connected to all situations. When the data is unbalanced or insufficient annotations are not available, the performance of traditional development trend forecasting models is usually not ideal. Summary of the Invention

[0003] In view of the above existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides a regional industry trend prediction method and system based on big data analysis to solve the problem that although traditional methods are effective to a certain extent, they are powerless in the face of complex practical application scenarios. The development trends of different industries are different. A single prediction for a certain industry cannot be connected to all situations. When the data is unbalanced or sufficient annotations cannot be obtained, the performance of traditional development trend prediction models is usually unsatisfactory.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a method for predicting regional industry trends based on big data analysis, comprising:

[0007] S1: Collect historical industry development data of the city and pre-process the historical data;

[0008] S2: Build a self-attention mechanism model with a multi-layer Transformer architecture and train it using preprocessed industry historical data to obtain a basic model for predicting industry development trends.

[0009] S3: Collect and pre-process specific data for the target industry to extract specialized trend data. Use the trained basic model as a transfer learning model to fine-tune the specific data for the target industry to obtain a prediction model that is more suitable for the target industry.

[0010] S4: Use the fine-tuned forecasting model to predict the development trend of the target industry and output the forecast results.

[0011] As a preferred solution of the regional industry trend forecasting method based on big data analysis described in the present invention, the historical industry development data includes historical development performance data of R industries, the historical development performance data includes an impact feature set and a development performance change, the impact feature set includes at least one impact feature, and the development performance change is one of industry scale, market share, profitability, and technological innovation capability.

[0012] As a preferred solution of the regional industrial trend prediction method based on big data analysis described in the present invention, wherein: the first layer in the self-attention mechanism model of the Transformer architecture includes an embedding layer, the second layer includes six stacked layers, and the third layer is the output layer, wherein each stacked layer includes two groups of sublayers, and the two groups of sublayers are multi-head self-attention sublayers and feedforward neural network sublayers, respectively. Residual connections and layer normalization are added to each sublayer, and the output layer is added to the last layer of the six stacked layers.

[0013] As a preferred solution of the regional industry trend forecasting method based on big data analysis of the present invention, wherein: the use of pre-processed industry historical data to train the model includes:

[0014] Divide historical industry development data into a historical training set and a historical test set, wherein the historical training set includes a common impact feature set and its corresponding development performance change;

[0015] Obtain a label annotation for the corresponding development performance change amount for each influencing feature set, wherein the annotation label includes "-1", "0" and "1", wherein "-1" indicates a decrease, "0" indicates no change, and "1" indicates an increase.

[0016] Initialize the self-attention mechanism model of the Transformer architecture, use the common influence feature set in the historical training set as the input of the self-attention mechanism model, and use the annotated labels in the general model training set as the output of the self-attention mechanism model, train the self-attention mechanism model, and obtain a trained self-attention mechanism model;

[0017] Use the historical test set to verify the trained self-attention mechanism model, and output the trained self-attention mechanism model that is greater than or equal to the preset test accuracy threshold as the general basic model;

[0018] The method for obtaining the common impact feature set is:

[0019] Obtain the influencing feature set of N industries and use the Pearson correlation coefficient to calculate the correlation between each feature in different industries;

[0020] Set a threshold, calculate the average value of the correlation between each feature and all other features, compare the calculated value with the established threshold, and take the set greater than the threshold as the common influence feature set.

[0021] As a preferred solution of the regional industry trend forecasting method based on big data analysis of the present invention, wherein: 5. the forecasting model that is more suitable for the target industry includes:

[0022] Obtain specific data and historical development performance data for the target industry;

[0023] Perform intersection calculation on the target industry's specific data and historical development performance data to obtain intersection data points, which are integers.

[0024] Perform difference calculation on the specific data of the target industry and the intersection data points to obtain difference data points, which are used as the change in the development performance of the target industry;

[0025] With the goal of minimizing the predefined loss function, the fine-tuned transfer learning model is trained using the changes in industry development performance to obtain a dedicated prediction model for predicting the target industry.

[0026] As a preferred solution of the regional industry trend forecasting method based on big data analysis described in the present invention, the fine-tuning of the transfer learning model includes:

[0027] Freeze the embedding layer and the first three stacking layers in the transfer learning model;

[0028] Remove the output layer in the transfer learning model and add a new output layer. The number of neurons in the new output layer is equal to the sum of the data points in the difference set.

[0029] In a second aspect, the present invention provides a regional industry trend forecasting system based on big data analysis, comprising:

[0030] Data collection module, which collects historical development data of corresponding urban industries and pre-processes the data;

[0031] Model training: Establish a self-attention mechanism model of a multi-layer Transformer architecture, input historical development data, train the multi-layer Transformer architecture, and obtain a general shape prediction model;

[0032] The model migration module migrates the trained prediction model and uses the specific data of the target industry to train the migrated model to obtain a dedicated prediction model;

[0033] The predictive analysis module uses a dedicated predictive model to predict industry development trends and screen out high-confidence results for output.

[0034] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the regional industry trend forecasting method based on big data analysis as described in the first aspect of the present invention.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the regional industry trend forecasting method based on big data analysis as described in the first aspect of the present invention.

[0036] The beneficial effects of the present invention are: historical industry development data is extracted from the preprocessed historical data, and the historical industry development data is trained using the Jixing only attention mechanism to obtain the most basic industry development prediction model. Then, the basic model is migrated, and the transfer learning model is fine-tuned. The output of the fine-tuned transfer learning model is connected to the new fully connected layer, the output of the new fully connected layer is activated, and with the goal of minimizing the predefined loss function, the fine-tuned transfer learning model is trained using the specific data of the target industry to obtain a dedicated prediction model. The dedicated prediction model is used to predict industry development. Through the combination of deep learning and transfer learning, the efficiency and accuracy of industry development prediction are achieved. By collecting historical industry development data, the generalization ability and scenario adaptability of the model are improved. The Transformer architecture of the self-attention mechanism enables the model to quickly process multi-dimensional features, and the transfer learning strategy reduces the dependence on large-scale labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a flow chart of the regional industry trend prediction method based on big data analysis in Example 1. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0041] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0042] Example 1, with reference to Figure 1 , which is the first embodiment of the present invention, provides a regional industry trend forecasting method based on big data analysis, comprising the following steps:

[0043] S1: Collect historical industry development data of the city and pre-process the historical data;

[0044] The historical industry development data includes historical development performance data of R industries, the historical development performance data includes an impact feature set and a change in development performance, the impact feature set includes at least one impact feature, and the change in development performance is one of industry scale, market share, profitability, and technological innovation capability;

[0045] It should be understood that industry size, market share, profitability, and technological innovation capabilities are specific indicators of an industry's development potential. Industry size refers to the total output or sales of all companies within an industry. This helps us understand the industry's overall development level and market potential. Profitability refers to a company's ability to generate profits, and is used to measure its operating efficiency and financial health. It is calculated by dividing operating profit by sales.

[0046] Among them, influencing characteristics include but are not limited to macroeconomic environment, policies and regulations, technological progress, market demand, and competitive situation.

[0047] S2: Build a self-attention mechanism model with a multi-layer Transformer architecture, train the model using preprocessed industry historical data, and obtain a basic model for predicting industry development trends.

[0048] Specifically, the first layer of the self-attention mechanism model of the Transformer architecture includes an embedding layer, the second layer includes six stacked layers, and the third layer is the output layer. Each stacked layer includes two groups of sublayers, namely multi-head self-attention sublayers and feedforward neural network sublayers. Each sublayer is followed by residual connections and layer normalization, and the output layer is added to the last layer of the six stacked layers.

[0049] It is clear that the Transformer architecture achieves efficient, stable, and accurate gas sensor aging detection by organically combining the embedding layer, six Transformer encoder modules, and the output layer. The model architecture takes into account deep feature learning, parallel processing, and generalization capabilities, ensuring that it can adapt to diverse sensor environments and providing a good foundation for transfer learning.

[0050] In an embodiment, the model is trained using pre-processed industry historical data, including:

[0051] The historical industry development data is divided into a historical training set and a historical test set. The historical training set includes a set of common influencing features and their corresponding development performance changes.

[0052] Obtain a label annotation for the corresponding development performance change amount for each influencing feature set, wherein the annotation labels include "-1", "0" and "1", where "-1" indicates a decrease, "0" indicates no change, and "1" indicates an increase.

[0053] Among them, the annotation labels can be manually labeled by AI data annotators or technical personnel, or they can be automatically labeled by a pre-set annotation program.

[0054] Initialize the self-attention mechanism model of the Transformer architecture, use the common influence feature set in the historical training set as the input of the self-attention mechanism model, and use the annotated labels in the general model training set as the output of the self-attention mechanism model, train the self-attention mechanism model, and obtain a trained self-attention mechanism model;

[0055] Use the historical test set to verify the trained self-attention mechanism model, and output the trained self-attention mechanism model that is greater than or equal to the preset test accuracy threshold as the general basic model;

[0056] The method for obtaining the common impact feature set is:

[0057] Obtain the influencing feature set of N industries and use the Pearson correlation coefficient to calculate the correlation between each feature in different industries;

[0058] Set a threshold, calculate the average value of the correlation between each feature and all other features, compare the calculated value with the established threshold, and take the set greater than the threshold as the common influence feature set.

[0059] The formula for the Pearson correlation coefficient is as follows: , where: is the Pearson correlation coefficient; is the value of the i-th influencing feature of the N-th industry; n is the number of data points; p is the value of the change in industry development performance, and are the means of X and Y respectively;

[0060] S3: Collect and pre-process specific data for the target industry to extract specialized trend data. Use the trained basic model as a transfer learning model to fine-tune the specific data for the target industry to obtain a prediction model that is more suitable for the target industry.

[0061] Specifically, 5. Obtain a forecasting model that is more suitable for the target industry, including:

[0062] Obtain specific data and historical development performance data for the target industry;

[0063] Perform intersection calculation on the target industry's specific data and historical development performance data to obtain intersection data points, which are integers.

[0064] Perform difference calculation on the specific data of the target industry and the intersection data points to obtain difference data points, which are used as the change in the development performance of the target industry;

[0065] In the embodiment, assuming that the impact feature set of the target industry is A={y1,y2,y3,y4,y5}, and its corresponding target industry development performance change is a={z1,z2, z3,z4,z5}, the historical development performance data has a common aging feature set B={y3,y4,y5,y6,y7} and its corresponding historical development performance change is b={z3,z4,z5,z6,z7}, then the intersection data point is {y3,y4,y5}, after calculating the difference set, that is, A−(A∩B)={y1,y2}, so the difference set data points are y1 and its corresponding z1 and y2 and its corresponding z2 as the difference set data points.

[0066] With the goal of minimizing the predefined loss function, the fine-tuned transfer learning model is trained using the changes in industry development performance to obtain a dedicated prediction model for predicting the target industry.

[0067] Specifically, fine-tune the transfer learning model, including:

[0068] Freeze the embedding layer and the first three stacking layers in the transfer learning model;

[0069] Remove the output layer in the transfer learning model and add a new output layer. The number of neurons in the new output layer is equal to the sum of the data points in the difference set.

[0070] S4: Use the fine-tuned forecasting model to predict the development trend of the target industry and output the forecast results.

[0071] This embodiment also provides a regional industry trend forecasting system based on big data analysis, including:

[0072] Data collection module, which collects historical development data of corresponding urban industries and pre-processes the data;

[0073] Model training: Establish a self-attention mechanism model of a multi-layer Transformer architecture, input historical development data, train the multi-layer Transformer architecture, and obtain a general shape prediction model;

[0074] The model migration module migrates the trained prediction model and uses the specific data of the target industry to train the migrated model to obtain a dedicated prediction model;

[0075] The predictive analysis module uses a dedicated predictive model to predict industry development trends and screen out high-confidence results for output.

[0076] This embodiment also provides a computer device, which is suitable for the regional industry trend prediction method based on big data analysis, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the regional industry trend prediction method based on big data analysis proposed in the above embodiment.

[0077] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0078] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the regional industry trend prediction method based on big data analysis proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0079] In summary, the present invention further strengthens the coupling association between features and states through modeling of conditional probabilities, thereby making up for the limitation that traditional Markov chains cannot handle complex contextual dependencies in multimodal systems. The long-term state is generated by weighted smoothing of local states, and the state smoothing factor controls the adjustment strength of the long-term state on the local state, that is, whether it relies more on local data or global trends. It not only enhances the long-term dependency modeling capability in the state sequence, but also solves the problem of local state fluctuations caused by time series noise. By embedding feature vectors from other modalities as context information and combining them with long-term states, a Bayesian network is constructed to model short-term dependencies, thereby achieving a comprehensive description of the observed data. The Bayesian network combines long-term and short-term state probabilities with contextual information, so that short-term predictions can simultaneously capture the dynamic changes of previous observation data and global states. By splicing the enhanced features of the modality with the long-term states of the state sequence, the global representation capability of the features is enhanced.

[0080] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A regional industry trend forecasting method based on big data analysis, characterized in that: include: S1: Collect historical industry development data of the city and pre-process the historical data; S2: Build a self-attention mechanism model with a multi-layer Transformer architecture and train it using preprocessed industry historical data to obtain a basic model for predicting industry development trends. S3: Collect and pre-process specific data for the target industry to extract specialized trend data. Use the trained basic model as a transfer learning model to fine-tune the specific data for the target industry to obtain a prediction model that is more suitable for the target industry. S4: Use the fine-tuned forecasting model to predict the development trend of the target industry and output the forecast results.

2. The regional industry trend forecasting method based on big data analysis according to claim 1, characterized in that: The historical industry development data includes historical development performance data of R industries, and the historical development performance data includes an impact feature set and a development performance change amount. The impact feature set includes at least one impact feature, and the development performance change amount is one of industry scale, market share, profitability, and technological innovation capability.

3. The regional industry trend forecasting method based on big data analysis according to claim 1, characterized in that: In the self-attention mechanism model of the Transformer architecture, the first layer includes an embedding layer, the second layer includes six stacked layers, and the third layer is the output layer. Each stacked layer includes two groups of sublayers, which are multi-head self-attention sublayers and feedforward neural network sublayers. Each sublayer is followed by residual connections and layer normalization. The output layer is added to the last layer of the six stacked layers.

4. The regional industry trend forecasting method based on big data analysis according to claim 3, characterized in that: The model is trained using the pre-processed industry historical data. include, Divide historical industry development data into a historical training set and a historical test set, wherein the historical training set includes a common impact feature set and its corresponding development performance change; Obtain a label for each influencing feature set corresponding to the development performance change, wherein the label includes "-1", "0" and "1", where "-1" indicates a decrease, "0" indicates no change, and "1" indicates an increase. Initialize the self-attention mechanism model of the Transformer architecture, use the common influence feature set in the historical training set as the input of the self-attention mechanism model, and use the annotated labels in the general model training set as the output of the self-attention mechanism model, train the self-attention mechanism model, and obtain a trained self-attention mechanism model; Use the historical test set to verify the trained self-attention mechanism model, and output the trained self-attention mechanism model that is greater than or equal to the preset test accuracy threshold as the general basic model; The method for obtaining the common impact feature set is: Obtain the influencing feature set of N industries and use the Pearson correlation coefficient to calculate the correlation between each feature in different industries; Set a threshold, calculate the average value of the correlation between each feature and all other features, compare the calculated value with the established threshold, and take the set greater than the threshold as the common influence feature set.

5. The regional industry trend forecasting method based on big data analysis according to claim 4, characterized in that: The prediction model that is more suitable for the target industry includes: Obtain specific data and historical development performance data for the target industry; Perform intersection calculation on the target industry's specific data and historical development performance data to obtain intersection data points, which are integers. Perform difference calculation on the specific data of the target industry and the intersection data points to obtain difference data points, which are used as the change in the development performance of the target industry; With the goal of minimizing the predefined loss function, the fine-tuned transfer learning model is trained using the changes in industry development performance to obtain a dedicated prediction model for predicting the target industry.

6. The regional industry trend forecasting method based on big data analysis according to claim 5, characterized in that: Fine-tuning the transfer learning model includes: Freeze the embedding layer and the first three stacking layers in the transfer learning model; Remove the output layer in the transfer learning model and add a new output layer. The number of neurons in the new output layer is equal to the sum of the data points in the difference set.

7. A regional industry trend forecasting system based on big data analysis, based on the regional industry trend forecasting method based on big data analysis according to any one of claims 1 to 6, characterized in that: include, Data collection module, which collects historical development data of corresponding urban industries and pre-processes the data; Model training: Establish a self-attention mechanism model of a multi-layer Transformer architecture, input historical development data, train the multi-layer Transformer architecture, and obtain a general shape prediction model; The model migration module migrates the trained prediction model and uses the specific data of the target industry to train the migrated model to obtain a dedicated prediction model; The predictive analysis module uses a dedicated predictive model to predict industry development trends and screen out high-confidence results for output.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the regional industry trend prediction method based on big data analysis according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the regional industry trend prediction method based on big data analysis according to any one of claims 1 to 6 are implemented.

Images