Company correlation modeling method based on artificial intelligence algorithm

Abstract

The present invention discloses a company relevance modeling method based on an artificial intelligence algorithm, which belongs to the field of network information processing technology and includes the following steps: step one, data selection; step two, data labeling; step three, sample data visualization analysis; step four, feature derivation, feature derivation of the above-obtained data, and construction of eight new feature variables based on the three original variables of title, text, and target company; step five, visualization analysis of derived features; step six, preprocessing of derived features, that is, converting derived features into numerical features; step seven, building a model; using cross-validation to divide the above numerical features, randomly breaking up the sample set and dividing it into a training set and a test set, repeating ten times, and then training with decision tree regression of machine learning, and finally obtaining a model. The present invention derives eight feature variables, preprocesses them, and establishes an efficient model that can quickly screen out highly relevant public opinions.

Description

Technical Field

[0001] The present invention relates to the technical field of network information processing, and in particular to a company relevance modeling method based on an artificial intelligence algorithm. Background Art

[0002] With the rapid development of Internet technology, the Internet has become one of the main carriers for reflecting public opinion. More and more users express their views, opinions, attitudes, opinions and emotions through the Internet. In order to facilitate companies to understand the public's thoughts in a timely and accurate manner and grasp the public's needs, public opinion push services have come into being.

[0003] The existing public opinion push service is based on the business logic of first matching keywords, then cleaning the rules and AI models, and finally pushing the public opinion.

[0004] However, the existing technology has the following problems: some public opinion information has a low relevance to the company and cannot meet the expectation of screening highly relevant information. Therefore, those skilled in the art provide a company relevance modeling method based on artificial intelligence algorithm to solve the problems raised in the above background technology. Summary of the invention

[0005] The purpose of the present invention is to provide a company relevance modeling method based on an artificial intelligence algorithm, by deriving eight characteristic variables and preprocessing them to establish an efficient model that can quickly screen out highly relevant public opinions, so as to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The company relevance modeling method based on artificial intelligence algorithm includes the following steps:

[0008] Step 1: Data selection;

[0009] Collect and select multiple sets of basic dependent data, that is, two features of public opinion information: title and text. These feature data belong to non-standard text type data, and then construct a third feature variable: target company. Obtain the list of currently listed companies through statistical sorting, extract the target company keywords matching the listed companies in the title and text, and complete the basic gross data extraction;

[0010] Step 2: Data labeling;

[0011] Label the sample data, judge the relevance of news and news subjects based on expert experience, and label them into values ​​between 0 and 1, and merge them into the original training data;

[0012] Step 3: Visual analysis of sample data;

[0013] Perform statistics and visualization on the correlation of the above sample data;

[0014] Step 4: Feature derivation;

[0015] The above data is used to derive features, and based on the three original variables of title, text, and target company, the following eight new feature variables are constructed:

[0016] 1) Number of target company title text matches;

[0017] 2) Number of hits in the target company’s main text;

[0018] 3) Title text string length;

[0019] 4) The length of the text string;

[0020] 5) Number of text matching hits of other target listed companies;

[0021] 6) The total number of other target listed companies after deduplication;

[0022] 7) The industry type to which the target company belongs;

[0023] 8) The percentage of target company matching hits;

[0024] Step 5: Visual analysis of derived features;

[0025] Step 6: Derived feature preprocessing, that is, converting derived features into numerical features;

[0026] Step 7: Build the model;

[0027] Use cross-validation to divide the above numerical features, randomly break up the sample set and divide it into training set and test set, repeat ten times, and then use machine learning decision tree regression for training to finally get the model.

[0028] The present invention derives eight characteristic variables and establishes an efficient model that can quickly screen out highly relevant public opinions after preprocessing them.

[0029] As a further solution of the present invention: in the step 2, the sample data with missing values ​​in the company names are deleted.

[0030] This setting is used to remove invalid sample data to ensure the accuracy of training data.

[0031] As a further solution of the present invention: in the step 2, the correlation is distributed between 0 and 1, and 0 is defined as no correlation, 0.5 as medium correlation, and 1 as high correlation.

[0032] This setting divides the relevance, making it easier for later staff to quickly identify the relevance of public opinion information to the company.

[0033] As a further solution of the present invention: in the step six, the industry to which the target company belongs in the feature variable is a text feature, which is converted into a numerical feature through LabelEncoder.

[0034] This setting enables the industry to which the target company belongs to be smoothly converted into numerical features, which is convenient for later training data.

[0035] As a further solution of the present invention: in step seven, the ratio of the training set to the test set is 8:2.

[0036] This ratio can effectively ensure the accuracy of training.

[0037] As a further solution of the present invention: in step 7, the specific process of training using decision tree regression of machine learning is as follows:

[0038] S1: build decision tree regression;

[0039] S2: build tree depth interval, defined from 1 to 50;

[0040] S3: Use r2 as the evaluation metric;

[0041] S4: Build a grid search and train the model;

[0042] S5: Get the model and tree depth parameters.

[0043] As a further solution of the present invention: after the model constructed in step seven is deployed online on the cloud server, a preset relevance value is set, and the public opinion information imported into the model is screened, and the public opinion information with a relevance value higher than the preset relevance value is screened out for push.

[0044] Staff can set relevant preset values ​​according to the company's needs, so as to better push relevant public opinion information to the company.

[0045] As a further solution of the present invention: the preset value of the correlation is 0.7.

[0046] As a further solution of the present invention: the model is deployed online in the form of an API interface, the request method is a post request, and the parameters are the title, body and target company name.

[0047] A company relevance modeling device based on an artificial intelligence algorithm includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. Push notifications only when highly relevant public opinions (such as those greater than 0.7) are captured, thus improving the public opinion push experience.

[0050] 2. Intelligent push: customers can set the relevance threshold independently and control the balance between comprehensiveness and accuracy.

[0051] 3. Real-time monitoring and data statistics of highly relevant public opinions, focusing on keeping up with current hot issues. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of the method for modeling company relevance based on artificial intelligence algorithms;

[0053] Figure 2 This is a diagram showing the fitting effect of the company correlation modeling method based on artificial intelligence algorithm. DETAILED DESCRIPTION

[0054] See also Figures 1-2 In an embodiment of the present invention, a company relevance modeling method based on an artificial intelligence algorithm includes the following steps:

[0055] Step 1: Data selection;

[0056] Collect and select multiple sets of basic dependent data, that is, two features of public opinion information: title and text. These feature data belong to non-standard text type data, and then construct a third feature variable: target company. Obtain the list of currently listed companies through statistical sorting, extract the target company keywords matching the listed companies in the title and text, and complete the basic gross data extraction;

[0057] Step 2: Data labeling;

[0058] Label the sample data, judge the relevance of news and news subjects based on expert experience, and label them into values ​​between 0 and 1, and merge them into the original training data;

[0059] Step 3: Visual analysis of sample data;

[0060] The correlation of the above sample data is counted and visualized; that is, statistics are made through pie charts, bar charts, and lists for viewing;

[0061] Step 4: Feature derivation;

[0062] The above data is used to derive features, and based on the three original variables of title, text, and target company, the following eight new feature variables are constructed:

[0063] 1) Number of target company title text matches;

[0064] 2) Number of hits in the target company’s main text;

[0065] 3) Title text string length;

[0066] 4) The length of the text string;

[0067] 5) Number of text matching hits of other target listed companies;

[0068] 6) The total number of other target listed companies after deduplication;

[0069] 7) The industry type to which the target company belongs;

[0070] 8) The percentage of target company matching hits;

[0071] Step 5: Visual analysis of derived features, i.e. statistical analysis through pie charts, bar charts, and lists for viewing;

[0072] Step 6: Derived feature preprocessing, that is, converting derived features into numerical features;

[0073] Step 7: Build the model;

[0074] Use cross-validation to divide the above numerical features, randomly break up the sample set and divide it into training set and test set, repeat ten times, and then use machine learning decision tree regression for training to finally get the model.

[0075] The present invention derives eight characteristic variables and establishes an efficient model that can quickly screen out highly relevant public opinions after preprocessing them.

[0076] In this implementation: In step 2, the sample data with missing values ​​in the company name is deleted. This setting is used to remove invalid sample data, thereby ensuring the accuracy of the training data.

[0077] In this implementation: In step 2, the relevance is distributed between 0 and 1, with 0 being irrelevant, 0.5 being moderately relevant, and 1 being highly relevant. This setting divides the relevance, making it easier for later staff to quickly identify the relevance of public opinion information to the company.

[0078] In this implementation: In step 6, the industry of the target company in the feature variable is a text feature, which is converted into a numerical feature through LabelEncoder. This setting enables the industry of the target company to be smoothly converted into a numerical feature, which is convenient for later training data.

[0079] In this implementation: In step 7, the ratio of the training set to the test set is 8:2. This ratio can effectively ensure the accuracy of training.

[0080] In this implementation: In step 7, the specific process of training using decision tree regression of machine learning is as follows:

[0081] S1: build decision tree regression;

[0082] S2: build tree depth interval, defined from 1 to 50;

[0083] S3: Use r2 as the evaluation metric;

[0084] S4: Build a grid search and train the model;

[0085] S5: Get the model and tree depth parameters, where Figure 2 As shown in the figure, when the tree depth is 15, the training score and testing socore scores are not very high, and the model is under-fitting; when it is greater than 25, the degree of fitting begins to tend to be excellent. When the tree depth is between 35 and 45, the training score curve and the testing score curve perform excellently, with the best degree of fitting, and there is no overfitting phenomenon with high training socore and low testing socore.

[0086] In this implementation: After the model built in step 7 is deployed on the cloud server, a preset value of relevance is set, and the public opinion information imported into the model is screened, and public opinion information with a relevance value higher than the preset value is screened out for push. The staff can set relevant preset values ​​according to the company's needs, so as to facilitate the push of relevant public opinion information to the company.

[0087] In this implementation: the preset value of the correlation is 0.7.

[0088] In this implementation: the model is deployed online in the form of an API interface, the request method is a post request, and the parameters passed are the title, body, and target company name.

[0089] The present invention provides a company relevance modeling device based on an artificial intelligence algorithm, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of any of the above methods are implemented.

[0090] The present invention constructs a company relevance calculation model based on an artificial intelligence algorithm. After the model is deployed online on a cloud server, when new news comes in, it will be automatically parsed and form eight feature variables. Then the trained machine learning model will automatically predict and output a result of 0-1. If it is greater than 0.7, it is judged as highly relevant public opinion. Subsequently, the highly relevant public opinion will be pushed to relevant companies, effectively improving the public opinion push experience.

[0091] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A company relevance modeling method based on artificial intelligence algorithm, characterized in that: The following steps are involved: Step 1: Data selection; Collect and select multiple sets of basic dependent data, that is, two features of public opinion information: title and text. These feature data belong to non-standard text type data, and then construct a third feature variable: target company. Obtain the list of currently listed companies through statistical sorting, extract the target company keywords matching the listed companies in the title and text, and complete the basic gross data extraction; Step 2: Data labeling; Label the sample data, judge the relevance of news and news subjects based on expert experience, and label them into values ​​between 0 and 1, and merge them into the original training data; Step 3: Visual analysis of sample data; Perform statistics and visualization on the correlation of the above sample data; Step 4: Feature derivation; The above data is used to derive features, and based on the three original variables of title, text, and target company, the following eight new feature variables are constructed: 1) Number of target company title text matches; 2) Number of hits in the target company’s main text; 3) Title text string length; 4) The length of the text string; 5) Number of text matching hits of other target listed companies; 6) The total number of other target listed companies after deduplication; 7) The industry type to which the target company belongs; 8) The percentage of target company matching hits; Step 5: Visual analysis of derived features; Step 6: Derived feature preprocessing, that is, converting derived features into numerical features; Step 7: Build the model; Use cross-validation to divide the above numerical features, randomly break up the sample set and divide it into training set and test set, repeat ten times, and then use machine learning decision tree regression for training to finally get the model.

2. The company relevance modeling method based on artificial intelligence algorithm according to claim 1 is characterized in that: In the step 2, sample data with missing values ​​in company names are deleted.

3. The company relevance modeling method based on artificial intelligence algorithm according to claim 1 is characterized in that: In the step 2, the correlation is distributed between 0 and 1, and 0 is defined as no correlation, 0.5 as medium correlation, and 1 as high correlation.

4. The company relevance modeling method based on artificial intelligence algorithm according to claim 1 is characterized in that: In the step 6, the industry to which the target company belongs in the feature variable is a text feature, which is converted into a numerical feature through LabelEncoder.

5. The company relevance modeling method based on artificial intelligence algorithm according to claim 1 is characterized in that: In step 7, the ratio of the training set to the test set is 8:

2.

6. The company relevance modeling method based on artificial intelligence algorithm according to claim 1 is characterized in that: In step 7, the specific process of training using decision tree regression of machine learning is as follows: S1: Construct decision tree regression; S2: build tree depth interval, defined from 1 to 50; S3: Use r2 as the evaluation metric; S4: Build a grid search and train the model; S5: Get the model and tree depth parameters.

7. The company relevance modeling method based on artificial intelligence algorithm according to claim 1 is characterized in that: After the model built in step seven is deployed on the cloud server, a preset relevance value is set to filter the public opinion information imported into the model, and the public opinion information with a higher relevance value is filtered out for push.

8. The company relevance modeling method based on artificial intelligence algorithm according to claim 7 is characterized in that: The preset value of the correlation is 0.

7.

9. The company relevance modeling method based on artificial intelligence algorithm according to claim 7 is characterized in that: The model is deployed online in the form of an API interface, the request method is a post request, and the parameters are the title, body and target company name.

10. A company relevance modeling device based on an artificial intelligence algorithm, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images