A text clustering method based on improved whale optimization algorithm

Abstract

The application relates to a text clustering method based on an improved whale optimization algorithm, and belongs to the fields of big data mining and machine learning. The method comprises the following steps: S1: obtaining scientific research text data from a scientific research text database by using Spark, and performing data cleaning; S2: performing word segmentation, stop word removal, feature selection, vectorization and dimension reduction operations on the text data after data cleaning, so that unstructured text data is converted into structured numerical data; and S3: calculating initial clustering centers by using a K-means algorithm, optimizing the clustering centers by using an improved whale optimization algorithm, and outputting clustering results. The global optimization capability of the improved whale optimization algorithm is utilized, the problem that current text clustering algorithms are prone to falling into local optimization is solved, the accuracy of clustering of scientific research text data is improved, the data dimension is reduced, and the clustering effect is enhanced.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of big data mining and machine learning, and relates to a text clustering method based on an improved whale optimization algorithm. BACKGROUND

[0002] In current scientific research management institutions, most scientific research achievements are not stored and presented in the form of structured numerical data, such as the abstracts of scientific research papers, keywords, abstracts of patents, project briefs and other data information. It is difficult to find the correlation between scientific research data for such unstructured text data, which is not conducive to the discovery and mining of innovation achievements. For the analysis and mining of scientific research achievements, it is necessary to first divide the huge scientific research text data and classify the scientific research data with high correlation into a class. The traditional text clustering method based on the K-means algorithm has the characteristics of simple principle, easy implementation and fast convergence speed, and is often used for clustering of text data. However, the K-means clustering algorithm is too sensitive to the initial centroid and is prone to local optimal solution, which reduces the accuracy of text clustering and cannot meet the requirements of current massive scientific research big data text clustering. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a text clustering method based on an improved whale optimization algorithm, which solves the technical problem that the current text clustering algorithm is prone to local optimal solution, resulting in inaccurate text clustering results.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] A text clustering method based on an improved whale optimization algorithm, comprising the following steps:

[0006] S1: obtaining text data from a text database using Spark and performing data cleaning;

[0007] S2: performing word segmentation, stop word removal, feature selection, vectorization and dimensionality reduction operations on the text data after data cleaning, and converting the unstructured text data into structured numerical data;

[0008] S3: calculating the initial clustering center using the K-means algorithm, optimizing the clustering center using the improved whale optimization algorithm, and outputting the clustering result.

[0009] Further, the text data in S1 includes paper abstracts, authorized patent abstracts and project briefs.

[0010] Further, the S2, specifically: using jieba word segmentation model, the continuous Chinese text is split into words and saved, load stop word text, remove the stop words without actual meaning, keep the words that can represent the key information of the text, calculate the weight of each word using TF-IDF algorithm, convert the text data into feature vector, realize dimension reduction of the text feature vector using PCA algorithm, select the 20 feature vectors A=[α1,α2,...,α 20 ] that can best represent the article content as the original input of clustering; wherein the calculation formula of TF-IDF algorithm is:

[0011]

[0012]

[0013] TF-IDF=TF*IDF (3)

[0014] Wherein, TF is the frequency of the word, n i,j is the number of times the word appears in the file, ∑ k n k,j is the sum of the number of times all words appear in the file; IDF is the inverse file word frequency, |D| is the total number of documents in the corpus, j is the number of documents containing the word, t i is the i-th word in the text, d j is the j-th text in the text library.

[0015] Further, the S3 uses the improved whale optimization algorithm to optimize the clustering center and outputs the clustering result, specifically:

[0016] S41: determine the number of text clustering cluster centers and the parameters of the whale optimization algorithm; the parameters of the number of text clustering cluster centers include: population size, maximum iteration number; the parameters of the whale optimization algorithm include: correction factor, random variable l, constant coefficient b;

[0017] S42: chaotic initialization is performed on the whale individuals of the improved whale optimization algorithm, and the characteristics of chaotic initialization are used to relatively uniformly distribute the whale individuals in the entire hunting space;

[0018] S43: calculate the distance of each text data individual to different cluster centers, assign each text data point to the nearest center, divide the data into K clusters, and complete the clustering process;

[0019] S44: calculate the intra-cluster distance of the K clusters, and take the distance as the fitness function of the improved whale optimization algorithm; the intra-cluster distance of the K clusters is formula:

[0020]

[0021] Wherein, N is the number of the same cluster of text data; x j is the jth text data, x i is the ith text data;

[0022] S45: According to the fitness function, the hunting behavior of the whale individual is judged, and the behavior parameter is set to a random number in the value range of [0, 1];

[0023] When the parameter is less than 0.5, the whale group hunts in the way of surrounding the prey or the improved random search strategy and updates the position;

[0024] When the parameter is greater than 0.5, the whale individual searches for the prey in the way of spiral search and updates the position;

[0025] S46: According to the updated position, each cluster center is updated, and the optimal fitness value is calculated, whether the whale population has completed the position update is judged, if not, jump to S85, otherwise, the next step is performed;

[0026] S47: Whether the iteration number is the maximum value or meets a certain convergence condition is judged;

[0027] If it is satisfied, the optimal cluster center and the text clustering result are output;

[0028] Otherwise, the iteration number is increased by 1, the current obtained cluster center is taken as the initial center, and the operation returns to S44 to continue the clustering operation until the convergence condition is met.

[0029] Further, in the S41, the whale optimization algorithm is used to simulate the behavior of the humpback whale group hunting prey, obtain the optimal solution of the fitness function, and the hunting behavior of the whale optimization algorithm includes: surrounding prey behavior, random search behavior and spiral search prey behavior;

[0030] The mathematical model of the surrounding prey behavior is:

[0031]

[0032] Wherein, i is the current iteration number, i max is the maximum iteration number, x i is the position of the current whale individual after the ith iteration, is the position of the best whale individual after the ith iteration, A, C are the coefficient vectors of the whale hunting behavior, a is a coefficient which is linearly reduced with the iteration number, and the value range is: [0, 2], rand1, rand2 are random variables with the value range of [0, 1];

[0033] The mathematical model of the random search behavior is:

[0034]

[0035] in, is the position vector of a randomly selected individual whale;

[0036] The mathematical model of the spiral search for prey behavior is:

[0037]

[0038]

[0039] Among them, b is a constant term that determines the shape of the logarithmic spiral, l is a random number with a value range in the range of [-1,1], and D represents the distance from the current whale individual to the best whale individual.

[0040] Furthermore, in S42, the improved whale optimization algorithm is optimized by simulating the hunting behavior of a group of whales. The improved whale optimization algorithm includes: optimizing the random initialization of the whale population into a chaotic initialization of the whale population, introducing the position information of the optimal individual as a reference indicator in the random search behavior, increasing the boundary search capability by introducing random variables, and introducing the Levy flight strategy.

[0041] Furthermore, the S45 is specifically as follows:

[0042] When the parameter p∈[0,1]<0.5 and |A|<1, the whales will take a very small step to approach the optimal whale position and surround the prey. The position after the action is:

[0043]

[0044] Among them, i is the current iteration number, i max is the maximum number of iterations, x i is the position of the current whale individual after the i-th iteration, is the position of the best whale individual after the i-th iteration, A, C are the coefficient vectors of whale hunting behavior, a is a coefficient that decreases linearly with the number of iterations, and its range is: [0,2], rand1, rand2 are random variables with a range of [0,1];

[0045] When the parameter p∈[0,1]<0.5 and |A|>1, the whale group will randomly search with the individual as the center and the best whale individual position as the reference. The updated position of the whale is:

[0046]

[0047] Where K is the correction factor;

[0048] When the parameter p∈[0,1] is greater than 0.5 and |A| is less than 1, the whale group will carry out a spiral search of the increased levy flight strategy, and the updated position is:

[0049]

[0050] L(s)~s -β ,1≤β≤3 (13)

[0051]

[0052] wherein levy(β) is the coefficient of the increased flight strategy, s is the flight step, l is a random number with a value range of [-1, 1], D represents the distance from the current whale individual to the best whale individual, β is a constant, L(s) is an expression of the levy flight strategy coefficient with s as the independent variable, Γ(x) is a gamma function about x,

[0053] Further, the S42 chaotic initialization formula is:

[0054] z i+1 =1-m(cos(n·cos -1 z i )) 2 ,z i ∈[-1,1] (15)

[0055] wherein m and n are two control variables.

[0056] Further, the K-means algorithm in S3 is specifically:

[0057] If the data set is X={x1,x2,...,x n}, n is the number of data sets, the cluster center set is Y={y1,y2,...,y k}, and K is the number of center points, then the steps of the K-means algorithm are as follows:

[0058] S91: pre-processing the data, which includes standardizing the data and filtering abnormal points;

[0059] S92: randomly selecting K center points from the pre-processed data, denoted as y1,y2,...,y k ;

[0060] S93: calculating the distance of each data to the K center points, and assigning the data to the center point with the smallest distance as a data cluster, and using the product of cosine similarity and Euclidean distance as a text similarity index, wherein the formula is:

[0061]

[0062] S93: calculate the average value of each cluster data, and take the average value of each cluster data as a new clustering center;

[0063] S94: cycle steps S92-S94 until the iteration number reaches the maximum value or a certain condition is met, and output the clustering center result.

[0064] The beneficial effects of the present application are:

[0065] First, the chaos initialization strategy of the whale population proposed in the present application increases the search ability of the original population in space.

[0066] Second, the present application improves the random search behavior model of the whale, and introduces the position of the best whale individual as a reference on the basis of the traditional model, thereby reducing the blindness to a certain extent.

[0067] Third, the present application adds a levy flight model in the spiral search prey behavior model, increases the global search ability, and reduces the risk of falling into a local optimal solution.

[0068] Fourth, the present application modifies the position of the whale individual that exceeds the boundary, increases the search ability of the whale individual around the boundary, and increases the possibility of searching for an optimal solution.

[0069] Fifth, the present application uses the Spark computing framework to obtain a large amount of scientific research text data from a scientific research achievement database, uses the jieba module to perform word segmentation and stop word removal on the text data, uses TF-IDF to calculate the word frequency of the text, and uses the PCA algorithm for dimensionality reduction processing, thereby obtaining key information representing the text; the global optimization ability of the improved whale optimization algorithm is used for clustering analysis, which can reduce the risk of the text clustering falling into a local optimal solution, and greatly improves the accuracy of scientific research text clustering.

[0070] Other advantages, objects and features of the present application will be set forth in part in the following specification, and in part will become apparent to those skilled in the art from the examination of the following, or can be learned from practice of the present application. The objects and other advantages of the present application can be realized and obtained by the following specification. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to make the objects, technical solutions and advantages of the present application clearer, the preferred detailed description of the present application will be made below in combination with the drawings, wherein:

[0072] Figure 1 is a flowchart of a scientific research big data text clustering method based on an improved whale optimization algorithm;

[0073] Figure 2 Flowchart and schematic diagram of the classic whale optimization algorithm;

[0074] Figure 3 Flowchart of the improved whale optimization algorithm. DETAILED DESCRIPTION

[0075] The present application can be implemented or applied in other different embodiments, and various modifications or changes can be made to the details based on different views and applications without departing from the spirit of the present application. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present application in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0076] The accompanying drawings are only used for illustrative purposes, and the representation is only a schematic diagram, not a physical diagram, and should not be understood as a limitation on the present application; in order to better illustrate the embodiments of the present application, some components in the drawings may be omitted, enlarged or reduced, and do not represent the actual product size; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0077] The same or similar reference numerals in the drawings of the embodiments of the present application correspond to the same or similar components; in the description of the present application, it should be understood that if the terms "upper", "lower", "left", "right", "front", "back" and the like indicate the orientation or positional relationship shown in the drawings, only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the terms describing the positional relationship in the drawings are only used for illustrative purposes, and cannot be understood as a limitation on the present application, for those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0078] Figure 1 The present application provides a scientific research big data text clustering method based on an improved whale optimization algorithm, comprising:

[0079] Step 1: data acquisition, using Spark to obtain scientific research text data from a scientific research text database, and performing data cleaning;

[0080] Specifically, the text data includes paper abstracts, authorized patent abstracts, project participation introductions, etc.

[0081] Step 2: For the obtained text data, word segmentation, stop word removal, feature selection, vectorization and dimensionality reduction operations are performed to convert unstructured text data into structured numerical data;

[0082] Specifically, Chinese text is not separated by spaces like English text, but words are closely connected to each other, so it is necessary to use the jieba word segmentation model to split the continuous Chinese text into words and save it, load the stop word text, remove the stop words without actual meaning, keep the words that can represent the key information of the text, calculate the weight of each word using the TF-IDF algorithm, which can convert the text data into a feature vector, and then use the PCA algorithm to realize dimensionality reduction of the text feature vector. The calculation formula of the TF-IDF algorithm is:

[0083]

[0084]

[0085] TF-IDF = TF * IDF (3)

[0086] Where, TF is the frequency of the word, n i,j is the number of times the word appears in the file, ∑ k n k,j is the sum of the number of times all words appear in the file; IDF is the inverse file word frequency, |D| is the total number of documents in the corpus, and j is the number of documents containing the word.

[0087] Step 3: Use the improved whale optimization algorithm to fuse K-means to cluster the dimensionality-reduced data, and then output the optimal clustering center and clustering result;

[0088] Where, the improved whale optimization algorithm still optimizes by simulating the hunting behavior of the whale group, which includes three processes: surrounding prey, random search, and spiral search. Four improvements have been made compared to the traditional whale optimization algorithm. The improved whale optimization algorithm optimizes the random initialization of the whale population to the chaotic initialization of the whale population. In order to reduce the randomness of random search, the position information of the optimal individual is introduced as a reference index in the random search behavior. Considering that the traditional whale optimization algorithm has a single way of handling individuals that exceed the boundary, the boundary search capability is increased by introducing a random variable. In order to reduce the possibility of falling into a local optimal solution as much as possible, the Levy flight strategy is introduced to change the step size and increase the possibility of jumping out of the local optimal solution. The hunting behavior of the improved whale optimization algorithm is as follows:

[0089] When the random variable p∈[0,1]<0.5 and |A|<1, the whales will approach the best whale position with a small step to surround the prey. The position after the action is:

[0090]

[0091] where i is the current iteration number, i max is the maximum iteration number, x i is the position of the current whale individual after the i-th iteration, is the position of the best whale individual after the i-th iteration, A, C are the coefficient vectors of the whale hunting behavior, a is a coefficient that linearly decreases with the iteration number, and the value range is: [0, 2], rand1, rand2 are random variables with a value range of [0, 1];

[0092] When the random variable p∈[0,1]<0.5 and |A|>1, the whale group will search randomly with the best whale individual position as the reference. The updated position of the whale is:

[0093]

[0094] where K is a correction factor, and the value is 2;

[0095] When the random variable p∈[0,1]>0.5 and |A|<1, the whale group will perform a spiral search with an increased levy flight strategy, and the updated position is:

[0096]

[0097] L(s)~s -β ,1≤β≤3 (7)

[0098]

[0099] where levy(β) is the coefficient of the increased flight strategy, s is the flight step, and β is a constant with a value of 1.5;

[0100] When the random variable p∈[0,1]>0.5 and |A|>1, the whale group will perform another form of spiral search, and the updated position is:

[0101] x i+1 =x i +D·e bl ·cos(2πl)·levy(β), (9)

[0102] L(s)~s -β ,1≤β≤3 (10)

[0103]

[0104] The pseudo code for improving whale optimization algorithm is as follows:

[0105] Start

[0106] Initialize the population size N, the maximum number of iterations T max , and the related coefficients D, b;

[0107] Chaotic initialization is performed on the whale individual x j,D (j=1, 2, 3,..., N), and the mathematical model of chaotic initialization is:

[0108] z i+1 =1-m(cos(n·cos -1 z i )) 2 ,z i ∈[-1,1] (12)

[0109] Wherein, m, n are two control variables, and the values are 2 and 4 respectively.

[0110] Initialize the optimal individual position x gbest , and calculate the value of the fitness function of the whale population:

[0111] When the number of iterations iter is less than the maximum number of iterations T max , enter the While loop

[0112] While: iter < T max

[0113] When j is equal to 1: N, enter the for1 loop to update a, A, C, p, and l

[0114] for1 j=1:N

[0115] Then update a, A, C, p, and l

[0116] If p∈[0,1]<0.5 and |A|<1

[0117] Then the hunting method of surrounding prey is used to update the whale position

[0118] If p∈[0,1]<0.5 and |A|>1

[0119] Then the improved random search hunting method is used to update the whale position

[0120] If p∈[0,1]>0.5 and |A|<1

[0121] The whale position is updated using a spiral search hunting method with the optimal solution position as a reference.

[0122] If p∈[0,1]>0.5 and |A|<1

[0123] The spiral search hunting method is used to update the whale's position

[0124] end for1

[0125] End for1 loop

[0126] Determine whether the updated whale individual is out of bounds. If so, update the position:

[0127]

[0128] Among them, UB is the upper boundary and LB is the lower boundary

[0129] When j is equal to 1:N, enter the for2 loop

[0130] for2 j=1:N

[0131] Determine whether the number of iterations is the maximum value or meets certain convergence conditions

[0132] if(x j,D )<(x gbest )

[0133] If it satisfies the requirement, the optimal cluster center and text clustering results are output.

[0134] x gbest =x j,D

[0135] end for2

[0136] End for2 loop

[0137] If it is not satisfied, the number of iterations iter=iter+1

[0138] iter=iter+1

[0139] Taking the current cluster center as the initial center, continue the clustering operation, that is, continue the while loop until the convergence condition is met.

[0140] end while

[0141] End the while loop

[0142] The K-means clustering algorithm is a traditional clustering analysis algorithm, and its main idea is to randomly initialize the clustering center by giving a data clustering center number K, then assign the data closest to the clustering center together, after the assignment is completed, recalculate the distance of each cluster, find the new clustering center, then repeat the above iteration process until the clustering center no longer changes or changes are small or reach the maximum iteration number. If the data set is X = {x1, x2,..., x n}, n is the number of data sets, the clustering center set is Y = {y1, y2,..., y k}, and K is the number of center points. The steps of the K-means algorithm are as follows:

[0143] Step 1: data preprocessing, mainly standardization, outlier filtering;

[0144] Step 2: randomly select K center points, denoted as y1, y2,..., y k ;

[0145] Step 3: calculate the distance of each data to K center points, and assign the data to the nearest center point as a data cluster;

[0146] In order to reduce the error between the text similarity, the product of cosine similarity and Euclidean distance is used as the text similarity index, and the formula is:

[0147]

[0148] Step 4: calculate the average value of each cluster data, and take this value as the new clustering center;

[0149] Step 5: loop steps 2-4 until the maximum iteration number is reached or a certain condition is met, and output the clustering result;

[0150] The improved whale optimization algorithm is used to optimize the K-means clustering effect, that is, the intra-cluster distance of each cluster is used as the fitness function of the improved whale optimization algorithm, and the global optimization ability of the improved whale optimization algorithm is used to find the optimal clustering center, so that the output clustering result is more accurate. The steps include:

[0151] Step 1: initialize various parameters, and use chaotic initialization to obtain whale population, determine the maximum iteration number, and each whale individual represents a solution, that is, a set of clustering centers. The chaotic initialization formula is:

[0152] z i+1 =1-m(cos(n·cos -1 z i )) 2 ,zi ∈ [-1, 1] (15)

[0153] Step 2: Calculate the distance of each data individual to each cluster center, assign each data point to the nearest center, and divide the data into K clusters;

[0154] Step 3: Calculate the intra-cluster distance of K clusters:

[0155]

[0156] Step 4: Calculate the fitness function to get the best whale individual, that is, the best cluster center group;

[0157] Step 5: According to the value of the random variable, select the whale hunting mode, update the position of the whale, then calculate the fitness function value, and update the global optimal solution;

[0158] Step 6: Determine whether the iteration number reaches the maximum value or meets certain conditions, if the conditions are met, end the iteration and output the clustering result; otherwise, increase the iteration number by 1 and continue the iteration process until the iteration number reaches the maximum value or meets certain conditions;

[0159] In order to embody the beneficial effects of the scientific research big data text clustering method based on the improved whale optimization algorithm, 4 different types of journal paper abstracts are manually screened from the China Knowledge Network Discovery Network Platform; among them, there are 100 paper abstracts in the field of medical image processing, 100 paper abstracts in the field of vehicle target detection, 100 paper abstracts in the field of unsupervised learning, and 50 paper abstracts in the field of education, with an average of about 300-500 words per abstract. The present method and the K-means clustering algorithm are used for clustering respectively, and the comparison experimental results are shown in Table 1.

[0160] Among them, the precision reflects the accuracy of the clustering result, the larger the value, the better the clustering effect, and the formula is:

[0161] N p N is the number of correct news texts in the clustering result, N e is the total number of news texts participating in clustering.

[0162] The text clustering result is shown in Table 1.

[0163] Table 1 is the comparison experimental results of the present method and the K-means clustering algorithm

[0164]

[0165] From table 1, it can be seen that the method proposed in the application improves the clustering effect in different fields by 31.69%, 39.61%, 40.60% and 25.49% respectively compared with the traditional K-means algorithm.

[0166] In conclusion, the method proposed in the application solves the problem that the K-means clustering algorithm is prone to fall into a local optimal solution in text clustering to a certain extent, and improves the accuracy of the traditional K-means algorithm in scientific research text data clustering.

[0167] Finally, it should be pointed out that the above examples are only used to illustrate the technical solutions of the application and are not limiting. Although the application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the application can be modified or replaced equivalently without departing from the purpose and scope of the technical solutions, and they should be covered in the scope of the claims of the application.

Claims

1. A text clustering method based on an improved whale optimization algorithm, characterized in that: The method comprises the following steps: S1: obtaining text data from a text database using Spark and performing data cleaning; S2: performing word segmentation, stop word removal, feature selection, vectorization and dimension reduction operations on the text data after data cleaning, and converting unstructured text data into structured numerical data; S3: calculating initial cluster centers using a K-means algorithm, optimizing the cluster centers using an improved whale optimization algorithm, and outputting the clustering results, specifically: S41: determine the number of cluster centers of the text clustering, parameters of the whale optimization algorithm; the parameters of the number of cluster centers of the text clustering include: population size, maximum iteration number; the parameters of the whale optimization algorithm include: correction factor, random variable l , constant coefficient b ; S42: the whale individuals of the improved whale optimization algorithm are chaotically initialized, the characteristics of chaotic initialization are utilized, and the whale individuals are relatively uniformly distributed in the entire hunting space; the chaotic initialization is represented as: wherein, are two control variables; S43: calculating the distance of each text data individual to different cluster center, assigning each text data point to the nearest center, dividing the data into K clusters, and completing the clustering process; S44: calculating the intra-cluster distance of the K clusters, and taking the distance as the fitness function of the improved whale optimization algorithm; the intra-cluster distance of the K clusters is calculated according to the formula: Wherein, N is the number of the same cluster of Chinese text data; is the jth text data, is the ith text data; S45: determining the hunting behavior of the whale individual according to the fitness function, and setting the behavior parameter to a random number in the range of [0, 1]; When the parameter is less than 0.5, the whale group hunts in a way of surrounding the prey or performs an improved random search strategy and updates the position; when the parameter is greater than 0.5, the whale individual searches for the prey in a spiral search manner and updates the position; S46: updating each cluster center according to the updated position, calculating the optimal fitness value, and determining whether the whale population has completed the position update; if not, jump to S85; otherwise, proceed to the next step; S47: determining whether the iteration number is the maximum value or meets a certain convergence condition; If yes, output the optimal cluster center and the text clustering result; otherwise, increase the iteration number by 1, take the current obtained cluster center as the initial center, and return to S44 to continue the clustering operation until the convergence condition is met.

2. The text clustering method based on the improved whale optimization algorithm according to claim 1, characterized in that: The S1 text data includes paper abstracts, authorized patent abstracts and project participation introductions.

3. The text clustering method based on the improved whale optimization algorithm according to claim 1, characterized in that: The S2, in particular, is: using a jieba word segmentation model to continuously split Chinese text into words and save, loading stop word text, removing stop words without actual meaning, retaining words that can represent key information of the text, using a TF-IDF algorithm to calculate the weight of each word, converting the text data into a feature vector, using a PCA algorithm to realize dimension reduction of the text feature vector, and selecting 20 feature vectors that can best represent the article content as the original input of clustering; wherein the calculation formula of the TF-IDF algorithm is: wherein, tfis the term frequency, is the number of times a term appears in a document, is the sum of the term frequencies of all words in a document; is the inverse document term frequency, is the total number of documents in the corpus, j is the number of documents containing the term, is the i-th term in the text, is the j-th document in the corpus.

4. The text clustering method based on improved whale optimization algorithm according to claim 1, characterized in that, In S41, the whale optimization algorithm is used to simulate the hunting behavior of a group of humpback whales to obtain the optimal solution of the fitness function, and the hunting behavior of the whale optimization algorithm includes surrounding prey behavior, random search behavior and spiral search prey behavior. The mathematical model of the surrounding prey behavior is: wherein, is the current iteration number, is the maximum iteration number, is the position of the current whale individual after the th iteration, is the position of the best whale individual after the th iteration, is the coefficient vector of the whale hunting behavior, is a coefficient that linearly decreases with the iteration number, with a value range of: [0, 2], is a random variable with a value range of [0, 1]; The mathematical model of the random search behavior is: wherein, is a randomly selected whale individual position vector; The mathematical model of the spiral search prey behavior is: wherein, is a constant term for determining the logarithmic spiral shape, is a random number with a value range in [-1, 1], denotes the distance of the current whale individual to the best whale individual.

5. The text clustering method based on improved whale optimization algorithm according to claim 1, characterized in that, In S42, the improved whale optimization algorithm is used to simulate the hunting behavior of a group of humpback whales to optimize, and the improved whale optimization algorithm includes optimizing the random initialization of the whale population to the chaotic initialization of the whale population, introducing the position information of the optimal individual as a reference index in the random search behavior, increasing the boundary search ability by introducing a random variable, and introducing a Levy flight strategy.

6. The text clustering method based on improved whale optimization algorithm according to claim 1, characterized in that, In S45, specifically: When the parameters and are satisfied, the whales will approach the best whale position with a small step to surround the prey, and the position after the action is: wherein, is the current iteration number, is the maximum iteration number, is the position of the current whale individual after the th iteration, is the position of the best whale individual after the th iteration, is the coefficient vector of the whale hunting behavior, is a coefficient that linearly decreases with the iteration number and has a value range of: [0, 2], is a random variable with a value range of [0, 1]; When the parameters and are satisfied, the whale group will search randomly with the individual as the center and the best whale individual position as the reference, and the updated position of the whale is: Wherein, K is a correction factor; When the parameters and are satisfied, the whale swarm will perform a spiral search with an increased levy flight strategy, and the updated position is: wherein, is the coefficient of the increasing flight strategy, is the flight step size, is a random number with a value range in [-1, 1], denotes the distance of the current whale individual to the best whale individual, is a constant, is the expression of the levy flight strategy coefficient with s as the argument, is the gamma function of .​ 7. The text clustering method based on improved whale optimization algorithm according to claim 1, characterized in that: In S3, the K-means algorithm is specifically: If the data set is , is the number of data sets, the cluster center set is , and K is the number of center points; the steps of the K-means algorithm are as follows: S91: pre-processing the data, which includes standardizing the data and filtering out abnormal points; S92: randomly select K center points from the pre-processed data, denoted as ; S93: Calculate the distance of each data to K center points, assign the data to the center point with the minimum distance as a data cluster, and use the product of cosine similarity and Euclidean distance as a measure of text similarity index, where the formula is: S93: Calculate the average value of each cluster data, and take the average value of each cluster data as the new clustering center; S94: Loop steps S92-S94 until the iteration number reaches the maximum value or meets certain conditions, and output the clustering center result.

Images