A Fast K-Nearest Neighbor Classifier Method for Large-Scale Data

Abstract

This invention discloses a fast K-nearest neighbor classifier method for large-scale data, belonging to the fields of data processing and machine learning technology. It proposes the concept of a particle sphere to introduce the DBSCAN method and the concept of optimization difference to introduce an adaptive KNN algorithm that adaptively selects the key K value. Specifically, the particle sphere concept and the optimization difference concept include a DBSCAN particle sphere generation algorithm based on density peaks and a density-based particle sphere GBSKNN algorithm based on optimization difference. The DBSCAN algorithm addresses the problem of low time complexity in traditional KNN algorithms when dealing with large-scale data processing by introducing the concept of particle sphere computation. Based on the traditional algorithm, a traditional particle sphere GBSKNN algorithm with optimization difference is proposed, where the particle sphere formation process uses particle sphere computation based on density peaks. While ensuring accuracy, its running time is much shorter than that of traditional KNN, maintaining excellent prediction performance while significantly reducing computation time.

Description

Technical Field

[0001] This invention relates to the technical field of data processing and machine learning, specifically to a fast K-nearest neighbor classifier method for large-scale data. Background Technology

[0002] The k-nearest neighbor (KNN) algorithm was first proposed by Cover and Hart in 1968. It is an algorithm based on the idea of ​​template matching. The mature theory and simple implementation of the K-nearest neighbor classification algorithm make it one of the classic machine learning algorithms. With the continuous development of information technology, the K-nearest neighbor algorithm has achieved excellent results in machine learning classification [2]. Its good classification performance and simple principle have made it continue to be used to this day. The Internet has been developing since the 1950s. Its information volume is getting bigger and bigger, and the amount of data generated is also getting bigger and bigger. When facing large-scale data processing, the K-nearest neighbor classification algorithm has low time efficiency, which has always been a disadvantage of the algorithm. It can be seen that in today's rapidly developing information age, the K-nearest neighbor algorithm can hardly meet the time efficiency requirements of the big data era. Secondly, as one of the underlying algorithms of machine learning, the time efficiency of the K-nearest neighbor algorithm has always affected the upper layer. With the increase of data volume, its time efficiency will also increase exponentially.

[0003] Taking the K-Nearest Neighbors (KNN) algorithm for image recognition as an example, the principle of KNN requires mathematical calculations between a sample image and every image in the set to obtain the corresponding classification result. However, as the number of images increases, the computational load also increases, leading to decreased time efficiency. Clearly, this makes it difficult to meet today's recognition speed requirements. In the context of today's information age and big data era, addressing the time efficiency disadvantage of the KNN algorithm when dealing with large-scale data processing is particularly urgent.

[0004] In the traditional K-nearest neighbor algorithm, the value of K is also determined manually, which is obviously inefficient. The best prediction effect is achieved when the value of K cannot be determined. When faced with large-scale data processing, there is a problem of inaccurate determination of the value of K or manual determination of the value of K, which leads to inefficiency.

[0005] This invention addresses two problems with the aforementioned K-nearest neighbor algorithm and the shortcomings of existing technologies by introducing the idea of ​​granular computing to granulate large-scale data and solve the problem of large-scale data. It also introduces the concept of optimization difference to solve the problem that the K-nearest neighbor algorithm requires manual determination of the value of K. Combining the two methods, this invention proposes a fast K-nearest neighbor algorithm for large-scale data. Summary of the Invention

[0006] The purpose of this invention is to overcome the aforementioned technical difficulties by proposing the concept of a particle-sphere to introduce the DBSCAN method, and by proposing the concept of optimization difference to introduce an adaptive KNN algorithm for adaptively selecting key K values, thus providing a fast K-nearest neighbor classifier method for large-scale data.

[0007] To achieve the above objectives, the technical solution adopted is as follows: a fast K-nearest neighbor classifier method for large-scale data: The method introduces the concept of a particle sphere into the DBSCAN algorithm, and introduces the concept of optimization difference into an adaptive KNN algorithm that adaptively selects the key K value. Specifically, the particle sphere concept and the optimization difference concept include a DBSCAN particle sphere generation algorithm based on density peaks and a density-based particle sphere GBSKNN algorithm based on optimization difference.

[0008] The steps of the DBSCAN particle generation algorithm based on density peak value are as follows:

[0009] (1) Input: Dataset D, purity threshold p, neighborhood radius eps value, minimum number of points within the neighborhood radius minpts value;

[0010] (2) Output: List of balls B;

[0011] ① Use the 2-means clustering algorithm to divide the dataset D into two clusters, D1 and D2, with n = 2 clusters;

[0012] ② For each sphere Di;

[0013] 1) The cluster center of sphere Di is the centroid of all sample points in that sphere;

[0014] 2) The radius of the sphere Di is equal to the average distance from all sample points in the sphere to its centroid;

[0015] 3) The purity of granules Di is equal to the percentage of the majority of the sample in the granules;

[0016] 4) If the purity of the granules is less than p;

[0017] a. Divide it into two spheres using the 2-means clustering algorithm, and increment the number of spheres n;

[0018] ③If the purity of each sphere is higher than p;

[0019] 1) Proceed to step 5;

[0020] ④ Otherwise: Return to step ②;

[0021] ⑤ For each Di;

[0022] 1) Calculate the number of clusterable spheres k using the DBSCAN algorithm with parameters eps and minpts;

[0023] 2) If the number of clusterable clusters k is greater than or equal to 2;

[0024] a. Use the k-means clustering algorithm to further divide the spheres into k spheres, with the number of spheres n = n + k - 1;

[0025] ⑥ If the number of clusters k for each sphere is less than 2;

[0026] 1) Proceed to step ⑧;

[0027] ⑦ Otherwise: Return to step ⑤;

[0028] ⑧ Termination;

[0029] The specific steps of the density-based sphere GBSKNN algorithm based on optimization difference are as follows:

[0030] (1) Input: Dataset D, a query point O outside dataset D, and a list of particles B calculated by the DBSCAN particle generation algorithm based on density peaks;

[0031] (2) Output: the label of O;

[0032] ① Calculate the distance from point O to each ball in the ball list B and sort them to obtain the ascending distance list DIS;

[0033] ②If the minimum distance DIS[0] is less than 0;

[0034] 1) Then the label of point O is equal to the label of the ball closest to it, go to step 9;

[0035] ③Otherwise: Initialize the distance list Dis to be calculated to [0, DIS[0]];

[0036] ④ Calculate the standard deviation of Dis, Std1;

[0037] ⑤ Initialize the standard deviation list STD = [Std1];

[0038] ⑥ Let i = 1;

[0039] ⑦ Add DIS[i] to Dis;

[0040] ⑧ Calculate the current standard deviation of Dis (Stdi);

[0041] 1) If Stdi <= Stdi-1;

[0042] a. Add Stdi to STD, i++;

[0043] b. Return to step ⑦;

[0044] 2) Otherwise: Calculate the number of elements n in the current STD;

[0045] a. Calculate the list of labels corresponding to the first n balls in the DIS list;

[0046] b. Then the label of point O is equal to the label with the largest number of labels;

[0047] 9. Termination;

[0048] Furthermore, the granularization concept is defined for large-scale data granulation as follows:

[0049] Definition 1: Given a dataset A n-1 ={a1,a2,...,a n-1}, a point o and a ball A n-1 ={a1,a2,...,a n-1},A n-1 ={a1,a2,...,a n-1 As the center of the sphere, A n-1 ={a1,a2,...,a n-1} represents the radius of the grain. The center of the grain is A. n-1 ={a1,a2,...,a n-1} is the centroid of all sample points, A n-1 ={a1,a2,...,a n-1} represents all sample points in the sphere to A n-1 ={a1,a2,...,a n-1 The average distance of}. Then we have:

[0050]

[0051] (radius A) n-1 ={a1,a2,...,a n-1 The main reason why the distance is defined as the average distance rather than the maximum or minimum distance is to make the size of the sphere less susceptible to the influence of outlier samples.

[0052] Definition 2 Given a dataset A n-1 ={a1,a2,...,a n-1}, point A n-1 ={a1,a2,...,a n-1} and a ball A n-1 ={a1,a2,...,a n-1},A n-1 ={a1,a2,...,a n-1 As the center of the sphere, A n-1 ={a1,a2,...,a n-1 Let} be the radius of the sphere. Then point A... n-1={a1,a2,...,a n-1} to particle A n-1 ={a1,a2,...,a n-1 The distance to} is defined as:

[0053]

[0054] Definition 3 Given a dataset A n-1 ={a1,a2,...,a n-1}, point A n-1 ={a1,a2,...,a n-1} and a ball A n-1 ={a1,a2,...,a n-1},A n-1 ={a1,a2,...,a n-1 As the center of the sphere, A n-1 ={a1,a2,...,a n-1 Let} be the radius of the sphere. Sphere A n-1 ={a1,a2,...,a n-1 The total label is defined as ball A. n-1 ={a1,a2,...,a n-1 The labels of most samples in the}, that is, the labels with the most quantity in the particles.

[0055] Furthermore, the concept of optimization difference is defined for automatic K value selection as follows:

[0056] Define and assume A n-1 ={a1,a2,...,a n-1}, A n ={a1,a2,...,a n-1 ,a n} represents two sets of numbers.

[0057] and A respectively n-1 and A n The mean.

[0058] and A respectively n-1 and A n The standard deviation.

[0059] For A n middle element a n The optimization difference is defined as:

[0060]

[0061] Furthermore, the concept of optimization difference is defined in relation to the automatic selection of K, and A... n Compared to A n-1 There is an extra element 'a' n . when a n Join A n-1 The middle part is A. n It may change A n-1 The degree of dispersion, and this change is due to the newly added element; therefore, the optimization difference reflects the effect of adding a new element. When z n When the value is greater than 0, element a is called... n The addition of z produced a negative effect; when z n When < 0, element a is called n The addition of z produced a positive effect; when z n When = 0, element a is called 0. n The addition of has no effect on the original set.

[0062] The beneficial effects of adopting the above scheme are as follows: This invention provides a fast K-nearest neighbor classifier method for large-scale data. It utilizes the DBSCAN algorithm to address the issue of low time complexity in traditional KNN algorithms when dealing with large-scale data processing. Furthermore, by employing the concept of optimization difference, it accurately determines the value of K, improving prediction accuracy while reducing time costs. It introduces the idea of ​​particle ball computation, proposing a traditional particle ball GBSKNN algorithm based on optimization difference. This algorithm uses particle ball computation based on density peaks to form particles, achieving significantly shorter running time than traditional KNN while maintaining accuracy. It not only maintains excellent prediction performance but also greatly reduces computation time. In today's rapidly developing big data landscape, this invention's model reduces the computational cost of big data development, making it more suitable for small and medium-sized enterprises with lower development costs. It can also save large big data enterprises substantial computational costs, reduce the consumption of computing resources, promote the development of the big data industry, and drive its progress. Attached Figure Description

[0063] Figure 1 This is a flowchart of the DBSCAN particle generation algorithm based on density peaks, which is a fast K-nearest neighbor classifier method for large-scale data according to the present invention.

[0064] Figure 2 This is a flowchart of the GBSKNN algorithm based on the optimization difference density sphere, which is a fast K-nearest neighbor classifier method for large-scale data according to the present invention.

[0065] Figure 3 This is a demonstration of particle formation in a fast K-nearest neighbor classifier method for large-scale data according to the present invention.

[0066] Figure 4This invention presents the KNN algorithm flow for a fast K-nearest neighbor classifier method for large-scale data.

[0067] Figure 5 This is a demonstration diagram of the density peak particle formation based on a fast K-nearest neighbor classifier method for large-scale data according to the present invention.

[0068] Figure 6 This invention presents a comparison of the accuracy of a fast K-nearest neighbor classifier method for large-scale data on different datasets.

[0069] Figure 7 This invention presents a comparison of the accuracy of a fast K-nearest neighbor classifier method for large-scale data on different datasets.

[0070] Figure 8 This invention presents a comparison of recall rates for different datasets using a fast K-nearest neighbor classifier method for large-scale data.

[0071] Figure 9 This invention presents a comparison of F1-Scores for different datasets of a fast K-nearest neighbor classifier method for large-scale data.

[0072] Figure 10 This invention presents a comparison of the running times of two algorithms for a fast K-nearest neighbor classifier method for large-scale data. Detailed Implementation

[0073] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments. The described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0074] Example 1

[0075] A fast K-nearest neighbor classifier method for large-scale data is proposed: The concept of granular spheres is introduced to incorporate the DBSCAN method, which is a density-based clustering method for applications with noise. The principle of the K-nearest neighbor classification algorithm is to calculate the distance from a sample point to all points in the set and find the k nearest entities to predict the category of the sample point. However, when the number of entities in the set is very large, the sample point needs to perform mathematical calculations with every entity in the set, which greatly increases the time cost of the algorithm. This invention introduces the concept of granular spheres based on the idea of ​​granular computation to granulate large-scale data. The sample point performs mathematical calculations on the granular spheres formed in the set.

[0076] This invention introduces an adaptive KNN algorithm that uses the concept of "optimization difference" to adaptively select the key value of K. Traditional K-nearest neighbor classification algorithms require manual determination of the K value during classification prediction, increasing time costs and making it difficult to control prediction accuracy. This invention uses optimization difference, based on the optimized difference relationship between the sample point and each particle's Euclidean distance, to determine the K value for the current prediction. This automatic K value selection not only reduces time costs but also significantly improves the accuracy of prediction results.

[0077] In summary, this invention combines the ideas of granular computing and the concept of granules to granulate large-scale data, and then uses the optimization difference to achieve automatic K-value selection, thus proposing a fast K-nearest neighbor classifier for large-scale data (hereinafter referred to as: GBSKNN algorithm).

[0078] The aforementioned particle-sphere concept and the aforementioned optimization difference concept specifically include the DBSCAN particle-sphere generation algorithm based on density peaks and the GBSKNN density-sphere generation algorithm based on optimization differences.

[0079] The steps of the DBSCAN particle generation algorithm based on density peak value are as follows:

[0080] (1) Input: Dataset D, purity threshold p, neighborhood radius eps value, minimum number of points within the neighborhood radius minpts value;

[0081] (2) Output: List of balls B;

[0082] ① Use the 2-means clustering algorithm to divide the dataset D into two clusters, D1 and D2, with n = 2 clusters;

[0083] ② For each sphere Di;

[0084] 1) The cluster center of sphere Di is the centroid of all sample points in that sphere;

[0085] 2) The radius of the sphere Di is equal to the average distance from all sample points in the sphere to its centroid;

[0086] 3) The purity of granules Di is equal to the percentage of the majority of the sample in the granules;

[0087] 4) If the purity of the granules is less than p;

[0088] a. Divide it into two spheres using the 2-means clustering algorithm, and increment the number of spheres n;

[0089] ③If the purity of each sphere is higher than p;

[0090] 1) Proceed to step 5;

[0091] ④ Otherwise: Return to step ②;

[0092] ⑤ For each Di;

[0093] 1) Calculate the number of clusterable spheres k using the DBSCAN algorithm with parameters eps and minpts;

[0094] 2) If the number of clusterable clusters k is greater than or equal to 2;

[0095] a. Use the k-means clustering algorithm to further divide the spheres into k spheres, with the number of spheres n = n + k - 1;

[0096] ⑥ If the number of clusters k for each sphere is less than 2;

[0097] 1) Proceed to step ⑧;

[0098] ⑦ Otherwise: Return to step ⑤;

[0099] ⑧ Termination;

[0100] The specific algorithm flow is as follows: Figure 1 As shown;

[0101] The specific steps of the density-based sphere GBSKNN algorithm based on optimization difference are as follows:

[0102] (1) Input: Dataset D, a query point O outside dataset D, and a list of particles B calculated by the DBSCAN particle generation algorithm based on density peaks;

[0103] (2) Output: the label of O;

[0104] ① Calculate the distance from point O to each ball in the ball list B and sort them to obtain the ascending distance list DIS;

[0105] ②If the minimum distance DIS[0] is less than 0;

[0106] 1) Then the label of point O is equal to the label of the ball closest to it, go to step 9;

[0107] ③Otherwise: Initialize the distance list Dis to be calculated to [0, DIS[0]];

[0108] ④ Calculate the standard deviation of Dis, Std1;

[0109] ⑤ Initialize the standard deviation list STD = [Std1];

[0110] ⑥ Let i = 1;

[0111] ⑦ Add DIS[i] to Dis;

[0112] ⑧ Calculate the current standard deviation of Dis (Stdi);

[0113] 1) If Stdi <= Stdi-1;

[0114] a. Add Stdi to STD, i++;

[0115] b. Return to step ⑦;

[0116] 2) Otherwise: Calculate the number of elements n in the current STD;

[0117] a. Calculate the list of labels corresponding to the first n balls in the DIS list;

[0118] b. Then the label of point O is equal to the label with the largest number of labels;

[0119] 9. Termination;

[0120] The specific algorithm flow is as follows: Figure 2 As shown

[0121] The granularization concept for large-scale data is defined as follows:

[0122] Definition 1: Given a dataset A n-1 ={a1,a2,...,a n-1}, a point o and a ball A n-1 ={a1,a2,...,a n-1},A n-1 ={a1,a2,...,a n-1 As the center of the sphere, A n-1 ={a1,a2,...,a n-1} represents the radius of the grain. The center of the grain is A. n-1 ={a1,a2,...,a n-1} is the centroid of all sample points, A n-1 ={a1,a2,...,a n-1} represents all sample points in the sphere to A n-1 ={a1,a2,...,a n-1 The average distance of}. Then we have:

[0123]

[0124] (radius A) n-1 ={a1,a2,...,a n-1 The main reason why the distance is defined as the average distance rather than the maximum or minimum distance is to make the size of the sphere less susceptible to the influence of outlier samples.

[0125] Definition 2 Given a dataset A n-1 ={a1,a2,...,a n-1}, point An-1 ={a1,a2,...,a n-1} and a ball A n-1 ={a1,a2,...,a n-1},A n-1 ={a1,a2,...,a n-1 As the center of the sphere, A n-1 ={a1,a2,...,a n-1 Let} be the radius of the sphere. Then point A... n-1 ={a1,a2,...,a n-1} to particle A n-1 ={a1,a2,...,a n-1 The distance to} is defined as:

[0126]

[0127] Definition 3 Given a dataset A n-1 ={a1,a2,...,a n-1}, point A n-1 ={a1,a2,...,a n-1} and a ball A n-1 ={a1,a2,...,a n-1},A n-1 ={a1,a2,...,a n-1 As the center of the sphere, A n-1 ={a1,a2,...,a n-1 Let} be the radius of the sphere. Sphere A n-1 ={a1,a2,...,a n-1 The total label is defined as ball A. n-1 ={a1,a2,...,a n-1 The labels of most samples in the}, that is, the labels with the most quantity in the particles.

[0128] Using sphere computation, the original dataset can be divided into different spheres. A demonstration of sphere formation is shown below. Figure 3 As shown:

[0129] The concept of optimization difference is defined for automatic K value selection as follows:

[0130] Define and assume A n-1 ={a1,a2,...,a n-1}, A n ={a1,a2,...,a n-1 ,a n} represents two sets of numbers.

[0131] and A respectively n-1 and A n The mean.

[0132] and A respectively n-1 and A n The standard deviation.

[0133] For A n middle element a n The optimization difference is defined as:

[0134]

[0135] The concept of optimization difference refers to the definition of automatic K value selection, where A... n Compared to A n-1 There is an extra element 'a' n . when a n Join A n-1 The middle part is A. n It may change A n-1 The degree of dispersion, and this change is due to the newly added element; therefore, the optimization difference reflects the effect of adding a new element. When z n When the value is greater than 0, element a is called... n The addition of z produced a negative effect; when z n When < 0, element a is called n The addition of z produced a positive effect; when z n When = 0, element a is called 0. n The addition of has no effect on the original set.

[0136] Implementation principle of K-nearest neighbor classification algorithm

[0137] The k-nearest neighbor classifier treats each example (training data, test data, and data to be predicted) as a data point in a d-dimensional space, where d refers to the number of attributes of the example [4]. In the d-dimensional space, there are different "distances" between points (there are many types of distances, such as Hamming distance, Eulerian distance, Minkowski distance, etc.). Based on the difference in the distance between these points, there will be a distinction between near and far. So the k-nearest neighbor classifier determines the prediction principle based on these k data points that are "closest" to it. If there are more A-class data points than B-class data points among the data points closest to it, then it is considered to be a data point of class B with a high probability, and the classification is completed. The KNN algorithm process is as follows: Figure 4 As shown.

[0138] Key to the KNN algorithm

[0139] (1) All features of the sample must be quantized for comparability: If there are non-numerical types in the sample features, methods must be taken to quantize them into numerical values. For example, if the sample features include color, distance calculation can be achieved by converting the color to grayscale values.

[0140] (2) Sample features need to be normalized: A sample has multiple parameters, each with its own domain and range of values. They have different effects on distance calculation. For example, the influence of parameters with larger values ​​will outweigh that of parameters with smaller values. Therefore, sample parameters must be scaled. The simplest way is to normalize the values ​​of all features.

[0141] (3) Determining the value of K: A K value that is too large can easily lead to underfitting, while a K value that is too small can easily lead to overfitting. Cross-validation is required to determine the K value. The particle-sphere concept is as follows:

[0142] The human brain's perception of the environment always progresses from large to small, from coarse to fine. If a computer wants to simulate human thinking, it must first simulate the human brain's thought process. Granular computing is an efficient and scalable method that closely resembles the human brain's thinking. The concept of granular computing can effectively address the progression of computer processing from coarse to fine granular. Currently, granular computing has been extensively developed in various fields, incorporating the aforementioned granular-sphere concept based on the K-means clustering algorithm.

[0143] First, the center points are determined using the K-means clustering algorithm, and then the sphere radius is determined based on the cluster set. To reduce overly lenient clustering during sphere formation, this invention introduces DBSCAN (Density-Based Spatial Clustering of Applications with Noise), a density-based spatial clustering algorithm. This algorithm divides regions with sufficient density into clusters and discovers clusters of arbitrary shapes in a noisy spatial database, defining a cluster as the largest set of density-connected points. In the DBSCAN algorithm, data points are divided into three categories:

[0144] Key point: If sample x i If the ε-neighborhood of a sample point x contains at least minpts samples, then the sample point x is said to be a sample point. i The core point.

[0145] Boundary point: If sample x i If the number of samples contained in the ε-neighborhood is less than minpts, but it is within the neighborhood of other core points, then it becomes a sample point x. i These are boundary points.

[0146] Noise point: A point that is neither a core point nor a boundary point.

[0147] The DBSCAN algorithm has two key variables: the neighborhood radius eps (i.e., the generated ε-neighborhood) and the number of minimum points minpts. Accuracy can be scaled by controlling these two variables.

[0148] To better understand the formation process of density peak spheres, a demonstration diagram of density peak sphere formation is provided as follows: Figure 5 As shown. The concept of optimization difference:

[0149] Rough sets, proposed by Professor Pawlak in 1982, are a mathematical tool for data mining and knowledge discovery, and have been widely applied in cutting-edge research in the field of artificial intelligence in recent years. A major advantage of rough sets is that they require no prior knowledge; by simply learning their approach to handling uncertainties and imprecise data, they can solve many uncertainties and ambiguities in the field of artificial intelligence. Currently, rough sets have been successfully applied in fields such as artificial intelligence, text recognition, predictive analytics, machine recognition, pattern recognition, and image processing, achieving significant results.

[0150] With the development of rough sets, scholars around the world have generated many new ideas for solving uncertainty problems based on rough set concepts. Three-way decision theory was proposed by Professor Yao Yiyu of the University of Regina, Canada. Three-way decision theory is a cognitive approach derived from rough sets, simulating human cognition. Its core idea is to solve the problem separately by dividing the domain into three regions, with different decision strategies for each region. As a current hot topic in artificial intelligence research, the three-way decision model has achieved good validation in solving uncertainty problems. Compared to two-way decisions, which only have two outcomes (yes or no), three-way decision theory cannot be easily solved by simply choosing between two results when facing complex real-world problems. It introduces a non-committal decision as a third option when information is insufficient, refusing to judge whether to say yes or no. This makes three-way decision theory more flexible and more in line with human cognition when solving complex and highly uncertain problems.

[0151] Every research object possesses its inherent attributes. To better record and analyze these objects, each attribute corresponds to an attribute value. Attribute values ​​can be categorized into numerical and character attributes. Furthermore, attributes can be classified by the number of attributes as multi-attribute or single-attribute, but the conversion between multi-attribute and single-attribute attributes does not affect the number of decisions. Numerous scholars have studied the conversion between character and numerical attributes, such as attribute reduction and weighted comprehensive evaluation. To better determine the thresholds α and β for dividing the three regions in a three-way decision, the concept of optimization difference is introduced.

[0152] By introducing rough sets, three-way decision-making, and attributes, we can gain a good understanding of the problem-solving approach for uncertain problems. Based on three-way decision-making, this paper proposes the concept of optimization difference to address the problem of determining the value of K in the K-nearest neighbor classification algorithm. Let there be a set A... n ={a1,a2,a3,...,a n} and set A n-1 ={a1,a2,a3,...,a n-1}, where A n The extra element a in the set n Make A n-1 The dispersion of values ​​within the set changes, and this change can be understood as the dispersion of element a. n Add to collection A n-1 The optimization difference is determined by the magnitude of this influence in the three regions of the three-way decision.

[0153] Experimental Example 1

[0154] 1. Experimental Environment

[0155] JetBrains PyCharm 2022.1.4 x64 was used as the IDE for code development in the program's backend; Test platform CPU: AMD Ryzen 7 5800H 3.20GH Graphics card: NVIDIA T600 4GB Operating system: Microsoft Windows 11; The prediction results and time of the algorithm's execution were recorded.

[0156] 2. Experimental Data

[0157] (1) All data used are prediction classification data, and all come from the UCI dataset (https: / / archive.ics.uci.edu).

[0158] (2) To better reflect the comparison of the time efficiency of the algorithm under different data volumes, the data volumes of the selected datasets are arranged in an increasing order. Details are shown in Table 1.

[0159]

[0160]

[0161] Table 1: Experimental Dataset Information

[0162] 3. Experimental Analysis

[0163] To verify the effectiveness of the adaptive K value for optimizing the difference, the accuracy of 10-fold cross-validation for different datasets with different K values ​​was calculated. The K value with the best prediction performance was selected and experimentally analyzed with the GBSKNNN model, as shown in Table 2.

[0164]

[0165] Table 2: Different K values ​​for different datasets

[0166] To better compare the prediction performance of the GBSKNNS algorithm and the KNN algorithm, four machine learning evaluation functions are defined:

[0167] Assume the total amount of information in the information system is:

[0168] Total sample size = TP + FP + TN + FN

[0169] Where TP represents the relevant information quantity in the positive domain, FP represents the relevant information quantity in the incorrect prediction of the positive domain, TN represents the relevant information quantity in the negative domain, and so on, conversely, FN represents the relevant information quantity in the negative domain.

[0170] Accuracy: The percentage of correctly predicted information in an information system; in other words, it measures the proportion of correct predictions. Comparisons of accuracy across different datasets are also possible. Figure 6 As shown.

[0171] Accuracy = (TP + TN) / (FN + TP + TN + FP)

[0172] Precision, also known as accuracy, refers to the percentage of predicted relevant information in an information system that is truly relevant. It involves comparing the precision of different datasets. Figure 7 As shown.

[0173] Accuracy = TP / (FP + TP)

[0174] Recall, also known as recall, measures the percentage of total relevant information in an information system that is predicted to be relevant. Comparisons of recall rates across different datasets are also relevant. Figure 8 As shown.

[0175] Recall = TP / (FN + TP)

[0176] F1-Score: A harmonic metric combining precision and recall, it measures a model's ability to find truly relevant information, aiming for both precision and comprehensiveness in finding target information. Comparisons of F1-Scores across different datasets are shown below. Figure 9 As shown.

[0177] F1-Score = 2 * Precision * Recall / (Precision + Recall)

[0178] By comparing the two algorithms on different datasets across four machine learning evaluation metrics, it can be seen that GBSKNN maintains good prediction performance. The runtime required by the algorithm for each dataset is represented by the CPU computation time consumed for each dataset. The runtime of the KNN and GBSKNN models differs across different datasets. A comparison of the runtime of the two algorithms is provided below. Figure 10 As shown.

[0179] Analysis of the experimental results shows that, by comparing four evaluation metrics for machine learning—accuracy, precision, recall, F1 score, and runtime—the GBSKNN algorithm, while ensuring good prediction results, significantly reduces computation time compared to the KNN algorithm as the amount of data increases, achieving excellent results.

[0180] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0181] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A fast K-nearest neighbor classifier for large-scale image data, comprising a processing system storing the following processing algorithms, characterized in that: This paper proposes the concept of a particle sphere and introduces the DBSCAN method. It also proposes the concept of optimization difference and introduces an adaptive KNN algorithm for selecting the key K value. Specifically, the particle sphere concept and the optimization difference concept include a DBSCAN particle sphere generation algorithm based on density peaks and a density-based particle sphere GBSKNN algorithm based on optimization differences. The steps of the DBSCAN particle sphere generation algorithm based on density peaks are as follows: (1) Input: Dataset D, purity threshold p, neighborhood radius eps value, minimum number of points within the neighborhood radius minpts value; (2) Output: List of particles B; ① Use the 2-means clustering algorithm to divide the dataset D into two clusters, D1 and D2, with n=2 clusters; ②For each sphere Di; 1) The cluster center of sphere Di is the centroid of all sample points in that sphere; 2) The radius of the sphere Di is equal to the average distance from all sample points in the sphere to its centroid; 3) The purity of granules Di is equal to the percentage of the majority of the sample in the granules; 4) If the purity of the granules is less than p; a. Divide it into two spheres using the 2-means clustering algorithm, and increment the number of spheres n; ③If the purity of each sphere is higher than p; 1) Proceed to step 5; ④ Otherwise: Return to step ②; ⑤ For each Di; 1) Calculate the number of clusterable spheres k using the DBSCAN algorithm with parameters eps and minpts; 2) If the number of clusterable clusters k is greater than or equal to 2; a. Use the k-means clustering algorithm to further divide the spheres into k spheres, with the number of spheres n = n + k - 1; ⑥ If the number of clusters k for each sphere is less than 2; 1) Proceed to step ⑧; ⑦ Otherwise: Return to step ⑤; ⑧ Termination; The specific steps of the density-based sphere GBSKNN algorithm based on optimization difference are as follows: (1) Input: Dataset D, a query point O outside dataset D, and a list of particles B calculated by the DBSCAN particle generation algorithm based on density peaks; (2) Output: the label of O; ① Calculate the distance from point O to each ball in the ball list B and sort them to obtain the ascending distance list DIS; ②If the minimum distance DIS[0] is less than 0; 1) Then the label of point O is equal to the label of the ball closest to it, go to step 9; ③Otherwise: Initialize the distance list Dis to be calculated to [0, DIS[0]]; ④ Calculate the standard deviation of Dis, Std1; ⑤ Initialize the standard deviation list STD=[Std1]; ⑥ Let i = 1; ⑦ Add DIS[i] to Dis; ⑧ Calculate the current standard deviation of Dis (Stdi); 1) If Stdi <= Stdi-1; a. Add Stdi to STD, i++; b. Return to step ⑦; 2) Otherwise: Calculate the number of elements n in the current STD; a. Calculate the list of labels corresponding to the first n balls in the DIS list; b. Then the label of point O is equal to the label with the largest number of labels; 9. Termination.

2. The fast K-nearest neighbor classifier for large-scale image data according to claim 1, characterized in that: The granularization concept for large-scale data is defined as follows: Definition 1 Given a dataset A dot and a ball , As the center of the granule The radius of the grain; the center of the grain. It is the centroid of all sample points. For all sample points in the sphere The average distance; then we have: ； radius The main reason for defining it as the average distance rather than the maximum or minimum distance is to make the size of the sphere less susceptible to the influence of outlier samples. Definition 2 Given a dataset ,a little and a ball , As the center of the granule Let be the radius of the sphere; then point . To the ball The distance is defined as: ； Definition 3 Given a dataset ,a little and a ball , As the center of the granule The radius of the sphere; sphere The total label is defined as a ball. The labels of the majority of samples, i.e., the labels with the highest number of particles.

3. The fast K-nearest neighbor classifier for large-scale image data according to claim 1, characterized in that: The concept of optimization difference is defined for automatic K value selection as follows: Define assumptions , Given two sets of numbers; and They are respectively and The mean; and They are respectively and Standard deviation; for medium elements The optimization difference is defined as: 。 4. The fast K-nearest neighbor classifier for large-scale image data according to claim 3, characterized in that: The concept of optimization difference refers to the definition of automatic K value selection. Compare One more element ;when join in The middle and later are It may change The degree of dispersion, and this change is due to the newly added element; therefore, the optimization difference reflects the effect of adding a new element. When, it is called an element The addition produced a negative effect; when When, it is called an element The addition produced a positive effect; when When, it is called an element The addition of has no effect on the original set.

Images