Clustering methods, apparatus, equipment and storage media for data samples

Abstract

This application discloses a data sample clustering method, apparatus, device, and storage medium, belonging to the field of data processing technology. The data sample clustering method includes: sampling multiple data samples to obtain multiple representative samples, which are used to describe the distribution shape of the multiple data samples in a feature space, where the feature space is a virtual space indicating the data dimensions of the multiple data samples; adjusting the positions of the multiple representative samples in the feature space according to their relative positions; performing a first clustering on the adjusted representative samples to obtain multiple first data clusters; and performing a second clustering on the multiple data samples based on the multiple first data clusters. By sampling multiple data samples to obtain representative samples, and using these representative samples to characterize the distribution shape of the multiple data samples in the feature space, this application can significantly reduce the data scale that needs to be processed during the clustering process and significantly improve clustering efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus, device and storage medium for clustering data samples. Background Technology

[0002] With the improvement of data acquisition and processing capabilities and the iteration of information technology, big data has become a core resource in many fields. For example, aerospace big data collected through remote sensing and satellite communication technologies plays a crucial role in areas such as Earth observation and disaster early warning, providing precise data support for decision-making.

[0003] Big data inherently possesses a cluster structure, meaning that data clusters naturally form due to data similarity. Furthermore, influenced by the complex distribution of data, these cluster structures exhibit diverse shapes, blurred boundaries, and high dimensionality. Therefore, to accurately uncover the inherent distribution patterns of big data and extract valuable information, related technologies typically require clustering based on data similarity. This allows for the structured summarization of big data and clarifies the boundaries and affiliations of data clusters.

[0004] However, due to the massive scale of big data and the fact that it often includes some erroneous or redundant data, methods that directly cluster big data are computationally expensive and have high complexity and time costs. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for clustering data samples to solve the technical problems existing in related technologies. Specifically, it includes the following technical solutions.

[0006] In a first aspect, this application provides a data sample clustering method, the method comprising: sampling multiple data samples to obtain multiple representative samples, the multiple representative samples being used to describe the distribution shape of the multiple data samples in a feature space, the feature space being a virtual space indicating the data dimensions of the multiple data samples; adjusting the positions of the multiple representative samples in the feature space according to their relative positions in the feature space; performing a first clustering on the adjusted multiple representative samples to obtain multiple first data clusters; and performing a second clustering on the multiple data samples based on the multiple first data clusters.

[0007] In some possible implementations, the plurality of data samples includes selected samples added to a first set and unselected samples other than the selected samples; the sampling of the plurality of data samples to obtain the plurality of representative samples includes: determining a center point of the plurality of data samples, the center point being a virtual center of the distribution shape of the plurality of data samples in the feature space; determining a first data sample among the plurality of data samples based on a first Euclidean distance between the plurality of data samples and the center point, and adding the first data sample to the first set; obtaining a plurality of second Euclidean distances corresponding to the unselected samples, any one of the plurality of second Euclidean distances being the minimum of the third Euclidean distances from any data sample among the unselected samples to the first set; adding the data sample corresponding to a fourth Euclidean distance among the unselected samples to the first set, the fourth Euclidean distance being the maximum of the plurality of second Euclidean distances; if the number of set elements in the first set reaches a first specified number, then the set elements are the plurality of representative samples.

[0008] In some possible implementations, the proportion of the plurality of representative samples in the plurality of data samples is not less than a specified proportion threshold.

[0009] In some possible implementations, adjusting the positions of the multiple representative samples in the feature space according to their relative positions includes: adjusting the positions of the multiple representative samples in the feature space multiple times according to their relative positions until the number of adjustments reaches a specified number; wherein, the position change of the multiple representative samples in the feature space corresponding to any one of the multiple adjustments is less than a specified threshold.

[0010] In some possible implementations, the step of clustering the adjusted multiple representative samples to obtain multiple first data clusters includes: generating multiple second sets based on the adjusted multiple representative samples; merging the multiple second sets according to a first boundary similarity of the multiple second sets to obtain the multiple first data clusters; wherein the first boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the multiple second sets in the feature space.

[0011] In some possible implementations, merging the plurality of second sets based on the first boundary similarity of the plurality of second sets includes: merging the two sets with the highest first boundary similarity among the plurality of second sets; stopping the merging process if the number of merged plurality of second sets reaches the second specified number; and continuing to merge the two sets with the highest second boundary similarity among the merged plurality of second sets until the number of merged plurality of second sets reaches the second specified number; wherein the second boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space among the merged plurality of second sets.

[0012] In some possible implementations, the step of performing secondary clustering on the plurality of data samples based on the plurality of first data clusters includes: determining the boundary range of the distribution shape of any one of the plurality of first data clusters in the feature space; if the position of any one of the plurality of data samples in the feature space falls within the boundary range, then performing secondary clustering on any one of the data samples based on any one of the first data clusters.

[0013] Secondly, this application provides a data sample clustering apparatus, the apparatus comprising a sampling module, an adjustment module, and a clustering module; the sampling module is configured to sample multiple data samples to obtain multiple representative samples, the multiple representative samples being used to describe the distribution shape of the multiple data samples in a feature space, the feature space being a virtual space indicating the data dimensions of the multiple data samples; the adjustment module is configured to adjust the positions of the multiple representative samples in the feature space according to their relative positions in the feature space; the clustering module is configured to perform a first clustering on the adjusted multiple representative samples to obtain multiple first data clusters; the clustering module is further configured to perform a second clustering on the multiple data samples based on the multiple first data clusters.

[0014] In some possible implementations, the plurality of data samples includes selected samples added to a first set and unselected samples other than the selected samples; when the sampling module samples the plurality of data samples to obtain the plurality of representative samples, it is configured to: determine the center point of the plurality of data samples, the center point being a virtual center of the distribution shape of the plurality of data samples in the feature space; determine a first data sample among the plurality of data samples based on a first Euclidean distance between the plurality of data samples and the center point, and add the first data sample to the first set; obtain a plurality of second Euclidean distances corresponding to the unselected samples, any one of the plurality of second Euclidean distances being the minimum of the third Euclidean distances from any data sample among the unselected samples to the first set; add the data sample corresponding to a fourth Euclidean distance among the unselected samples to the first set, the fourth Euclidean distance being the maximum of the plurality of second Euclidean distances; when the number of set elements in the first set reaches a first specified number, the set elements are the plurality of representative samples.

[0015] In some possible implementations, the proportion of the plurality of representative samples in the plurality of data samples is not less than a specified proportion threshold.

[0016] In some possible implementations, when the adjustment module adjusts the positions of the multiple representative samples in the feature space according to their relative positions in the feature space, it is configured to: adjust the positions of the multiple representative samples in the feature space multiple times according to their relative positions until the number of adjustments reaches a specified number; wherein, the position change of the multiple representative samples in the feature space corresponding to any one of the multiple adjustments is less than a specified threshold.

[0017] In some possible implementations, when the clustering module performs a clustering operation on the adjusted multiple representative samples to obtain multiple first data clusters, it is configured to: generate multiple second sets based on the adjusted multiple representative samples; merge the multiple second sets according to a first boundary similarity of the multiple second sets to obtain the multiple first data clusters; wherein the first boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the multiple second sets in the feature space.

[0018] In some possible implementations, when the clustering module merges the plurality of second sets based on the first boundary similarity of the plurality of second sets, it is configured to: merge the two sets with the highest first boundary similarity among the plurality of second sets; if the number of the merged plurality of second sets reaches the second specified number, then stop merging; if the number of the merged plurality of second sets does not reach the second specified number, then continue to merge the two sets with the highest second boundary similarity among the merged plurality of second sets until the number of the merged plurality of second sets reaches the second specified number; wherein, the second boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space.

[0019] In some possible implementations, when the clustering module performs secondary clustering on the plurality of data samples based on the plurality of first data clusters, it is configured to: determine the boundary range of the distribution shape of any one of the plurality of first data clusters in the feature space; if the position of any one of the plurality of data samples in the feature space falls within the boundary range, then perform secondary clustering on any one of the data samples based on any one of the first data clusters.

[0020] Thirdly, this application provides an electronic device for clustering data samples, comprising: a memory storing at least one program instruction for clustering data samples; and a processor, wherein when the program instruction is executed by the processor, the device implements the method of the first aspect of this application or any possible embodiment of the first aspect.

[0021] Fourthly, this application provides a computer program (product) including computer program / instructions, which are executed by a processor to cause a device to implement the method of the first aspect of this application or any possible implementation of the first aspect.

[0022] Fifthly, this application provides a computer-readable storage medium having stored thereon program instructions for clustering data samples, which, when executed by one or more processors, cause a device to implement the method of the first aspect or any possible implementation of the first aspect of this application.

[0023] In a sixth aspect, this application provides a vehicle that includes the apparatus described in the second aspect of this application or any possible embodiment of the second aspect.

[0024] The beneficial effects of the technical solution provided in this application include at least the following: The technical solution provided in this application, on the one hand, obtains representative samples by sampling multiple original data samples, and uses these representative samples to characterize the distribution shape of the multiple data samples in the feature space. This helps to significantly reduce the data scale that needs to be processed during clustering, reduce the computational power consumption and time cost of subsequent clustering, and significantly improve clustering efficiency. On the other hand, adjusting the position of the multiple representative samples in the feature space based on their relative positions can correct sampling bias, optimize the uniformity of the distribution of the multiple representative samples, ensure that the multiple representative samples accurately restore the characteristics of the original data clusters, and guarantee the accuracy of clustering. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of an implementation scenario provided in the embodiments of this application; Figure 2 This is a flowchart of the clustering method for data samples provided in the embodiments of this application; Figure 3 This is a schematic diagram of the clustering results of different data samples provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the clustering device for data samples provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of an electronic device for clustering data samples provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] Figure 1 This is a schematic diagram of an implementation scenario provided in an embodiment of this application. (Reference) Figure 1 The implementation scenario provided in this application includes an onboard sensor 11 and a control unit 12. It should be noted that... Figure 1 The implementation scenarios shown are for illustrative purposes only and are not limiting. The solutions provided in this application can also be applied to any scenario that requires clustering of multiple data points in the collected data, such as Earth observation, disaster early warning, and weather monitoring.

[0030] The vehicle-mounted sensor 11 includes, for example, millimeter-wave radar, lidar, camera, ultrasonic radar, and other sensors installed in the vehicle for collecting data related to the vehicle's surrounding environment; gyroscope, accelerometer, GPS (global positioning system) module, vehicle speed sensor, and other sensors for collecting data related to the vehicle's motion characteristics; voltage sensor, current sensor, temperature sensor, and other sensors for collecting data related to the operating status of key functional modules such as the engine and power battery in the vehicle; or any other type of sensor for collecting vehicle-related data. This application makes no limitation in this regard.

[0031] The control unit 12 can be a single server, a server cluster consisting of multiple servers, or an in-vehicle terminal capable of performing control functions. The control unit 12 communicates with the in-vehicle sensor 11 via wired or wireless means, enabling the control unit 12 to perform clustering processing on multiple data samples from the data collected by the in-vehicle sensor 11.

[0032] Those skilled in the art should understand that the above-described vehicle sensor 11 and control unit 12 are merely examples. Other existing or future vehicle sensors or control units that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0033] Those skilled in the art should understand that the above-described vehicle 11, vehicle-mounted drone 12, and hangar 13 are merely examples. Other existing or future vehicles, vehicle-mounted drones, and hangars that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0034] Figure 2 This is a flowchart of a clustering method for data samples provided in an embodiment of this application. This method can, for example, be... Figure 1 The control unit shown performs the actions, and this application makes no limitations in this regard. See also Figure 2 The data sample clustering method provided in this application embodiment may include steps S210-S240.

[0035] Step S210: Sample multiple data samples to obtain multiple representative samples. The multiple representative samples are used to describe the distribution shape of the multiple data samples in the feature space. The feature space is a virtual space indicating the data dimensions of the multiple data samples.

[0036] For example, multiple data samples are used to indicate the feature set of a target object in different scenarios, and are used, but not limited to, to describe the feature performance of the target object in a specific scenario. The data dimension of any one of the multiple data samples can be one-dimensional data, meaning any data sample is data with a single data dimension, or it can be multi-dimensional data, meaning any data sample is data with multiple data dimensions. This application does not impose any restrictions here. The data dimension indicates the number of independent attributes describing the features of any data sample. Each dimension corresponds to an independent feature of any data sample. The more data dimensions a data sample has, the richer its information. For example, when the multiple data samples are data collected from multiple vehicles, any one data sample may be three-dimensional data formed by brand, price, and friendliness dimensions; when the multiple data samples are aerospace big data collected from remote sensing pixel samples, any one data sample may be five-dimensional data formed by longitude, latitude, elevation, shooting time, and spectral reflectance dimensions.

[0037] The feature space is a virtual space indicating the data dimensions of multiple data samples. It's an abstract space constructed using each independent feature of the data samples as a coordinate axis. Any data sample is mapped to a point in the feature space. It's used, but not limited to, to intuitively characterize the feature distribution patterns of multiple data samples and to measure the similarity and differences between them. The data dimensions of the multiple data samples can be the same or different. When the data dimensions are the same, the multiple data samples are directly mapped to the same feature space, and the positional relationship and distance between them directly reflect feature similarity. When the data dimensions are different, preprocessing methods such as feature mapping and dimensionality increase / decrease alignment are needed to map the multiple data samples to a unified feature space before analysis.

[0038] The distribution shape of multiple data samples in the feature space is used to indicate the characteristics of the original data clusters, such as clustering patterns, boundaries of the original data clusters, and overall distribution characteristics. Representative samples are typical data samples selected from the multiple data samples that can characterize the features of the original data clusters. They can be used, but are not limited to, to replace multiple data samples in cluster analysis and to reconstruct the distribution shape of multiple data samples in the feature space. By using multiple representative samples to describe the distribution shape of multiple data samples in the feature space, the embodiments of this application can significantly reduce the data processing scale, reduce computing power consumption and time costs, and improve the clustering efficiency of multiple data samples while preserving the characteristics of the original data clusters.

[0039] In some embodiments, the method of sampling multiple data samples to obtain multiple representative samples includes: sampling the multiple data samples using a uniform continuous sampling method to obtain the multiple representative samples. The uniform continuous sampling method refers to a sampling method that continuously and uniformly selects samples from the original data sample sequence according to a preset sampling interval or sampling ratio (e.g., traversing the data sample set with a fixed step size, selecting one sample every N samples as a representative sample, or uniformly extracting samples from different distribution areas of the data samples according to a fixed ratio). Through uniform continuous sampling, this embodiment can ensure that the representative samples uniformly cover the distribution range of the original data samples in the feature space, avoiding the loss of local features due to sampling bias, while effectively controlling the scale of the number of representative samples. While preserving the overall distribution trend of the original data, it reduces the complexity of subsequent clustering calculations and improves processing efficiency. Furthermore, the rules of uniform continuous sampling are simple and easy to execute, adaptable to different types of data sample sets (such as time-series data from vehicle sensors, spatial distribution data of aerospace big data), and have strong versatility.

[0040] The following will further illustrate the method of sampling multiple data samples to obtain multiple representative samples in the embodiments of this application, using an exemplary sampling method.

[0041] In some embodiments, the plurality of data samples includes selected samples added to a first set and unselected samples other than the selected samples, the first set being an empty set used for sampling the plurality of data samples.

[0042] In this case, the method of sampling multiple data samples to obtain multiple representative samples may include steps S211-S213 as shown below. Step S211: Determine the center point of multiple data samples. The center point is the virtual center of the distribution shape of the multiple data samples in the feature space.

[0043] For example, multiple data samples X are represented as X = {X1, X2, X3, ..., Xn}, and any data sample Xi is represented as Xi = {Xi1, Xi2, Xi3, ..., Xid}. Here, n is the number of data samples in the multiple data samples, and d is the data dimension of any data sample.

[0044] Based on this, the center point X of multiple data samples mean ={X mean，1 X mean，2 , ..., X mean，d The method for determining} can refer to the following formula (1): (1) in, Center point The value taken in the j-th data dimension, It represents the maximum value in the set of values ​​for all data samples in the j-th data dimension among multiple data samples X; It represents the minimum value in the set of values ​​for all data samples in the j-th data dimension among multiple data samples X.

[0045] Step S212: Determine the first data sample among the multiple data samples based on the first Euclidean distance between the multiple data samples and the center point, and add the first data sample to the first set.

[0046] For example, the first data sample is, for instance, one of multiple data samples X that is closest to the center point. The first Euclidean distance between them is dis(Xi, The data sample corresponding to the minimum value in (). After adding the first data sample to the first set C, the first set is no longer empty.

[0047] Step S213: Obtain multiple second Euclidean distances corresponding to the unselected samples; add the data samples corresponding to the fourth Euclidean distances among the unselected samples to the first set; if the number of elements in the first set reaches a first specified number, then the set elements are multiple representative samples. Wherein, any one of the multiple second Euclidean distances is the minimum of the third Euclidean distances from any data sample in the unselected samples to the first set, and the fourth Euclidean distance is the maximum of the multiple second Euclidean distances. The first specified number is the same as the number of multiple representative samples.

[0048] It should be noted that step S213 is a loop process, as follows: a) For any data sample in the unselected sample set, calculate the Euclidean distance between this data sample and all existing data samples in the first set, obtaining multiple second Euclidean distances for this data sample; b) For each unselected sample, select the minimum value from its multiple second Euclidean distances as the shortest distance from this sample to the first set; c) From the unselected sample set, select the data sample with the largest shortest distance, record its corresponding shortest distance as the fourth Euclidean distance, and add this data sample to the first set; d) Remove the data sample added to the first set from the unselected sample set; e) Determine whether the number of elements in the first set has reached a first specified number. If it has, exit the loop; if not, return to step a and continue executing the loop. Based on this, when the number of elements in the first set reaches the first specified number, the loop ends, and the elements in the first set are the multiple representative samples selected.

[0049] For example, the unselected sample XC is the set of data samples from multiple data samples X that have not been added to the first set C, i.e. For example, when the first set C is empty, the unselected sample XC = multiple data samples X; when the first data sample is added to the first set C, the unselected sample XC becomes the set of multiple data samples X excluding the first data sample.

[0050] The third Euclidean distance indicates the Euclidean distance between any data sample Xi in XC and all data sample points in the first set C. Any one of the multiple second Euclidean distances is the minimum of the third Euclidean distance, denoted as dismin(Xi, C). The fourth Euclidean distance is the minimum of the multiple second Euclidean distances, i.e., the minimum of multiple dismin(Xi, C). After sampling stops, the elements of the first set C are the multiple representative samples; at this point, the first set C equals the multiple representative samples.

[0051] In some embodiments, the calculation method for any second Euclidean distance can refer to the following formula (2): (2) in, This represents the value of the t-th dimension of the i-th data sample Xi in the unselected sample XC; Let Xc represent the value of the t-th dimension of the c-th data sample Xc in the first set C.

[0052] In the above method, by using the center point of multiple data samples as the sampling starting point, it can be ensured that the sampling range starts from the core region of the distribution of multiple data samples in the feature space, avoiding the initial sampling from deviating from the main features of the data clusters of multiple data samples. Then, the data sample with the closest first Euclidean distance to the center point among the multiple data samples is used as the initial set element of the first set. The data samples added to the first set are determined by the multiple second Euclidean distances and the fourth Euclidean distance corresponding to the unselected samples. This can ensure that the first set always contains the key feature points of the distribution of multiple data samples, such as the core points and boundary points of the data clusters, while gradually expanding the coverage of the first set. At the same time, it avoids the repeated selection of redundant data samples, so that the final multiple representative samples can highly restore the distribution shape of multiple data samples in the feature space.

[0053] In some embodiments, the proportion of multiple representative samples in multiple data samples is not less than a specified proportion threshold, so as to avoid the loss of shape features of data clusters due to insufficient sample quantity. The value of the specified proportion threshold can be adjusted according to the actual application scenario, and this application does not impose any restrictions in this regard.

[0054] Step S220: Adjust the positions of multiple representative samples in the feature space according to their relative positions in the feature space.

[0055] For example, the relative positions of multiple representative samples in the feature space are used to indicate the spatial relationship, distance, and density of distribution among the representative samples. For instance, some representative samples may cluster together to form local clusters, while others may be discretely distributed. By adjusting the relative positions of multiple representative samples in the feature space, this embodiment of the application can optimize the distribution shape of multiple representative samples, eliminate local density or sparsity problems caused by sampling bias, and make multiple representative samples more uniformly fit the contour features of the original data clusters of multiple data samples. At the same time, it enhances the ability of multiple representative samples to characterize the data boundaries of multiple data samples, thereby improving the accuracy of subsequent clustering analysis.

[0056] In some embodiments, adjusting the positions of multiple representative samples in the feature space based on their relative positions includes: adjusting the positions of the multiple representative samples in the feature space multiple times based on their relative positions until the number of adjustments reaches a specified number. The change in the position of the multiple representative samples in the feature space corresponding to any one of the adjustments is less than a specified threshold, to avoid over-adjustment causing the multiple representative samples to deviate from the characteristics of the original data cluster, or insufficient adjustment leaving residual sampling bias, thus failing to accurately restore the distribution shape and boundaries of the original data cluster. The specific values ​​of the specified number of adjustments and the specified threshold can be adjusted according to the actual application scenario, and this application does not impose any restrictions in this regard.

[0057] Optionally, any representative sample X from the plurality of representative samples C i After T adjustments, it becomes X. i T The sample obtained after adjusting each of the multiple representative samples C T times, i.e., the adjusted multiple representative samples, is denoted as CT. X i T The determination method is shown in the following formula (3): (3) in, Let C be any one of the multiple representative samples. Let T be any representative sample after T adjustments. Let α be any representative sample after T-1 adjustments, and let α be the magnitude of the position change of any representative sample in the feature space during each of the T adjustments. For any representative sample among multiple representative samples C and another representative sample The Euclidean distance between them As a representative sample Any sample in the neighborhood of any representative sample. The neighborhood is defined as the set of k representative samples whose Euclidean distance to any representative sample is less than a specific threshold.

[0058] Step S230: Perform a clustering operation on the adjusted representative samples to obtain multiple first data clusters.

[0059] As mentioned above, multiple representative samples can be used, but are not limited to, to characterize the features of the original data cluster. In this case, by performing a clustering operation on the adjusted multiple representative samples, the resulting multiple first data clusters can accurately map the features of the original data cluster in the feature space of the multiple data samples, and fully preserve the shape features, boundary information and distribution patterns of the original data cluster.

[0060] The number of primary data clusters can be determined based on the desired number of categories into which the data samples are classified after secondary clustering. For example, if after secondary clustering of multiple data samples, it is desired that the data samples are classified into n categories, then the number of primary data clusters is n.

[0061] In some embodiments, clustering the adjusted representative samples to obtain multiple first data clusters includes: generating multiple second sets based on the adjusted representative samples; and merging the multiple second sets according to a first boundary similarity to obtain multiple first data clusters. The first boundary similarity indicates the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space.

[0062] For example, the set of multiple second sets is denoted as CTS, and any element in CTS is any second set among the multiple second sets. The boundary similarity between any two elements in CTS, that is, the boundary similarity between any two elements in the multiple second sets, is denoted as sim(CTS). i CTS j Considering that higher boundary similarity indicates stronger overlap or proximity of the boundary regions of two second sets in the feature space, their corresponding data clusters already belong to the same continuous cluster structure in the original distribution. Therefore, when merging multiple second set elements, the two second set elements with the highest boundary similarity can be merged to gradually restore the complete cluster structure of the original data clusters, avoiding cluster splitting caused by a single clustering step, while ensuring that the merged clusters accurately reflect the original distribution of the original data clusters in the feature space.

[0063] In this case, the multiple second sets are merged based on the first boundary similarity, including: merging the two sets with the highest first boundary similarity among the multiple second sets; if the number of merged multiple second sets reaches a second specified number, the merging stops; if the number of merged multiple second sets does not reach the second specified number, the two sets with the highest second boundary similarity among the merged multiple second sets are merged again until the number of merged multiple second sets reaches the second specified number; wherein, the second boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space among the merged multiple second sets.

[0064] Step S240: Perform secondary clustering on multiple data samples based on multiple first data clusters.

[0065] As mentioned earlier, multiple first data clusters can accurately map the features of the original data clusters in the feature space for multiple data samples. Therefore, the multiple first data clusters obtained by clustering representative samples once can be directly used as simplified representations of the original data clusters. By matching each of the multiple data samples to the corresponding first data clusters, the clustering of multiple data samples can be completed quickly, significantly reducing the computational cost of clustering multiple data samples, while ensuring that the clustering results retain the features of the original data clusters.

[0066] Based on this, a method for secondary clustering of multiple data samples according to multiple first data clusters includes, for example: determining the boundary range of the distribution shape of any first data cluster in the feature space; if the position of any data sample in the feature space falls within the boundary range, then secondary clustering is performed on any data sample according to any first data cluster.

[0067] For example, multiple first data clusters are mapped to n different regions in the feature space. This means that the feature space, indicating the data dimensions of multiple data samples, is divided into n different regions, each with distinct boundary ranges. Different data labels are assigned to each of the n boundary ranges, resulting in a total of n data labels. If the position of any data sample in the feature space falls within a certain boundary range, that data sample is labeled with the corresponding data label. This process continues until all data samples in the multiple data samples have been labeled with their corresponding data labels. At this point, the multiple data samples are divided into n classes, completing the secondary clustering.

[0068] Figure 3 This is a schematic diagram of the clustering results of different data samples provided in the embodiments of this application.

[0069] Table 1 shows data samples where at least one of the following four data dimensions, the number of data samples, and the number of categories after secondary clustering are different, as provided in the embodiments of this application. The results obtained after clustering using the clustering method for the data samples provided in the embodiments of this application are as follows: Figure 3 As shown.

[0070] Table 1

[0071] Depend on Figure 3 It is known that, for data samples with different data dimensions, sample size, and number of categories after secondary clustering, the clustering method for data samples provided in this application can maintain the characteristics of the original data clusters after clustering the data samples. It has high clustering efficiency, as well as good clustering accuracy and robustness.

[0072] The technical solution provided in this application, on the one hand, obtains representative samples by sampling multiple original data samples, and uses these representative samples to characterize the distribution shape of the multiple data samples in the feature space. This helps to significantly reduce the data scale that needs to be processed during clustering, reduce the computational power consumption and time cost of subsequent clustering, and significantly improve clustering efficiency. On the other hand, adjusting the position of the multiple representative samples in the feature space based on their relative positions can correct sampling bias, optimize the uniformity of the distribution of the multiple representative samples, ensure that the multiple representative samples accurately restore the characteristics of the original data clusters, and guarantee the accuracy of clustering.

[0073] In some other possible implementations, this application also provides a clustering apparatus for data samples. Figure 4 This is a schematic diagram of the structure of a clustering device for data samples provided in an embodiment of this application. (Reference) Figure 4 The data sample clustering device provided in this application embodiment includes a sampling module 410, an adjustment module 420, and a clustering module 430.

[0074] The sampling module 410 is configured to sample multiple data samples to obtain multiple representative samples. The multiple representative samples are used to describe the distribution shape of the multiple data samples in the feature space, which is a virtual space indicating the data dimensions of the multiple data samples.

[0075] The adjustment module 420 is configured to adjust the position of multiple representative samples in the feature space based on their relative positions in the feature space.

[0076] Clustering module 430 is configured to perform a single clustering of the adjusted multiple representative samples to obtain multiple first data clusters.

[0077] Clustering module 430 is also configured to perform secondary clustering of multiple data samples based on multiple first data clusters.

[0078] In some possible implementations, the multiple data samples include selected samples added to the first set and unselected samples other than the selected samples; when sampling the multiple data samples to obtain multiple representative samples, the sampling module 410 is configured to: determine the center point of the multiple data samples, the center point being a virtual center of the distribution shape of the multiple data samples in the feature space; determine a first data sample among the multiple data samples based on a first Euclidean distance between the multiple data samples and the center point, and add the first data sample to the first set; obtain multiple second Euclidean distances corresponding to the unselected samples, any one of the multiple second Euclidean distances being the minimum of the third Euclidean distances from any data sample in the unselected samples to the first set; add the data sample corresponding to a fourth Euclidean distance among the unselected samples to the first set, the fourth Euclidean distance being the maximum of the multiple second Euclidean distances; if the number of set elements in the first set reaches a first specified number, then the set elements are multiple representative samples.

[0079] In some possible implementations, the proportion of multiple representative samples in multiple data samples is not less than a specified proportion threshold.

[0080] In some possible implementations, when adjusting the positions of multiple representative samples in the feature space according to their relative positions, the adjustment module 420 is configured to: adjust the positions of multiple representative samples in the feature space multiple times according to their relative positions until the number of adjustments reaches a specified number; wherein, the change in the position of the multiple representative samples in the feature space corresponding to any one of the multiple adjustments is less than a specified threshold.

[0081] In some possible implementations, when clustering the adjusted multiple representative samples to obtain multiple first data clusters, the clustering module 430 is configured to: generate multiple second sets based on the adjusted multiple representative samples; merge the multiple second sets according to the first boundary similarity of the multiple second sets to obtain multiple first data clusters; wherein the first boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space.

[0082] In some possible implementations, when the clustering module 430 merges multiple second sets based on the first boundary similarity of the multiple second sets, it is configured to: merge the two sets with the highest first boundary similarity among the multiple second sets; if the number of merged multiple second sets reaches a second specified number, then stop merging; if the number of merged multiple second sets does not reach the second specified number, then continue to merge the two sets with the highest second boundary similarity among the merged multiple second sets until the number of merged multiple second sets reaches the second specified number; wherein, the second boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space among the merged multiple second sets.

[0083] In some possible implementations, when performing secondary clustering on multiple data samples based on multiple first data clusters, the clustering module 430 is configured to: determine the boundary range of the distribution shape of any one of the multiple first data clusters in the feature space; if the position of any one of the multiple data samples in the feature space falls within the boundary range, then perform secondary clustering on any one of the data samples based on any one of the first data clusters.

[0084] It should be understood that the data sample clustering device and the data sample clustering method provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the data sample clustering method embodiments.

[0085] In some other possible implementations, this application also provides an electronic device for clustering data samples. Figure 5 This is a schematic diagram of the structure of an electronic device for clustering data samples provided in an embodiment of this application. See also... Figure 5 The electronic device for clustering data samples provided in this application includes the following structure.

[0086] Memory 510 stores at least one program instruction for clustering data samples. Processor 520 executes the aforementioned program instruction, causing the device to perform the above-described combination. Figure 2The steps of the described method and its various embodiments are described below. Depending on the implementation, the processor 420 may be one or more types of processors, including but not limited to DSP (digital signal processor), ASIC (application-specific integrated circuit), FPGA (field-programmable gate array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and the number of such devices may be determined according to actual needs.

[0087] In some other possible implementations, this application also provides a computer program (product) comprising computer programs / instructions, which are executed by a processor to cause the device to perform the above-described combination. Figure 2 The steps of the described method and its various embodiments.

[0088] In some other possible implementations, this application also provides a computer-readable storage medium having program instructions stored thereon for clustering data samples, wherein when the program instructions are executed by one or more processors, the device performs the above-described combination. Figure 2 The steps of the described method and its various embodiments are described. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0089] In some other possible implementations, this application also provides a vehicle, the vehicle including Figure 4 The apparatus described in several embodiments thereof.

[0090] It should also be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0091] The term "and / or" in the embodiments of this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0092] The above description is only for the purpose of enabling those skilled in the art to understand the technical solution of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application shall be included within the scope of protection of this application.

Claims

1. A clustering method for data samples, characterized in that, The method includes: Multiple data samples are sampled to obtain multiple representative samples, which are used to describe the distribution shape of the multiple data samples in a feature space, where the feature space is a virtual space indicating the data dimensions of the multiple data samples; The positions of the multiple representative samples in the feature space are adjusted according to their relative positions in the feature space; The adjusted representative samples are clustered once to obtain multiple first data clusters; The multiple data samples are then clustered a second time based on the multiple first data clusters.

2. The method according to claim 1, characterized in that, The plurality of data samples includes selected samples added to the first set and unselected samples other than the selected samples; The sampling of the plurality of data samples to obtain the plurality of representative samples includes: Determine the center point of the plurality of data samples, wherein the center point is a virtual center of the distribution shape of the plurality of data samples in the feature space; A first data sample is determined from the plurality of data samples based on the first Euclidean distance between the plurality of data samples and the center point, and the first data sample is added to the first set; Obtain multiple second Euclidean distances corresponding to the unselected samples, where any one of the multiple second Euclidean distances is the minimum of the third Euclidean distances from any data sample in the unselected samples to the first set; Add the data sample corresponding to the fourth Euclidean distance in the unselected samples to the first set, where the fourth Euclidean distance is the maximum value among the plurality of second Euclidean distances; If the number of elements in the first set reaches a first specified number, then the elements in the set are the plurality of representative samples.

3. The method according to claim 1, characterized in that, The proportion of the multiple representative samples in the multiple data samples is not less than a specified proportion threshold.

4. The method according to claim 1, characterized in that, The step of adjusting the positions of the multiple representative samples in the feature space according to their relative positions in the feature space includes: The positions of the multiple representative samples in the feature space are adjusted multiple times according to the relative positions until the number of adjustments reaches a specified number. Wherein, the position change of the multiple representative samples in the feature space corresponding to any one of the multiple adjustments is less than a specified threshold.

5. The method according to claim 1, characterized in that, The process of clustering the adjusted representative samples to obtain multiple first data clusters includes: Multiple second sets are generated based on the adjusted representative samples; The multiple second sets are merged based on the first boundary similarity of the multiple second sets to obtain the multiple first data clusters; Wherein, the first boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space among the plurality of second sets.

6. The method according to claim 5, characterized in that, The step of merging the multiple second sets based on the first boundary similarity of the multiple second sets includes: Merge the two sets with the highest first boundary similarity among the plurality of second sets; If the number of merged second sets reaches the second specified number, then the merging process stops; If the number of the merged second sets does not reach the second specified number, then continue to merge the two sets with the highest second boundary similarity among the merged second sets until the number of the merged second sets reaches the second specified number. The second boundary similarity is used to indicate the degree of fit between the boundaries of the distribution shapes of any two sets in the feature space of the merged plurality of second sets.

7. The method according to claim 1, characterized in that, The step of performing secondary clustering on the multiple data samples based on the multiple first data clusters includes: Determine the boundary range of the distribution shape of any one of the plurality of first data clusters in the feature space; If the position of any data sample among the plurality of data samples falls within the boundary range in the feature space, then a secondary clustering is performed on any data sample based on any first data cluster.

8. A data sample clustering device, characterized in that, The device includes a sampling module, an adjustment module, and a clustering module; The sampling module is configured to sample multiple data samples to obtain multiple representative samples, which are used to describe the distribution shape of the multiple data samples in a feature space, where the feature space is a virtual space indicating the data dimensions of the multiple data samples. The adjustment module is configured to adjust the position of the plurality of representative samples in the feature space according to their relative positions in the feature space; The clustering module is configured to perform a single clustering of the adjusted representative samples to obtain multiple first data clusters; The clustering module is further configured to perform secondary clustering on the plurality of data samples based on the plurality of first data clusters.

9. An electronic device, characterized in that, include: A memory, wherein program instructions for clustering data samples are stored; as well as, A processor, when the program instructions are executed by the processor, causes the vehicle to perform the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores program instructions for clustering data samples, which, when executed by one or more processors, cause the device to implement the method of any one of claims 1-7.

Images