A method and system for visualizing a sample
By modeling the similarity relationships of samples in a circular packaging layout, a compact and non-overlapping circular packaging layout is generated, which solves the problem of insufficient sample similarity perception in the prior art and realizes more effective data visualization analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2023-07-10
- Publication Date
- 2026-06-09
Smart Images
Figure CN116701945B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data visualization technology, and specifically proposes a method and system for visualizing samples. Background Technology
[0002] Data visualization can present abstract data in an intuitive way, helping users understand patterns in the data and discover potential problems, thereby improving the efficiency of data analysis. For example, by mapping high-dimensional data to a low-dimensional space and displaying it as a scatter plot, users can intuitively observe the distribution of the data, understand the classification of the data, and more easily find misclassified points in the data, which are called outliers. However, scatter plots suffer from problems such as over-drawing and overlapping visual labels, which are not conducive to users' observation and analysis of samples of interest. Circular wrapping is a commonly used display format in data visualization, with advantages such as simple and easy-to-understand visual encoding and aesthetically pleasing layout. In circular wrapping, each sample is represented as a circle, the size (radius) of which usually encodes certain numerical attributes in the data (such as the importance of the sample), and the color of the circle indicates its initial classification category. These circles of different sizes are tightly wrapped together. Because there is no overlap between different circles in circular wrapping, it can be used to conveniently perform tasks that require exploring samples, such as correcting misclassifications in image data. Summary of the Invention
[0003] The following description includes exemplary methods, systems, techniques, and sequences of instructions embodying the techniques of this invention. However, it should be understood that the described invention can be practiced in one or more aspects without these specific details. In other instances, well-known protocols, structures, and techniques have not been shown in detail so as not to obscure the invention. Those skilled in the art will understand that the described techniques and mechanisms can be applied to various architectures for ordering values.
[0004] According to one aspect of the present invention, a system for visualizing samples is proposed, comprising: a receiving module configured to receive a plurality of samples, wherein information of each sample in the plurality of samples includes the importance of the sample, a feature vector corresponding to the sample, and an initial classification category of the sample; a projection module configured to obtain, based on the feature vector corresponding to each sample in the plurality of samples, a respective projection point of each sample in the plurality of samples on a two-dimensional plane; a planar plot generation module configured to obtain an initial planar plot constructed by the respective projection points of each sample in the plurality of samples on the two-dimensional plane; and a sub-region generation module configured to, based on the feature vector of each sample in the plurality of samples... The system obtains multiple sub-regions on the two-dimensional plane based on the projection points of each sample and the initial planar diagram, wherein each sub-region corresponds to each sample among the multiple samples; a circular packaging layout generation module is configured to obtain a circle corresponding to each sample on the two-dimensional plane based on the obtained multiple sub-regions and the importance and initial classification of each sample among the multiple samples, wherein the parameters of each circle include the center, radius, and color; and a visualization module is configured to visualize the multiple samples on the two-dimensional plane based on the corresponding circles of each sample among the multiple samples.
[0005] According to another aspect of the present invention, a method for visualizing samples is proposed, comprising: receiving a plurality of samples, wherein information of each sample includes the importance of the sample, a feature vector corresponding to the sample, and an initial classification category of the sample; obtaining a projection point of each sample on a two-dimensional plane based on the feature vector corresponding to each sample; obtaining an initial planar map constructed by the projection points of each sample on the two-dimensional plane; obtaining a plurality of sub-regions on the two-dimensional plane based on the projection points of each sample and the initial planar map, wherein each sub-region corresponds to each sample; obtaining a circle corresponding to each sample on the two-dimensional plane based on the obtained sub-regions, the importance of each sample, and the initial classification category, wherein parameters of each circle include a center, a radius, and a color; and visualizing the plurality of samples on the two-dimensional plane based on the circles corresponding to each sample.
[0006] According to another aspect of the invention, a computer-readable storage medium for visualizing samples is provided, the computer-readable storage medium having program instructions stored therein, the program instructions being executable by a computing device to cause the computing device to perform the method described above. Attached Figure Description
[0007] The invention itself, its preferred modes of use, objectives, features, and advantages can be better understood by referring to the following detailed description of illustrative embodiments, in which:
[0008] Figure 1 A shows 7 samples from a dataset of clothing images;
[0009] Figure 1 B shows Figure 1 The results of visualizing the circular packaging layout of the 7 samples in A;
[0010] Figure 2 The visualization results of circular packaging layouts for several image samples using existing methods are shown;
[0011] Figure 3 A structural block diagram of a system for visualizing samples according to one or more embodiments of the present invention is shown;
[0012] Figure 4 A shows the projection points corresponding to the five samples projected onto the two-dimensional plane;
[0013] Figure 4 B shows Figure 4 The initial plan view result established by the projection points in A;
[0014] Figure 5 A flowchart illustrating the division of a convex hull region into multiple sub-regions according to one or more embodiments of the present invention is shown;
[0015] Figure 6 A-6D shows that Figure 4 The process of dividing the convex hull region of B into multiple sub-regions;
[0016] Figure 7 A-7C shows the results obtained Figure 6 The process of obtaining the circle corresponding to each sample on a two-dimensional plane from multiple sub-regions of D;
[0017] Figure 8 A flowchart of a method for visualizing a sample according to one or more embodiments of the present invention is shown;
[0018] Figure 9 The diagram shows an interface for visualizing an image dataset containing outliers in a visualization system developed based on the method of this invention. Detailed Implementation
[0019] Embodiments of the present invention will now be described with reference to the accompanying drawings. Numerous specific details are set forth in the following description to provide a more complete understanding of the invention. However, it will be apparent to those skilled in the art that implementations of the invention may not include some of these specific details. Furthermore, it should be understood that the invention is not limited to the specific embodiments described. Rather, the invention can be practiced with any combination of the following features and elements, regardless of whether they relate to different embodiments. Moreover, the steps of the method are not limited to the described order, and the order of many steps can be adjusted. Therefore, the following aspects, features, embodiments, and advantages are for illustrative purposes only and should not be construed as elements or limitations of the appended claims unless expressly set forth in the claims.
[0020] With the development of network technology, big data research has become a hot topic, and the demand for big data classification and in-depth analysis is increasing. Data visualization can present abstract data in an intuitive way, helping users understand patterns in the data and discover potential problems, thereby improving the efficiency of data analysis. Circular wrapping is a commonly used display format in data visualization, with advantages such as simple and easy-to-understand visual encoding and aesthetically pleasing layout. In circular wrapping, each sample is represented as a circle. The size (radius) of the circle usually encodes certain numerical attributes in the data (such as the importance of the sample), and the color of the circle indicates its initial classification category. These circles of different sizes are tightly wrapped together. Because there is no overlap between different circles in circular wrapping, it can be used to conveniently perform sample exploration tasks, such as correcting mislabeled classes in image data and finding outliers.
[0021] For example, Figure 1 Image A shows seven samples from a clothing image dataset, where samples 101-107 represent shirt images. Each sample corresponds to a feature set (also called a feature vector) and an initial classification category (this initial classification category can be manually labeled or obtained by other methods, and is not necessarily obtained by classifying based on the feature vector corresponding to that sample). The initial classification category of sample 101 is jacket, and the initial classification category of samples 102-107 is shirt. However, if classified according to the feature vector of samples 101-107, the classification category of samples 101-107 should all be shirt. Figure 1 B shows Figure 1 The results of visualizing the 7 samples in A using their feature vectors through a circular wrapping layout are as follows: Figure 1 The circles corresponding to samples 101-107 in A are respectively Figure 1 Circles 110-170 in B. Circle 110 has a different color from circles 120-170 (color indicates category), making it an outlier and indicating an initial classification error.
[0022] because Figure 1 In a small sample size, the position of the circle has little impact on user perception; users can easily identify outliers through color differences. However, when a large number of image samples need to be visualized, the position of the circle has a significant impact on user perception. Figure 2 This document presents a visualization of circular wrapping layouts for several image samples using existing methods. Figure 2 In this visualization, the grayscale value of the circle represents its color, indicating the initial classification category of the image. It can be seen that users find it difficult to perceive the similarity between samples and their corresponding classifications (the classification categories calculated based on feature vectors) in this visualization, making it difficult to identify outliers. The inventors of this invention, through analysis, believe that the main reason is that the circular packaging layout does not consider preserving the similarity between samples (i.e., the similarity between the feature vectors of the samples). This results in similar samples appearing irregularly in the circular packaging layout visualization, making it difficult to discern the classification within the samples and to identify outliers.
[0023] The inventors of this invention believe that in circular packaging layouts, the similarity between data corresponding to the circles can be conveyed through the adjacency relationships between the circles. In existing technologies, the modeling of proximity relationships between data is not considered. Therefore, during the subsequent generation of sub-regions and circular packaging layouts, the proximity relationships between the sub-regions corresponding to each data point and the circles corresponding to each data point are not maintained. This results in circles representing similar data potentially not being placed in adjacent positions in the final generated circular packaging layout, disrupting the user's perception of sample similarity and hindering the user's analysis of clustering structures and outliers in the data. This invention proposes a method for visualizing circular packaging layouts of multiple samples. This method models the data and the proximity relationships between data as nodes and edges in a planar graph. During the subsequent generation of sub-regions and circular packaging layouts, it strives to maintain the proximity relationships between nodes in the planar graph. Based on the information from the planar graph, circles representing similar data are placed in adjacent positions, thereby helping users perceive the similarity of samples. Furthermore, to generate aesthetically pleasing and effective circular packaging layouts to aid analysis, this method simultaneously considers generating circular packaging layouts with good compactness, non-overlapping properties, overall convexity, and clustering convexity. The method of this invention can help users visually analyze the clustering structure and outliers in samples, which is beneficial for users to analyze mislabeled and difficult-to-distinguish samples in data (including but not limited to images, videos, tables, documents, etc.).
[0024] This invention proposes a system and method for visualizing the circular layout of multiple samples. The system and method place circles representing similar data in adjacent positions to help users perceive the similarity of samples. Figure 3 A structural block diagram of a system 300 for visualizing samples according to one or more embodiments of the present invention is shown.
[0025] like Figure 3 As shown, system 300 includes a receiving module 310, a projection module 320, a plan view generation module 330, a sub-region generation module 340, a circular package layout generation module 350, and a visualization module 360.
[0026] exist Figure 3 In this embodiment, the receiving module 310 is configured to receive multiple samples 301. The information for each sample 301 includes its importance, its corresponding feature vector, and its initial classification category. The initial classification category may be based on the feature vector, a manually labeled category, or a category obtained using any clustering method. In one implementation, the receiving module 310 can directly read the multiple samples 301 stored somewhere (local or remote disk). In another implementation, the receiving module 310 can obtain the multiple samples 301 transmitted to it via a network.
[0027] The projection module 320 is configured to obtain the respective projection point 302 of each of the plurality of samples 301 on a two-dimensional plane based on the feature vector corresponding to each sample in the plurality of samples 301. In one embodiment, projecting each of the plurality of samples 301 onto the two-dimensional plane can be achieved using the dimensionality reduction algorithm t-SNE. Those skilled in the art will know that projecting each of the plurality of samples 301 onto the two-dimensional plane can also be achieved using other dimensionality reduction algorithms, such as MDS, PCA, etc. Figure 4 A shows the projection points corresponding to a set of samples 401-405 projected onto a two-dimensional plane 400.
[0028] The planar graph generation module 330 is configured to obtain an initial planar graph 303 constructed on the two-dimensional plane by the respective projection points 302 of each of the plurality of samples 301. In one embodiment, the initial planar graph 303 can be constructed by calculating the Delaunay triangulation result of the respective projection points 302 of each of the plurality of samples 301. Those skilled in the art will know that the initial planar graph 303 can also be constructed by other methods. Figure 4 B shows Figure 4 The initial planar graph result established by the projection points in A contains 5 nodes 401'-405' and 7 edges obtained after calculating the Delaunay triangulation of nodes 401'-405'. Nodes 401'-405' represent... Figure 4In sample A, the projection points corresponding to samples 401-405 are represented by each edge, indicating the nearest neighbor relationship between samples. In existing technologies, the data and the nearest neighbor relationships between data points are not modeled as nodes and edges in a planar graph. Therefore, the nearest neighbor relationships between nodes in the planar graph cannot be well maintained during subsequent sub-region generation and circular wrapping layout generation. This results in circles representing similar data not being placed in adjacent positions, disrupting the user's perception of sample similarity.
[0029] The sub-region generation module 340 is configured to obtain a plurality of sub-regions 304 on the two-dimensional plane based on the respective projection point 302 of each of the plurality of samples 301 and the initial planar map 303, wherein each sub-region 304 corresponds to each of the plurality of samples 301.
[0030] In existing technologies, obtaining multiple sub-regions on a two-dimensional plane involves first calculating a square or circular region containing the projection points of each of the multiple samples on the two-dimensional plane. Then, methods such as CentroidalPower Diagram are used to directly divide the square or circular region into multiple sub-regions, where each sub-region corresponds to each of the multiple samples. However, this process of dividing the square or circular region into multiple sub-regions can significantly disrupt the nearest neighbor relationships between nodes in the initial planar graph, resulting in poor similarity preservation.
[0031] In one embodiment of the present invention, obtaining multiple sub-regions 304 on a two-dimensional plane can be achieved by first calculating the convex hull region of the projection points 302 of each of the multiple samples 301 on the two-dimensional plane, and then dividing the convex hull region into multiple sub-regions 304, where each sub-region corresponds to each of the multiple samples 301. Since the convex hull region is the smallest polygonal region containing all the projection points 302 of the multiple samples 301, its area is smaller than that of a square or circular region. Therefore, in the process of dividing the convex hull region into multiple sub-regions 304, the displacement amplitude of the projection points of each of the multiple samples 301 is smaller, and the nearest neighbor relationships between corresponding nodes in the established initial planar graph can be preserved more, thereby improving the similarity preservation effect.
[0032] In one implementation, the convex hull region of the respective projection point 302 of each of the multiple samples 301 in a two-dimensional plane can be calculated using the Graham algorithm. Those skilled in the art will understand that calculating the convex hull region of the respective projection point 302 of each of the multiple samples 301 in a two-dimensional plane can also be achieved using other algorithms, such as divide-and-conquer, incremental methods, etc. Figure 4B, where the convex hull region of nodes 401'-405' is the smallest convex polygon region formed around these nodes, namely the polygon region 401'-402'-403'-405'-404'.
[0033] Figure 5 A flowchart 500 is shown, illustrating a method for dividing a convex hull region into a plurality of sub-regions according to one or more embodiments of the present invention.
[0034] like Figure 5 As shown, in step 510, based on the distance between the respective projection points 302 of each sample in the multiple samples 301 corresponding to the two-dimensional plane, a clustering structure of all projection points corresponding to the multiple samples 301 is obtained. Each category of the clustering structure includes several projection points corresponding to the samples. In one embodiment, the clustering structure of all projection points 302 corresponding to the multiple samples 301 can be obtained by the K-Means clustering algorithm. Those skilled in the art will know that the clustering structure of all projection points 302 corresponding to the multiple samples 301 can also be obtained by other clustering algorithms, such as K-Medoids, Meanshift, etc. Figure 6 A-6D shows that Figure 4 The process of dividing the convex hull region of B into multiple sub-regions, where projection points 601-605 correspond to Figure 4 Projection points 401'-405' in B. Assume... Figure 6 In cluster A, projection points 601-602 form one cluster category, and projection points 603-605 form another cluster category.
[0035] In step 520, the convex hull region 601-602-603-605-604 is divided into multiple super-sub-regions according to the clustering structure, such that the multiple super-sub-regions satisfy a first condition. Each super-sub-region corresponds to a cluster category in the clustering structure. The first condition includes that the ratio of the area of each super-sub-region to the area of the convex hull region is equal to the ratio of the sum of the importance scores of several samples included in the cluster category corresponding to that super-sub-region to the sum of the importance scores of several samples. Assume... Figure 6 In region A, the importance of the samples corresponding to projection points 601-605 are 1, 4, 2, 5, and 3, respectively. Therefore, the area ratio of the super-sub-regions corresponding to projection points 601 and 602 to the area of the convex hull region in the two divided super-sub-regions is: The area ratio of the hypersubregion corresponding to projection points 603-605 to the area of the convex hull region is: There are multiple ways to partition the supersubregions to satisfy this area ratio. In one implementation, dividing the convex hull region into multiple supersubregions can be achieved using the existing Power Diagram for calculating the capacity constraint of the convex hull region. The Power Diagram for calculating the capacity constraint of the convex hull region requires an initial site and a capacity constraint for each supersubregion. The initial site for each supersubregion is the center of the projection points of multiple samples in the category corresponding to that supersubregion, and the capacity constraint for each supersubregion is the sum of the importance of multiple samples in the category corresponding to that supersubregion. For example... Figure 6 As shown in Figure B, using this method, the convex hull region can be divided into two super sub-regions 610 and 620. Those skilled in the art should understand that other methods can be used to divide the convex hull region into multiple super sub-regions; as long as these multiple super sub-regions satisfy the first condition, they are all within the scope of protection of this invention.
[0036] In step 530, several projection points corresponding to several samples included in each of the multiple super sub-regions are simultaneously rotated, translated, and / or scaled to obtain several first updated projection points, such that these several first updated projection points satisfy the second condition, thereby obtaining the first updated subplane map. All the first updated subplane maps constitute the first updated (i.e., the first update of the initial plane map) plane map. The second condition includes that the several first updated projection points included in each super sub-region are all located in that super sub-region, the connection relationship of the several first updated projection points included in each super sub-region remains unchanged in the first updated subplane map, and the several first updated projection points in the first updated subplane map cannot be scaled by a scaling ratio greater than 1. In one implementation, the following method can be used to simultaneously rotate, translate, and / or scale several projection points corresponding to several samples within each super sub-region to obtain several first updated projection points: For each super sub-region, firstly, calculate the centers of multiple projection points within the super sub-region and the center of the super sub-region itself. Then, translate these projection points so that the two centers coincide. Finally, calculate the scaling transformation of all projection points at each rotation angle using a grid search, ensuring that the first updated projection points remain within the super sub-region, and that the connectivity between the first updated projection points in the super sub-region remains unchanged in the first updated sub-plane graph. Furthermore, scaling transformations with a scaling ratio greater than 1 cannot be performed, thereby obtaining the first updated projection point corresponding to the largest scaling ratio. The above is a brute-force search method; however, various other brute-force search methods can be used in the prior art. Figure 6 Projection points 601-605 in A and Figure 6 In superregions 610-620 of B, the method in step 530 is first applied to obtain... Figure 6The first updated projection points in C are 601'-605'; as shown in... Figure 6 As shown in C, then for the super sub-region 610, the brute-force search method in step 530 is applied, transforming the positions of projection points 601 and 602 into the first updated projection points 601' and 602', and projection points 601' and 602' cannot undergo scaling transformations with a scaling ratio greater than 1; then for the super sub-region 620, the same brute-force search method in step 530 is applied, transforming the positions of projection points 603-605 into the first updated projection points 603'-605', and projection points 603'-605' cannot undergo scaling transformations with a scaling ratio greater than 1. Thus, the first updated planar graph 601'-605' is obtained, which includes the updated projection points 601'-605' and as shown in Figure C. Figure 6 The edge shown is C.
[0037] In step 540, for each of the multiple super-sub-regions, based on the first updated projection points obtained in that super-sub-region, the super-sub-region is divided into n sub-regions, where n is the number of samples included in the super-sub-region, and each sub-region corresponds to each sample in that super-sub-region. In one embodiment, dividing each super-sub-region into n sub-regions can be achieved using existing methods for calculating the Centroidal Power Diagram. Calculating the Centroidal Power Diagram requires initial sites and initial weights corresponding to the n sub-regions, where the initial site corresponding to each sub-region is the first updated projection point of the sample corresponding to that sub-region, and the initial weight corresponding to each sub-region is the importance of the sample corresponding to that sub-region. Those skilled in the art should know that other methods can be used to divide each super-sub-region into n sub-regions, such as the Centroidal Voronoi Diagram. Figure 6 As shown in D, super subregion 610 is divided into subregions 611 and 612, corresponding to the first updated projection points 601' and 602', respectively. Super subregion 620 is divided into subregions 623-625, corresponding to the first updated projection points 603'-605', respectively.
[0038] Module 304 utilizes the convex hull region and divides it into multiple sub-regions. Compared with existing methods, the multiple sub-regions obtained by module 304 retain more adjacency relationships between projection points in the initial planar graph and can ensure that the super sub-region corresponding to each cluster category is a convex polygon, so that the subsequently generated circular packaging layout can better reflect the cluster category of the sample.
[0039] Back Figure 3The circular layout generation module 350 is configured to obtain a circle 305 corresponding to each sample in the plurality of samples 301 on a two-dimensional plane based on the obtained plurality of sub-regions 304 and the importance and initial classification category of each sample in the plurality of samples 301. The parameters of each circle include its center, radius, and color. Similar to existing technologies, the radius of each circle represents the importance of the sample corresponding to that circle, and the color of the circle represents the initial classification category of the sample corresponding to that circle. Unlike existing technologies, in this layout, the distance between any two circles represents the similarity between the two samples corresponding to those two circles. Obtaining the circle corresponding to each sample in the plurality of samples on the two-dimensional plane includes initial circle layout processing and post-processing. The initial circle layout processing can utilize existing circle layout methods, for example, first calculating the centroid of each sub-region of the plurality of sub-regions, then calculating the circle with the largest radius centered at the centroid and completely located within that sub-region in each sub-region, and finally calculating the maximum radius of the circle corresponding to each sample in the plurality of samples on the two-dimensional plane, such that the importance of each sample in the plurality of samples is related to the radius of the circle corresponding to each sample in the plurality of samples. The radius of the circle corresponding to the i-th sample can be obtained by the following formula:
[0040]
[0041]
[0042] Figure 7 A shows the pair Figure 6 Subregions 611, 612, and 623-625 in D are processed using the initial circular layout to obtain circles 701-705.
[0043] In post-processing, existing methods can also be used. For example, firstly, based on the obtained multiple sub-regions, a first updated planar diagram is obtained, resulting in a second updated planar diagram. In the second updated planar diagram, each of the multiple second updated projection points corresponds to each of the multiple samples, and the edges between the second updated projection points within a super-sub-region represent the adjacency relationship between the n sub-regions within that super-sub-region. Then, based on the second updated planar diagram, the position of the circle corresponding to each sample in the multiple samples on the two-dimensional plane is adjusted using a force-directed method, so that the multiple circles satisfy predetermined conditions. Commonly used predetermined conditions in existing technologies are that the compactness of the circular packaging layout formed by the multiple circles is as high as possible, and that the circles do not overlap.
[0044] In one embodiment, the predetermined conditions include that the circular packaging layout consisting of multiple circles is as compact as possible, that the circles do not overlap, and that the circles corresponding to samples with similar feature vectors are arranged in close proximity on the two-dimensional plane.
[0045] In one implementation, a force-directed method for adjusting the position of each of the multiple circles on a two-dimensional plane adds a constant-magnitude gravity pointing towards the center of the layout to each of the multiple circles, and adds a gravity pointing towards the center of the other circle to the circles corresponding to a pair of samples with connected edges based on similarity according to the second updated planar graph. Then, the method is optimized by gradient descent, so that the compactness of the circular packaging layout composed of multiple circles is as high as possible, the circles do not overlap, and the circles corresponding to samples with similar feature vectors are placed in close proximity as much as possible.
[0046] Those skilled in the art should know that the position of each of the multiple circles on the two-dimensional plane can be adjusted by other methods of defining forces and other optimization methods, and by using force-directed methods. As long as the multiple circles can meet the predetermined conditions, they are all within the protection scope of this invention.
[0047] Figure 7 B shows Figure 7 The direction of the gravitational force applied in the force-directed method for circles 701-705 in A. Figure 7 C shows Figure 7 The positions of circles 701-705 in A after post-processing are shown. It is evident that the adjusted positions of the circles obtained after post-processing result in a higher degree of compactness, ensuring that the circles do not overlap and that circles representing similar samples are placed in closer proximity, helping users better perceive the similarity of the samples.
[0048] Back Figure 3 The visualization module 360 is configured to visualize multiple samples in a two-dimensional plane based on the respective circles 305 corresponding to each of the multiple samples 301.
[0049] In one embodiment, system 300 further includes an outlier determination module 370, configured to determine that a specific sample is an outlier in response to the fact that the color of the circle corresponding to a specific sample on the two-dimensional plane is different from the color of a plurality of adjacent circles of the same color. Since the similarity between samples is preserved in the circular packaging layout generated by this invention, circles corresponding to similar samples are located close to each other in the layout, which demonstrates the classification of the samples. Figure 2 Compared to the circular packaging layout results generated by existing technologies, the system of this invention enables users to easily identify outliers in the sample.
[0050] The visualization system 300 can be implemented as an application on a general computer system, or as an application on a server system, or as a network application, or as an application on a cloud platform.
[0051] Based on the same inventive concept, this invention also discloses a method for visualizing samples. Figure 8 A flowchart of a method 800 for visualizing a sample according to one or more embodiments of the present invention is shown.
[0052] according to Figure 8 In step 810, multiple samples are received. The information of each sample includes the importance of the sample, the feature vector corresponding to the sample, and the initial classification category of the sample.
[0053] In step 820, based on the feature vector corresponding to each of the multiple samples, the projection point of each of the multiple samples on the two-dimensional plane is obtained.
[0054] In step 830, an initial planar map is obtained by constructing the projection points of each of the multiple samples on a two-dimensional plane.
[0055] In step 840, multiple sub-regions on a two-dimensional plane are obtained based on the respective projection points of each of the multiple samples and the initial planar diagram, wherein each of the multiple sub-regions corresponds to each of the multiple samples.
[0056] In step 850, based on the obtained multiple sub-regions and the importance and initial classification of each sample in the multiple samples, a corresponding circle for each sample in the multiple samples is obtained on the two-dimensional plane, wherein the parameters of each circle include the center, radius, and color.
[0057] In step 860, the multiple samples are visualized on the two-dimensional plane according to the respective circles corresponding to each sample.
[0058] In one implementation, the radius of each circle represents the importance of the sample corresponding to that circle, the color of each circle represents the initial classification category of the sample corresponding to that circle, and the distance between any two circles represents the similarity between the two samples corresponding to those two circles.
[0059] In one embodiment, method 800 further includes step 870, in response to the fact that the color of the circle corresponding to a particular sample on the two-dimensional plane is different from the color of a plurality of adjacent circles of the same color, determining the particular sample as an outlier.
[0060] In one embodiment, step 840 includes: firstly, calculating the convex hull region of all projection points of the multiple samples on the two-dimensional plane based on the respective projection points of each sample in the multiple samples and the initial planar diagram; then dividing the convex hull region into multiple sub-regions, each of the multiple sub-regions corresponding to a sample.
[0061] In one implementation, the convex hull region is divided into multiple sub-regions, each sub-region corresponding to a sample. This involves: first, obtaining a clustering structure of all projection points corresponding to the multiple samples based on the distances between their respective projection points on the two-dimensional plane; each category of the clustering structure including several projection points corresponding to samples; then, dividing the convex hull region into multiple super-sub-regions based on the clustering structure, such that the multiple super-sub-regions satisfy a first condition, wherein each super-sub-region corresponds to each clustering category of the clustering structure. The first condition includes that the ratio of the area of each super-sub-region to the area of the convex hull region is equal to the ratio of the sum of the importance of the samples included in the clustering category corresponding to the super-sub-region to the sum of the importance of the multiple samples. Then, the projection points corresponding to the samples included in each super-sub-region are... Rotation, translation, and / or scaling are performed to obtain several first updated projection points, such that these several first updated projection points satisfy a second condition, thereby obtaining a first updated subplane map. All first updated subplane maps constitute the first updated plane map. The second condition includes that the several first updated projection points included in each super sub-region are all located in that super sub-region, the connection relationship of the several first updated projection points included in each super sub-region remains unchanged in the first updated subplane map, and the several first updated projection points in the first updated subplane map cannot be scaled by a scaling ratio greater than 1. Finally, for each of the multiple super sub-regions, based on the first updated projection points obtained in that super sub-region, the super sub-region is divided into n sub-regions, where n is the number of samples included in that super sub-region, and each sub-region corresponds to each sample in that super sub-region.
[0062] In one implementation, based on the obtained multiple sub-regions and the importance and initial classification of each sample in the multiple samples, the respective circle corresponding to each sample in the multiple samples on the two-dimensional plane is obtained, including initial circle layout processing and post-processing.
[0063] In one implementation, the initial circle layout process includes: firstly calculating the centroid of each of the multiple sub-regions; then calculating the circle with the largest radius centered on the centroid and completely located within the sub-region; and finally calculating the maximum radius of the circle corresponding to each sample in the multiple samples on the two-dimensional plane, such that the importance of each sample in the multiple samples is related to the radius of the circle corresponding to each sample in the multiple samples.
[0064] In one implementation, the post-processing includes: firstly, updating the first updated planar graph based on the multiple sub-regions corresponding to the multiple samples to obtain a second updated planar graph. In the second updated planar graph, multiple nodes represent each sample among the multiple samples corresponding to each of the multiple sub-regions, and the edges between nodes represent the adjacency relationships between the multiple sub-regions. Then, based on the second updated planar graph, adjusting the positions of the circles corresponding to each sample in the multiple samples on the two-dimensional plane so that the multiple circles satisfy predetermined conditions. It should be noted that the step of obtaining the second updated planar graph in the post-processing can occur before, during, or after the initial circle layout processing. It is sufficient that the process occurs before adjusting the positions of the circles corresponding to each sample in the multiple samples on the two-dimensional plane to ensure that the multiple circles satisfy the predetermined conditions.
[0065] In one embodiment, the predetermined conditions include that the circular packaging layout consisting of multiple circles corresponding to multiple samples is as compact as possible, that the circles do not overlap, and that the circles corresponding to samples with similar feature vectors are arranged in close proximity on the two-dimensional plane.
[0066] Based on the method of this invention, a visualization system was developed to analyze image data. Figure 9 This image shows the interface of the visualization system for visualizing a clothing dataset containing outliers. For example... Figure 9 As shown, the color of circle 901 is inconsistent with the colors of its neighboring circles 902-907, indicating that the initial classification of circle 901 is inconsistent with the initial classification of its neighboring circles. However, examining the corresponding images reveals that image 901', corresponding to circle 901, and images 902'-907', corresponding to its neighboring circles 902-907, both depict shirts. Therefore, it can be determined that the initial classification of circle 901 is incorrect, and the data corresponding to circle 901 is an outlier. This interface allows users to explore the clustering structure and outliers of samples in a dataset, identify potential erroneous classifications, and correct them.
[0067] This invention can be a system, a method, and / or a computer-readable storage medium. The computer-readable storage medium carries computer-readable program instructions for causing a processor to implement various aspects of the invention. The methods of this invention can be executed on a standalone computer system, on a distributed computing system, or even on a cloud platform.
[0068] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer-readable storage media according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0069] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable storage media according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.
[0070] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A system for visualizing samples, the system comprising: The receiving module is configured to receive multiple samples, each of which includes information about its importance, its corresponding feature vector, and its initial classification category. The projection module is configured to obtain the respective projection point of each of the plurality of samples on a two-dimensional plane based on the feature vector corresponding to each of the plurality of samples; A planar plot generation module is configured to obtain an initial planar plot constructed on the two-dimensional plane by the respective projection points of each of the plurality of samples; The sub-region generation module is configured to obtain multiple sub-regions on the two-dimensional plane based on the respective projection points of each of the multiple samples and the initial planar map, wherein each of the multiple sub-regions corresponds to each of the multiple samples; The circular packaging layout generation module is configured to obtain a corresponding circle for each of the multiple samples on the two-dimensional plane based on the obtained multiple sub-regions and the importance and initial classification category of each of the multiple samples, wherein the parameters of each circle include the center, radius, and color; as well as The visualization module is configured to visualize the plurality of samples on the two-dimensional plane based on the respective circle corresponding to each of the plurality of samples. The radius of each circle represents the importance of the sample corresponding to that circle; Obtaining multiple sub-regions on the two-dimensional plane includes: Based on the respective projection points of each of the plurality of samples and the initial planar diagram, calculate the convex hull region of all projection points of the plurality of samples on the two-dimensional plane; Based on the distance between the respective projection points of each of the plurality of samples in the two-dimensional plane, a clustering structure of all projection points corresponding to the plurality of samples is obtained, wherein each category of the clustering structure includes several projection points corresponding to the samples. The convex hull region is divided into multiple super sub-regions according to the clustering structure, such that the multiple super sub-regions satisfy a first condition, wherein each of the multiple super sub-regions corresponds to each cluster category of the clustering structure, and the first condition includes that the ratio of the area of each of the multiple super sub-regions to the area of the convex hull region is equal to the ratio of the sum of the importance of a plurality of samples included in the cluster category corresponding to the super sub-region to the sum of the importance of the plurality of samples; For each of the plurality of super sub-regions, several projection points corresponding to several samples are simultaneously rotated, translated, and / or scaled to obtain several first updated projection points, such that the several first updated projection points satisfy a second condition, thereby obtaining a first updated subplane map. All first updated subplane maps constitute a first updated plane map, wherein the second condition includes that the several first updated projection points included in each super sub-region are all located in the super sub-region, the connection relationship of the several first updated projection points included in each super sub-region remains unchanged in the first updated subplane map, and the several first updated projection points in the first updated subplane map cannot undergo scaling transformations with a scaling ratio greater than 1; and For each of the plurality of super sub-regions, based on the first updated projection points obtained in the super sub-region, the super sub-region is divided into n sub-regions, where n is the number of samples included in the super sub-region, and each sub-region corresponds to each sample in the super sub-region; Furthermore, based on the obtained multiple sub-regions and the importance and initial classification category of each sample among the multiple samples, obtaining the respective circle corresponding to each sample on the two-dimensional plane includes: Calculate the centroid of each of the plurality of sub-regions; Calculate the circle with the largest radius that is completely located within the sub-region, centered at the centroid, in each of the plurality of sub-regions; Calculate the maximum radius of the circle corresponding to each of the plurality of samples on the two-dimensional plane, such that the importance of each of the plurality of samples is related to the radius of the circle corresponding to each of the plurality of samples; Based on the multiple sub-regions corresponding to the multiple samples, the first updated planar graph is updated to obtain a second updated planar graph. In the second updated planar graph, each of the multiple second updated projection points corresponds to each of the multiple samples. The edges connecting the second updated projection points within a super-sub-region represent the adjacency relationships between the n sub-regions within that super-sub-region. Based on the second updated plan view, adjust the position of the circle corresponding to each of the plurality of samples on the two-dimensional plane so that the plurality of circles meet the predetermined conditions; The predetermined conditions include: The circular packaging layout consisting of multiple circles corresponding to the multiple samples is highly compact, with no overlap between the circles, and the circles corresponding to samples with similar feature vectors are arranged in close positions on the two-dimensional plane.
2. In the system according to claim 1, the color of each circle represents the initial classification category of the sample corresponding to that circle, and the distance between any two circles represents the similarity between the two samples corresponding to those two circles.
3. The system according to claim 1, further comprising: The outlier identification module is configured to identify a specific sample as an outlier in response to the fact that the color of the circle corresponding to a specific sample on the two-dimensional plane is different from the color of a plurality of adjacent circles of the same color.
4. The system according to claim 1, wherein the sample is at least one of the following: image; video; Tables; and document.
5. A method for visualizing samples, comprising: Receive multiple samples, and the information of each sample includes the importance of the sample, the feature vector corresponding to the sample, and the initial classification category of the sample; Based on the feature vector corresponding to each of the plurality of samples, obtain the respective projection point of each of the plurality of samples on the two-dimensional plane; Obtain an initial planar map constructed on the two-dimensional plane by the respective projection points of each of the plurality of samples; Based on the respective projection points of each of the plurality of samples and the initial planar diagram, a plurality of sub-regions on the two-dimensional plane are obtained, wherein each of the plurality of sub-regions corresponds to each of the plurality of samples; Based on the obtained multiple sub-regions and the importance and initial classification category of each sample in the multiple samples, a corresponding circle is obtained for each sample in the multiple samples on the two-dimensional plane, wherein the parameters of each circle include the center, radius, and color; and Based on the respective circle corresponding to each of the plurality of samples, the plurality of samples are visualized on the two-dimensional plane; The radius of each circle represents the importance of the sample corresponding to that circle; Obtaining multiple sub-regions on the two-dimensional plane includes: Based on the respective projection points of each of the plurality of samples and the initial planar diagram, calculate the convex hull region of all projection points of the plurality of samples on the two-dimensional plane; Based on the distance between the respective projection points of each of the plurality of samples in the two-dimensional plane, a clustering structure of all projection points corresponding to the plurality of samples is obtained, wherein each category of the clustering structure includes several projection points corresponding to the samples. The convex hull region is divided into multiple super sub-regions according to the clustering structure, such that the multiple super sub-regions satisfy a first condition, wherein each of the multiple super sub-regions corresponds to each cluster category of the clustering structure, and the first condition includes that the ratio of the area of each of the multiple super sub-regions to the area of the convex hull region is equal to the ratio of the sum of the importance of a plurality of samples included in the cluster category corresponding to the super sub-region to the sum of the importance of the plurality of samples; For each of the plurality of super sub-regions, several projection points corresponding to several samples are simultaneously rotated, translated, and / or scaled to obtain several first updated projection points, such that the several first updated projection points satisfy a second condition, thereby obtaining a first updated subplane map. All first updated subplane maps constitute a first updated plane map, wherein the second condition includes that the several first updated projection points included in each super sub-region are all located in the super sub-region, the connection relationship of the several first updated projection points included in each super sub-region remains unchanged in the first updated subplane map, and the several first updated projection points in the first updated subplane map cannot undergo scaling transformations with a scaling ratio greater than 1; and For each of the plurality of super sub-regions, based on the first updated projection points obtained in the super sub-region, the super sub-region is divided into n sub-regions, where n is the number of samples included in the super sub-region, and each sub-region corresponds to each sample in the super sub-region; Furthermore, based on the obtained multiple sub-regions and the importance and initial classification category of each sample among the multiple samples, obtaining the respective circle corresponding to each sample on the two-dimensional plane includes: Calculate the centroid of each of the plurality of sub-regions; Calculate the circle with the largest radius that is completely located within the sub-region, centered at the centroid, in each of the plurality of sub-regions; Calculate the maximum radius of the circle corresponding to each of the plurality of samples on the two-dimensional plane, such that the importance of each of the plurality of samples is related to the radius of the circle corresponding to each of the plurality of samples; Based on the multiple sub-regions corresponding to the multiple samples, the first updated planar graph is updated to obtain a second updated planar graph. In the second updated planar graph, each of the multiple second updated projection points corresponds to each of the multiple samples. The edges connecting the second updated projection points within a super-sub-region represent the adjacency relationships between the n sub-regions within that super-sub-region. Based on the second updated plan view, adjust the position of the circle corresponding to each of the plurality of samples on the two-dimensional plane so that the plurality of circles meet the predetermined conditions; The predetermined conditions include: The circular packaging layout consisting of multiple circles corresponding to the multiple samples is highly compact, with no overlap between the circles, and the circles corresponding to samples with similar feature vectors are arranged in close positions on the two-dimensional plane.
6. The method according to claim 5, wherein the color of each circle represents the initial classification category of the sample corresponding to that circle, and the distance between any two circles represents the similarity between the two samples corresponding to those two circles.
7. The method according to claim 5, further comprising: In response to the fact that the color of the circle corresponding to a specific sample on the two-dimensional plane is different from the color of several adjacent circles of the same color, the specific sample is identified as an outlier.
8. The method of claim 5, wherein the sample is at least one of the following: image; video; Tables; and document.
9. A computer-readable storage medium for visualizing samples, the computer-readable storage medium having program instructions stored therein, the program instructions being executable by a computing device to cause the computing device to perform the method as described in any one of claims 5-8.
10. A system for visualizing samples, comprising: Memory; as well as At least one processor is operatively coupled to a memory and configured to perform the method as described in any one of claims 5-8.