Systems, methods, and computer program products for visualizing a large dataset in a grid layout

Abstract

The application discloses a computer system, method and program product for visualizing grid layout of a large-scale data set, the method comprising: obtaining a large-scale data set with a category hierarchy and a sample number K displayed on a display page, the large-scale data set containing N samples, the category hierarchy and the membership of the samples to categories; randomly sampling a first group of K samples from the N samples; calculating the membership of each sample in the first group of K samples to each category in a first category; calculating the layout position of each sample in the first group of K samples on the display page; assigning a color to each category in the first category; and displaying the first group of K samples on the display page. The system, method and program product can maintain visual stability in the process of exploring the large-scale data set with the category hierarchy, and place confusing samples at the boundary of a confusing cluster category, so that a user can easily track and analyze data.

Description

Technical Field

[0001] This disclosure relates to data visualization, and in particular, to a computer system, method, and computer program product for visualizing a grid layout of a large-scale dataset with a hierarchical structure of categories. Background Technology

[0002] Grid layout is a commonly used visualization method, offering advantages such as high space utilization and non-overlapping elements, making it widely applied in machine learning. Recent work has introduced clustering-oriented concepts into grid layout, placing similar data samples in adjacent grid positions while enhancing the compactness and convexity of the grid graph corresponding to each cluster, helping users perceive sample similarity and cluster structure. However, when dealing with massive datasets of hundreds of thousands or even millions, this method struggles to complete visualization layouts promptly. An effective way to visualize large-scale data is to utilize hierarchical clustering structures and employ interactive, step-by-step zooming to explore the data. However, existing clustering-oriented grid layout methods, when applying this approach, struggle to maintain visual stability during step-by-step zooming, making it difficult for users to track and analyze the data. Summary of the Invention

[0003] The following description includes exemplary methods, systems, techniques, and sequences of instructions embodying the technology of this disclosure. However, it should be understood that the described invention can be practiced in one or more aspects without these specific details. In other instances, well-known protocols, structures, and techniques have not been shown in detail so as not to obscure the invention. Those skilled in the art will understand that the described techniques and mechanisms can be applied to various architectures for visualizing samples.

[0004] According to one aspect of this disclosure, a computer system for visualizing a grid layout of a large-scale dataset with a hierarchical category structure is proposed, comprising: an acquisition module configured to acquire a large-scale dataset with a hierarchical category structure and a number K of samples to be displayed on a display page, the large-scale dataset containing N samples, a hierarchical category structure, and membership degrees of samples to categories, wherein each of the N samples contains sample features, the hierarchical category structure in which the N samples are classified into a multi-level structure, the category of a child node in the next level of the hierarchical category structure being a further classification result of the category of the parent node in the previous level, wherein in the hierarchical category structure, the category of each node in the next level below the root node is a first category, M is the number of categories in the bottom level of the hierarchical category structure, and the membership degrees of samples to categories are... The membership degree includes the membership degree of each sample in the N samples to each category in the M categories; the extraction module is configured to randomly extract a first group of K samples from the N samples; the membership degree calculation module is configured to calculate the membership degree of each sample in the first group of K samples to each category in the first category; the layout module is configured to calculate the layout position of each sample in the first group of K samples on the display page based on the calculated membership degree of each sample in the first group of K samples to each category in the first category and the sample features of each sample in the first group of K samples; the color allocation module is configured to assign a color to each category in the first category; and the display module is configured to display the first group of K samples on the display page according to the layout position and the assigned color.

[0005] According to another aspect of this disclosure, a method for visualizing a grid layout of a large-scale dataset with a hierarchical category structure includes: acquiring a large-scale dataset with a hierarchical category structure and the number K of samples to be displayed on a display page, wherein the large-scale dataset contains N samples, a hierarchical category structure, and the membership degree of samples to categories, wherein each of the N samples contains sample features, and in the hierarchical category structure, the N samples are classified into a multi-level structure, wherein the category of the child node of the next level in the multi-level structure is a further classification result of the category of the parent node of the previous level, wherein the category of each node in the next level below the root node in the hierarchical category structure is a first category, and M is the class into which the N samples are classified. The number of categories at the bottom layer of the hierarchical structure is determined. The membership degree of each sample to a category includes the membership degree of each of the N samples to each of the M categories. A first group of K samples is randomly selected from the N samples. The membership degree of each of the first group of K samples to each of the first categories is calculated. Based on the calculated membership degree of each of the first group of K samples to each of the first categories and the sample features of each of the first group of K samples, the layout position of each of the first group of K samples on the display page is calculated. A color is assigned to each category in the first category. The first group of K samples is displayed on the display page according to the layout position and the assigned color.

[0006] According to another aspect of this disclosure, a computer program product for visualizing the grid layout of a large-scale dataset with a hierarchical structure of categories is proposed, the computer program product comprising program instructions executable by a computing device to cause the computing device to perform the method described above.

[0007] The aforementioned systems, methods, and program products not only satisfy the concept of clustering while exploring large-scale datasets with hierarchical class structures, but also maintain visual stability and place confused samples at the boundaries of confused cluster categories, making it easy for users to track and analyze the data. Attached Figure Description

[0008] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0009] Figure 1 This schematically illustrates a category hierarchy;

[0010] Figure 2 A schematic illustration shows the existing method's approach to... Figure 1 A visual layout of some samples in a category hierarchy;

[0011] Figure 2 B schematically illustrates the... Figure 1 The layout when further exploring the hierarchical structure;

[0012] Figure 3 A block diagram of a computer system for visualizing a grid layout of a large dataset with a hierarchical class structure is shown, according to one or more embodiments of the present disclosure.

[0013] Figure 4 A flowchart is shown illustrating a method for visualizing a grid layout of a large-scale dataset with a category hierarchy according to one or more embodiments of the present disclosure;

[0014] Figure 5 A flowchart is shown of a method for calculating the layout position of each sample in a first group of K samples on a display page, according to one or more embodiments of the present disclosure.

[0015] Figure 6 A-6F shows examples of the process and results of dividing the first category into regions and transforming the regions into a grid shape using different methods;

[0016] Figure 7 A flowchart is shown illustrating a method for calculating the area allocated to each category in a display page for each category in a first category using an iterative horizontal and vertical partitioning method according to one or more embodiments of the present disclosure;

[0017] Figure 8 A schematic illustration shows an example of a confused sample at a confused boundary in a grid shape;

[0018] Figure 8 B illustrates an example of selecting a specific grid area in a display page;

[0019] Figure 9 A flowchart is shown of a method for calculating the layout position of each sample in a second group of K samples on a display page, according to one or more embodiments of the present disclosure;

[0020] Figure 10 A-10C shows that... Figure 8 An example of the process and result of determining the location of the category in the second category after a specific grid region in B is selected using step 930;

[0021] Figure 11 A-11D illustrates the specific grid area and... Figure 10 This is an example of the process and result of dividing the second category into regions using the powerdiagram method to calculate the category position in C.

[0022] Figure 12A flowchart is shown illustrating a method for calculating the area allocated to each category in a display page for each category in a second category using an iterative horizontal and vertical partitioning approach, according to one or more embodiments of the present disclosure.

[0023] Figure 13 A-13D shows an example of the h function used to calculate the stability cost. Detailed Implementation

[0024] Embodiments of the present invention will now be described with reference to the accompanying drawings. Numerous specific details are set forth in the following description to provide a more complete understanding of the invention. However, it will be apparent to those skilled in the art that implementations of the invention may not include some of these specific details. Furthermore, it should be understood that the invention is not limited to the specific embodiments described. Rather, the invention can be practiced with any combination of the following features and elements, regardless of whether they relate to different embodiments. Therefore, the following aspects, features, embodiments, and advantages are illustrative only and should not be construed as elements or limitations of the appended claims unless expressly set forth in the claims.

[0025] As mentioned earlier, when faced with massive datasets of hundreds of thousands or even millions of records, grid layout methods struggle to complete visualization in a timely manner. An effective way to visualize large-scale data is to utilize hierarchical clustering structures within the data and employ interactive, step-by-step zooming to explore the data. However, existing clustering-oriented grid layout methods, when applying this approach, struggle to maintain visual stability during step-by-step zooming, making it difficult for users to track and analyze the data.

[0026] Meanwhile, according to fuzzy set theory, a sample can belong to multiple different clusters with different probabilities (e.g., the prediction confidence of a model), and the sum of these different probabilities is 1. For confusing samples that have a high probability in two or more clusters, existing methods cannot effectively place these samples (called confusing samples) at the boundaries of two or more cluster categories, interfering with the user's judgment of the confusion nature.

[0027] Figure 1 The diagram illustrates a category hierarchy represented as a tree, where nodes 100–113 are all nodes in the hierarchy, with node 100 being the root node. The other nodes represent the category of the sample at that level. Leaf nodes 102, 106, and 108–113 represent the categories at the bottom level of the hierarchy.

[0028] Figure 1This is merely an illustrative representation of the hierarchical structure of categories in a large dataset. In practice, the number of samples in such a dataset can be extremely large, for example, reaching the millions. Therefore, the hierarchical structure of categories may have many layers, and the samples can be data types such as images, videos, tables, and documents. Those skilled in the art should know that the samples can also be other types of data, such as chat logs from social software, data on product quality control within an enterprise, and so on.

[0029] In the Figure 1 In the process of visualizing the grid layout of a large-scale dataset with a hierarchical category structure, as shown, Figure 2 A schematic illustration shows the existing method's approach to... Figure 1 This is a visual layout of a subset of samples within a category hierarchy. It shows the layout of the 16 samples selected during the initial exploration on the visual page. This initial exploration focuses on the first level of categories under the root node of the hierarchy. It displays... Figure 1 The first-level nodes 101, 102, and 103 under the root node have 16 selected samples, with 8, 4, and 4 samples corresponding to the three categories, respectively. Figure 2 The black, gray, and white grid in A. Figure 2 The layout of A already satisfies the concept of clustering. Figure 2 B schematically illustrates the... Figure 1 The layout for further exploration of the hierarchical structure, which is selected by the user. Figure 2 This was generated after grid region 210 (the four grids in the upper left corner) in area A. Grid region 210 contains black and white grids, corresponding to... Figure 1 The categories of nodes 101 and 103 are as follows: black grid occupies the bottom left three grids of node 210, and white grid occupies the top right one grid of node 210. Further exploration of the layout... Figure 2 B showcased Figure 1 The categories of nodes 104, 105, and 106 are given, where the categories of nodes 104 and 105 are derived from the category of the original node 101. Figure 2 The layout of B includes 4 samples and 6 samples respectively. The 4 samples are displayed as a light gray grid and correspond to the category of node 104, while the 6 samples are displayed as a dark gray grid and correspond to the category of node 105. The category of node 106 is obtained by expanding the category of the original node 103. Figure 2 The layout of B includes 6 samples, which are displayed as a white grid. Figure 2 The 16 samples shown in B are the samples from 4 grids in grid region 210 plus 8 newly selected samples. It can be noted that because the categories of nodes 104 and 105 are expanded from the original category of node 101, therefore... Figure 2The shape of the light gray and dark gray grid in B should be consistent with... Figure 2 In A, the black grid corresponds to the shape formed by the three grids in the lower left corner of grid region 210; and the category of node 106 is obtained by expanding the category of the original node 103, therefore Figure 2 The shape formed by the white grid in B should be consistent with... Figure 2 The white grid in A corresponds to the shape formed by the upper right grid in grid region 210. However, in Figure 2 In B, this correspondence (i.e., layout stability during exploration) is not visually maintained, neither in shape nor position. Figure 2 In section B, the shape composed of light gray and dark gray grids should have been in the lower left corner of the layout, arranged in a similar manner. Figure 2 In A, the black grid forms an approximate triangle shape with three grids in the lower left corner of grid region 210; while Figure 2 The shape formed by the white grid in B should have been arranged in the upper right corner of the layout. It is evident that... Figure 2 The layout of B makes it difficult for users to track and analyze the corresponding categories during the exploration process.

[0030] also, Figure 2 In A, grids 221 and 222 are respectively and Figure 2 In grid B, identical samples are placed on grids 231 and 232. Note that the top-to-bottom order of 231 and 232 is reversed compared to 221 and 222, which also results in instability in the sample layout. Figure 2 The sample placed on grid 233 in B is a confusing sample between the categories of nodes 104 and 106, but it is placed on a grid position far away from the boundary between the shapes formed by the light gray and white grids corresponding to the two categories, which is not conducive to the user judging the confusion of the sample.

[0031] To address the problems existing in the prior art, this disclosure proposes a system, method, and computer program product for visualizing the grid layout of a large-scale dataset with a hierarchical structure. The method includes: acquiring a large-scale dataset with a hierarchical category structure and the number K of samples displayed on a display page. The large-scale dataset contains N samples, a hierarchical category structure, and the membership degree of each sample to its category. Each of the N samples contains sample features. In the hierarchical category structure, the N samples are classified into multiple levels, and the category of a child node in the next level of the hierarchical structure is a further classification result of the category of its parent node in the previous level. In this hierarchical category structure, each node in the next level below the root node belongs to the first category. The number of categories at the bottom level of the category hierarchy is M. The membership degree of a sample to a category includes the membership degree of each of the N samples to each of the M categories. A first group of K samples is randomly selected from these N samples. The membership degree of each of the first group of K samples to each of the first categories is calculated. The layout position of each of the first group of K samples on the display page is calculated. A color is assigned to each category in the first category. The first group of K samples is displayed on the display page according to the layout position and assigned color. This system, method, and program product can not only satisfy the concept of clustering while maintaining visual stability during the exploration of large-scale datasets with a hierarchical category structure (including sample extraction and layout, color assignment to categories, display on the display page, and interactive operations supporting user selection of local areas and further exploration), but also places confusing samples at the boundaries of confusing cluster categories, making it easy for users to track and analyze the data.

[0032] Figure 3 A structural block diagram of a computer system 300 for visualizing a grid layout of a large-scale dataset with a hierarchical category structure, according to one or more embodiments of the present disclosure, is shown. Figure 3 As shown, system 300 includes an acquisition module 310, an extraction module 320, a membership calculation module 330, a layout module 340, a color allocation module 350, and a display module 360.

[0033] exist Figure 3In this process, the acquisition module 310 can be configured to acquire a large-scale dataset 301 with a hierarchical category structure and the number of samples K 308 displayed on the display page. Here, the large-scale dataset 301 contains N samples, a hierarchical category structure, and the membership degree of each sample to its category. Each of the N samples contains sample features (e.g., F-dimensional features). In the hierarchical category structure, these N samples are classified into multiple levels. The category of the child node in the next level of this hierarchical category structure is a further classification result of the category of the parent node in the previous level. The category of each node in the level below the root node in the hierarchical category structure is the first category. The number of categories in the bottom layer of the hierarchical category structure is M. The membership degree of each sample to its category includes the membership degree of each of the N samples to each of the M categories. The categories in this hierarchical category structure can be predicted categories by the model based on the feature vector corresponding to the sample, manually labeled categories, or classification categories obtained by any clustering method. The membership degree of a sample to a category can be the confidence level of the model's prediction based on the feature vector corresponding to the sample, or any membership degree that satisfies the definition of fuzzy set theory (i.e., the sum of the membership degrees of any sample to the categories corresponding to all bottom-level nodes in the category hierarchy is 1). Sample features can be represented by a data matrix of size N×F, or by any other effective method. The category hierarchy can be represented by a tree composed of category nodes, or by any other effective method.

[0034] For example in Figure 1 In this context, assume the parameters are N = 100 and K = 16. Figure 1 Child nodes 101 to 103 below the root node Figure 1 The category 101 is designated as the first category. Nodes 102, 106, 108-113 have no child nodes; these eight nodes are leaf nodes in the category hierarchy, corresponding to the bottom-level categories. Therefore, M = 8 in the category hierarchy. Assume each sample contains 7-dimensional features, which can be physical features of the sample, or data features (e.g., obtained through neural network processing), etc.

[0035] In one embodiment, the acquisition module 310 can directly read a large-scale dataset 301 with a hierarchical class structure stored somewhere (local or remote disk) and the number of samples K 308 displayed on the display page. In another embodiment, the receiving module 310 can obtain the large-scale dataset 301 with a hierarchical class structure and the number of samples K 308 displayed on the display page transmitted to it via a network.

[0036] Figure 3The extraction module 320 can be configured to randomly select a first group of K samples 302 from the N samples. In one embodiment, randomly selecting the first group of K samples from the N samples can be achieved through equal probability random sampling. Those skilled in the art will know that randomly selecting the first group of K samples from the N samples can also be achieved through other existing random sampling algorithms, such as blue noise sampling algorithms.

[0037] Figure 3 The membership calculation module 330 can be configured to calculate the membership degree 303 of each sample in the first group of K samples 302 to each category in the first category. The membership degree of a specific sample to a specific category can be calculated by summing the membership degrees of that specific sample to all the corresponding categories of the bottom-level category nodes contained in the category hierarchy. Since no two categories in the first category contain duplicate bottom-level category nodes in the category hierarchy, and the union of all bottom-level category nodes contained in the category hierarchy of all categories in the first category is the bottom-level category node in the category hierarchy, the sum of the membership degrees of any sample to all categories in the first category is 1.

[0038] by Figure 1 For example, suppose that for a specific sample, its membership degrees to the categories of nodes 102, 106, 108~113 are 0, 0.5, 0, 0.1, 0.1, 0.1, 0.1, 0.1, respectively. Then its membership degree to the category of node 101 in the first category is 0+0.1+0.1+0.1=0.3, and its membership degree to the category of node 103 in the first category is 0.5+0.1+0.1=0.7.

[0039] Figure 3 The layout module 340 can be configured to calculate the layout position 304 of each sample in the first group of K samples 302 on the display page based on the membership degree 303 of each sample in the first group of K samples 302 to each category in the first category and the features of each sample in the first group of K samples 302 in the large-scale dataset 301. The specific implementation of the layout module 340 will be described in detail later.

[0040] Figure 3 The color allocation module 350 can be configured to assign a color to each category in the first category, thereby obtaining the display color 305 corresponding to the first category. The color allocation process can employ any color allocation method in the prior art.

[0041] Figure 3The display module 360 ​​can be configured to display the first group of K samples 302 on the display page according to the layout position 304 of the first group of K samples 302 and the display color 305 corresponding to the first category.

[0042] Correspondingly, this disclosure also discloses a method for visualizing the grid layout of large-scale datasets with a hierarchical class structure. Figure 4 A flowchart illustrating a method 400 for visualizing a grid layout of a large-scale dataset with a hierarchical category structure according to one or more embodiments of the present disclosure is shown. Figure 4 In step 410, a large-scale dataset with a category hierarchy and the number K samples to be displayed on the display page are obtained. The large-scale dataset contains N samples, a category hierarchy, and the membership degree of each sample to its category. Each of the N samples contains sample features (e.g., F-dimensional features). In the category hierarchy, the N samples are classified into a multi-level structure. The category of the child node in the next level of the category hierarchy is a further classification result of the category of the parent node in the previous level. In the category hierarchy, the category of each node in the next level below the root node is the first category. The number of categories in the bottom layer of the category hierarchy is M. The membership degree of each sample to its category includes the membership degree of each of the N samples to each of the M categories. In step 420, a first group of K samples is randomly selected from the N samples. In step 430, the membership degree of each sample in the first group of K samples to each category in the first category is calculated. In step 440, based on the calculated membership degree of each sample in the first group of K samples to each category in the first category and the sample features of each sample in the first group of K samples, the layout position of each sample in the first group of K samples on the display page is calculated. In step 450, a color is assigned to each category in the first category. In step 460, the first group of K samples is displayed on the display page according to the layout position and the assigned color. Figure 4 The specific implementation of each step and Figure 3 The implementations of each module are corresponding; therefore, only specific descriptions will be provided below. Figure 3 The implementation of each module.

[0043] In one implementation, Figure 3 The layout module 340 can be further configured to adopt Figure 5 The method 500 for calculating the layout position 304 shown is as follows: Figure 4 Step 440 can also follow the process of method 500. Figure 5 A flowchart is shown of a method 500 for calculating the layout position 304 of each sample in a first group of K samples 302 on a display page, according to one or more embodiments of the present disclosure.

[0044] like Figure 5 As shown, in step 510, based on the calculated membership degree of each sample in the first group of K samples to each category in the first category, the specific category to which each sample in the first group of K samples belongs is determined. In one implementation, for a specific sample, the category with the highest membership degree is selected as the category to which the sample belongs. For example, when the model prediction confidence is used as the membership degree, the category with the highest prediction confidence in the first category is selected as the specific category to which the sample belongs in the first category.

[0045] In step 520, the average sample feature of each sample in the first category within the first group of K samples is calculated. Taking a specific category within the first category as an example, assuming that the category contains K' samples in the first group of K samples, the average sample feature of these K' samples is calculated as the average feature of the category.

[0046] In step 530, the position of each category in the first category on the display page is calculated, such that categories containing samples with similar average sample feature values ​​are placed in close proximity. In one implementation, step 530 can be performed using an existing KK-Layout algorithm. Specifically, in the KK-Layout algorithm, the difference between any two categories in the first category is first calculated. In one implementation, let the average feature values ​​of the two categories be f... i with f j The degree of difference can then be calculated using the Euclidean distance:

[0047] d ij =||f i -f j ||

[0048] If there are M' categories in the first category, then the calculated difference matrix D will be of size M'×M'. This difference matrix can also be calculated by any other method, and can even incorporate other information such as prediction confidence.

[0049] Secondly, after calculating the difference, the following cost function is established according to the KK-Layout algorithm:

[0050]

[0051] Where c1 to c M′ Let c1 be the position of each category in the first category on the display page. For the above cost function, use any optimization method to find a set of values ​​from c1 to c2. M′This minimizes the cost function value. In one implementation, the L-BFGS algorithm can be used for optimization. This optimization problem can also be solved using any other method. Those skilled in the art will know that step 530 can also be accomplished using other existing algorithms, such as SGD, BFGS, etc.

[0052] In step 540, based on the position of each category in the first category on the display page and the specific category within the first group of K samples to which each sample belongs in the first category, regions are divided on the display page such that the area ratio of the region corresponding to each category in the first category is consistent with the proportion of samples contained in that category to the total number of samples in the first group of K samples. Specifically, for step 540, firstly, based on the specific category within the first category to which each sample in the first group of K samples belongs, the proportion of samples contained in each category within the first category to the total number of samples in the first group of K samples is calculated. Taking a specific category in the first category as an example, assuming that this category contains K' samples in the first group of K samples, then the proportion of this category is... Subsequently, based on the position of each category in the first category on the display page and the proportion of the number of samples contained in each category in the first category to the number of samples in the first group of K, different methods can be used to obtain the corresponding area of ​​each category in the first category.

[0053] Specifically, in one implementation method, the existing power diagram method can be used for calculation. Its cost function is:

[0054]

[0055] Among them, Ω1 to Ω M′ Let Ω1 represent the region allocated to each category in the first category on the display page. These regions must be non-overlapping, their union equal to the entire display page, and the proportion of a particular category's region on the display page equal to the proportion of samples in that category out of the total number of samples in the first group of K samples. For the above cost function, use any optimization method to find a set of Ω1 to Ω2. M′ The optimization aims to minimize the cost function value, thereby determining the region allocated to each category on the display page. In one implementation, the optimization method may employ the L-BFGS algorithm. Those skilled in the art will recognize that other existing algorithms, such as SGD, BFGS, etc., can also be used for optimization.

[0056] Figure 6 A-6F illustrates examples of the process and results of dividing the first category into regions and transforming those regions into a grid shape using different methods. Figure 6 A and Figure 6 B illustrates the process and results of region division using the power diagram method. Figure 6 A represents the position of the category in the first category calculated in step 530 on the display page 600, denoted as 601 to 603, respectively corresponding to... Figure 1 The categories of nodes 101-103. Assuming K=16, in the first category... Figure 1 The categories of nodes 101-103 contain 8, 4, and 4 samples from the first group of K samples, respectively, resulting in a proportion of 1 / 2, 1 / 4, and 1 / 4 for each category. Based on positions 601-603 and the proportions of each category, the power diagram method, as shown in Figure B, is used to divide the area into regions 611-613, corresponding to the three categories, with an area ratio of 2:1:1. The power diagram method ensures that the region allocated to each category is a compact convex polygon.

[0057] In addition to the existing power diagram method, another implementation method can be to use an iterative horizontal and vertical division method to calculate the area allocated to each category on the display page. Figure 7 A flowchart is shown of a method 700 for calculating the area to be divided in a display page for each category in a first category using an iterative horizontal and vertical division method according to one or more embodiments of the present disclosure.

[0058] like Figure 7 As shown, in step 710, the page will be displayed in relation to all categories in the first category as the initialization of the iteration.

[0059] In step 720, it is determined whether each planar space is associated with only one category. If so, the partitioned region corresponding to each category is obtained, and the process ends; otherwise, it proceeds to iterative step 730.

[0060] In step 730, a planar space associated with more than one category and its associated categories are selected. These categories are then divided into two parts, either vertically or horizontally, based on the position of each category in the first category on the display page. The current planar space is further divided according to the sum of the proportions of each category on each side relative to the number of samples in the first group of K samples, ensuring that the aspect ratios of the two resulting planar spaces are as close as possible. The categories on each side are then associated with their respective resulting planar spaces. Step 720 is then returned to, and the iteration continues until each planar space is associated with only one category, at which point method 700 ends.

[0061] Figure 6 A, Figure 6 D and Figure 6 E illustrates the process and results of region division using an iterative horizontal and vertical partitioning method. Specifically, according to step 710, Figure 6Categories 601 through 603 in A are initially associated with display page 600. Subsequently, as... Figure 6 As shown in D, in step 730, display page 600 and its three associated categories are selected and divided into left and right sections. Category 601 is assigned to the left, associated with the partitioned space 631, accounting for 1 / 2; categories 602 and 603 are assigned to the right, associated with the partitioned space 632, accounting for 1 / 4 + 1 / 4 = 1 / 2. In the next iteration, as... Figure 6 As shown in E, space 632 and its two associated categories are selected and divided into upper and lower parts. Category 603 is assigned to the upper part, associated with space 613', occupying 1 / 4 of the area; category 602 is assigned to the lower part, associated with space 612', also occupying 1 / 4. At this point, each space is associated with only one category. Renaming the previous space 631 as 611', spaces 611' to 613' correspond to three categories respectively, with an area ratio of 2:1:1. This method can obtain rectangular divisions and also achieve compact and convex region shapes.

[0062] Back Figure 5 In step 550, based on the area allocated to each category on the display page in the first category, the area is converted into a grid shape, such that the number of grids in each grid shape is consistent with the number of samples contained in each category of the first group of K samples. In one implementation, a greedy method is used, the steps of which include:

[0063] (1) Divide the display page into K grids; (2) Calculate the coverage relationship between the area corresponding to each category in the first category and each grid in the K grids based on the K grids and the area divided on the display page for each category in the first category. This coverage relationship can be represented as an M'×K matrix, where the i-th row and j-th column of the matrix is ​​the overlap area between the area corresponding to the i-th category and the j-th grid; (3) Sort the M'×K matrix items of the coverage relationship in descending order of overlap area, and assign the specific grid corresponding to each item to the specific category corresponding to that item in turn, while ensuring that the area of ​​the grid area assigned to each category in the first category does not exceed the area of ​​the area divided on the display page for that category, until all K grids are assigned, and at this time the grid shape of each category in the first category on the display page is obtained.

[0064] Figure 6 C and Figure 6 F shows the result of the mesh shape transformation performed in step 550, where Figure 6 The grid shapes 621 to 623 in C are respectively Figure 6The result of the transformation of regions 611 to 613 in B is that the number of grids in the three regions are 8, 4, and 4, respectively. Figure 6 The grid shapes 621' to 623' in F are respectively Figure 6 The result of the transformation of regions 611' to 613' in E is that the number of grids in the three regions are 8, 4, and 4, respectively.

[0065] In step 560, grid positions are assigned to the first group of K samples based on the grid shape of each category in the first category on the display page, the sample features of the first group of K samples, and the membership degree of the first group of K samples to each category in the first category, such that samples with similar sample features are placed at close grid positions in the grid shape of the region, and confused samples are placed at positions close to the confusion boundary of the grid shape of the region.

[0066] Here, confused samples can be defined based on the membership degrees of the first group of K samples to each category in the first category. In one implementation, a threshold τ can be set, and for a specific sample, if its membership degree to its respective category (i.e., the maximum value of its membership degree to each category in the first category) is less than the threshold τ, it is determined to be a confused sample.

[0067] For a specific confused sample, its confusion boundary to a specific category can be defined as the boundary between the mesh shape corresponding to the membership category of the specific sample and the mesh shape corresponding to the specific category. Layouts generated by existing methods Figure 2 One problem with B is that the scrambled samples are not placed on the grid near the corresponding scrambled boundaries.

[0068] Figure 8 A schematic illustration shows an example of obfuscated samples within an obfuscated boundary in a mesh shape. (See diagram.) Figure 8 As shown in A, Figure 6 Taking the grid shape in C as an example, assuming that a certain confused sample belongs to the category of grid shape 621, then its confusion boundary with the category corresponding to grid shape 623 is 801, that is, the boundary between grid shapes 621 and 623.

[0069] Step 560 includes step 1: Based on the sample features of the first group of K samples, calculate the high-dimensional similarity between any two samples in each category of the first category within the first group of K samples. Taking a specific category as an example, assuming that the specific category contains K' samples, the high-dimensional similarity between any two samples can be represented as a K'×K' matrix, where the item p in the i-th row and j-th column of the matrix... ijThis represents the high-dimensional similarity between the i-th sample and the j-th sample within that specific category. In one implementation, according to the existing T-SNE method, the high-dimensional similarity can be calculated using a Gaussian distribution.

[0070] Step 2: Based on the grid shape of each category in the first category on the display page, calculate the low-dimensional similarity between any two grids in the corresponding grid shape of each category in the first category. Taking a specific category as an example, assuming that the specific category contains K' samples, the corresponding grid shape contains K' grids. The low-dimensional similarity between any two grids can be represented as a K'×K' matrix, where the term q in the i-th row and j-th column of the matrix... ij This represents the low-dimensional similarity between the i-th and j-th grids contained in the grid shape corresponding to that specific category. In one implementation, the low-dimensional similarity can be calculated using a T-distribution according to the existing T-SNE method.

[0071] Step 3: Based on the grid shape of each category in the first category on the display page and the membership degree of the first group of K samples to each category in the first category, obtain the confusion cost function from each confused sample to the confusion boundary. Taking a specific category in the first category as an example, assuming that the specific category contains K' samples, the corresponding grid shape contains K' grids. Assuming that the i-th sample it contains is a confused sample, and it is placed on the j-th grid in the grid shape corresponding to the specific category, then the confusion cost of this sample to the t-th category in the first category is f. it *r jt , where f it Let r be the membership degree of the i-th sample to the t-th category in the first category. jt Let r be the squared distance from the j-th grid to the boundary between the grid shape corresponding to this specific category and the grid shape corresponding to the t-th category. If the shapes corresponding to the two categories do not border each other, or if the two categories are the same, then r... jt =0. The smaller the confusion cost, the closer the i-th sample is to its confusion boundary.

[0072] Step 4: Based on the high-dimensional similarity between any two samples in each category of the first group of K samples, the low-dimensional similarity between any two grids in the grid shape corresponding to each category of the first group, and the confusion cost function from each confused sample to the confusion boundary, obtain the grid position corresponding to each sample in the first group of K samples. In one implementation, a quadratic assignment problem solution method can be used to solve for the grid position corresponding to each sample in the first group of K samples. Specifically, taking a specific category in the first group as an example, assuming that the specific category contains K' samples, the optimization objective of the quadratic assignment problem is expressed as:

[0073]

[0074] Where the 0-1 variable δ ij This indicates whether the i-th sample is placed on the j-th grid, with 1 indicating yes and 0 indicating no, and requires ∑ i δ ij =1,∑ j δ ij =1. The first term of the above optimization objective is the proximity cost term, which, referring to the existing T-SNE method, is calculated using the KL divergence between high-dimensional similarity and positional similarity. The second term is the confusion cost term, which uses variable i in the above formula to enumerate and calculate all K' samples included in the specific category. In one implementation, it is also possible to enumerate and calculate only the confused samples among the K' samples. The coefficient ω1 is the weight coefficient of the confusion cost term, which can be set to a fixed value or set using any multi-objective optimization method.

[0075] For the optimization objective of the aforementioned quadratic allocation problem, any optimization method can be used to obtain a set of valid δ values ​​that correctly represent the correspondence between samples and grid cells. 11 to δ K′K′ This minimizes the total cost, thereby placing each of the K' samples onto one of the K' grids. In one implementation, an existing FAQ algorithm is used to solve the problem. The method is applied to each category in the first category, thereby placing each of the first group of K samples onto one of the K grids.

[0076] Through the above process 500, the layout position 304 of each sample in the first group of K samples on the display page is calculated. In this layout, the shapes corresponding to categories with similar feature averages in the first category are located close to each other. The grid layout of samples belonging to the same category in the first category forms a compact and convex shape. Similar samples in the same category are placed on grids with close positions, and confused samples are placed near the confusion boundary.

[0077] Because layout position 304 fully considers the convexity and compactness of the grid shape corresponding to the category, similar samples are placed in close positions and confused samples are placed near the boundary. Therefore, the displayed results can help users better perceive the similarity and clustering structure of the samples, and at the same time correctly judge the confusion nature of the samples.

[0078] In one implementation, Figure 4 Step 440 of method 400 further includes: firstly, determining the specific category of the first category to which each sample in the first group of K samples belongs, based on the calculated membership degree of each sample in the first group of K samples to each category in the first category; secondly, calculating the average sample feature of the samples contained in each category of the first category in the first group of K samples; thirdly, calculating the position of each category in the first category on the display page, such that categories with similar average sample feature values ​​are placed in close proximity; then, dividing the display page into regions based on the position of each category in the first category on the display page and the specific category to which each sample in the first group of K samples belongs, such that the area ratio of the region corresponding to each category in the first category is proportional to the proportion of the number of samples contained in that category in the first group of K samples. The proportions are consistent; then, based on the area divided on the display page for each category in the first category, the area is transformed into a grid shape, such that the number of grids in each grid shape is consistent with the number of samples contained in each category of the first group of K samples; finally, based on the grid shape of each category in the first category on the display page, the sample features of the first group of K samples, and the membership degree of the first group of K samples to each category in the first category, grid positions are assigned to the first group of K samples, such that samples with similar features are placed in close grid positions in the grid shape of the area, and confused samples are placed near the confusion boundary of the grid shape of the area, wherein the confused samples and the confusion boundary are defined according to the membership degree of the first group of K samples to each category in the first category and the grid shape of each category in the first category on the display page.

[0079] In one implementation, grid positions are assigned to the first group of K samples based on the grid shape of each category in the first category on the display page, the sample features of the first group of K samples, and the membership degree of the first group of K samples to each category in the first category. Samples with similar features are placed at adjacent grid positions within the grid shape of the region, and confused samples are placed near the confusion boundary of the grid shape of the region. The confused samples and confusion boundary are defined based on the membership degree of the first group of K samples to each category in the first category and the grid shape of each category in the first category on the display page, including: firstly, based on the sample features of the first group of K samples, calculating the values ​​of any two samples from each category within the first group of K samples. The algorithm calculates the high-dimensional similarity between samples; secondly, based on the grid shape of each category in the first category on the display page, it calculates the low-dimensional similarity between any two grids in the corresponding grid shape of each category in the first category; thirdly, based on the grid shape of each category in the first category on the display page and the membership degree of the first group of K samples to each category in the first category, it obtains the confusion cost function from each confused sample to the confusion boundary; finally, based on the high-dimensional similarity between any two samples in each category of the first category in the first group of K samples, the low-dimensional similarity between any two grids in the corresponding grid shape of each category in the first category, and the confusion cost function from each confused sample to the confusion boundary, it obtains the grid position corresponding to each sample in the first group of K samples. In one embodiment, a quadratic assignment problem solving method can be used to solve for the grid position corresponding to each sample in the first group of K samples. Those skilled in the art will know that other methods can also be used to solve for the grid position corresponding to each sample in the first group of K samples.

[0080] In one implementation, system 300 further includes an association module 370, configured to, for the first group of K samples displayed on the display page, calculate the similarity between each sample in the remaining NK samples (excluding the first group of K samples) of the large-scale dataset and each sample in the first group of K samples; and, based on the calculated similarity, associate each of the NK samples with the most similar sample in the first group of K samples. This module will obtain the association 307 between the remaining samples and the first group of K samples to facilitate subsequent interactive exploration.

[0081] Simultaneously, system 300 also allows users to select areas on the display page and further explore the dataset. Specifically, in response to the selection of a specific grid area on the display page, modules 310 to 370 are further configured and executed.

[0082] Specifically, the acquisition module 310 is further configured to acquire samples corresponding to each grid in a specific grid region, wherein the number of samples corresponding to each grid in the specific grid region is A.

[0083] Figure 8 B illustrates an example of selecting a specific grid area on a display page. For example... Figure 1 and Figure 8 As shown in B, in Figure 6 Taking the layout result of placing samples on the grid shape shown in C as an example, in Figure 8 In B, a specific grid region 810 is selected, which contains 4 grids, corresponding to 4 samples in the first group of K samples, i.e., A=4.

[0084] The extraction module 320 is further configured to randomly extract KA samples from the samples associated with the samples corresponding to each grid in a specific grid region, wherein the KA samples and the KA samples constitute a second group of K samples 302. Simultaneously, the KA samples and the samples associated with the KA samples constitute a second sample exploration space.

[0085] The membership calculation module 330 is further configured to obtain the categories of the nodes in the category hierarchy corresponding to the second group of K samples 302, as the second category, and to calculate the membership degree 303 of each sample in the second group of K samples 302 to each category in the second category. The second category is obtained by expanding down from the first category, that is, the category corresponding to the child nodes of the node corresponding to the first category in the category hierarchy. The membership degree of a specific sample to a specific category is calculated by summing the membership degrees of that specific sample to all the categories corresponding to the bottom-level category nodes contained below that specific category in the category hierarchy.

[0086] like Figure 1 and Figure 8 As shown in B, Figure 8 In B, the four samples in a specific grid region 810 correspond to the categories of nodes 101 and 103 in the first category, and together with the eight samples collected from the remaining samples, form the second group of K samples. Figure 1 In the first category, node set 101' expands downwards into node set 102' of the second category. Node 101 expands downwards into child nodes 104 and 105, node 103 expands downwards into 106 and 107, and node 102 is the bottom node in the hierarchical structure and is not expanded.

[0087] The layout module 340 is further configured to calculate the layout position 304 of each sample in the second group of K samples on the display page based on the membership degree 303 of each sample in the second group of K samples 302 to each category in the second category, the features of each sample in the second group of K samples 302 in the large-scale dataset 301, and the selected specific grid region.

[0088] The color allocation module 350 is further configured to assign a color to each category in the second category, thereby obtaining the display color 305 corresponding to the second category.

[0089] The display module 360 ​​is further configured to display the second group of K samples 302 on the display page according to the layout position 304 of the second group of K samples 302 and the display color 305 corresponding to the second category, based on the layout position and the assigned color.

[0090] In one implementation, the layout module 340 can be further configured to adopt... Figure 9 The method 900 for calculating the layout position 304 is shown. Figure 9 A flowchart is shown of a method 900 for calculating the layout position 304 of each sample in a second group of K samples 302 on a display page, according to one or more embodiments of the present disclosure. Figure 5 Compared to the illustrated process 500, the first category and the first group of K samples in process 500 are replaced with the second category and the second group of K samples in process 900.

[0091] Specifically, in step 910, based on the calculated membership degree of each sample in the second group of K samples to each category in the second category, the specific category to which each sample in the second group of K samples belongs in the second category is determined. In one implementation, for a specific sample, the category with the highest membership degree is selected as the category to which the sample belongs. For example, when the model prediction confidence is used as the membership degree, the category with the highest prediction confidence in the second category is selected as the specific category to which the sample belongs in the second category.

[0092] In step 920, the average sample feature of each sample in the second category within the second group of K samples is calculated. Taking a specific category in the second category as an example, assuming that the category contains K' samples in the second group of K samples, the average sample feature of these K' samples is calculated as the average feature of the category.

[0093] In step 930, the position of each category in the second category on the display page is calculated such that categories with similar average sample feature values ​​are placed in close proximity, and the position of each category in the second category remains stable relative to the layout of A samples in the selected specific grid area.

[0094] In step 940, based on the selected specific grid area, the position of each category in the second category on the display page, and the specific category that each sample in the second group of K samples belongs to in the second category, the display page is divided into regions such that the area ratio of the region corresponding to each category in the second category is consistent with the ratio of the number of samples contained in that category to the number of samples in the second group of K samples, and the region corresponding to each category in the second category remains stable compared to the region corresponding to the first category in the selected specific grid area.

[0095] In step 950, based on the area allocated to each category on the display page in the second category, the area is converted into a grid shape, such that the number of grids in each grid shape is consistent with the number of samples contained in each category of the second group of K samples. In one implementation, step 950 can be implemented in essentially the same way as step 550, the only difference being that step 950 uses the second group of K samples and the second category instead of the first group of K samples and the first category in step 550.

[0096] In step 960, based on the grid shape of each category in the second category on the display page, the sample features of the second group of K samples, and the membership degree of the second group of K samples to each category in the second category, grid positions are assigned to the second group of K samples, such that samples with similar sample features are placed at close grid positions in the grid shape of the region, and confused samples are placed at positions close to the confusion boundary of the grid shape of the region, wherein the confused samples and the confusion boundary are defined according to the membership degree of the second group of K samples to each category in the second category, while the positions assigned to the A samples in the selected specific grid region remain stable relative to their relative positions in the selected specific grid region.

[0097] In one implementation, step 930 can be performed by introducing a stability cost based on the existing KK-Layout algorithm. Specifically, the difference between any two categories in the second category is first calculated. In one implementation, let the feature averages of the two categories be f. i with f j The degree of difference can then be calculated using the Euclidean distance:

[0098] d ij =||f i -f j ||

[0099] If there are M' categories in the second category, then the calculated difference matrix D will be of size M'×M'. This difference matrix can also be calculated in any other way, even by incorporating other information such as prediction confidence.

[0100] After calculating the dissimilarity, the following proximity cost function is established according to the KK-Layout algorithm:

[0101]

[0102] Where c1 to c M′ This refers to the position of each category in the second category on the display page.

[0103] Subsequently, based on the selected specific grid region and A samples within that specific grid region, the average position of each sample from the second category within those A samples is calculated within the specific grid region. Taking a specific category within the second category as an example, assuming it contains A' samples within those A samples, the average position of that category within the specific grid region is the average of the layout positions of those A' samples within the specific grid region. If a specific category within the second category does not contain any samples within those A samples, its average position is not calculated. Let Z = {z1, ..., z2} be the set of categories within the second category that contain at least one sample within those A samples. |Z|}, its average position in a specific grid area is To facilitate subsequent calculations, when calculating the average position, a specific grid area is scaled to the same size as the display page, such that {c1, ..., c M′}and They are in the same size range.

[0104] Then, based on the calculated average position The following stability cost function is defined:

[0105]

[0106] The advantage of optimizing this stability cost function is that it minimizes the difference between the newly calculated position of each category in the second category and its position in the selected specific grid region, thereby maintaining stability.

[0107] Therefore, based on the proximity cost function and the stability cost function, the following overall optimization objective needs to be obtained:

[0108]

[0109] Where μ is the weight coefficient of the stability cost function, which can be set to a fixed value or set using any multi-objective optimization method.

[0110] Finally, for the overall optimization objective mentioned above, use any optimization method to find a set of values ​​from c1 to C. M′The goal is to minimize the overall optimization objective value. In one implementation, the L-BFGS algorithm can be used for optimization. This optimization problem can also be solved using any other method. Those skilled in the art will know that step 930 can also be accomplished using other existing algorithms, such as SGD, BFGS, etc.

[0111] Figure 10 A-10C shows that... Figure 8 This is an example of the process and result of determining the category position in the second category after a specific grid region 810 in B is selected, using step 930. Assuming that in the second category, based on the category to which each sample in the second group of K samples belongs, as determined in step 910, only the category of nodes 104-106 contains samples in the second group of K samples, then only these three categories will be considered in the subsequent layout. Figure 10 A shows a specific grid area, compared to Figure 8 The values ​​in 810 are consistent. Suppose that in A samples within a specific grid region, the sample at grid 1001 belongs to the category of node 104, the samples at grids 1002-1003 belong to the category of node 105, and the sample at grid 1004 belongs to the category of node 106. Here, a different [classification] is used. Figure 8 The color B is used to distinguish them. Figure 10 B shows the average positions 1011 to 1013 of the three categories in the specific grid area after scaling the specific grid area to the same size as the display page 1000. These positions correspond to the categories of nodes 104 to 106, where position 1011 is the position of grid 1001, position 1012 is the average position of grids 1002 to 1003, and position 1013 is the position of grid 1004. Figure 10 C shows the positions 1021 to 1023 of the categories calculated based on the proximity cost function and the stability cost function on the display page, corresponding to the categories of nodes 104 to 106, respectively.

[0112] In one implementation, step 940 can first calculate the proportion of samples in each of the second categories out of the total number of samples in the second group of K samples, based on the specific category each sample in the second group of K samples belongs to within the second category. Taking a specific category within the second category as an example, assuming that this category contains K' samples in the second group of K samples, then the proportion of this category is...

[0113] Subsequently, based on the hierarchical relationship between the first and second categories and the proportion of samples contained in each category of the second category relative to the total number of samples in the second group of K, the proportion of samples contained in each category of the first category relative to the total number of samples in the second group of K is calculated. Taking a specific category in the first category as an example, assuming that this category contains M” categories from the second category in the category hierarchy, then the proportion of this category is the sum of the proportions of these M” categories.

[0114] Then, based on the position of each category in the second category on the display page, the proportion of samples contained in each category in the second category to the total number of samples in the second group of K samples, and the proportion of samples contained in each category in the first category to the total number of samples in the second group of K samples, different methods can be used to obtain the corresponding region for each category in the second category. In one implementation method, the existing power diagram method can be used for calculation.

[0115] Specifically, suppose that in a certain partitioning operation, the current space is to be divided into regions corresponding to M” categories, and the position of each category corresponding to each region is c. i Its cost function is:

[0116]

[0117] Among them, Ω1 to Ω M″ The current space is divided into regions. These regions must be non-overlapping, their union equal to the current space, and the proportion of a region corresponding to a certain category within the planar space must be proportional to the proportion of samples of that category in the second group of K samples. For the above cost function, use any optimization method to find a set of Ω1 to Ω... M′ This minimizes the cost function value. In one implementation, the L-BFGS algorithm can be used for optimization. This optimization problem can also be solved using any other method. The power diagram method ensures that the region assigned to each category is a compact convex polygon.

[0118] To ensure the stability of the partitioned regions compared to specific grid areas, the partitioning will be performed in two steps. In the first step, based on the region corresponding to the first category within the selected specific grid area and the proportion of samples in each category within the first category relative to the K samples in the second group, the power diagram method is used to partition the entire display page in the same way as the region corresponding to the first category within the specific grid area (i.e., using the same parameters c as in the previous execution of steps 540 or 940). i(Calculations are performed), ensuring that the area ratio of each category in the first category is consistent with the sample number ratio, thus obtaining the corresponding division area of ​​each category in the first category on the current display page. In the second step, based on the corresponding division area of ​​each category in the first category on the current display page, and according to the position of each category in the second category on the display page and the proportion of the number of samples contained in each category in the second category to the number of samples in the second group of K samples, the powerdiagram method is used to further divide the corresponding division area of ​​the first category on the current display page into the corresponding division area of ​​each category in the second category (i.e., based on the parameter c obtained in step 930 of this execution). i (Perform calculations).

[0119] Figure 11 A-11C illustrates the specific grid area and... Figure 10 This example demonstrates the process and results of calculating category positions in C and using the powerdiagram method to partition the region for the second category. Figure 11 A shows the division regions 1101 to 1102 corresponding to the first category in a specific grid area, respectively. Figure 1 The categories of nodes 101 and 103 are defined. Assume that in the second group of K samples, 4 samples belong to the category of node 104, 6 samples belong to the category of node 105, and 6 samples belong to the category of node 106. Then the sample percentages for each category in the second category are 1 / 4, 3 / 8, and 3 / 8, respectively. In the first category, the sample percentage for node 101 is 1 / 4 + 3 / 8 = 5 / 8, and the sample percentage for node 103 is 3 / 8. Figure 11 B shows the result of the first step of the partitioning, which is based on... Figure 11 In the same way as in A, the display page will be divided into regions 1111 and 1112, which correspond to the categories of nodes 101 and 103 in the first category, respectively, and the ratio of the areas of the two regions is 5:3. Figure 11 C shows the result of the second step of partitioning. In the second step, the region 1111 corresponding to the category of node 101 is determined according to... Figure 10 Positions 1021 and 1022 in C are divided into two regions 1121 and 1122, which correspond to the categories of nodes 104 and 105 in the second category, respectively. The category of node 103 in the first category only includes the category of node 106 in the second category. Therefore, region 1112 is re-recorded as 1123, corresponding to the category of node 106 in the second category.

[0120] In addition to the existing power diagram method, in another implementation, step 930 can also employ an iterative horizontal and vertical division method to calculate the area allocated to each category on the display page. This is based on... Figure 12The flowchart of method 1200 is shown. Figure 12 A flowchart illustrating a method for calculating the area allocated to each category in a display page for each category in a second category using an iterative horizontal and vertical partitioning approach, according to one or more embodiments of this disclosure, is shown. Figure 7 Compared to the illustrated process 700, the first category and the first group of K samples in process 700 are replaced with the second category and the second group of K samples in process 1200.

[0121] Specifically, step 1210 is further configured to divide the entire display page in the same way as the area corresponding to the first category in the selected specific grid area, and the proportion of the number of samples contained in each category in the first category to the number of samples in the second group of K samples, using an iterative horizontal and vertical division method (i.e., dividing in the same way as when step 540 or 940 was executed last time), and ensuring that the area ratio of each category in the first category is consistent with the sample number ratio, thereby obtaining the corresponding division area of ​​each category in the first category in the current display page, and associating the categories contained in the second category of each category in the first category with the division area corresponding to that category in the first category as the initialization for further iterations.

[0122] In step 1220, it is determined whether all planar spaces are associated with only one category. If so, the partitioned region corresponding to each category is obtained, and the process ends; otherwise, it proceeds to iterative step 1230.

[0123] In step 1230, a planar space associated with more than one category and its associated categories are selected. These categories are then divided into two parts, either vertically or horizontally, based on the position of each category in the second category on the display page. The current planar space is further divided according to the sum of the proportions of each category on both sides of the second group of K samples, ensuring that the aspect ratios of the two resulting planar spaces are as close as possible. The categories on each side are then associated with the planar spaces on their respective sides. Step 1220 is then returned to, and the iteration continues until each planar space is associated with only one category, at which point method 1200 ends.

[0124] Method 1200 can obtain rectangular division regions, and can also obtain compact and convex region shapes, while keeping the region corresponding to each category of the second category stable compared to the region corresponding to the first category in the selected specific grid region.

[0125] Figure 11 D shows the... Figure 11 C executes step 950 to perform the mesh shape transformation, where mesh shapes 1131 to 1133 are respectively Figure 11The result of transforming regions 1121 to 1123 in C is that the number of grids in the three regions are 4, 6, and 6, respectively.

[0126] In one implementation, the confusing samples in step 960 can be defined based on the membership degrees of the second group of K samples to each category in the second category. In one implementation, a threshold τ can be set, and for a specific sample, it is determined to be a confusing sample when its membership degree to its category (i.e., the maximum value of its membership degree to each category in the second category) is less than the threshold τ.

[0127] For a specific confused sample, its confusion boundary to a specific category is defined as the boundary between the grid shape corresponding to the class to which the sample belongs and the grid shape corresponding to the specific class.

[0128] Step 960 further includes step 1': Based on the sample features of the second group of K samples, calculate the high-dimensional similarity between any two samples in each category of the second category within the second group of K samples. Taking a specific category as an example, assuming that the specific category contains K' samples, the high-dimensional similarity between any two samples can be represented as a K'×K' matrix, where the item p in the i-th row and j-th column of the matrix... ij This represents the high-dimensional similarity between the i-th sample and the j-th sample within that specific category. In one implementation, according to the existing T-SNE method, the high-dimensional similarity can be calculated using a Gaussian distribution.

[0129] Step 2': Based on the grid shape of each category in the second category on the display page, calculate the low-dimensional similarity between any two grids in the corresponding grid shape of each category in the second category. Taking a specific category as an example, assuming that the specific category contains K' samples, the corresponding grid shape contains K' grids, and the low-dimensional similarity between any two grids can be represented as a K'×K' matrix, where the term q in the i-th row and j-th column of the matrix... ij This represents the low-dimensional similarity between the i-th and j-th grids contained in the grid shape corresponding to that specific category. In one implementation, the low-dimensional similarity can be calculated using a T-distribution according to the existing T-SNE method.

[0130] Step 3': Based on the grid shape of each category in the second category on the display page and the membership degrees of the second set of K samples to each category in the second category, obtain the confusion cost function for each confused sample to the confusion boundary. Taking a specific category in the second category as an example, assuming that the specific category contains K' samples, the corresponding grid shape contains K' grids. Assuming that the i-th sample it contains is a confused sample, and it is placed on the j-th grid in the grid shape corresponding to the specific category, then the confusion cost of this sample to the t-th category in the second category is f. it δ jt , where f it Let r be the membership degree of the i-th sample to the t-th category in the second category. jt Let r be the squared distance from the j-th grid to the boundary between the grid shape corresponding to this specific category and the grid shape corresponding to the t-th category. If the shapes corresponding to the two categories do not border each other, or if the two categories are the same, then r... jt =0. The smaller the confusion cost, the closer the i-th sample is to its confusion boundary.

[0131] Step 4': Based on the positions of the A samples within a specific grid region, obtain the stability cost function for those A samples. Taking a specific category within the second category as an example, assume that this specific category contains K' samples, the corresponding grid shape contains K' grids, and the index set of the samples included in the A samples of this specific category within the K' samples is O = {o1, ..., o}. |O| Suppose that for O, the o-th sample in the K' samples... i The sample and the oth j There are 1 sample, denoted as _ . and Its position within a specific grid area is and And it is placed on the kth and 1st grids of the grid shape corresponding to this specific category, denoted as v. k With v l Then the stability cost can be defined as:

[0132]

[0133] The parameters of the h function are the positions of the two samples within a specific grid region (the first parameter pair) and the positions of the two samples within the current grid shape (the second parameter pair). The h function is defined as the number of times the order of the two sets of positions is reversed on the horizontal or vertical coordinates. Taking the horizontal coordinate as an example, if the first sample is located to the left of the second sample in the specific grid region (i.e., in the formula...), then... The x-coordinate is less than The x-coordinate), and in the current grid shape, it is located to the right of the second sample (i.e., v in the formula). k The x-coordinate is greater than v l If the x-coordinate is not specified, then it is determined that the order has been reversed on the horizontal coordinate. The minimum value of the h function is 0 (i.e., no order has been reversed on either the horizontal or vertical coordinates), and the maximum value is 2 (i.e., the order has been reversed on both the horizontal and vertical coordinates).

[0134] Figure 13 A-13D shows an example of the h function used to calculate the stability cost. Figure 13 A shows the category in the second category within a specific grid area (corresponding to...). Figure 1 The two samples under node 105 in the graph correspond to grids 1301 and 1302, which are related to... Figure 10 A is consistent, and grids 1301 and 1302 correspond to respectively. Figure 10 In sample A, both samples 1002 and 1003 belong to the second category. Figure 1 The category of node 105. Since the two samples have the same ordinate in a specific grid area, only the case where their order on the abscissa is reversed is considered. Figure 13 B to Figure 13 The grid shape shown in D, which contains 6 grids, is... Figure 11 The grid shape in D is 1132, which is... Figure 1 The mesh shape corresponding to the category of node 105. For example... Figure 13 As shown in B and 13C, when the samples corresponding to grids 1301 and 1302 are placed in the positions of grids 1311 and 1312, or grids 1321 and 1322 respectively, the order of the horizontal coordinates remains consistent with the order of grids 1301 and 1302 in the specific grid region, and the h function value is 0. Figure 13 As shown in D, when the samples corresponding to grids 1301 and 1302 are placed at positions 1331 and 1332 respectively, the order of the horizontal coordinates is reversed compared to the order of grids 1301 and 1302 in a specific grid region, and the h function value is 1.

[0135] Step 5': Based on the high-dimensional similarity between samples in each category of the second class within the second set of K samples, the low-dimensional similarity between grids in the grid shape corresponding to each category of the second class, the confusion cost function from each confused sample to the confusion boundary, and the stability cost function of A samples in a specific grid region, obtain the grid position corresponding to each sample in the second set of K samples. In one implementation, a quadratic assignment method can be used to solve for the grid position corresponding to each sample in the second set of K samples. Specifically, taking a specific category in the second class as an example, assuming that the specific category contains K' samples, the optimization objective of the quadratic assignment problem is expressed as:

[0136]

[0137] Where the 0-1 variable δ ij This indicates whether the i-th sample is placed on the j-th grid, with 1 indicating yes and 0 indicating no, and requires ∑ i δ ij =1,∑ j δ ij =1. The first term of the above optimization objective is the proximity cost term, which, referring to the existing T-SNE method, is calculated using the KL divergence of high-dimensional similarity and positional similarity. The second term is the confusion cost term, which uses variable i in the above formula to enumerate all K' samples included in the specific category. In one implementation, only the confused samples among the K' samples can be enumerated. The third term is the stability cost term, which counts the number of times the order of the current sample layout is reversed compared to the sample layout in a specific grid area. The coefficient ω1 is the weight coefficient of the confusion cost term, and the coefficient ω2 is the weight coefficient of the stability cost term. They can be set to fixed values ​​or set using any multi-objective optimization method.

[0138] For the optimization objective of the aforementioned quadratic allocation problem, any optimization method can be used to obtain a set of valid δ values ​​that correctly represent the correspondence between samples and grid cells. 11 to δ K′K′ This minimizes the total cost by placing each of the K' samples onto one of the K' grids. In one implementation, an existing FAQ algorithm is used to solve the problem. The method is applied to each category in the second category, thus placing each of the second group of K samples onto one of all K grids.

[0139] Through the above process 900, the layout position 304 of each sample in the second group of K samples on the display page is calculated. In this layout, the shapes corresponding to categories with similar feature averages in the second category are located close to each other. The grid layout of samples belonging to the same category in the second category forms a compact and convex shape. Compared with the stability of the layout shape and position before the interaction, similar samples in the same category are laid out on grids with close positions, confused samples are laid out near the confusion boundary, and the relative positions between selected samples in a specific area remain stable.

[0140] Because layout position 304 fully considers the convexity and compactness of the grid shape corresponding to the category, similar samples are placed in close positions, and confused samples are placed near the boundary, and the layout remains stable compared to the layout before interaction, the displayed results can help users better perceive the similarity and clustering structure of samples, smoothly track and analyze samples during the exploration process, and correctly judge the confusion nature of samples.

[0141] In one implementation, the association module 370 is further configured to, for the second group of K samples displayed on the display page and the second sample exploration space in the further configured extraction module 320, calculate the similarity between each sample in the remaining samples after removing the second group of K samples and each sample in the second group of K samples; and, based on the calculated similarity, associate each sample in the remaining samples with the most similar sample in the second group of K samples. This module will obtain the association 307 between the remaining samples and the second group of K samples to facilitate subsequent interactive exploration.

[0142] Simultaneously, system 300 also supports users selecting areas on the display page and further exploring the dataset, namely exploring subcategories of the current first or second category, generating a new second category and a new second group of K samples, forming a new layout and displaying it. In one embodiment, this disclosure also discloses a computer program product, the computer program product including program instructions that can be executed by a computing device to cause the computing device to perform the method described in any of the above methods.

[0143] This invention can be a computer system, method, and / or computer program product. The method of this invention can be executed on a standalone computer system, on a distributed computing system, or even on a cloud platform.

[0144] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of computer systems, methods, and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of computer systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0146] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer system for visualizing the grid layout of a large-scale dataset with a hierarchical class structure, the system comprising: The acquisition module is configured to acquire a large-scale dataset with a category hierarchy and the number K of samples to be displayed on the display page. The large-scale dataset contains N samples, a category hierarchy, and the membership degree of each sample to a category. Each of the N samples contains sample features. In the category hierarchy, the N samples are classified into a multi-level structure. The category of the child node in the next level of the category hierarchy is a further classification result of the category of the parent node in the previous level. In the category hierarchy, the category of each node in the next level below the root node is the first category. The number of categories in the bottom level of the category hierarchy is M. The membership degree of each sample to a category includes the membership degree of each of the N samples to each of the M categories. The extraction module is configured to randomly extract a first group of K samples from the N samples; The membership calculation module is configured to calculate the membership degree of each sample in the first group of K samples to each category in the first category; The layout module is configured to calculate the layout position of each sample in the first group of K samples on the display page based on the calculated membership degree of each sample in the first group of K samples to each category in the first category and the sample features of each sample in the first group of K samples; and to assign grid positions to the first group of K samples based on the grid shape of each category in the first category on the display page, the sample features of the first group of K samples, and the membership degree of the first group of K samples to each category in the first category, such that samples with similar features are placed at close grid positions in the grid shape of the region, and that confused samples are placed near the confusion boundary of the grid shape of the region, wherein the confused samples and the confusion boundary are defined according to the membership degree of the first group of K samples to each category in the first category and the grid shape of each category in the first category on the display page, specifically including: Based on the sample features of the first group of K samples, calculate the high-dimensional similarity between any two samples in each category of the first category in the first group of K samples; Based on the grid shape of each category in the first category on the display page, calculate the low-dimensional similarity between any two grids in the grid shape corresponding to each category in the first category; Based on the grid shape of each category in the first category on the display page and the membership degree of the first group of K samples to each category in the first category, obtain the confusion cost function of each confused sample to the confusion boundary; Based on the high-dimensional similarity between any two samples in each category of the first group of K samples, the low-dimensional similarity between any two grids in the grid shape corresponding to each category of the first group, and the confusion cost function from each confused sample to the confusion boundary, the grid position corresponding to each sample in the first group of K samples is obtained. The color assignment module is configured to assign a color to each category in the first category; The display module is configured to display the first group of K samples on the display page according to the layout position and the assigned color.

2. The computer system according to claim 1, wherein the layout module is further configured as follows: Based on the calculated membership degree of each sample in the first group of K samples to each category in the first category, determine the specific category in the first category to which each sample in the first group of K samples belongs; Calculate the average sample feature of the samples in each category of the first category within the first group of K samples; Calculate the position of each category in the first category on the display page, such that categories with similar average sample feature values ​​are placed in close proximity; Based on the position of each category in the first category on the display page and the specific category in the first group of K samples to which each sample belongs, the display page is divided into regions such that the area ratio of the region corresponding to each category in the first category is consistent with the ratio of the number of samples contained in that category to the number of samples in the first group of K samples. Based on the area that each category in the first category is divided into on the display page, the area is transformed into a grid shape such that the number of grids in each grid shape is consistent with the number of samples contained in each category of the first category in the first group of K samples.

3. The computer system according to any one of claims 1-2, further comprising: The association module is configured to, for the first group of K samples displayed on the display page, calculate the similarity between each sample in the remaining NK samples in the large-scale dataset (excluding the first group of K samples) and each sample in the first group of K samples. And based on the calculated similarity, each of the NK samples is associated with the most similar sample in the first group of K samples.

4. The computer system of claim 3, wherein in response to a specific grid area in the display page being selected: The acquisition module is further configured to obtain a sample corresponding to each grid in the specific grid region, wherein the number of samples corresponding to each grid in the specific grid region is A; The extraction module is further configured to randomly extract KA samples from the samples associated with the samples corresponding to each grid in the specific grid region, wherein the A samples and the KA samples constitute a second group of K samples; The membership calculation module is further configured to obtain the category of the node in the category hierarchy corresponding to the second group of K samples, as the second category, and to calculate the membership degree of each sample in the second group of K samples to each category in the second category. The layout module is further configured to calculate the layout position of each sample in the second group of K samples on the display page based on the calculated membership degree of each sample in the second group of K samples to each category in the second category, the sample features of each sample in the second group of K samples, and the selected specific grid area. The color allocation module is further configured to assign a color to each category in the second category; The display module is further configured to display the second group of K samples on the display page according to the layout position and the assigned color.

5. The computer system according to claim 4, wherein the layout module is further configured as follows: Based on the calculated membership degree of each sample in the second group of K samples to each category in the second category, determine the specific category in the second category to which each sample in the second group of K samples belongs; Calculate the average sample feature of the samples in each category of the second category within the second group of K samples; Calculate the position of each category in the second category on the display page, such that categories containing samples with similar average sample features are placed in close proximity, and such that the position of each category in the second category remains stable relative to the layout of the A samples in the selected specific grid area; Based on the selected specific grid area, the position of each category in the second category on the display page, and the specific category to which each sample in the second group of K samples belongs, the display page is divided into regions such that the area ratio of the region corresponding to each category in the second category is consistent with the ratio of the number of samples contained in that category to the number of samples in the second group of K samples, and the region corresponding to each category in the second category remains stable compared to the region corresponding to the first category in the selected specific grid area; Based on the area that each category in the second category is divided into on the display page, the area is transformed into a grid shape, such that the number of grids in each grid shape is consistent with the number of samples contained in each category of the second category in the second group of K samples; Based on the grid shape of each category in the second category on the display page, the sample features of the second group of K samples, and the membership degree of the second group of K samples to each category in the second category, grid positions are assigned to the second group of K samples such that samples with similar features are placed at close grid positions in the grid shape of the region, and confused samples are placed at positions close to the confusion boundary of the grid shape of the region, wherein the confused samples and the confusion boundary are defined according to the membership degree of the second group of K samples to each category in the second category, while the positions assigned to the A samples in the selected specific grid region remain stable relative to their relative positions in the selected specific grid region.

6. A method for visualizing the grid layout of a large-scale dataset with a hierarchical class structure, the method comprising: Obtain a large-scale dataset with a hierarchical category structure and the number K of samples to be displayed on the display page. The large-scale dataset contains N samples, a hierarchical category structure, and the membership degree of each sample to its category. Each of the N samples contains sample features. In the hierarchical category structure, the N samples are classified into a multi-level structure. The category of the child node in the next level of the hierarchical category structure is a further classification result of the category of the parent node in the previous level. In the hierarchical category structure, the category of each node in the next level below the root node is the first category. The number of categories in the bottom level of the hierarchical category structure is M. The membership degree of each sample to its category includes the membership degree of each of the N samples to each of the M categories. Randomly select the first group of K samples from the N samples; Calculate the membership degree of each sample in the first group of K samples to each category in the first category; Based on the calculated membership degree of each sample in the first group of K samples to each category in the first category and the sample features of each sample in the first group of K samples, the layout position of each sample in the first group of K samples on the display page is calculated; based on the grid shape of each category in the first category on the display page, the sample features of the first group of K samples, and the membership degree of the first group of K samples to each category in the first category, grid positions are assigned to the first group of K samples, such that samples with similar features are placed at close grid positions in the grid shape of the region, and confused samples are placed near the confusion boundary of the grid shape of the region, wherein the confused samples and the confusion boundary are defined according to the membership degree of the first group of K samples to each category in the first category and the grid shape of each category in the first category on the display page, specifically including: Based on the sample features of the first group of K samples, calculate the high-dimensional similarity between any two samples in each category of the first category in the first group of K samples; Based on the grid shape of each category in the first category on the display page, calculate the low-dimensional similarity between any two grids in the grid shape corresponding to each category in the first category; Based on the grid shape of each category in the first category on the display page and the membership degree of the first group of K samples to each category in the first category, obtain the confusion cost function of each confused sample to the confusion boundary; Based on the high-dimensional similarity between any two samples in each category of the first group of K samples, the low-dimensional similarity between any two grids in the grid shape corresponding to each category of the first group, and the confusion cost function from each confused sample to the confusion boundary, the grid position corresponding to each sample in the first group of K samples is obtained. Assign a color to each category in the first category; The first group of K samples are displayed on the display page according to the layout position and assigned color.

7. The method according to claim 6, wherein calculating the layout position of each sample in the first group of K samples on the display page based on the calculated membership degree of each sample in the first group of K samples to each category in the first category and the sample features of each sample in the first group of K samples includes: Based on the calculated membership degree of each sample in the first group of K samples to each category in the first category, determine the specific category in the first category to which each sample in the first group of K samples belongs; Calculate the average sample feature of the samples in each category of the first category within the first group of K samples; Calculate the position of each category in the first category on the display page, such that categories with similar average sample feature values ​​are placed in close proximity; Based on the position of each category in the first category on the display page and the specific category in the first group of K samples to which each sample belongs, the display page is divided into regions such that the area ratio of the region corresponding to each category in the first category is consistent with the ratio of the number of samples contained in that category to the number of samples in the first group of K samples. Based on the area that each category in the first category is divided into on the display page, the area is transformed into a grid shape such that the number of grids in each grid shape is consistent with the number of samples contained in each category of the first category in the first group of K samples.

8. A computer program product comprising program instructions executable by a computing device to cause the computing device to perform the method as described in any one of claims 6-7.

Images