A system for visualizing grid layouts of multiple samples

Abstract

The present invention discloses a system for visualizing a grid layout of multiple samples, comprising a sample acquisition module for acquiring multiple image samples; a feature extraction module for extracting high-dimensional feature vectors of the multiple image samples; a dimension reduction module for performing similarity calculation on the high-dimensional feature vectors of the multiple image samples, and reducing the dimensions of the high-dimensional feature vectors to a two-dimensional space based on the similarity; and a visualization layout module for performing grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on the similarity. The present invention measures the consistency of the similarity distribution between the high-dimensional space and the low-dimensional space by introducing KL divergence in the dimension reduction process, and updates the two-dimensional coordinate matrix by gradient descent optimization, thereby significantly improving the fidelity of the sample distribution in the low-dimensional space to the original similarity in the high-dimensional space, so that the visualization grid layout of the image samples is more accurate in reflecting the content similarity between the samples.

Description

Technical Field

[0001] The invention relates to a visualization system, in particular to a system for performing grid layout visualization on multiple samples. Background Art

[0002] In the field of data analysis and visualization, dimensionality reduction technology is widely used to process high-dimensional data so that it can be presented in a more intuitive way in two-dimensional or three-dimensional space. Patent document with patent publication number CN113326874A discloses a data dimensionality reduction method and device based on t-SNE algorithm, which makes the similarity of sample data with high similarity higher and the similarity of sample data with low similarity lower. The low-dimensional samples after dimensionality reduction can better reflect the spatial distribution of samples in high-dimensional space;

[0003] However, it has significant deficiencies in dimensionality reduction of image samples in visualization scenarios. The high-dimensional features of image samples usually contain their content information, while traditional dimensionality reduction techniques such as PCA (Principal Component Analysis) and t-SNE (Distributed Stochastic Neighbor Embedding) often only focus on the dimensionality reduction of high-dimensional feature vectors, while ignoring the preservation of similarities between samples in high-dimensional space. As a result, in the visualization grid layout of image samples, the spatial position of the samples may be inconsistent with the similarity of their content. In addition, it is difficult for existing technologies to maintain the correlation between image samples and their identification information after dimensionality reduction, which further limits the application of dimensionality reduction technology in image sample visualization. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention provides a system for visualizing a grid layout of multiple samples, which solves the technical problems raised in the background technology by calculating similarity.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0006] A system for visualizing a grid layout of multiple samples, comprising:

[0007] A sample acquisition module, used to acquire multiple image samples;

[0008] Feature extraction module, used to extract high-dimensional feature vectors of multiple image samples;

[0009] The dimension reduction module is used to calculate the similarity of high-dimensional feature vectors of multiple image samples and reduce their dimension to two-dimensional space based on the similarity;

[0010] The visualization layout module is used to perform a grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on similarity.

[0011] In some embodiments, the step of extracting high-dimensional feature vectors of multiple image samples by the feature extraction module includes:

[0012] S2-1, assigning consecutive identification IDs to multiple image samples;

[0013] S2-2, preprocessing the plurality of image samples to generate a plurality of image sample tensors;

[0014] S2-3, encode the continuous identification ID to obtain the encoded ID feature;

[0015] S2-4, inputting the image sample tensor into a convolutional model to extract high-dimensional features of the image sample;

[0016] S2-5. Concatenate the high-dimensional features of the image sample and the encoded ID features to obtain a high-dimensional feature vector of the image sample.

[0017] In some embodiments, the dimension reduction module calculates similarity of high-dimensional feature vectors of multiple image samples and reduces their dimension to two-dimensional space based on the similarity, including:

[0018] S3-1, combining the high-dimensional feature vectors of the plurality of image samples into a grid feature matrix; wherein the row vector of the grid feature matrix is ​​the high-dimensional feature vector of each image sample, and the high-dimensional feature vector of the image sample is regarded as a sample of the grid feature matrix;

[0019] The expression of the grid feature matrix is:

[0020] F = {f1, f2, ..., f N}, f i ∈R d ;

[0021] Where F represents the grid feature matrix, represents a set of real numbers, N is the total number of image samples, and d represents the dimension of the feature vector;

[0022] S3-2, calculating the high-dimensional similarity of the sample pair according to the row vector of any sample pair in the grid feature matrix; wherein any sample pair represents the high-dimensional feature vector of any two image samples;

[0023] The calculation expression of the high-dimensional similarity is:

[0024]

[0025] Among them, S ij represents the unnormalized high-dimensional similarity between sample i and sample j, f i and f jRepresent the high-dimensional feature vectors of samples i and j respectively, ||f i -f j || 2 represents the square of the Euclidean distance between sample i and sample j in high-dimensional space; σ represents the Gaussian kernel width of high-dimensional similarity, which is used to adjust the sensitivity of high-dimensional similarity to Euclidean distance;

[0026] S3-3, constructing a high-dimensional similarity matrix according to the high-dimensional similarity of the sample pairs; wherein the matrix elements of the high-dimensional similarity matrix represent the high-dimensional similarity of any sample pair;

[0027] The expression of the high-dimensional similarity matrix is:

[0028] S={S(i, j)}, i, j∈{1,...,N};

[0029] Among them, S represents a high-dimensional similarity matrix;

[0030] S3-4, projecting the high-dimensional similarity matrix into a two-dimensional space, and constructing a target two-dimensional coordinate matrix in the two-dimensional space.

[0031] In some embodiments, projecting the high-dimensional similarity matrix into a two-dimensional space and constructing a target two-dimensional coordinate matrix in the two-dimensional space includes:

[0032] S3-4-1, symmetrizing the high-dimensional similarity matrix to obtain a symmetric high-dimensional matrix; wherein the matrix elements in the symmetric high-dimensional matrix are symmetric similarities;

[0033] The expression of the symmetric similarity is:

[0034]

[0035] Among them, S(i, j) represents the high-dimensional similarity between sample i and sample j, S(j, i) represents the high-dimensional similarity between sample j and sample i, and S sym (i, j) represents the symmetric similarity between sample i and sample j;

[0036] S3-4-2, normalizing the symmetric similarity in the symmetric high-dimensional matrix and defining it as a high-dimensional similarity probability distribution;

[0037] The expression of the high-dimensional similarity probability distribution is:

[0038]

[0039] Among them, P ijrepresents the high-dimensional similarity probability distribution of sample i and sample j, representing the similarity ratio of sample i and sample j in the high-dimensional space; (k, l) represents the index of sample i and sample j, representing the traversal of any two samples from 1 to N respectively; k≠1 means ignoring the matrix diagonal; It represents the sum of the high-dimensional similarities of all two samples in the symmetric high-dimensional matrix;

[0040] S3-4-3, generating initial two-dimensional coordinates for each sample, collecting the initial two-dimensional coordinates of all samples, and constructing an initial two-dimensional coordinate matrix; wherein the matrix elements of the initial two-dimensional coordinate matrix are the tuples of the ID features and the initial two-dimensional coordinates;

[0041] The expression of the initial two-dimensional coordinate matrix is:

[0042]

[0043] Among them, Z (0) Represents the initial two-dimensional coordinate matrix, ID i represents the ID feature of sample i, represents the initial two-dimensional coordinates of sample i;

[0044] S3-4-4. In the initial two-dimensional coordinate matrix, define the inverse of the square of the Euclidean distance between sample i and sample j as the low-dimensional similarity, and calculate the low-dimensional similarity probability distribution of sample i and sample j by normalization;

[0045] The expression of the low-dimensional similarity probability distribution is:

[0046]

[0047] Among them, z i =(x i ,y i ) represents the coordinate vector of sample i in two-dimensional space, z j =(x j ,y j ) represents the coordinate vector of sample j in two-dimensional space, ||z i -z j || 2 represents the square of the Euclidean distance between sample i and sample j in two-dimensional space; (1+||z i -z j || 2 ) -1 It means that the inverse of the square of the Euclidean distance is defined as the unnormalized low-dimensional similarity; represents the sum of low-dimensional similarities between any two samples in two-dimensional space; Q ijRepresents the low-dimensional similarity probability distribution of sample i and sample j in two-dimensional space;

[0048] S3-4-5, minimizing the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution, updating each sample two-dimensional coordinate to update the initial two-dimensional coordinate matrix to a target two-dimensional coordinate matrix; wherein the matrix elements of the target two-dimensional coordinate matrix are the target two-dimensional coordinates and the ID features of the sample;

[0049] The target two-dimensional coordinate matrix represents: the maximum matching degree of similarity of sample pairs in high-dimensional space in two-dimensional space.

[0050] In some embodiments, minimizing the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution, updating each sample two-dimensional coordinate to update the initial two-dimensional coordinate matrix to the target two-dimensional coordinate matrix, includes:

[0051] S3-4-5-1. Define the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution as KL divergence;

[0052] The expression of the KL divergence is:

[0053]

[0054] Where C represents KL divergence;

[0055] S3-4-5-2. Calculate the gradient of the KL divergence relative to each sample coordinate in the two-dimensional coordinate matrix;

[0056] The expression of the gradient of each sample coordinate is:

[0057]

[0058] in, Represents the gradient of KL divergence with respect to the coordinates of sample i;

[0059] S3-4-5-3. Use the gradient descent method to update the two-dimensional coordinates of each sample until the minimum KL divergence is reached, and generate the target two-dimensional coordinate matrix.

[0060] In some embodiments, the two-dimensional coordinates of each sample are updated using a gradient descent method until the minimum KL divergence is reached to generate the target two-dimensional coordinate matrix, including:

[0061] S3-4-5-3-1. Based on the gradient of the KL divergence for each sample's two-dimensional coordinate, update each sample's two-dimensional coordinate to obtain an updated coordinate vector;

[0062] The expression of the updated coordinate vector is:

[0063]

[0064] in, represents the coordinate vector of sample i at the tth iteration, represents the updated coordinate vector of sample i at the t+1th iteration, η represents the learning rate, which is used to control the step size of each coordinate update. represents the gradient of KL divergence with respect to the coordinates of sample i at the tth iteration;

[0065] S3-4-5-3-2, using the updated coordinate vector as a matrix element to construct a sample coordinate update matrix;

[0066] S3-4-5-3-3. Use the sample coordinates to update the matrix and recalculate the updated low-dimensional similarity probability distribution;

[0067] S3-4-5-3-4. Calculate the updated KL divergence according to the updated low-dimensional similarity probability distribution;

[0068] S3-4-5-3-5. When the updated KL divergence reaches the minimum KL divergence, each sample two-dimensional coordinate is deemed to be updated to the target two-dimensional coordinate;

[0069] S3-4-5-3-6. Combine the target two-dimensional coordinates of each sample to generate the target two-dimensional coordinate matrix.

[0070] In some embodiments, the step of the visualization layout module performing a grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on similarity includes:

[0071] S4-1, obtaining a target two-dimensional coordinate matrix;

[0072] S4-2, extracting the target two-dimensional coordinates of the target two-dimensional coordinate matrix;

[0073] S4-3. Perform a visual grid layout on the multiple image samples according to the target two-dimensional coordinates.

[0074] In some of the embodiments, performing a visual grid layout on the plurality of image samples according to the target two-dimensional coordinates includes:

[0075] S4-3-1. Obtain normalized sample coordinates according to the target two-dimensional coordinates;

[0076] S4-3-2. Generate a visual grid layout of multiple image samples based on the normalized sample coordinates.

[0077] In some of the embodiments, obtaining the normalized sample coordinates according to the target two-dimensional coordinates includes:

[0078] S4-3-1-1, select the coordinate range of the horizontal and vertical coordinates in the target two-dimensional coordinate matrix; the coordinate range is characterized by: the maximum and minimum values ​​of the horizontal coordinate, and the maximum and minimum values ​​of the vertical coordinate;

[0079] S4-3-1-2, define the spatial scale of the grid layout according to the coordinate range of the horizontal and vertical coordinates;

[0080] S4-3-1-3. Linearly scale the target two-dimensional coordinates and map them to the spatial scale of the grid layout to generate normalized sample coordinates that fit the grid layout.

[0081] In some of the embodiments, generating a visual grid layout of a plurality of image samples according to the normalized sample coordinates includes:

[0082] S4-3-2-1. Draw a two-dimensional layout diagram of the normalized sample coordinates according to the normalized sample coordinates;

[0083] S4-3-2-2, according to the identification ID, associating the normalized sample coordinates with the image sample;

[0084] S4-3-2-3. Overlay the image samples with the associated normalized sample coordinates onto the two-dimensional layout diagram to generate a visual grid layout of the multiple image samples.

[0085] The present invention provides a system for visualizing a grid layout of multiple samples, which has the following beneficial effects:

[0086] The present invention introduces KL divergence in the dimensionality reduction process to measure the consistency of the similarity distribution between high-dimensional space and low-dimensional space, and updates the two-dimensional coordinate matrix through gradient descent optimization, which significantly improves the fidelity of the sample distribution in the low-dimensional space to the original similarity in the high-dimensional space, making the visual grid layout of the image samples more accurate in reflecting the content similarity between samples;

[0087] Furthermore, the present invention ensures the complete retention of sample identification information during the dimensionality reduction process by encoding the image sample identification ID and splicing it with the high-dimensional feature vector; at the same time, the linear scaling method is used to normalize the two-dimensional coordinates, and combined with the image sample overlay technology, the image samples are intuitively distributed in the two-dimensional layout diagram, generating a visual grid layout based on content similarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A structural block diagram of a system for visualizing a grid layout of multiple samples according to the present invention;

[0089] Figure 2 A schematic diagram of a visualization process of a system for visualizing a grid layout of multiple samples according to the present invention;

[0090] Figure 3 A schematic diagram of the process of constructing a target two-dimensional coordinate matrix according to the present invention;

[0091] Figure 4 Schematic diagram of updating the two-dimensional coordinates of the sample of the present invention. DETAILED DESCRIPTION

[0092] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0093] First, the prior art and related concepts involved in the embodiments of the present invention are described:

[0094] KL divergence: Kullback-Leibler (KL) divergence, also known as relative entropy, is a concept in information theory and statistics that measures the difference between two probability distributions P and Q. Specifically, it quantifies the additional information loss when probability distribution Q is used to represent data originally generated by probability distribution P.

[0095] Example 1: Please refer to Figures 1 to 4 The present invention provides a system for visualizing a grid layout of multiple samples, comprising:

[0096] A sample acquisition module, used to acquire multiple image samples;

[0097] Feature extraction module, used to extract high-dimensional feature vectors of multiple image samples;

[0098] The dimension reduction module is used to calculate the similarity of high-dimensional feature vectors of multiple image samples and reduce their dimension to two-dimensional space based on the similarity;

[0099] The visualization layout module is used to perform a grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on similarity.

[0100] This embodiment obtains multiple image samples, extracts their high-dimensional feature vectors, calculates similarity based on the high-dimensional feature vectors and reduces the dimension to a two-dimensional space, and finally realizes a grid layout of multiple image samples based on similarity in a two-dimensional space, which can effectively and intuitively display the correlation and distribution pattern between image samples and improve the visualization effect of sample data.

[0101] In this embodiment, the step of extracting high-dimensional feature vectors of multiple image samples by the feature extraction module includes:

[0102] S2-1, assigning consecutive identification IDs to multiple image samples;

[0103] S2-2, preprocessing the plurality of image samples to generate a plurality of image sample tensors;

[0104] S2-3, encode the continuous identification ID to obtain the encoded ID feature;

[0105] S2-4, inputting the image sample tensor into a convolutional model to extract high-dimensional features of the image sample;

[0106] S2-5, concatenate the high-dimensional features of the image sample and the encoded ID features to obtain a high-dimensional feature vector of the image sample. After the concatenation operation, the high-dimensional feature vector is normalized to ensure that features of different dimensions have a balanced impact on subsequent similarity calculations.

[0107] This embodiment ensures the uniqueness and traceability of each sample by assigning continuous identification IDs to image samples; preprocesses the image samples to convert them into a tensor form suitable for convolutional model input, laying the foundation for subsequent efficient feature extraction; encodes the identification ID and concatenates it with high-dimensional features, thereby enhancing the ability of the feature vector to distinguish samples while retaining the identification information of the samples.

[0108] In this embodiment, the dimension reduction module calculates the similarity of high-dimensional feature vectors of multiple image samples and reduces their dimension to a two-dimensional space based on the similarity, including:

[0109] S3-1, combining the high-dimensional feature vectors of the plurality of image samples into a grid feature matrix; wherein the row vector of the grid feature matrix is ​​the high-dimensional feature vector of each image sample, and the high-dimensional feature vector of the image sample is regarded as a sample of the grid feature matrix;

[0110] The expression of the grid feature matrix is:

[0111] F = {f1, f2, ..., f N}, f i ∈R d ;

[0112] Where F represents the grid feature matrix, represents a set of real numbers, N is the total number of image samples, and d represents the dimension of the feature vector;

[0113] S3-2, calculating the high-dimensional similarity of the sample pair according to the row vector of any sample pair in the grid feature matrix; wherein any sample pair represents the high-dimensional feature vector of any two image samples;

[0114] The calculation expression of the high-dimensional similarity is:

[0115]

[0116] Among them, S ij represents the unnormalized high-dimensional similarity between sample i and sample j, f i and f j Represent the high-dimensional feature vectors of samples i and j respectively, ||f i -f j || 2 represents the square of the Euclidean distance between sample i and sample j in high-dimensional space; σ represents the Gaussian kernel width of high-dimensional similarity, which is used to adjust the sensitivity of high-dimensional similarity to Euclidean distance; that is, the smaller σ is, the more sensitive high-dimensional similarity is to changes in Euclidean distance, and the larger σ is, the less sensitive high-dimensional similarity is to changes in Euclidean distance. The Gaussian kernel width σ can be selected using the mean distance of the data distribution as the initial width.

[0117] S3-3, constructing a high-dimensional similarity matrix according to the high-dimensional similarity of the sample pairs; wherein the matrix elements of the high-dimensional similarity matrix represent the high-dimensional similarity of any sample pair;

[0118] The expression of the high-dimensional similarity matrix is:

[0119] S={S(i, j)}, i, j∈{1,...,N};

[0120] Among them, S represents a high-dimensional similarity matrix;

[0121] S3-4, projecting the high-dimensional similarity matrix into a two-dimensional space, and constructing a target two-dimensional coordinate matrix in the two-dimensional space.

[0122] This embodiment effectively organizes the high-dimensional feature structure of the samples by combining the high-dimensional feature vectors of multiple image samples into a grid feature matrix; by calculating the high-dimensional similarity of any sample pair based on the row vector, the Euclidean distance characteristics between samples are fully utilized, combined with the dynamic selection of the Gaussian kernel width, the sensitivity of the similarity calculation is improved; further, a high-dimensional similarity matrix is ​​constructed to clearly characterize the similarity relationship between all sample pairs; finally, through dimensionality reduction projection, the complex high-dimensional similarity information is effectively mapped to the two-dimensional space to generate the target two-dimensional coordinate matrix.

[0123] Furthermore, the specific steps of step S3-4 also include:

[0124] S3-4-1, symmetrizing the high-dimensional similarity matrix to obtain a symmetric high-dimensional matrix; wherein the matrix elements in the symmetric high-dimensional matrix are symmetric similarities;

[0125] The expression of the symmetric similarity is:

[0126]

[0127] Among them, S(i, j) represents the high-dimensional similarity between sample i and sample j, S(j, i) represents the high-dimensional similarity between sample j and sample i, and S sym (i, j) represents the symmetric similarity between sample i and sample j;

[0128] S3-4-2, normalizing the symmetric similarity in the symmetric high-dimensional matrix and defining it as a high-dimensional similarity probability distribution;

[0129] The expression of the high-dimensional similarity probability distribution is:

[0130]

[0131] Among them, P ij represents the high-dimensional similarity probability distribution of sample i and sample j, representing the similarity ratio of sample i and sample j in the high-dimensional space; (k, l) represents the index of sample i and sample j, representing the traversal of any two samples from 1 to N respectively; k≠1 means ignoring the matrix diagonal, that is, ignoring the high-dimensional similarity of the sample itself; It represents the sum of the high-dimensional similarities of all two samples in the symmetric high-dimensional matrix;

[0132] S3-4-3, generating initial two-dimensional coordinates for each sample, collecting the initial two-dimensional coordinates of all samples, and constructing an initial two-dimensional coordinate matrix; wherein the matrix elements of the initial two-dimensional coordinate matrix are the tuples of the ID features and the initial two-dimensional coordinates;

[0133] The expression of the initial two-dimensional coordinate matrix is:

[0134]

[0135] Among them, Z (0) Represents the initial two-dimensional coordinate matrix, ID i represents the ID feature of sample i, represents the initial two-dimensional coordinates of sample i;

[0136] S3-4-4. In the initial two-dimensional coordinate matrix, define the inverse of the square of the Euclidean distance between sample i and sample j as the low-dimensional similarity, and calculate the low-dimensional similarity probability distribution of sample i and sample j by normalization;

[0137] The expression of the low-dimensional similarity probability distribution is:

[0138]

[0139] Among them, z i =(x i ,y i ) represents the coordinate vector of sample i in two-dimensional space, z j =(x j ,y j ) represents the coordinate vector of sample j in two-dimensional space, ||z i -z j || 2 represents the square of the Euclidean distance between sample i and sample j in two-dimensional space; (1+||z i -z j || 2 ) -1 It means that the inverse of the square of the Euclidean distance is defined as the unnormalized low-dimensional similarity; represents the sum of low-dimensional similarities between any two samples in two-dimensional space; Q ij Represents the low-dimensional similarity probability distribution of sample i and sample j in two-dimensional space;

[0140] Specifically, Q ij Represents the low-dimensional similarity probability distribution of sample i and sample j in the initial two-dimensional coordinate matrix in the two-dimensional space. The relationship between sample i and sample j is determined by their Euclidean distance; by defining the inverse of the Euclidean distance as the low-dimensional similarity, the following relationship can be expressed:

[0141] When the Euclidean distance of any pair of samples is close, their low-dimensional similarity is large, indicating that the correlation between sample i and sample j in the two-dimensional space is strong;

[0142] On the contrary, when the Euclidean distance of any pair of samples is far, their low-dimensional similarity is small, indicating that the correlation between sample i and sample j in the two-dimensional space is weak.

[0143] S3-4-5, minimizing the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution, updating each sample two-dimensional coordinate to update the initial two-dimensional coordinate matrix to a target two-dimensional coordinate matrix; wherein the matrix elements of the target two-dimensional coordinate matrix are the target two-dimensional coordinates and the ID features of the sample;

[0144] The target two-dimensional coordinate matrix represents: the maximum matching degree of similarity of sample pairs in high-dimensional space in two-dimensional space.

[0145] This embodiment eliminates the deviation that may be caused by asymmetry and ensures the symmetry and consistency of sample similarity by symmetrizing the high-dimensional similarity matrix; further constructs the high-dimensional similarity probability distribution through normalization, and clarifies the relative similarity ratio of each pair of samples in the high-dimensional space; on this basis, generates initial two-dimensional coordinates for each sample, and defines the low-dimensional similarity probability distribution as the inverse of the square of the Euclidean distance, thereby establishing a correlation model between high-dimensional and low-dimensional similarities; finally, by minimizing the difference between the high-dimensional and low-dimensional similarity probability distributions, dynamically adjusts the two-dimensional coordinates of each sample, generates a target two-dimensional coordinate matrix, and realizes the accurate projection of high-dimensional similarity relationships in two-dimensional space.

[0146] Furthermore, the specific steps of step S3-4-5 also include:

[0147] S3-4-5-1. Define the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution as KL divergence;

[0148] The expression of the KL divergence is:

[0149]

[0150] Where C represents KL divergence;

[0151] S3-4-5-2. Calculate the gradient of the KL divergence relative to each sample coordinate in the two-dimensional coordinate matrix;

[0152] The expression of the gradient of each sample coordinate is:

[0153]

[0154] in, Represents the gradient of KL divergence with respect to the coordinates of sample i;

[0155] S3-4-5-3. Use the gradient descent method to update the two-dimensional coordinates of each sample until the minimum KL divergence is reached, and generate the target two-dimensional coordinate matrix.

[0156] Furthermore, the specific steps of step S3-4-5-3 also include:

[0157] S3-4-5-3-1. Based on the gradient of the KL divergence for each sample's two-dimensional coordinate, update each sample's two-dimensional coordinate to obtain an updated coordinate vector;

[0158] The expression of the updated coordinate vector is:

[0159]

[0160] in, represents the coordinate vector of sample i at the tth iteration, represents the updated coordinate vector of sample i at the t+1th iteration, η represents the learning rate, which is used to control the step size of each coordinate update. represents the gradient of KL divergence with respect to the coordinates of sample i at the tth iteration;

[0161] S3-4-5-3-2, using the updated coordinate vector as a matrix element to construct a sample coordinate update matrix;

[0162] S3-4-5-3-3. Use the sample coordinates to update the matrix and recalculate the updated low-dimensional similarity probability distribution;

[0163] S3-4-5-3-4. Calculate the updated KL divergence according to the updated low-dimensional similarity probability distribution;

[0164] S3-4-5-3-5. When the updated KL divergence reaches the minimum KL divergence, each sample two-dimensional coordinate is deemed to be updated to the target two-dimensional coordinate;

[0165] S3-4-5-3-6. Combine the target two-dimensional coordinates of each sample to generate the target two-dimensional coordinate matrix.

[0166] This embodiment defines KL divergence as the difference measure between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution, and combines the gradient descent method to gradually optimize the two-dimensional coordinates of each sample, ensuring that the high-dimensional similarity distribution characteristics in the dimensionality reduction process can achieve maximum matching in the two-dimensional space. By dynamically adjusting the step size of each update, the optimization process converges quickly and stably, and the target two-dimensional coordinate matrix finally generated can restore the similarity relationship of samples in the high-dimensional space with high fidelity.

[0167] Embodiment 2: The technical solution of Embodiment 2 is different from that of Embodiment 1 in that the visualization layout module performs a grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on similarity, including:

[0168] S4-1, obtaining a target two-dimensional coordinate matrix;

[0169] S4-2, extracting the target two-dimensional coordinates of the target two-dimensional coordinate matrix;

[0170] S4-3. Perform a visual grid layout on the multiple image samples according to the target two-dimensional coordinates.

[0171] Furthermore, the specific steps of step S4-3 also include:

[0172] S4-3-1. Obtain normalized sample coordinates according to the target two-dimensional coordinates;

[0173] S4-3-2. Generate a visual grid layout of multiple image samples based on the normalized sample coordinates.

[0174] Furthermore, the specific steps of step S4-3-1 also include:

[0175] S4-3-1-1, select the coordinate range of the horizontal and vertical coordinates in the target two-dimensional coordinate matrix; the coordinate range is characterized by: the maximum and minimum values ​​of the horizontal coordinate, and the maximum and minimum values ​​of the vertical coordinate;

[0176] S4-3-1-2, define the spatial scale of the grid layout according to the coordinate range of the horizontal and vertical coordinates;

[0177] S4-3-1-3. Linearly scale the target two-dimensional coordinates and map them to the spatial scale of the grid layout to generate normalized sample coordinates that fit the grid layout.

[0178] Since the range of the original target 2D coordinates may not be suitable for direct grid layout (for example, it exceeds the image boundary or is too small to make the distribution unclear), these coordinates need to be linearly transformed to map them to a fixed target range. The linear transformation can be expressed using the MMS formula:

[0179]

[0180] Among them, x scaled represents the normalized sample coordinates, x represents the original coordinates, and x min Indicates the minimum value of the original coordinate, x max Indicates the maximum value of the original coordinates, scale max Indicates the maximum value of the target coordinates, scale min Identify the minimum value of the target coordinate. Use the MMS method to linearly map the target two-dimensional coordinate from the original coordinate range of the target two-dimensional coordinate [x min ,x max ]Convert to target range[s calemin, scale max ] to generate normalized sample coordinates.

[0181] Furthermore, the specific steps of step S4-3-2 also include:

[0182] S4-3-2-1. Draw a two-dimensional layout diagram of the normalized sample coordinates according to the normalized sample coordinates;

[0183] A point is drawn using the two-dimensional coordinates of each normalized sample so that each point can reflect its relative position relationship in the two-dimensional space.

[0184] S4-3-2-2. According to the identification ID, the normalized sample coordinates are associated with the image sample; thereby establishing a mapping relationship between the normalized sample coordinates and the original image sample to ensure that the image sample can be correctly superimposed on its corresponding position. Each normalized coordinate corresponds to its unique identification ID, and the identification ID is assigned in the previous processing (such as S2-1 or when extracting features). The correspondence between the ID feature and the normalized coordinate can be established through a data structure (such as a dictionary, a key-value pair, or a database).

[0185] S4-3-2-3. Overlay the image samples with the associated normalized sample coordinates onto the two-dimensional layout diagram to generate a visual grid layout of the multiple image samples.

[0186] The image samples are superimposed on the positions of the normalized coordinate points to generate an intuitive grid layout visualization result. Each normalized coordinate point can be used as the center point of the image sample, and the size of the image sample can be adjusted to avoid overlap. In the drawn two-dimensional layout diagram, each point is replaced with the corresponding image sample, and the final visualization result is presented as a grid layout of the image samples.

[0187] This embodiment solves the problem that the original coordinate range is not suitable for direct visualization by performing a grid layout on multiple image samples in a two-dimensional space and utilizing the normalization operation and linear mapping technology of the target two-dimensional coordinate matrix. By introducing the MMS linear scaling method, the target two-dimensional coordinates are mapped to the target layout range, and the normalized sample coordinates are generated, thereby achieving accurate positioning of the sample points in the two-dimensional layout diagram. Based on the normalized coordinates, the image samples are further associated with the identification ID, and the image samples are superimposed on the normalized coordinate positions, ultimately generating an intuitive image grid layout visualization effect. This embodiment improves the visualization effect of the grid layout while effectively retaining the relative position relationship between samples.

[0188] The above embodiments can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0189] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0190] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division of a waterway underwater terrain change analysis system and method. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A system for visualizing a grid layout of multiple samples, characterized in that: include: A sample acquisition module, used to acquire multiple image samples; Feature extraction module, used to extract high-dimensional feature vectors of multiple image samples; The step of extracting high-dimensional feature vectors of multiple image samples by the feature extraction module includes: S2-1, assigning consecutive identification IDs to multiple image samples; S2-2, preprocessing the plurality of image samples to generate a plurality of image sample tensors; S2-3, encode the continuous identification ID to obtain the encoded ID feature; S2-4, inputting the image sample tensor into a convolutional model to extract high-dimensional features of the image sample; S2-5, concatenating the high-dimensional features of the image sample and the encoded ID features to obtain a high-dimensional feature vector of the image sample; The dimension reduction module is used to calculate the similarity of high-dimensional feature vectors of multiple image samples and reduce their dimension to two-dimensional space based on the similarity; The visualization layout module is used to perform a grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on similarity.

2. A system for visualizing a grid layout of multiple samples according to claim 1, characterized in that: The dimension reduction module calculates similarity of high-dimensional feature vectors of multiple image samples and reduces their dimension to two-dimensional space based on the similarity, including: S3-1, combining the high-dimensional feature vectors of the plurality of image samples into a grid feature matrix; wherein the row vector of the grid feature matrix is ​​the high-dimensional feature vector of each image sample, and the high-dimensional feature vector of the image sample is regarded as a sample of the grid feature matrix; The expression of the grid feature matrix is: ； Where F represents the grid feature matrix, represents a set of real numbers, N is the total number of image samples, and d represents the dimension of the feature vector; S3-2, calculating the high-dimensional similarity of the sample pair according to the row vector of any sample pair in the grid feature matrix; wherein any sample pair represents the high-dimensional feature vector of any two image samples; The calculation expression of the high-dimensional similarity is: ； in, represents the unnormalized high-dimensional similarity between sample i and sample j, and Represent the high-dimensional feature vectors of samples i and j respectively, Represents the square of the Euclidean distance between sample i and sample j in high-dimensional space; The width of the Gaussian kernel representing high-dimensional similarity is used to adjust the sensitivity of high-dimensional similarity to Euclidean distance; S3-3, constructing a high-dimensional similarity matrix according to the high-dimensional similarity of the sample pairs; wherein the matrix elements of the high-dimensional similarity matrix represent the high-dimensional similarity of any sample pair; The expression of the high-dimensional similarity matrix is: ； Among them, S represents a high-dimensional similarity matrix; S3-4, projecting the high-dimensional similarity matrix into a two-dimensional space, and constructing a target two-dimensional coordinate matrix in the two-dimensional space.

3. A system for visualizing a grid layout of multiple samples according to claim 2, characterized in that: Projecting the high-dimensional similarity matrix into a two-dimensional space and constructing a target two-dimensional coordinate matrix in the two-dimensional space includes: S3-4-1, symmetrizing the high-dimensional similarity matrix to obtain a symmetric high-dimensional matrix; wherein the matrix elements in the symmetric high-dimensional matrix are symmetric similarities; The expression of the symmetric similarity is: ； in, represents the high-dimensional similarity between sample i and sample j, represents the high-dimensional similarity between sample j and sample i, Represents the symmetric similarity between sample i and sample j; S3-4-2, normalizing the symmetric similarity in the symmetric high-dimensional matrix and defining it as a high-dimensional similarity probability distribution; The expression of the high-dimensional similarity probability distribution is: ； in, Represents the high-dimensional similarity probability distribution of sample i and sample j, representing the similarity ratio of sample i and sample j in the high-dimensional space; Represents the index of sample i and sample j, representing the traversal of any two samples from 1 to N respectively; Indicates that the matrix diagonal is ignored; It represents the sum of the high-dimensional similarities of all two samples in the symmetric high-dimensional matrix; S3-4-3, generating initial two-dimensional coordinates for each sample, collecting the initial two-dimensional coordinates of all samples, and constructing an initial two-dimensional coordinate matrix; wherein the matrix elements of the initial two-dimensional coordinate matrix are the tuples of the ID features and the initial two-dimensional coordinates; The expression of the initial two-dimensional coordinate matrix is: ； in, represents the initial two-dimensional coordinate matrix, represents the ID feature of sample i, represents the initial two-dimensional coordinates of sample i; S3-4-4. In the initial two-dimensional coordinate matrix, define the inverse of the square of the Euclidean distance between sample i and sample j as the low-dimensional similarity, and calculate the low-dimensional similarity probability distribution of sample i and sample j by normalization; The expression of the low-dimensional similarity probability distribution is: ； in, represents the coordinate vector of sample i in two-dimensional space, represents the coordinate vector of sample j in two-dimensional space, Represents the square of the Euclidean distance between sample i and sample j in two-dimensional space; It means that the inverse of the square of the Euclidean distance is defined as the unnormalized low-dimensional similarity; Represents the sum of low-dimensional similarities between any two samples in two-dimensional space; Represents the low-dimensional similarity probability distribution of sample i and sample j in two-dimensional space; S3-4-5, minimizing the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution, updating each sample two-dimensional coordinate to update the initial two-dimensional coordinate matrix to a target two-dimensional coordinate matrix; wherein the matrix elements of the target two-dimensional coordinate matrix are the target two-dimensional coordinates and the ID features of the sample; The target two-dimensional coordinate matrix represents: the maximum matching degree of similarity of sample pairs in high-dimensional space in two-dimensional space.

4. The system for visualizing a grid layout of multiple samples according to claim 3, characterized in that: Minimizing the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution, updating each sample two-dimensional coordinate to update the initial two-dimensional coordinate matrix to a target two-dimensional coordinate matrix, including: S3-4-5-1. Define the difference between the high-dimensional similarity probability distribution and the low-dimensional similarity probability distribution as KL divergence; The expression of the KL divergence is: ； Where C represents KL divergence; S3-4-5-2. Calculate the gradient of the KL divergence relative to each sample coordinate in the two-dimensional coordinate matrix; The expression of the gradient of each sample coordinate is: ； in, Represents the gradient of KL divergence with respect to the coordinates of sample i; S3-4-5-3. Use the gradient descent method to update the two-dimensional coordinates of each sample until the minimum KL divergence is reached, and generate the target two-dimensional coordinate matrix.

5. The system for visualizing a grid layout of multiple samples according to claim 4, characterized in that: Use the gradient descent method to update the two-dimensional coordinates of each sample until the minimum KL divergence is reached to generate the target two-dimensional coordinate matrix, including: S3-4-5-3-1. Based on the gradient of the KL divergence for each sample's two-dimensional coordinate, update each sample's two-dimensional coordinate to obtain an updated coordinate vector; The expression of the updated coordinate vector is: ； in, represents the coordinate vector of sample i at the tth iteration, represents the updated coordinate vector of sample i at the t+1th iteration, Represents the learning rate, which is used to control the step size of each coordinate update. represents the gradient of KL divergence with respect to the coordinates of sample i at the tth iteration; S3-4-5-3-2, using the updated coordinate vector as a matrix element to construct a sample coordinate update matrix; S3-4-5-3-3. Use the sample coordinates to update the matrix and recalculate the updated low-dimensional similarity probability distribution; S3-4-5-3-4. Calculate the updated KL divergence according to the updated low-dimensional similarity probability distribution; S3-4-5-3-5. When the updated KL divergence reaches the minimum KL divergence, each sample two-dimensional coordinate is deemed to be updated to the target two-dimensional coordinate; S3-4-5-3-6. Combine the target two-dimensional coordinates of each sample to generate the target two-dimensional coordinate matrix.

6. The system for visualizing a grid layout of multiple samples according to claim 5, characterized in that: The step of the visualization layout module performing a grid layout on the multiple image samples in the two-dimensional space to obtain a visualization grid layout of the multiple samples based on similarity includes: S4-1, obtaining a target two-dimensional coordinate matrix; S4-2, extracting the target two-dimensional coordinates of the target two-dimensional coordinate matrix; S4-3. Perform a visual grid layout on the multiple image samples according to the target two-dimensional coordinates.

7. The system for visualizing a grid layout of multiple samples according to claim 6, characterized in that: Performing a visual grid layout on the multiple image samples according to the target two-dimensional coordinates includes: S4-3-1. Obtain normalized sample coordinates according to the target two-dimensional coordinates; S4-3-2. Generate a visual grid layout of multiple image samples based on the normalized sample coordinates.

8. The system for visualizing a grid layout of multiple samples according to claim 7, characterized in that: According to the target two-dimensional coordinates, obtain the normalized sample coordinates, including: S4-3-1-1, select the coordinate range of the horizontal and vertical coordinates in the target two-dimensional coordinate matrix; the coordinate range is characterized by: the maximum and minimum values ​​of the horizontal coordinate, and the maximum and minimum values ​​of the vertical coordinate; S4-3-1-2, define the spatial scale of the grid layout according to the coordinate range of the horizontal and vertical coordinates; S4-3-1-3. Linearly scale the target two-dimensional coordinates and map them to the spatial scale of the grid layout to generate normalized sample coordinates that fit the grid layout.

9. The system for visualizing a grid layout of multiple samples according to claim 7, characterized in that: Generate a visual grid layout of multiple image samples based on the normalized sample coordinates, including: S4-3-2-1. Draw a two-dimensional layout diagram of the normalized sample coordinates according to the normalized sample coordinates; S4-3-2-2, according to the identification ID, associating the normalized sample coordinates with the image sample; S4-3-2-3. Overlay the image samples with the associated normalized sample coordinates onto the two-dimensional layout diagram to generate a visual grid layout of the multiple image samples.

Images