A generalized kernel two-sample test method and system based on optimal transmission rank

Abstract

This application discloses a generalized kernel two-sample test method and system based on optimal transmission rank. The method includes: acquiring a first sample set and a second sample set; mapping the merged samples to a preset reference point set through optimal transmission to obtain the optimal transmission rank of each sample point; calculating the intra-group similarity and inter-group similarity of the optimal transmission rank based on the kernel function; constructing a first bias component and a second bias component accordingly; performing nonlinear aggregation on the two bias components to obtain a test statistic; directly determining the theoretical critical value based on the asymptotic distribution of the test statistic under the assumption that the two sample sets come from the same distribution; and outputting the judgment result after comparing the observed value with the critical value. This invention solves the problems of decreased power of linear combination statistics due to signal cancellation and low efficiency caused by relying on resampling to determine the critical value in two-sample tests, and achieves a nonparametric test with enhanced power, high computational efficiency, and distribution independence.

Description

Technical Field

[0001] This application relates to the field of statistics, and in particular to a generalized nuclear two-sample test method and system based on optimal transmission rank. Background Technology

[0002] In the fields of statistical inference and machine learning, the two-sample test is used to determine whether two independent samples come from the same distribution, and it is a fundamental tool in tasks such as medical diagnosis and generative model evaluation. Non-parametric methods have attracted much attention due to their relaxed assumptions about data distribution.

[0003] In recent years, multivariate rank methods based on optimal transport theory have been introduced into two-sample tests. For example, the rank-based maximum mean discrepancy (RMMD) test obtains the multivariate rank by mapping the original data to a reference distribution and constructs a linear combination of within-group and between-group similarities as a statistic. This test does not require the moment assumption, is robust to heavy-tailed data and outliers, and is distribution-independent.

[0004] However, the RMMD statistic is composed of the linear sum of two bias components. Under alternative assumptions such as scaling, the two components often exhibit opposite shifts, causing the signals to cancel each other out and severely weakening the test power. Furthermore, its null distribution is complex, making it impossible to directly provide a critical value. Practical applications still require permutation tests or Monte Carlo simulations, which incur significant computational costs. Summary of the Invention

[0005] This application provides a generalized kernel two-sample test method and system based on optimal transmission rank. It solves the problems in multivariate two-sample tests where the linear combination statistic cancels out due to the opposite signs of the two bias components under the scale alternative hypothesis, thus reducing the test power. It also addresses the problem of low computational efficiency caused by relying on resampling methods to determine the critical value. The method achieves enhanced test power by avoiding signal cancellation through nonlinear aggregation while maintaining distribution independence and nonparametric robustness. Furthermore, it improves computational efficiency by directly determining the theoretical critical value using the known asymptotic distribution under the null hypothesis.

[0006] This application provides a generalized kernel two-sample test method based on optimal transmission rank, the method comprising: Obtain the first and second sample sets to be tested; By solving the optimal transmission problem between the merged samples of the first and second sample sets and the preset reference point set, the optimal transmission rank corresponding to each sample point is obtained. The intra-group similarity and inter-group similarity of the optimal transmission rank are calculated based on the kernel function; Based on the intra-group similarity and the inter-group similarity, a first deviation component and a second deviation component are constructed; The first and second deviation components are nonlinearly aggregated to obtain a test statistic. The nonlinear aggregation ensures that the signals do not cancel each other out when the two deviation components have opposite values. Based on the asymptotic distribution of the test statistic under the assumption that the first and second sample sets come from the same distribution, the theoretical critical value is directly determined, and the two-sample test result is output based on the comparison between the observed value of the test statistic and the theoretical critical value.

[0007] Optionally, the step of solving the optimal transmission problem between the merged samples of the first and second sample sets and the preset reference point set includes: By minimizing the sum of squared Euclidean distances between the merged samples and the preset reference point set, a reference point is matched for each sample point, and this reference point is used as the optimal transmission rank for that sample point.

[0008] Optionally, the preset reference point set is generated using a deterministic low-difference sequence, and the empirical distribution of the deterministic low-difference sequence converges to the preset reference distribution.

[0009] Optionally, the step of constructing the first deviation component and the second deviation component based on the intra-group similarity and the inter-group similarity includes: The difference between the mean of similarity within the first sample set and the mean of similarity between the first sample set and the second sample set is taken as the first deviation component; The difference between the mean of similarity within the second sample set and the mean of similarity between the first and second sample sets is taken as the second deviation component.

[0010] Optionally, the step of nonlinearly aggregating the first deviation component and the second deviation component to obtain the test statistic includes: Calculate the conditional variances of the first and second deviation components respectively; The conditional variance is used to standardize the corresponding deviation components. The test statistic is obtained by performing a sum of squares operation on the standardized first and second deviation components.

[0011] Optionally, before the step of nonlinearly aggregating the first deviation component and the second deviation component, the method further includes: Based on the kernel matrix corresponding to the optimal transmission rank, calculate the sum of squares, the sum of third-order products, and the sum of fourth-order products of the kernel matrix elements; Based on the sum of squares, the sum of third-order products, and the sum of fourth-order products, and in conjunction with the sample sizes of the first and second sample sets, the conditional variances of the first and second bias components are determined.

[0012] Optionally, the kernel function is a Gaussian radial basis function, and the bandwidth parameter of the Gaussian radial basis function is the median of the distances between all pairs of sample points after the first sample set and the second sample set are merged.

[0013] Optionally, the step of directly determining the theoretical critical value based on the asymptotic distribution of the test statistic under the assumption that the first and second sample sets come from the same distribution includes: The asymptotic distribution of the test statistic, under the assumption that the first and second sample sets come from the same distribution, is determined to be a chi-square distribution with twice the degrees of freedom (1). ; Determine the significance level based on the preset significance level. The distribution corresponds to the quantiles, and these quantiles are used as the theoretical critical values.

[0014] Optionally, under the assumption that the first sample set and the second sample set come from the same distribution, the asymptotic correlation coefficient between the first bias component and the second bias component approaches negative 1 as the sample size increases.

[0015] Furthermore, to achieve the above objectives, embodiments of the present invention also provide a generalized kernel two-sample test system based on optimal transfer rank, the system comprising: The data acquisition module is used to acquire the first and second sample sets to be tested. The transmission rank calculation module is used to obtain the optimal transmission rank corresponding to each sample point by solving the optimal transmission problem between the merged sample of the first sample set and the second sample set and the preset reference point set. The similarity calculation module is used to calculate the intra-group similarity and inter-group similarity of the optimal transmission rank based on the kernel function; A deviation component construction module is used to construct a first deviation component and a second deviation component based on the intra-group similarity and the inter-group similarity. The statistical construction module is used to perform nonlinear aggregation on the first deviation component and the second deviation component to obtain the test statistic. The nonlinear aggregation ensures that the signals do not cancel each other when the two deviation components have opposite values. The test judgment module is used to directly determine the theoretical critical value based on the asymptotic distribution of the test statistic under the assumption that the first sample set and the second sample set come from the same distribution, and output the two-sample test judgment based on the comparison result between the observed value of the test statistic and the theoretical critical value.

[0016] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. By performing nonlinear aggregation on the two bias components, such as by summing the squares after standardization, the contributions of the two bias components can still accumulate to a positive value even if they have opposite signs under alternative assumptions such as scale difference. This avoids the problem of positive and negative signals canceling each other out in traditional linear combination statistics and enhances the detection power of the test under alternative assumptions such as distribution scale difference.

[0017] 2. This application proves that the asymptotic distribution of the test statistic under the null hypothesis is a chi-square distribution with two degrees of freedom and 1. Based on this, the theoretical critical value can be obtained directly from the chi-square distribution quantile according to the significance level without the need to perform resampling, which greatly reduces the computational cost and is especially suitable for large-scale datasets or application scenarios that require fast response.

[0018] 3. By solving the optimal transfer problem of the merged sample and reference point set of the first and second sample sets, the optimal transfer rank obtained makes the zero distribution of the test statistic independent of the specific distribution of the original data, and does not require any moment conditions to be imposed on the sample distribution. It has good adaptability and stability for heavy-tailed distributions, skewed distributions and data containing outliers. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the framework of the generalized kernel two-sample test method based on the optimal transmission rank in this application; Figure 2 This is a schematic diagram of the generalized kernel two-sample test system based on the optimal transmission rank in this application. Figure 3 This is a schematic diagram of the terminal structure of the hardware operating environment involved in one embodiment of this application. Detailed Implementation

[0020] In multivariate two-sample tests, the linear combination-based statistic, under the scale alternative hypothesis, suffers from signal cancellation due to the opposite signs of the two bias components, thus reducing the test power. Furthermore, the aforementioned methods rely on resampling to determine the critical value, leading to low computational efficiency. To address these issues, this application provides a generalized kernel two-sample test method based on optimal transmission rank. First, it solves the optimal transmission problem of the merged sample and reference point set of the first and second sample sets. Based on the kernel function, it calculates the intra-group and inter-group similarities to construct the first and second bias components. Then, it performs nonlinear aggregation of the two bias components, specifically standardization followed by summation of squares, to obtain the test statistic. Utilizing the property that this statistic asymptotically follows a chi-square distribution with two degrees of freedom (1) under the null hypothesis, the theoretical critical value is directly determined from the chi-square distribution quantiles based on the significance level, without any resampling process. The technical solution of this application improves the testing power by using nonlinear aggregation in the form of sum of squares, so that even if two bias components have opposite signs under the scale alternative assumption, their signals will not cancel each other out. It directly obtains the critical value by using the asymptotic zero distribution in analytical form, avoiding computationally intensive resampling and greatly improving computational efficiency. At the same time, the framework based on the optimal transmission rank gives the method complete distribution independence and robustness to heavy-tailed distributions and outliers, and does not depend on any moment assumptions.

[0021] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0022] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0023] Example 1 In this embodiment, a generalized kernel two-sample test method based on optimal transmission rank is provided.

[0024] Reference Figure 1 The generalized kernel two-sample test method based on optimal transmission rank in this embodiment includes the following steps: Step S1: Obtain the first and second sample sets to be tested; In this embodiment, the first sample set and the second sample set are two sets of independent observation data to be compared, denoted as follows: and , where each sample point The two sample sets are independent and identically distributed, and come from different distributions. and The null hypothesis to be tested by this method That is, the two groups of samples come from the same distribution; alternative hypothesis This means that the two groups of samples come from different distributions. Therefore, the test is to determine whether there is a difference in the population distribution represented by these two groups of samples.

[0025] As an optional implementation, the method of obtaining the sample set depends on the specific application scenario. For example, in medical diagnosis, the first sample set may be cell characteristic data from patients with benign tumors, and the second sample set may be cell characteristic data from patients with malignant tumors; in quality control, the first sample set may be product parameters under standard processes, and the second sample set may be product parameters after process adjustments; in machine learning model evaluation, the first sample set may be real data, and the second sample set may be data generated by the generative model. The sizes m and n of the sample sets can be equal or unequal, and the total number of samples N = m + n.

[0026] Step S2: By solving the optimal transmission problem of the merged sample and reference point set of the first and second sample sets, the optimal transmission rank corresponding to each sample point is obtained. In this embodiment, to overcome the difficulty of directly sorting multivariate data, this step utilizes optimal transport theory to define the optimal transport rank for multivariate data. At the global level, for an absolutely continuous distribution, McCann's theorem guarantees the existence of a unique optimal transport mapping that maps the data distribution to a reference distribution. When the distribution has an additional finite second moment, this mapping is the minimization solution to the Monge problem. At the sample level, by solving the discrete optimal transport problem between the merged samples and the preset reference point set, a reference point is matched for each sample point; this reference point is the optimal transport rank for that sample point. Through this transformation, the original data is converted to a standardized reference space, thereby making subsequent statistics distribution-independent.

[0027] As an optional implementation, the first sample set and the second sample set are combined into a mixed sample set. Then, a fixed number of preset reference point sets are generated within the reference distribution space. The reference distribution is preferably a unit cube. The points are uniformly distributed on the reference point set. The preset reference point set is generated using deterministic low-discrepancy sequences, such as Halton sequences, whose empirical distribution weakly converges to the preset reference distribution. The pairing of mixed sample points to the reference point set is then established by solving the following minimization problem:

[0028] in Indicates all There are several permutations. This problem is equivalent to minimizing the sum of squared Euclidean distances between the mixed sample set and the reference point set. After pairing, each sample point... Obtain a corresponding reference point This reference point is used as The optimal transmission rank, denoted as For those belonging to the first sample set Second sample set Their transmission ranks are respectively and .

[0029] Step S3: Calculate the intra-group similarity and inter-group similarity of the optimal transmission rank based on the kernel function; In this embodiment, intra-group similarity refers to the similarity between different sample points within the same sample set, including the sum of similarities within the first sample set. and the sum of similarities within the second sample set. Inter-group similarity refers to the sum of similarities between sample points in the first sample set and the second sample set. Through kernel functions This measure the similarity between two optimal transfer ranks. Commonly used kernel functions include the Gaussian radial basis function kernel and the Laplace kernel. This step calculates the sum of similarities within the first sample set, within the second sample set, and between the two sample sets. These similarities will be used to construct the bias component.

[0030] As an optional implementation, an N×N kernel matrix is ​​constructed based on the optimal transmission rank obtained in step S2. Its elements are , Then calculate the sum of the following three similarities: Sum of similarities within the first sample set ; The sum of similarities within the second sample set ; Sum of similarities between two sample sets .

[0031] In the above formula, when i=j, the internal similarity summation is not included to avoid self-similarity bias. The kernel function can be chosen as an eigenfunction kernel to ensure consistency of the test. For example, a Gaussian radial basis kernel can be used. Among them, bandwidth parameters The median can be used as a heuristic.

[0032] Step S4: Based on the intra-group similarity and the inter-group similarity, construct a first deviation component and a second deviation component, wherein the first deviation component reflects the difference between the first sample set and the whole, and the second deviation component reflects the difference between the second sample set and the whole; In this embodiment, the internal average similarity of each of the two sample sets is compared with the average similarity between the two sample sets to construct two bias components. The first bias component is the mean internal similarity of the first sample set minus the mean similarity between the first and second sample sets. The second bias component is the mean internal similarity of the second sample set minus the mean similarity between the first and second sample sets. Under the null hypothesis, i.e., when the distribution from which the first and second sample sets originate is the same, the expected values ​​of both bias components are zero. Under the alternative hypothesis, i.e., when the first and second sample sets originate from different distributions, the first and second bias components will deviate from zero. Furthermore, when there is a scale difference between the distribution from which the first and second sample sets originate, for example, when the variance of one distribution is greater than the variance of the other, the first and second bias components often exhibit opposite signs, i.e., one is positive and the other is negative.

[0033] As an optional implementation, the sum of similarities within the first sample set calculated in step S3 is used. The sum of similarities within the second sample set and the sum of similarities between the two sample sets. Calculate the following three means: The mean similarity within the first sample set = ; The mean similarity within the second sample set = ; The mean similarity between the first and second sample sets = .

[0034] Construct two more deviation components: first deviation component ; Second deviation component .

[0035] For example, in a simulation setting, the dimension d=2, the sample size m=n=100, the first sample set comes from a normal distribution with covariance matrix Σ, and the second sample set comes from a distribution with covariance matrix Σ. The normal distribution, where , Under the null hypothesis =0.3, take the alternative hypothesis =0.6. At this point, the scale of the distribution from which the second sample set comes is larger than the scale of the distribution from which the first sample set comes, and the calculated first bias component... Typically a negative value, the second deviation component. The values ​​are usually positive, with opposite signs. The first and second deviation components exhibit opposite signs, illustrating the necessity of using nonlinear aggregation rather than nonlinear combination.

[0036] Step S5: Nonlinearly aggregate the first deviation component and the second deviation component to obtain the test statistic. The nonlinear aggregation ensures that the signals do not cancel each other out when the two deviation components have opposite values. In this embodiment, traditional linear combinations cancel each other out when the first and second deviation components have opposite signs, leading to a decrease in test power. This step employs nonlinear aggregation, specifically a sum-of-squares form, ensuring that the contribution of each deviation component is positive regardless of its sign, thereby accumulating the signal. To eliminate the influence of different component dimensions, each deviation component is first standardized by dividing it by its conditional standard deviation before performing the sum-of-squares operation.

[0037] As an optional implementation method, each is determined and conditional variance of and The conditional variance depends on the following predefined quantity: Let the total sample size N = m + n; let... Let N×N be a deterministic kernel matrix, and its elements be... ,in For the first A preset reference point. Then define three geometric invariants:

[0038]

[0039]

[0040] in, This indicates the number of permutations.

[0041] Define the coefficient related to sample size:

[0042]

[0043]

[0044] The conditional variance of the first deviation component is:

[0045] The conditional variance of the second bias component is:

[0046] Standardize the two deviation components:

[0047]

[0048] Finally, construct the test statistic: This statistic is the sum of the squares of the two standardized deviation components. Due to the squaring operation, even if the first and second deviation components have opposite signs, their squares are still positive, thus avoiding signal cancellation.

[0049] Step S6: Based on the asymptotic distribution of the test statistic under the assumption that the first and second sample sets come from the same distribution, directly determine the theoretical critical value, and output the two-sample test judgment based on the comparison result between the observed value of the test statistic and the theoretical critical value.

[0050] In this embodiment, the asymptotic distribution obtained through theoretical analysis is used to obtain the critical value at a specified significance level without resampling, thereby significantly improving computational efficiency. Under the null hypothesis... Below, the test statistic The distribution converges to a chi-square distribution with twice the number of degrees of freedom, i.e. Furthermore, under the null hypothesis, the first deviation component With the second deviation component The asymptotic correlation coefficient approaches negative 1 as the sample size increases, which provides theoretical support for the distribution of the sum of squares statistic.

[0051] As an alternative implementation, given a significance level , can be set From the chi-square distribution table with 1 degree of freedom, we can find... The theoretical critical value is .For example, , The critical value is approximately 7.682. This is calculated in step S5. The observed value is compared with the theoretical critical value. If If the null hypothesis is true, then reject the null hypothesis and determine that the first and second sample sets come from different distributions; otherwise, accept the null hypothesis and consider that the two sample sets come from the same distribution.

[0052] This embodiment does not rely on the moment assumption of the sample distribution and is robust to heavy-tailed distributions and outliers. Furthermore, the testing method is consistent.

[0053] For example, in a hypothetical analysis of the Wisconsin diagnostic breast cancer dataset, if a significance level is set... The theoretical critical value is approximately 7.682. If the observed value of the hypothesis test statistic is greater than this critical value, the null hypothesis is rejected, and it is determined that there is a significant difference in the characteristic distribution of benign and malignant patients.

[0054] In this embodiment, the optimal transmission rank is obtained by solving the optimal transmission problem between the merged sample and the preset reference point set. Under the null hypothesis, the rank distribution of each sample point is independent of the specific distribution form of the original data, thus making the test statistic distribution distribution-independent, requiring no moment or parametric assumptions. At the same time, since the asymptotic distribution of the test statistic under the null hypothesis, or under the assumption that the two sample sets come from the same distribution, is analytically proven to be a chi-square distribution with 1 degree of freedom, the critical value can be directly determined by the quantile of the chi-square distribution without the need for resampling, which greatly improves the computational efficiency. In addition, by constructing two bias components and using nonlinear aggregation in the form of a sum of squares, the power loss caused by the opposite signs and mutual cancellation of the two bias components under the scale alternative hypothesis is effectively avoided. It can maintain high test power for various alternative hypotheses such as scale difference and distribution shape difference. Furthermore, the nonparametric framework based on the optimal transmission rank is independent of moment conditions and has good robustness to heavy-tailed distributions, skewed distributions, and sample data containing outliers. Therefore, this method, while ensuring consistency of the test, integrates multiple advantages such as distribution independence, computational efficiency, excellent power and strong robustness, providing a reliable and scalable technical solution for multivariate two-sample tests.

[0055] Based on the same inventive concept, this application also provides a generalized kernel two-sample test system based on the optimal transmission rank corresponding to the method in Embodiment 1, as shown in Embodiment 2.

[0056] Example 2 Based on Embodiment 1, another embodiment of this application is proposed, with reference to... Figure 2 The generalized kernel two-sample test system based on optimal transfer rank in this embodiment includes the following modules: The data acquisition module is used to acquire the first and second sample sets to be tested. In this embodiment, the data acquisition module can read two sets of sample data from local storage, a remote database, or a real-time data stream, such as benign and malignant cell characteristic data in medical image analysis, and normal and abnormal equipment parameters in industrial testing.

[0057] The optimal transmission rank calculation module is used to obtain the optimal transmission rank corresponding to each sample point by solving the optimal transmission problem between the merged sample of the first sample set and the second sample set and the preset reference point set. In this embodiment, the optimal transmission rank calculation module first merges the first sample set and the second sample set into a mixed sample set. Then, it generates a preset reference point set within the reference distribution space, for example, using a Halton sequence to generate uniformly distributed points on a unit cube. By minimizing the sum of squared Euclidean distances between the mixed sample set and the reference point set, a reference point is matched for each sample point, and this reference point is used as the optimal transmission rank for that sample point. The output of this module is the optimal transmission rank for all sample points.

[0058] The similarity calculation module is used to calculate the intra-group similarity and inter-group similarity of the optimal transmission rank based on the kernel function; In this embodiment, the similarity calculation module constructs a kernel matrix. Its elements are kernel function values. The kernel function calculates the sum of similarities within the first sample set, the sum of similarities within the second sample set, and the sum of similarities between the first and second sample sets. A characteristic kernel function, such as a Gaussian radial basis kernel, can be used.

[0059] A deviation component construction module is used to construct a first deviation component and a second deviation component based on the intra-group similarity and the inter-group similarity. In this embodiment, the bias component construction module calculates the mean similarity within the first sample set, the mean similarity within the second sample set, and the mean similarity between the two sets, and then constructs the first bias component and the second bias component. Under the null hypothesis, the expected values ​​of both bias components are zero; under the scale alternative hypothesis, they often have opposite signs.

[0060] The statistical construction module is used to perform nonlinear aggregation on the first deviation component and the second deviation component to obtain the test statistic. The nonlinear aggregation ensures that the signals do not cancel each other when the two deviation components have opposite values. In this embodiment, the first deviation component is first determined. Second deviation component conditional variance of and Then, the bias components are standardized, and finally the test statistic is output. .

[0061] The test judgment module is used to directly determine the theoretical critical value based on the asymptotic distribution of the test statistic under the assumption that the first sample set and the second sample set come from the same distribution, and output the two-sample test judgment based on the comparison result between the observed value of the test statistic and the theoretical critical value.

[0062] In this embodiment, the test and judgment module uses the output of the statistical quantity construction module. The observed values, and based on the significance level Find the theoretical critical value from the chi-square distribution table, compare the two values. If the observed value is greater than the critical value, reject the null hypothesis and output that the two samples come from different distributions; otherwise, accept the null hypothesis and output that the two samples come from the same distribution.

[0063] Since the system described in Embodiment 2 of this application is a system used to implement the method of Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system based on the method described in Embodiment 1 of this application, and therefore will not be described again here. All systems used in the method of Embodiment 1 of this application fall within the scope of protection of this application.

[0064] Example 3 In this application embodiment, a generalized kernel two-sample test device based on optimal transmission rank is proposed.

[0065] Reference Figure 3 , Figure 3 This is a schematic diagram of the terminal structure of the hardware operating environment involved in one embodiment of this application.

[0066] like Figure 3 As shown, the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be high-speed RAM or stable non-volatile memory, such as disk storage. Alternatively, the memory 1004 may be a storage device independent of the aforementioned processor 1001.

[0067] Those skilled in the art will understand that Figure 3 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0068] like Figure 3 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a network communication module, and a generalized kernel two-sample test program based on optimal transmission rank.

[0069] exist Figure 3 In the hardware structure of the generalized kernel two-sample test device based on optimal transfer rank shown, the processor 1001 can call the generalized kernel two-sample test program based on optimal transfer rank stored in the memory 1004 and perform the following operations: Obtain the first and second sample sets to be tested; By solving the optimal transmission problem between the merged samples of the first and second sample sets and the preset reference point set, the optimal transmission rank corresponding to each sample point is obtained. The intra-group similarity and inter-group similarity of the optimal transmission rank are calculated based on the kernel function; Based on the intra-group similarity and the inter-group similarity, a first deviation component and a second deviation component are constructed; The first and second deviation components are nonlinearly aggregated to obtain a test statistic. The nonlinear aggregation ensures that the signals do not cancel each other out when the two deviation components have opposite values. Based on the asymptotic distribution of the test statistic under the assumption that the first and second sample sets come from the same distribution, the theoretical critical value is directly determined, and the two-sample test result is output based on the comparison between the observed value of the test statistic and the theoretical critical value.

[0070] Optionally, the processor 1001 may call the generalized kernel two-sample test program based on optimal transfer rank stored in the memory 1004, and further perform the following operations: By minimizing the sum of squared Euclidean distances between the merged samples and the preset reference point set, a reference point is matched for each sample point, and this reference point is used as the optimal transmission rank for that sample point.

[0071] Optionally, the processor 1001 may call the generalized kernel two-sample test program based on optimal transfer rank stored in the memory 1004, and further perform the following operations: The difference between the mean of similarity within the first sample set and the mean of similarity between the first sample set and the second sample set is taken as the first deviation component; The difference between the mean of similarity within the second sample set and the mean of similarity between the first and second sample sets is taken as the second deviation component.

[0072] Optionally, the processor 1001 may call the generalized kernel two-sample test program based on optimal transfer rank stored in the memory 1004, and further perform the following operations: Calculate the conditional variances of the first and second deviation components respectively; The conditional variance is used to standardize the corresponding deviation components. The test statistic is obtained by performing a sum of squares operation on the standardized first and second deviation components.

[0073] Optionally, the processor 1001 may call the generalized kernel two-sample test program based on optimal transfer rank stored in the memory 1004, and further perform the following operations: Based on the kernel matrix corresponding to the optimal transmission rank, calculate the sum of squares, the sum of third-order products, and the sum of fourth-order products of the kernel matrix elements; Based on the sum of squares, the sum of third-order products, and the sum of fourth-order products, and in conjunction with the sample sizes of the first and second sample sets, the conditional variances of the first and second bias components are determined.

[0074] Optionally, the processor 1001 may call the generalized kernel two-sample test program based on optimal transfer rank stored in the memory 1004, and further perform the following operations: The asymptotic distribution of the test statistic, under the assumption that the first and second sample sets come from the same distribution, is determined to be a chi-square distribution with twice the degrees of freedom (1). ; Determine the significance level based on the preset significance level. The distribution corresponds to the quantiles, and these quantiles are used as the theoretical critical values.

[0075] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0076] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0077] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0078] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0079] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. This application can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, third, etc., does not indicate any order. These words can be interpreted as names.

[0080] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0081] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of the invention. Therefore, if these modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A generalized kernel two-sample test method based on optimal transfer rank, characterized in that, The method includes: Obtain the first and second sample sets to be tested; By solving the optimal transmission problem between the merged samples of the first and second sample sets and the preset reference point set, the optimal transmission rank corresponding to each sample point is obtained. The intra-group similarity and inter-group similarity of the optimal transmission rank are calculated based on the kernel function; Based on the intra-group similarity and the inter-group similarity, a first deviation component and a second deviation component are constructed; The first and second deviation components are nonlinearly aggregated to obtain a test statistic. The nonlinear aggregation ensures that the signals do not cancel each other out when the two deviation components have opposite values. Based on the asymptotic distribution of the test statistic under the assumption that the first and second sample sets come from the same distribution, the theoretical critical value is directly determined, and the two-sample test result is output based on the comparison between the observed value of the test statistic and the theoretical critical value.

2. The method as described in claim 1, characterized in that, The steps for solving the optimal transmission problem between the merged samples of the first and second sample sets and the preset reference point set include: By minimizing the sum of squared Euclidean distances between the merged samples and the preset reference point set, a reference point is matched for each sample point, and this reference point is used as the optimal transmission rank for that sample point.

3. The method as described in claim 2, characterized in that, The preset reference point set is generated using deterministic low-difference sequences, and the empirical distribution of the deterministic low-difference sequences converges to the preset reference distribution.

4. The method as described in claim 1, characterized in that, The step of constructing the first deviation component and the second deviation component based on the intra-group similarity and the inter-group similarity includes: The difference between the mean of similarity within the first sample set and the mean of similarity between the first sample set and the second sample set is taken as the first deviation component; The difference between the mean of similarity within the second sample set and the mean of similarity between the first and second sample sets is taken as the second deviation component.

5. The method as described in claim 1, characterized in that, The step of nonlinearly aggregating the first deviation component and the second deviation component to obtain the test statistic includes: Calculate the conditional variances of the first and second deviation components respectively; The conditional variance is used to standardize the corresponding deviation components. The test statistic is obtained by performing a sum of squares operation on the standardized first and second deviation components.

6. The method as described in claim 5, characterized in that, Before the step of nonlinearly aggregating the first deviation component and the second deviation component, the method further includes: Based on the kernel matrix corresponding to the optimal transmission rank, calculate the sum of squares, the sum of third-order products, and the sum of fourth-order products of the kernel matrix elements; Based on the sum of squares, the sum of third-order products, and the sum of fourth-order products, and in conjunction with the sample sizes of the first and second sample sets, the conditional variances of the first and second bias components are determined.

7. The method as described in claim 1, characterized in that, The kernel function is a Gaussian radial basis function, and the bandwidth parameter of the Gaussian radial basis function is the median of the distances between all sample point pairs after the first sample set and the second sample set are merged.

8. The method as described in claim 1, characterized in that, The step of directly determining the theoretical critical value based on the asymptotic distribution of the test statistic under the assumption that the first and second sample sets come from the same distribution includes: The asymptotic distribution of the test statistic, under the assumption that the first and second sample sets come from the same distribution, is determined to be a chi-square distribution with twice the degrees of freedom (1). ; Determine the significance level based on the preset significance level. The distribution corresponds to the quantiles, and these quantiles are used as the theoretical critical values.

9. The method as described in claim 8, characterized in that, Under the assumption that the first and second sample sets come from the same distribution, the asymptotic correlation coefficient between the first and second bias components approaches negative 1 as the sample size increases.

10. A generalized kernel two-sample test system based on optimal transfer rank, characterized in that, The system includes: The data acquisition module is used to acquire the first and second sample sets to be tested. The transmission rank calculation module is used to obtain the optimal transmission rank corresponding to each sample point by solving the optimal transmission problem between the merged sample of the first sample set and the second sample set and the preset reference point set. The similarity calculation module is used to calculate the intra-group similarity and inter-group similarity of the optimal transmission rank based on the kernel function; A deviation component construction module is used to construct a first deviation component and a second deviation component based on the intra-group similarity and the inter-group similarity. The statistical construction module is used to perform nonlinear aggregation on the first deviation component and the second deviation component to obtain the test statistic. The nonlinear aggregation ensures that the signals do not cancel each other when the two deviation components have opposite values. The test judgment module is used to directly determine the theoretical critical value based on the asymptotic distribution of the test statistic under the assumption that the first sample set and the second sample set come from the same distribution, and output the two-sample test judgment based on the comparison result between the observed value of the test statistic and the theoretical critical value.

Images