A method for constructing a kernel Gaussian mixture ridge regression model for one-to-many mapping
By fusing Gaussian hybrid model and nuclear ridge regression model, the nuclear Gaussian hybrid ridge regression model is solved, and the shortcomings of poor applicability of Gaussian hybrid model and the inability to deal with one-to-many problems in the existing technology are solved, and to achieve to-many data Effective fitting and processing of sets.
Patent Information
- Application Number
- CN202410272973.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-11
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-03-11
AI Technical Summary
The existing Gaussian hybrid model is not very suitable for regression tasks, and the nuclear ridge regression model cannot effectively deal with the one-to-many problem.
A nuclear Gaussian mixed ridge regression model (KGMRR) is proposed. By fusing the Gaussian mixed model and the kernel ridge regression model, the data set is fitted using multiple curves, assuming that the label y comes from the mixed Gaussian distribution, and the maximum posterior estimation is solved by EM algorithm parameter.
It realizes effective processing of multiple data sets, can fit multiple curves, process one-to-many relationships, provide the probability that each sample belongs to each data set, and judges noise through the covariance matrix to realize the initial division of the data set and Noise reduction.
Smart Images

Figure CN118133237B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for constructing a kernel Gaussian mixture ridge regression model, and belongs to the technical field of machine learning. Background Art
[0002] Gaussian mixture model (GMM) is often used for clustering or data preprocessing. When calculating the posterior probability of the class label y, due to the assumption of Gaussian distribution, the xy of each cluster can only be a linear relationship; if nonlinear fitting is to be achieved, a large number of clusters are required. Under limited sample conditions, the number of samples in each cluster is small, which is very easy to overfit and has poor applicability in regression tasks.
[0003] Kernel Ridge Regression (KRR) is commonly used for regression tasks. Usually a curve is used to fit a dataset X and its labels Y. KRR cannot handle one-to-many problems caused by the characteristics of the model itself or by mixing several unstructured / unstandardized datasets. Summary of the invention
[0004] A brief overview of the present invention is provided below in order to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important parts of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to a more detailed description discussed later.
[0005] In view of this, in order to solve the technical problems that the existing Gaussian mixture model in the prior art is not suitable for use and the kernel ridge regression model cannot achieve one-to-many, the present invention provides a method for constructing a kernel Gaussian mixture ridge regression model. The kernel Gaussian mixture ridge regression (KGMRR) of the present invention is a fusion of GMM and KRR. KGMRR assumes that the label y in the data set is π j The probability comes from the mixed multi-Gaussian distribution N(μ j (x),Σ j ), where μ j (x) Use kernel ridge regression μ j (x) = w j Φ(x) calculation. KGMRR can achieve clustering and regression, using multiple curves to fit the data set, thus solving one-to-many problems in complex situations. The parameters of KGMRR are θ=(π,w j ,Σ j ) is obtained by solving the maximum a posteriori estimate p(θ|x,y) through the EM algorithm.
[0006] Solution 1: A method for constructing a kernel Gaussian mixture ridge regression model. The Gaussian mixture model GMM uses multiple Gaussian distributions to describe the data distribution, and the kernel ridge regression KRR uses a curve to fit the data set. The Gaussian mixture model is integrated with the ridge regression model, and the mean of each Gaussian distribution in the Gaussian mixture model GMM is set to the output of a kernel ridge KRR. A one-to-many mapping kernel Gaussian mixture ridge regression model is obtained. The model is:
[0007] p(y|x)=∑ j π j N(y|w j Φ(x),Σ j )
[0008] Among them, y is the label, x is the model input, and π j is the prior probability that the sample belongs to the jth data set, w j is the parameter of the xy relationship represented by kernel ridge regression in the jth data set, Φ is an arbitrary mapping, Σ j is the noise covariance matrix of the jth data set;
[0009] The parameter iteration solution is performed based on the kernel Gaussian mixture ridge regression model: Assume: weight w j The prior probability is: Among them, n w is the degrees of freedom of the prior probability;
[0010] The prior probability of the covariance matrix is: p(Σ j )~IW(Σ j ;Σ0,n Σ ), where IW is the inverse Wishart distribution, Σ0 is the prior of the covariance matrix, and n Σ is the degrees of freedom of the inverse Wishart distribution;
[0011] The prior probability of each Gaussian distribution sampling probability is: p(π j )~Dir(α), where Dir(α) is the Dirichlet distribution with parameter α;
[0012] Therefore, when the model input value is x, the probability distribution of the model output value is:
[0013] p(y|x,π,w,Σ)=∑ j π j N(y|w j Φ(x),Σ j );
[0014] Among them, π is (π j ) j=1,2,... ,w is (w j ) j=1,2,... ;
[0015] Solve for the parameters (π, w j ,Σ j ) j=1,2,... , define θ=(π,w j ,Σ j ) j=1,2,... :
[0016]
[0017] Define the hidden variable z ij A logical variable indicating that data i comes from the jth curve; let γ ij =q(z ij ), then at each iteration E step, we have:
[0018]
[0019] Among them, π k is the prior probability that xy belongs to the kth curve, w k is the weight of the kth curve, x i is the i-th input value, Σ k is the covariance matrix of the kth Gaussian distribution, γ ij is (x i ,y i ) belongs to the posterior probability of the jth curve;
[0020] Each time the M-step iteration is performed, the upper bound of the objective function is maximized, and the solution is:
[0021] w k =(∑ i γ ik y i Φ(x i )')(∑ i γ ik Φ(x i )Φ(x i )'+n w I) -1
[0022]
[0023]
[0024] Among them, n w is the degrees of freedom of the weight prior, α k is the parameter of the Dirichlet distribution in the kth dimension, α j is the parameter of Dirichlet distribution in the jth dimension, k is the curve number;
[0025] Written in matrix form, we have:
[0026]
[0027] in:
[0028]
[0029]
[0030] Therefore, using the kernel method:
[0031] K(x i ,x j )=Φ(x i )'Φ(x j )
[0032] Among them, the kernel function K(x i ,x j )Use any function:
[0033] K(x i ,x j )=x i 'x j +c
[0034] or a Gaussian kernel:
[0035]
[0036] in, is the input matrix of the new data, X is the input matrix of the training set, Y is the label matrix of the training set, γ 1k is the posterior probability that the first sample belongs to the kth curve, γ Nk is the posterior probability that the Nth data belongs to the kth curve, x1' is the transpose of the first input value, x N ' is the transpose of the Nth input value, y1' is the transpose of the first label, y N ' is the transpose of the Nth label, x j is the jth input value, x i ' is the transpose of the i-th input value;
[0037] The kernel Gaussian mixture ridge regression model uses multiple curves to fit data and is applied to regression tasks to realize the processing of multiple data sets; the regression tasks include image segmentation, noise detection, data fitting, data preprocessing, time series prediction, classification recognition, industrial detection, target contour, inverse problem solving, data set segmentation or medical imaging diagnosis.
[0038] Solution 2: An electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method described in Solution 1 when executing the computer program.
[0039] Solution three: A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method described in Solution one when executed by a processor.
[0040] The beneficial effects of the present invention are as follows:
[0041] 1. The kernel Gaussian mixed ridge regression model of the present invention can fit multiple curves at the same time, realizing multiple interpretations of the data set;
[0042] 2. The kernel Gaussian mixture ridge regression model of the present invention can still perform well when the data set is polluted, multiple data sets are merged, and there is a one-to-many relationship such as a single sample and multiple labels in the task;
[0043] 3. When multiple data sets are merged, the kernel Gaussian mixture ridge regression model of the present invention can not only be solved well, but also obtain the probability of each sample belonging to each data set through the parameter γ, thereby obtaining the initial division of the data set;
[0044] 4. The kernel Gaussian mixture ridge regression model of the present invention can determine which points on the curve may be noise by observing the norm of the fitted covariance matrix, thereby achieving preprocessing such as data set denoising;
[0045] 5. The kernel Gaussian mixture ridge regression model KGMRR of the present invention can not only obtain the predicted value, but also obtain the confidence interval of the predicted value. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0047] Figure 1 It is a schematic diagram comparing the fitting results of the present invention and kernel ridge regression. DETAILED DESCRIPTION
[0048] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.
[0049] Example 1: Reference Figure 1 This embodiment describes a method for constructing a kernel Gaussian mixture ridge regression model, which is applied to regression tasks to achieve processing of multiple data sets and solve related problems with one-to-one and one-to-many mapping relationships;
[0050] Existing technologies such as kernel ridge regression, Gaussian process regression, support vector regression and other models all consider f as a single injection, and can only solve the regression problem where one x corresponds to one y. The steps to implement regression are relatively simple. First, obtain the data set; then fit the data set. When one x may correspond to multiple y, the above models are usually not applicable (such as Figure 1 a). Taking Gaussian mixture regression as an example, (x, y) is first clustered into Gaussian mixtures, and then the posterior probability p(y|x) of y with respect to x is calculated to achieve regression. This method can achieve one-to-many regression, but the solution process is relatively complicated, and it is impossible to intuitively describe which xy points belong to a fitting curve; and, because it is a Gaussian distribution, the distribution of xy in each cluster can only be linear, and nonlinear fitting is not possible.
[0051] The KGMRR regression model assumes that f is a one-to-many mapping, that y is a Gaussian mixture distribution that depends on x, and uses kernel techniques to fit the data set with multiple curves at the same time (e.g. Figure 1 b). KGMRR can give the probability of each output y given the input value x;
[0052] The Gaussian mixture model GMM uses multiple Gaussian distributions to describe the data distribution, and the kernel ridge regression KRR uses a curve to fit the data set. The Gaussian mixture model is integrated with the ridge regression model, and the mean of each Gaussian distribution in the Gaussian mixture model GMM is set to the output of a kernel ridge KRR. A kernel Gaussian mixture ridge regression model that fits the data set with multiple curves is obtained. The model is:
[0053] p(y|x)=∑ j π j N(y|w j Φ(x),Σ j )
[0054] Among them, y is the label, x is the model input, and π j is the prior probability that the sample belongs to the jth data set, w j is the parameter of the xy relationship represented by kernel ridge regression in the jth data set, Φ is an arbitrary mapping, Σ j is the noise covariance matrix of the jth data set;
[0055] The parameter iteration solution is performed based on the kernel Gaussian mixture ridge regression model: Assume: weight w j The prior probability is: Among them, n w is the degrees of freedom of the prior probability;
[0056] The prior probability of the covariance matrix is: p(Σ j )~IW(Σ j ;Σ0,n Σ), where IW is the inverse Wishart distribution, Σ0 is the prior of the covariance matrix, and n Σ is the degrees of freedom of the inverse Wishart distribution;
[0057] The prior probability of each Gaussian distribution sampling probability is: p(π j )~Dir(α), where Dir(α) is the Dirichlet distribution with parameter α;
[0058] Therefore, when the model input value is x, the probability distribution of the model output value is:
[0059] p(y|x,π,w,Σ)=∑ j π j N(y|w j Φ(x),Σ j );
[0060] Among them, π is (π j ) j=1,2,... ,w is (w j ) j=1,2,... ;
[0061] Solve for the parameters (π, w j ,Σ j ) j=1,2,... , define θ=(π,w j ,Σ j ) j=1,2,... :
[0062]
[0063] Define the hidden variable z ij A logical variable indicating that data i comes from the jth curve; let γ ij =q(z ij ), then at each iteration E step, we have:
[0064]
[0065] Among them, π k is the prior probability that xy belongs to the kth curve, w k is the weight of the kth curve, x i is the i-th input value, Σ k is the covariance matrix of the kth Gaussian distribution, γ ij is (x i ,y i ) belongs to the posterior probability of the jth curve;
[0066] Each time the M-step iteration is performed, the upper bound of the objective function is maximized, and the solution is:
[0067] w k =(∑i γ ik y i Φ(x i )')(∑ i γ ik Φ(x i )Φ(x i )'+n w I) -1
[0068]
[0069]
[0070] Among them, n w is the degrees of freedom of the weight prior, α k is the parameter of the Dirichlet distribution in the kth dimension, α j is the parameter of Dirichlet distribution in the jth dimension, k is the curve number;
[0071] Written in matrix form, we have:
[0072]
[0073] in:
[0074]
[0075]
[0076] Therefore, using the kernel method:
[0077] K(x i ,x j )=Φ(x i )'Φ(x j )
[0078] Among them, the kernel function K(x i ,x j )Use any function:
[0079] K(x i ,x j )=x i 'x j +c
[0080] or a Gaussian kernel:
[0081]
[0082] in, is the input matrix of the new data, X is the input matrix of the training set, Y is the label matrix of the training set, γ 1k is the posterior probability that the first sample belongs to the kth curve, γNk is the posterior probability that the Nth data belongs to the kth curve, x1' is the transpose of the first input value, x N ' is the transpose of the Nth input value, y1' is the transpose of the first label, y N ' is the transpose of the Nth label, x j is the jth input value, x i ' is the transpose of the i-th input value;
[0083] The kernel Gaussian mixture ridge regression model outputs multiple curves, which are used to fit data and applied to regression tasks to realize the processing of multiple data sets; the regression tasks include image segmentation, noise detection, data fitting, data preprocessing, time series prediction, classification recognition, industrial detection, target contour, inverse problem solving, data set segmentation or medical imaging diagnosis.
[0084] Hyperparameters such as the number of curves k of the present invention can be selected by methods such as cross-validation;
[0085] The kernel function K(x i ,x j ) can be any function, such as linear kernel function, Gaussian kernel function, etc.;
[0086] The parameters of the present invention can also be solved by variational inference, gradient descent, Newton's method and other methods;
[0087] The mapping Φ(x) of the present invention can also be expressed explicitly, such as polynomial mapping.
[0088] The regression tasks include image segmentation, noise detection, data fitting, data preprocessing, time series prediction, classification recognition, industrial detection, target contour, inverse problem solving, data set segmentation or medical image diagnosis.
[0089] The kernel Gaussian mixed ridge regression model of the present invention performs clustering in the clustering field by the relationship between x and y, rather than clustering only by the characteristics of the samples themselves. The clustering process is the same as the data set division process.
[0090] Reference Figure 1 , we can see that kernel ridge regression can only fit a single curve, while the present invention can fit multiple curves;
[0091] Embodiment 2: The computer device of the present invention may be a device including a processor and a memory, such as a single chip microcomputer including a central processing unit, etc. Furthermore, the processor is used to implement the steps of the method described in Embodiment 1 when executing a computer program stored in the memory.
[0092] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0093] The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0094] Embodiment 3: Computer readable storage medium embodiment.
[0095] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by a processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. A computer program is stored on the computer-readable storage medium. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the method described in Example 1 can be implemented.
[0096] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.
[0097] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is selected primarily for readability and teaching purposes, rather than for explaining or defining the subject matter of the present invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is illustrative, not restrictive, with respect to the scope of the present invention, which is defined by the appended claims.
Claims
1. A method for constructing a kernel Gaussian mixture ridge regression model, characterized in that: The Gaussian mixture model GMM uses multiple Gaussian distributions to describe the data distribution, and the kernel ridge regression KRR uses a curve to fit the data set. The Gaussian mixture model is integrated with the ridge regression model, and the mean of each Gaussian distribution in the Gaussian mixture model GMM is set to the output of a kernel ridge KRR. This results in a one-to-many mapping kernel Gaussian mixture ridge regression model. The model is: p(y|x)=∑ j p j N(y|w j Φ(x),Σ j ) Among them, y is the label, x is the model input, and π j is the prior probability that the sample belongs to the jth data set, w j is the parameter of the xy relationship represented by kernel ridge regression in the jth data set, Φ is an arbitrary mapping, Σ j is the noise covariance matrix of the jth data set; Use the kernel Gaussian mixture ridge regression model to iterate the parameters: Assume: weight w j The prior probability is: Among them, n w is the degrees of freedom of the prior probability; The prior probability of the covariance matrix is: p(Σ j )~IW(Σ j ;Σ0,n Σ ), where IW is the inverse Wishart distribution, Σ0 is the prior of the covariance matrix, and n Σ is the degrees of freedom of the inverse Wishart distribution; The prior probability of each Gaussian distribution sampling probability is: p(π j )~Dir(α), where Dir(α) is the Dirichlet distribution with parameter α; Therefore, when the model input value is x, the probability distribution of the model output value is: p(y|x,π,w,Σ)=∑ j p j N(y|w j Φ(x),Σ j ); Among them, π is (π j ) j=1,2,... ,w is (w j ) j=1,2,... ; Solve for the parameters (π, w j , Σ j ). j=1,2,... , define θ = (π, w j , Σ j ). j=1,2,... : Define the hidden variable z ij A logical variable indicating that data i comes from the jth curve; let γ ij =q(z ij ), then at each iteration E step, we have: Among them, π k is the prior probability that xy belongs to the kth curve, w k is the weight of the kth curve, x i is the i-th input value, Σ k is the covariance matrix of the kth Gaussian distribution, γ ij For x i ,y i The posterior probability of belonging to the jth curve; Each time the M-step iteration is performed, the upper bound of the objective function is maximized, and the solution is: w k =(∑ i c ik y i Φ(x i )')(∑ i c ik Φ(x i )Φ(x i )'+n w I) -1 Among them, n w is the degrees of freedom of the weight prior, α k is the parameter of the Dirichlet distribution in the kth dimension, α j is the parameter of Dirichlet distribution in the jth dimension, k is the curve number; Written in matrix form, we have: in: Therefore, using the kernel method: K(x i ,x j )=Φ(x i )'Φ(x j ) Among them, the kernel function K(x i ,x j )Use any function: K(x i ,x j )=x i 'x j +c or a Gaussian kernel: in, is the input matrix of the new data, X is the input matrix of the training set, Y is the label matrix of the training set, γ 1k is the posterior probability that the first sample belongs to the kth curve, γ Nk is the posterior probability that the Nth data belongs to the kth curve, x1' is the transpose of the first input value, x N ' is the transpose of the Nth input value, y1' is the transpose of the first label, y N ' is the transpose of the Nth label, x j is the jth input value, x i ' is the transpose of the i-th input value; The kernel Gaussian mixture ridge regression model uses multiple curves to fit data and is applied to regression tasks to achieve processing of multiple data sets; the regression tasks include image segmentation or medical image diagnosis.
2. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to claim 1 when executing the computer program.
3. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to claim 1 is implemented.
Citation Information
Patent Citations
Wind speed forecasting method and device based on G-L mixed noise characteristic kernel ridge regression technology
CN106405683A
Ridge regression soft measurement modeling method based on distributed parallel local modeling mechanism
CN110188433A
Fixed lens basketball goal detection and identification method based on Gaussian mixture model
CN116030383A
GMM (Gaussian Mixture Model) and LightGBM (LightGBM)-based auxiliary power transaction decision-making method
CN116796165A
