A multi-source image data feature extraction method, system, device and storage medium

Abstract

The application discloses a multi-source image data feature extraction method, system, device and storage medium, original view data is mapped into a high-dimensional space through a kernel method, and a multi-source data set is constructed; the self-expression learning of each view data and the similarity between the view data are used as constraints to calculate the corresponding graph information matrix of each view; for the graph information matrix corresponding to different views, the importance of different views is considered to give corresponding weights, and an optimal consistency graph adjacency matrix is calculated through a self-expression learning and adaptive fusion method; based on a graph constraint canonical correlation analysis algorithm, and through the fused graph structure information as a constraint, multi-view data canonical correlation analysis is carried out, common consistency features of each view data are fused, and a complete feature extraction objective function is constructed. Nonlinear information of data can be better captured, sample data is linearly separable in the high-dimensional space, and similarity calculation and graph structure construction are facilitated.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of machine learning, and relates to a multi-source image data feature extraction method, system, device and storage medium. BACKGROUND

[0002] In the field of computer vision, the same target, such as a face, an animal or a satellite, can be observed from different angles. Each data obtained by observing the target in different ways or angles is a view. The information contained in different view data has complementary characteristics. Feature extraction on multi-view picture data is beneficial to better recognition of the target object. In recent years, many algorithms for feature extraction on multi-view data have been proposed and applied to image recognition.

[0003] The graph multi-view canonical correlation analysis algorithm (GMCCA) based on graph constraint significantly improves the algorithm performance of multi-view canonical correlation analysis and enhances the clustering and classification effect of the algorithm by using the similarity of the graph structure information of each view data to construct a graph constraint term. However, the existing GMCCA algorithm uses a predefined graph structure, relies on priori, and the graph construction method is suboptimal. Moreover, the similarity difference between different data is not considered in graph fusion, which leads to the inability to obtain a graph structure reflecting the consistency of the characteristics of multi-view data, and defects exist. Therefore, the multi-view canonical correlation analysis algorithm (MCCA) with graph learning and fusion (GFMCCA) is proposed, which uses a graph learning method based on self-expression to obtain a graph structure and gives different weights to different view information to obtain an optimal fusion graph. The algorithm effectively utilizes the view data information and solves the defects of the GMCCA algorithm. Figure One

[0004] However, in actual applications, the view data is linearly inseparable in a low-dimensional space, the non-linear space is not clear how to construct, and thus the similarity calculation of the non-linear mapping features and the construction of the graph are affected. SUMMARY

[0005] The purpose of the present application is to overcome the above-mentioned defects of the prior art and provide a multi-source image data feature extraction method, system, device and storage medium, which can better capture the non-linear information of the data, make the sample data linearly separable in the high-dimensional space, and facilitate similarity calculation and graph structure construction.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0007] A multi-source image data feature extraction method, comprising the following processes: ​

[0008] S1: The original image view data is mapped to a high-dimensional space using the kernel method to construct a multi-source image dataset, which includes multiple high-dimensional linearly separable view data.

[0009] S2: By utilizing the self-expressive learning characteristics and similarity of each view data in the multi-source image dataset, calculate the corresponding graph information matrix for each input view;

[0010] S3: Consider the importance of different views and assign weights to the corresponding graph information matrix. Calculate the optimal consistent graph adjacency matrix through a method of self-expressive learning and adaptive fusion.

[0011] S4: Based on the graph constraint canonical correlation analysis algorithm, and using the fused graph structure information as constraints, perform canonical correlation analysis on multi-view data, fuse the common consistency features of each view data, and construct a complete image data feature extraction objective function.

[0012] Preferably, the specific process of S1 is as follows:

[0013] Use the original view data Mapped to Φ m It is a D m A kernel mapping from 3D space to infinite-dimensional space, using a kernel matrix. To calculate Φ m (X m ) T Φ m (X m ),Will The kernel matrix K obtained after centering m ;

[0014] The expression is as follows:

[0015]

[0016] K m This represents high-dimensional linearly separable data after kernel function mapping, where N is the number of samples; i, j, k represent the matrix rows / columns, and M represents the total number of views.

[0017] Preferably, the specific process of S2 is as follows: for each high-dimensional view data obtained after kernel mapping, the similarity graph information matrix is ​​learned according to its self-expressive characteristics, and regularization constraints are applied to the similarity graph information to calculate the graph information matrix corresponding to each view.

[0018] Preferably, the specific process of S3 is: according to the importance of different views, corresponding weights are given to the views, through the graph information matrix of each view data, a consistent graph adjacency matrix is adaptively fused and learned, and the optimal consistent graph adjacency matrix of all graphs is calculated by combining the regularization constraint on the weights.

[0019] Preferably, in S4, the consistent feature is smooth on the optimal consistent graph adjacency matrix, and βTr(CL G C) is introduced as a graph constraint term for maintaining data locality.

[0020] The complete feature extraction target function is:

[0021]

[0022]

[0023] Wherein, α, β, γ, η are regularization factors, and the value range is [0, ∞]; N represents the sample number; K m represents the view data that needs to be extracted features after being mapped by the kernel function; M represents the total number of views; is a weight coefficient; is the dual of the original MCCA projection matrix; C represents the common consistent feature of each view data; Tr(CL G C) represents a graph constraint term for maintaining locality, wherein represents the trace of the matrix; G represents the optimal consistent graph adjacency matrix generated after multi-graph fusion, which is a real symmetric matrix, g ij represents the weight of the edge between any two nodes; L G is the Laplacian matrix of the optimal consistent graph adjacency matrix G; Z m represents the graph information matrix corresponding to each view; d m represents the importance of different views; represents that the F-norm is used to impose a regularization constraint on the similar graph information; represents that the 2-norm is used to impose a regularization constraint on the weight; I represents the unit matrix.

[0024] Preferably, after S4 is completed, the complete feature extraction target function is optimized by using an alternating optimization method.

[0025] Further, the specific process of optimization is:

[0026] (1) fixing Z m , d, G, solving A m and C, the target function is as follows:

[0027]

[0028] stCC T =I

[0029] Z m Let d represent the graph information matrix corresponding to each view, d represent the importance of different views, G represent the optimal consistency graph adjacency matrix, and A represent the graph information matrix corresponding to each view. m C represents the dual of the original projection matrix, and C represents the common consistency feature of the view data.

[0030] The following intermediate variables are introduced:

[0031]

[0032] If Q is decomposed into its eigenvalues, then C can be constructed from the eigenvectors corresponding to the p largest eigenvalues ​​of Q.

[0033] Substituting C into the following equation, we get A. m :

[0034] A m =(K m +∈ m I) -1 C T

[0035] (2) Fix A m Given C, d, and G, solve for Z. m :

[0036] Z m =(ηK) m +αI+d m I) -1 (d m G+ηK m )

[0037] (3) Fix A m C, d, Z m Solve for G:

[0038] c i Let g be the i-th column vector of C. ij and G and Z respectively m The (i, j)th element in the vector r; define vector r i Let its j-th element The problem can then be transformed into the following form:

[0039]

[0040] stg i .1=1, g ij ≥0

[0041] in Z m the ith row vector of A

[0042] (4) fixing A m , C, G, Z m , solving d

[0043] The objective function is:

[0044]

[0045] s.t.d T 1=1, d≥0

[0046] Let The above formula can be equivalent to:

[0047]

[0048] s.t.d T 1=1, d≥0

[0049] The optimization problem can be converted into the following optimization problem with the same form:

[0050]

[0051]

[0052] The optimal a can be obtained by:

[0053] a i = max{b i -θ, 0}

[0054] Repeat the alternating iterative calculation process until the objective function converges, that is, the dual projection matrix A m and the consistency feature C can be obtained.

[0055] A multi-source image data feature extraction system, comprising:

[0056] A data set construction module: for mapping the original image view data to a high-dimensional space by a kernel method, constructing a multi-source image data set, and including a plurality of high-dimensional linearly separable view data in the multi-source image data set;

[0057] A graph information module: for calculating the corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set;

[0058] An optimal consistency graph adjacency matrix module: for considering the importance of different views to give weights to the corresponding graph information matrix, and calculating the optimal consistency graph adjacency matrix by the self-expression learning and adaptive fusion method;

[0059] Objective function module: used for performing multi-view data canonical correlation analysis based on graph constraint canonical correlation analysis algorithm and by taking fused graph structure information as constraint, fusing common consistency features of each view data, and constructing complete image data feature extraction objective function.

[0060] A computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the multi-source image data feature extraction method when executing the computer program.

[0061] A computer readable storage medium stores a computer program, and the computer program implements the steps of the multi-source image data feature extraction method when executed by a processor.

[0062] Compared with the prior art, the present application has the following beneficial effects:

[0063] The present application proposes a multi-source image data feature extraction method based on nonlinear fusion graph canonical correlation analysis, effectively utilizes the graph structure information of multi-view, processes the problem of complex data nonlinearity, and solves the deficiencies of existing algorithms. The original data is mapped to a high-dimensional space through a kernel function, thereby better capturing the nonlinear information of the data, making the sample data linearly separable in the high-dimensional space, facilitating similarity calculation and graph structure construction. A consistent optimal graph is obtained through self-expression learning and adaptive fusion. Unlike the method of predefining structure and trying to construct a similarity matrix multiple times, the present algorithm learns the graph structure based on the self-expression characteristics of data, has small dependence on prior knowledge, and enhances the ease of use of the algorithm. When forming an optimal consistency graph adjacency matrix through fusion, an adaptive weight mechanism fusion is performed considering the different importance of each view, and the fused graph structure information is taken as a constraint to obtain an effective constraint consistency feature graph structure, retain local structure, and not lose implicit information. The computational complexity is reduced, and the performance of the multi-view canonical correlation analysis algorithm is improved. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 It is a multi-source image data feature extraction method framework of the present application. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0066] It should be noted that the words "front", "back", "left", "right", "upper" and "lower" used in the following description refer to the directions in the drawings, and the words "inner" and "outer" refer to the directions towards or away from the geometric center of a particular component.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0068] As shown in the figure, the multi-source image data feature extraction method based on nonlinear fusion graph canonical correlation analysis of the application comprises the following steps: Figure 1

[0069] S1: Map the original image view data into a high-dimensional space by a kernel method, construct a multi-source image data set, and include multiple high-dimensional linearly separable view data in the multi-source image data set.

[0070] S2: Calculate the corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set.

[0071] S3: Consider the importance of different views to give the corresponding weight of the graph information matrix, and calculate the optimal consistency graph adjacency matrix by the self-expression learning and adaptive fusion method.

[0072] S4: Based on the graph-constrained canonical correlation analysis algorithm, and by using the fused graph structure information as a constraint, perform multi-view data canonical correlation analysis, fuse the common consistency features of each view data, and construct a complete image data feature extraction objective function.

[0073] Map the original data into a high-dimensional space by a kernel method. Calculate the corresponding graph information matrix of each view by using the self-expression learning of each view data and using the similarity between the view data as a constraint. For the graph information matrix corresponding to different views, consider the importance of different views to give the corresponding weight, and calculate the optimal consistency graph adjacency matrix by the self-expression learning and adaptive fusion method. By using the fused graph structure information as a constraint, establish a graph constraint term to constrain the correlation analysis of different views. Based on the graph-constrained canonical correlation analysis algorithm, perform multi-view data canonical correlation analysis, and fuse the common consistency features of each view data.

[0074] As a preferred scheme of the multi-source image data feature extraction method based on nonlinear fusion graph canonical correlation analysis of the application: ​

[0075] The process is as follows:

[0076] 1. Map the original view data into high-dimensional space by kernel method.

[0077] Map the original data into Φ m is a kernel mapping from D m -dimensional space to infinite-dimensional space. Use kernel matrix to calculate Φ m (X m ) T Φ m (X m ), and the kernel matrix K m obtained after centering.

[0078] The expression is as follows:

[0079]

[0080] K m represents the high-dimensional linearly separable view data after kernel function mapping, and N is the number of samples. i, j, k represent the matrix row / column.

[0081] 2. According to the self-expression characteristics of each high-dimensional view data K m obtained after kernel mapping, learn the similar graph information matrix, and apply regularization constraint to the similar graph information, calculate the final graph information matrix Z m corresponding to each view.

[0082]

[0083] s.t.Z m ≥0, m = 1, 2,..., M

[0084] α, η are regularization factors, randomly valued, with a value range of [0, ∞].

[0085] 3. According to the importance of different views, give them corresponding weights d m , learn the consistent graph adjacency matrix by adaptively fusing the graph information matrix Z m of each view data, and combine the regularization constraint on the weight to calculate the optimal consistent graph adjacency matrix G of all graphs.

[0086]

[0087] where γ is a regularization factor, randomly valued, with a value range of [0, ∞].

[0088] ​4. Hypothesis consistency feature On the graph G is smooth, the introduction of beta Tr(CL G C) The graph constraint term is used to keep the data locality. Based on the original graph constrained canonical correlation analysis algorithm (GMCCA), the kernel function is used for data mapping, the high-dimensional space is constructed, and the adaptive fused graph structure information is used as the constraint to perform multi-view data canonical correlation analysis, fuse the common consistency features of each view data, and form the complete KMGMCCA algorithm. The objective function is as follows:

[0089]

[0090]

[0091] Where, α, β, γ, η are regularization factors, the value range is [0, ∞]; N represents the sample number; K m represents the view data that needs to be extracted after the high-dimensional linear separable kernel function mapping; M represents the total number of views; is the weight coefficient; is the dual of the original MCCA projection matrix; C represents the common consistency feature of each view data; Tr(CL G C) represents the graph constraint term used to keep the locality, where represents the trace of the matrix; G represents the optimal consistency graph adjacency matrix generated after multi-graph fusion, which is a real symmetric matrix, g ij represents the weight of the edge between any two nodes; L G is the Laplacian matrix of the optimal consistency graph adjacency matrix G; Z m represents the graph information matrix corresponding to each view; d m represents the importance of different views; represents the use of F-norm to impose regularization constraints on similar graph information; represents the use of 2-norm to impose regularization constraints on weights; I represents the unit matrix.

[0092] The proposed objective function is optimized by the alternating optimization method:

[0093] Z m represents the graph information matrix corresponding to each view, d represents the importance of different views, G represents the optimal consistency graph adjacency matrix, A m represents the dual of the original projection matrix, and C represents the common consistency feature of each view data.

[0094] The steps of the alternating iterative optimization objective function are:

[0095] (1) Fix Z m , d, G, and solve A mand C, the objective function is as follows:

[0096]

[0097] s.t.CC T = I

[0098] Introduce intermediate variables as follows:

[0099]

[0100] Decompose Q, then C can be constructed by the eigenvectors corresponding to the largest p eigenvalues of Q.

[0101] Substitute C into the following formula to obtain A m :

[0102] A m = (K m + ∈ m I) -1 C T

[0103] (2) Fix A m , C, d, G, and solve Z m :

[0104] Z m = (ηK m + αI + d m I) -1 (d m G + ηK m )

[0105] (3) Fix A m , C, d, Z m , and solve G:

[0106] c i is the i-th column vector of C, g ij and are the (i, j)-th elements of G and Z m respectively. Define vector r i , let its j-th element then the problem can be transformed into the following form:

[0107]

[0108] where is the i-th row vector of Z m .

[0109] (4) Fix A m , C, G, Z m , and solve d,

[0110] The objective function is:

[0111]

[0112] s.t.d T 1=1,d≥0

[0113] Let The above formula can be equivalent to:

[0114]

[0115] s.t.d T 1=1,d≥0

[0116] The optimization problem can be converted into the following optimization problem with the same form:

[0117]

[0118]

[0119] The optimal a can be obtained by the following formula:

[0120] a i =max{b i -θ,0}

[0121] Repeat the alternating iteration process until the objective function converges, that is, the dual projection matrix A m and the consistency feature C can be obtained.

[0122] The following is an embodiment of the device of the application, which can be used to execute the method embodiment of the application. For details not covered in the device embodiment, please refer to the method embodiment of the application.

[0123] In another embodiment of the application, a multi-source image data feature extraction system is provided, which can be used to implement the multi-source image data feature extraction method described above. Specifically, the multi-source image data feature extraction system includes an image data set construction module, a graph information module, an optimal consistency graph adjacency matrix module, and an objective function module.

[0124] The data set construction module is used to map the original image view data to a high-dimensional space through a kernel method, and construct a multi-source image data set, which includes a plurality of high-dimensional linearly separable view data.

[0125] The graph information module is used to calculate the corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set.

[0126] The optimal consistency graph adjacency matrix module is used to consider the importance of different views to give the corresponding graph information matrix a weight, and to calculate the optimal consistency graph adjacency matrix by a self-expression learning and adaptive fusion method.

[0127] The objective function module is used to perform canonical correlation analysis of multi-view data based on a graph constraint canonical correlation analysis algorithm, and to perform canonical correlation analysis of multi-view data by taking the fused graph structure information as a constraint, to fuse the common consistency features of each view data, and to construct a complete image data feature extraction objective function.

[0128] In another embodiment of the present application, a terminal device is provided, which comprises a processor and a memory, the memory is used to store a computer program, the computer program comprises program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement a corresponding method process or a corresponding function; the processor in the embodiment of the present application can be used for the operation of the multi-source image data feature extraction method, comprising: S1, mapping the original image view data to a high-dimensional space by a kernel method to construct a multi-source image data set, and the multi-source image data set comprises a plurality of high-dimensional linearly separable view data; S2, calculating the corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set; S3, considering the importance of different views to give the corresponding graph information matrix a weight, and calculating the optimal consistency graph adjacency matrix by a self-expression learning and adaptive fusion method; S4, performing canonical correlation analysis of multi-view data based on a graph constraint canonical correlation analysis algorithm, and performing canonical correlation analysis of multi-view data by taking the fused graph structure information as a constraint, fusing the common consistency features of each view data, and constructing a complete image data feature extraction objective function.

[0129] In another embodiment, the present application also provides a computer readable storage medium (Memory), which is a memory device in the terminal device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0130] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the multi-source image data feature extraction method in the above embodiments. The one or more instructions stored in the computer readable storage medium are loaded and executed by the processor to perform the following steps: S1: mapping the original image view data into a high-dimensional space by a kernel method, constructing a multi-source image data set, and including a plurality of high-dimensional linearly separable view data in the multi-source image data set; S2: calculating the corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set; S3: considering the importance of different views to give the corresponding graph information matrix a weight, and calculating the optimal consistency graph adjacency matrix by the self-expression learning and adaptive fusion method; and S4: performing multi-view data canonical correlation analysis based on the graph constrained canonical correlation analysis algorithm, and taking the fused graph structure information as a constraint, fusing the common consistency features of each view data, and constructing a complete image data feature extraction objective function.

[0131] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0132] The present application is described in reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to an embodiment of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products. Figure One one or more flows and / or blocks. Figure One one or more flows and / or blocks.

[0133] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction means, which implement the functions specified in the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products. Figure One one or more flows and / or blocks. Figure One one or more flows and / or blocks.

[0134] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products. Figure One one or more flows and / or blocks. Figure One one or more flows and / or blocks.

[0135] The present application discloses a specific embodiment:

[0136] Given multi-view image data to be extracted features where M≥2, D m is the dimension of the mth view data sample, and N represents the number of samples. The original data Kernel mapping to high-dimensional space After centering, the kernel matrix K m is obtained. Based on the self-expression features of the data and the adaptive graph fusion learning, the corresponding view information matrix and the optimal consistency graph adjacency matrix are obtained. The fused graph information is introduced as a constraint term to perform canonical correlation analysis on the multi-source data set. The objective function of the complete KMGMCCA algorithm is as follows:

[0137]

[0138]

[0139] Wherein, alpha, beta, gamma, eta are regularization coefficients, Z m Indicate the graph structure corresponding to each view data, G indicates the optimal consistency graph adjacency matrix.

[0140] The specific operation includes the following steps:

[0141] Step one: input multi-view image data N is the number of data set samples, the subspace dimension is d, and the regularization coefficients are alpha, beta, gamma and eta.

[0142] Step two: initialize M-dimensional vector d, each component is d m =1 / M. Use kernel method to obtain high-dimensional linear separable data K m , the kernel width is sigma, and the weight coefficient is epsilon m .

[0143] Step three: initialize each view graph structure using Gaussian kernel function

[0144] Step four: use Z m to obtain the adaptive fusion similarity matrix G.

[0145] Step five: use Z m , d, G, to solve the transformation matrix A m .

[0146] Step six: input A m , and convert the C solving into an eigenvalue decomposition problem, define the matrix Q, and construct C T using the eigenvectors corresponding to the d largest eigenvalues of the matrix Q.

[0147] Step seven: use A m , C, d, G, to solve Z m .

[0148] Step eight: use Z m , C, d, A m to solve G.

[0149] Step nine: use Z m , C, G, A m to solve d.

[0150] Step ten: repeat steps five to nine until the objective function converges.

[0151] The effect of the application can be further illustrated by the following experiments on real databases.

[0152] The classification and clustering effects of the low-dimensional features extracted by the multi-source image data feature extraction method (KMGMCCA) based on nonlinear fusion graph canonical correlation analysis are tested on image data sets such as the Mnist data set, the ORL-mtv data set, and the Caltech20, and the classification and clustering effects of the low-dimensional features are compared with those of the multi-view canonical correlation analysis algorithm (MCCA), the graph-constrained multi-view canonical correlation analysis algorithm (GMCCA), and the multi-view canonical correlation analysis algorithm based on a kernel function (KMCCA), so as to verify the superiority of the algorithm.

[0153] The classification accuracy results are shown in Table 1, and it can be seen that the algorithm has obvious advantages compared with other models. This shows that the kernel method and graph regularization can effectively improve the performance of the MCCA algorithm.

[0154] Table 1 Average classification accuracy of KMGMCCA and its comparative algorithms on different data sets

[0155] Dataset Name MCCA KMCCA GMCCA GKMCCA KMGMCCA Ad 0.9170 0.9327 0.9275 0.9457 0.9476 Mnist 0.9268 0.9327 0.9326 0.9300 0.9583 ORL-mtv 0.2693 0.4795 0.4761 0.5177 0.7247 Cal20 0.7151 0.7849 0.7977 0.8041 0.8447 NUS 0.5098 0.5164 0.5066 0.5121 0.5406

[0156] In the data set construction part, the original data is mapped to a high-dimensional space through a kernel function, so that the nonlinear information of the data can be better captured, the sample data is linearly separable in the high-dimensional space, the problem of constructing a nonlinear data feature space is solved, and the similarity of the nonlinear data mapping features and the graph structure are calculated.

[0157] In the graph learning and fusion part, unlike the method of predefining a structure and trying to construct a similarity matrix multiple times, the algorithm learns a graph structure based on the self-expression characteristics of data, and has little dependence on prior knowledge. When the optimal consistency graph adjacency matrix is fused, an adaptive weight mechanism fusion is performed according to the different importance of each view, and the fused graph structure information is used as a constraint to obtain an effective constraint consistency feature graph structure, and the local structure of the view information is retained. The algorithm performance is improved by reducing the computational complexity.

[0158] It should be noted that, in this document, relational terms such as first and second and the like can merely be used to distinguish one entity or action from another, without necessarily requiring or implying that the entities or actions are in any way mutually exclusive or in any way in a sequence, unless the context clearly suggests otherwise. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0159] It is to be understood that the above description is intended to be illustrative, and not restrictive. Many embodiments and applications other than the examples provided would be apparent to those of skill in the art upon reading the above description. The scope of the technology should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. The disclosures of all articles and references, including patent applications and publications, are incorporated by reference for all purposes. The omission in the foregoing description of any aspect of the subject matter disclosed herein is not a disclaimer of such subject matter, nor should it be regarded that the applicant has disclaimed any such subject matter, nor should any such omission be regarded as affecting the scope of the claimed teaching.

Claims

1. A multi-source image data feature extraction method, characterized in that, The method comprises the following steps: S1: mapping original image view data into a high-dimensional space by a kernel method to construct a multi-source image data set, and the multi-source image data set comprises a plurality of high-dimensional linearly separable view data; The specific process is as follows: Use the original view data Mapped to , It is a general A kernel mapping from 3D space to infinite-dimensional space, using a kernel matrix. To calculate ,Will The kernel matrix obtained after centering ; The expression is as follows: represents the view data that is linearly separable in high dimension after mapping by the kernel function, is the number of samples; represents the matrix row / column, represents the total number of views; S2: calculating a corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set; S3: considering the importance of different views to give a weight to the corresponding graph information matrix, and calculating an optimal consistency graph adjacency matrix by a self-expression learning and adaptive fusion method; S4: performing canonical correlation analysis on multi-view data based on a graph-constrained canonical correlation analysis algorithm and by taking the fused graph structure information as a constraint, fusing common consistency features of the view data, and constructing a complete image data feature extraction objective function; Assumption of consistency features On the optimal consistency graph adjacency matrix is smooth, introduce Graphing constraint terms for maintaining data locality; The complete feature extraction objective function is as follows: wherein, is a regularization factor, taking value range [0, ]; represents the view data that needs to extract features after mapping through the kernel function, which is high-dimensional linearly separable; represents the total number of views; is a weight coefficient; is the dual of the original multi-view canonical correlation analysis algorithm MCCA projection matrix; represents the common consistency features of each view data; represents a graph constraint term for maintaining locality, wherein represents the trace of a matrix; represents the optimal consistency graph adjacency matrix generated after multi-graph fusion, which is a real symmetric matrix, represents the weight of the edge between any two nodes; is the Laplacian matrix of the optimal consistency graph adjacency matrix ; represents the graph information matrix corresponding to each view; represents the importance of different views; represents the regularization constraint on the similar graph information using the F-norm; represents the regularization constraint on the weight using the 2-norm; represents the unit matrix.

2. The multi-source image data feature extraction method of claim 1, wherein, The specific process of S2 is as follows: According to the self-expression characteristics of each high-dimensional view data obtained after kernel mapping, the similar graph information matrix is learned, and a regularization constraint is applied to the similar graph information to calculate a final graph information matrix corresponding to each view.

3. The multi-source image data feature extraction method of claim 1, wherein, The specific process of S3 is as follows: According to the importance of different views, a corresponding weight is given, a consistency graph adjacency matrix is adaptively fused and learned based on the graph information matrix of each view data, and a final optimal consistency graph adjacency matrix of all graphs is calculated by applying a regularization constraint to the weight.

4. The multi-source image data feature extraction method of claim 1, wherein, After S4 is completed, an alternating optimization method is used to optimize the complete image feature extraction objective function.

5. The multi-source image data feature extraction method of claim 4, wherein, The specific optimization process is as follows: (1) Fixing , solving and , the objective function is as follows: represents an optimal consistency graph adjacency matrix, represents a dual of the original projection matrix, represents a common consistency feature of the view data; An intermediate variable is introduced as follows: To perform eigen decomposition, then can be constructed by the largest eigenvalue corresponding to the eigenvector Bringing in the following formula can be obtained : (2) fixed , solve : (3) fixed , solve : for The column vectors, and They are respectively and The Middle One element; define a vector , ordering its first element The problem can then be transformed into the following form: wherein is the thcolumn vector; (4) fixed , solve , The objective function is as follows: Let The above equation can be equivalent to: The optimization problem can be converted into an optimization problem with the same form as follows: optimal From the following equation: The iterative calculation process is repeated alternately until the objective function converges, i.e. the dual of the projection matrix is obtained , and the consistency feature .

6. A multi-source image data feature extraction system, comprising: It comprises: A data set construction module for mapping original image view data into a high-dimensional space by a kernel method to construct a multi-source image data set, and the multi-source image data set comprises a plurality of high-dimensional linearly separable view data; The specific process is as follows: mapping the original view data to , is a kernel mapping from a d-dimensional space to an infinite dimensional space using a kernel matrix to compute , the kernel matrix obtained after centering ; The expression is as follows: represents the view data that is linearly separable in high dimension after kernel mapping, is the number of samples; represents the matrix row / column, represents the total number of views; A graph information module for calculating a corresponding graph information matrix of each view by using the self-expression learning characteristics and similarity of each view data in the multi-source image data set; An optimal consistency graph adjacency matrix module for considering the importance of different views to give a weight to the corresponding graph information matrix, and calculating an optimal consistency graph adjacency matrix by a self-expression learning and adaptive fusion method; An objective function module for performing canonical correlation analysis on multi-view data based on a graph-constrained canonical correlation analysis algorithm and by taking the fused graph structure information as a constraint, fusing common consistency features of the view data, and constructing a complete image data feature extraction objective function; Assumption of consistency features On the optimal consistency graph adjacency matrix is smooth, introduce Graphing constraint terms for maintaining data locality; The complete feature extraction objective function is as follows: wherein, is a regularization factor, and the value range is [0, ]; represents the view data that needs to extract features after being mapped by the kernel function and being linearly separable in high dimension; represents the total number of views; is a weight coefficient; is the dual of the original multi-view canonical correlation analysis algorithm MCCA projection matrix; represents the common consistency features of each view data; represents a graph constraint term for maintaining locality, wherein represents the trace of a matrix; represents the optimal consistency graph adjacency matrix generated after multi-graph fusion, which is a real symmetric matrix, represents the weight of the edge between any two nodes; is the Laplacian matrix of the optimal consistency graph adjacency matrix ; represents the graph information matrix corresponding to each view; represents the importance of different views; represents the regularization constraint on the similar graph information using the F-norm; represents the regularization constraint on the weight using the 2-norm; represents an identity matrix.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the multi-source image data feature extraction method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program is executed by the processor to realize the steps of the multi-source image data feature extraction method according to any one of claims 1 to 5.

Images