A Dual-Graph Driven Supervised Modeling Approach for Multi-Source Domain Transfer

Through the dual-graph-driven supervised modeling method, the dual-graph neural network model with invariance relationship alignment is used to solve the negative transmission problems caused by the integration of multi-source domain data and distribution differences, and realize the knowledge migration from the source domain to the target domain. It improves the efficiency and information utilization of transfer learning, and is suitable for a variety of soft measurement tasks.

CN118586434BActive Publication Date: 2024-12-03ZHEJIANG ZHENENG ELECTRIC POWER CO LTD XIAOSHAN POWER PLANT +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411062943.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-12-03
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

Existing transfer learning methods have problems of inefficiency and information loss in negative transmission problems caused by data integration and distribution differences in multi-source domains, especially in the case of limited labeled data, which is difficult to effectively transfer knowledge.

Method used

Using a dual-graph-driven supervised modeling method, a public-private graph construction module and information alignment module are constructed through a dual-graph neural network model that is aligned with invariance relationships, and a cross-domain migratory relationship is extracted using topological similarity and feature orthogonal constraints. , realize knowledge migration from the source domain to the target domain.

Benefits of technology

It effectively solves the negative transmission problem caused by the integration and distribution differences of multi-source domain information, realizes the effective migration of knowledge, makes full use of multi-source knowledge, is suitable for soft measurement tasks such as voice and industrial sensors, and improves the generalization ability and migration effect of the model. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118586434B_ABST
    Figure CN118586434B_ABST
Patent Text Reader

Abstract

The present invention discloses a dual-graph driven supervised modeling method for multi-source domain migration, which is mainly a dual-graph neural network model based on invariant relationship alignment, specifically including: extracting cross-domain invariant relationships and domain-specific relationships between variables through a public-private graph construction module, extracting cross-domain transferable relationships through a public graph information alignment module, extracting and filtering domain-specific information between variables through a private graph information filtering module, and collaboratively realizing the extraction of invariant relationships and the migration of multi-source domain knowledge to a target domain through a public graph information alignment module and a private graph information filtering module. The present invention can solve the negative transfer problem caused by the integration and distribution differences of multi-source domain information in traditional transfer learning, and realize effective knowledge migration from the source domain to the target domain; it can be applied to industrial soft measurement tasks of time series data such as speech and industrial sensor parameters in various transfer learning scenarios, and has good learning effect and generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of industrial soft measurement technology, and specifically to the field of industrial soft measurement technology for time series data such as speech and industrial sensor measurement point data in a multi-source domain migration scenario, and more particularly to a dual-graph driven supervised modeling method for multi-source domain migration. Background Art

[0002] In recent years, driven by the rapid development of data acquisition and processing technology, deep learning has received great attention and research in various fields. Deep neural networks have shown their advantages over traditional machine learning algorithms in classification and regression. However, the excellent performance of deep neural networks mainly depends on sufficient labeled data. It also requires that the training samples and test samples follow the same distribution. However, in practice, domain differences between training and test datasets are common, and it is costly to obtain sufficient labeled training samples. To this end, transfer learning provides a facilitating solution to solve the distribution difference by extracting and migrating knowledge from the source domain (original stored data) to the target domain (newly acquired data), and has been widely used in classification, segmentation, soft sensors and other fields.

[0003] Existing transfer learning methods learn domain-invariant features through distribution alignment or adversarial learning. However, in real applications, the training data may also have different distributions, resulting in multi-source data. In addition, in practice, the available training data may also have different distributions, resulting in multi-source data. The motivation of multi-source domain transfer learning is to build models from multiple different source domains and generalize them to related target domains, which can be mainly divided into feature-based and knowledge-based methods.

[0004] Feature-based methods align the data distribution of cross-domain data by mapping the data into a domain-invariant space. Although feature-based methods can reduce the distance between cross-domain data distributions, they may still encounter negative transfer, especially when there are significant data distribution differences between the source domain and the source-target domain.

[0005] Unlike feature-based methods, knowledge-based methods selectively utilize source domain data knowledge to assist target domain modeling. For example, by building a multi-source domain model, and filtering consensus knowledge based on the model output using voting and weighting strategies, and selecting and filtering input data, and selecting data that is very similar to the target domain for modeling, etc. However, these methods may lead to the loss of useful information in the data, or even the loss of information in the entire domain. At the same time, in contrast to previous work that mainly considered completely unlabeled target domain data, in practice, it is often encountered that the labeled data that can be accessed is very limited. However, using these limited labeled data to enhance the construction of the target domain model has been relatively neglected in previous research. Summary of the invention

[0006] The purpose of the present invention is to provide a dual-graph driven supervised modeling method for multi-source domain migration in order to solve the negative transfer problem caused by the integration and distribution differences of multi-source domain information in traditional transfer learning in the prior art. The dual-graph neural network model based on invariant relationship alignment constructed by the present invention uses an invariant graph network to extract cross-domain transferable relations and a specific graph network to capture domain features between variables, thereby realizing effective knowledge transfer from the source domain to the target domain.

[0007] The objective of the present invention is achieved through the following technical solution: A dual-graph driven supervised modeling method for multi-source domain migration, comprising the following steps:

[0008] (1) Obtain multiple source domain datasets and target domain datasets;

[0009] (2) constructing a dual-graph neural network model based on invariant relationship alignment, wherein the dual-graph neural network model includes a public-private graph construction module, a public graph information alignment module, and a private graph information filtering module; wherein the public graph information alignment module and the private graph information filtering module are connected in parallel after the public-private graph construction module;

[0010] (3) Using the multi-source domain dataset and the target domain dataset obtained in step (1) to train the dual-graph neural network model, with the goal of minimizing the loss function of the dual-graph neural network model, optimizing the parameters of the dual-graph neural network model until a preset number of training rounds is reached to obtain a trained dual-graph neural network model;

[0011] (4) Use the trained dual-graph neural network model to perform soft measurement tasks in different target domains.

[0012] Furthermore, the multi-source domain dataset includes K source domain datasets in total, and the kth source domain dataset of the multi-source domain dataset is represented as ,in , and They represent the i-th sample and the corresponding label in the k-th source domain dataset, respectively. , Representation sample The dimensional measurement value, d represents the sample There are d-dimensional measurement samples, Representation sample The l-th dimension measurement in Contains continuous Time series data, Represents the kth source domain dataset The total number of samples in ;

[0013] The target domain dataset is represented as ,in and Respectively represent the i-th sample and the corresponding label in the target domain dataset, Represents the total number of samples in the target domain dataset.

[0014] Furthermore, the public-private graph construction module includes K public graph learning layers arranged in parallel. and K private graph learning layers , where K represents the number of source domains, and the kth public graph learning layer is represented as , the kth private graph learning layer is represented as ; The data in each domain Input into the public-private graph construction module and pass through K public graph learning layers respectively and K private graph learning layers , the last public graph learning layer Output the public graph corresponding to the domain , the last private graph learning layer Output the private graph corresponding to this domain .

[0015] Furthermore, the public-private graph construction module obtains the public graph and the private graph through the following sub-steps:

[0016] (2.1) The source domain data in the K source domain datasets and the target domain data in the target domain dataset Input into the public-private graph construction module and enter the K public graph learning layers When , the variable is calculated according to the following formula and variables The quantitative invariant relationship between :

[0017]

[0018]

[0019] in, It uses a public graph learning layer Parameterized Sigmoid function, Indicates a connection operation. represents the sparsification threshold of the public graph learning layer. Only retain variables when they are strong enough and variables The connection relationship between them;

[0020] (2.2) According to the quantized invariant relationship calculated in step (2.1) , the public graph is calculated based on the following formula :

[0021]

[0022] in, represents the identity matrix, It is composed of elements The adjacency matrix composed of

[0023] (2.3) The source domain data in the K source domain datasets and the target domain data in the target domain dataset Input into the public-private graph construction module and enter the K private graph learning layers When , the variable is calculated according to the following formula and variables Domain-specific quantitative relationships between :

[0024]

[0025]

[0026] in, Is to use a private graph learning layer Parameterized Sigmoid function, Indicates a connection operation. represents the sparsification threshold of the private graph learning layer. Greater than the thinning threshold Only retain variables and variables The connection relationship between them;

[0027] (2.4) According to the quantitative relationship calculated in step (2.3) , the private graph is calculated based on the following formula :

[0028]

[0029] in, represents the identity matrix, It is composed of elements The adjacency matrix.

[0030] Furthermore, the public graph information alignment module includes K graph neural networks arranged in parallel. , the parameters of the graph neural network are shared, the public graph information alignment module extracts domain invariant relations and their corresponding graph representations through graph neural network parameter sharing and topological similarity constraints; the public graph output by the public-private graph construction module is Input to K graph neural networks In the final output, the domain invariant features corresponding to the public graph , which is calculated as:

[0031]

[0032] in, Represents input data, which here represents the data input by the public graph information alignment module; represents the activation function; Graph neural networks representing different domains share weight parameters.

[0033] Furthermore, the private graph information filtering module includes K graph neural networks arranged in parallel , the parameters of the graph neural network are not shared, the private graph information filtering module is used to learn and model the variable relationship in the private domain in each domain, and obtain a private graph different from the public graph through feature decoupling; the private graph output by the public-private graph construction module Input to K graph neural networks In the final output, the domain invariant features corresponding to the private graph , which is calculated as:

[0034]

[0035] in, Represents input data, which here represents the data input by the private graph information filtering module; represents the activation function; Represents the non-shared weight parameters of graph neural networks from different domains.

[0036] Furthermore, the step (3) specifically includes:

[0037] The source domain data in the K source domain data sets and the target domain data in the target domain data sets are input into the dual-graph neural network model, first entering the public-private graph construction module, and then passing through the K public graph learning layers. and K private graph learning layers , get the public graph corresponding to each source domain and target domain and private graph ; Then the public map The domain-invariant features corresponding to the public graph are obtained through the public graph information alignment module , private graph The domain-invariant features corresponding to the private graph are obtained through the private graph information filtering module ; Then according to the quantitative invariant relationship Domain-invariant features corresponding to the public graph Calculate the public graph information alignment loss ; According to the domain invariant features corresponding to the public graph Domain-invariant features corresponding to private graphs Calculating private graph information filtering loss ; Align loss based on public graph information and private graph information filtering loss Calculate the loss function of the dual-graph neural network model; with the goal of minimizing the loss function of the dual-graph neural network model, use the SGD optimizer to adjust and optimize the parameters of the dual-graph neural network model until the preset training rounds are reached to obtain the trained dual-graph neural network model.

[0038] Furthermore, the calculation formula of the loss function of the dual-graph neural network model is:

[0039]

[0040] in, represents the loss function of the dual graph neural network model, represents the public graph information alignment loss, represents the private graph information filtering loss;

[0041] The public graph information alignment loss The calculation formula is:

[0042]

[0043]

[0044]

[0045] in, Indicates the source domain Domain invariant features corresponding to all samples in , Indicates the source domain The labels corresponding to the public graphs of all samples in ; Represents the domain-invariant features corresponding to the common graph of all samples in the target domain, Represents the labels corresponding to all samples in the target domain; is the root mean square error loss, It is a downstream task network with fully connected layers; Indicates that the mth variable is in the source domain and source domain The quantitative similarity between Source Domain The corresponding topological vector of the mth variable in the invariant graph is expressed as:

[0046]

[0047] in, Source Domain Variables in and variables The quantitative invariant relationship between Source Domain Variables in and variables The quantitative invariant relationship between Source Domain Variables in and variables The quantitative invariant relationship between

[0048] The private graph information filtering loss The calculation formula is:

[0049]

[0050]

[0051]

[0052] in, Indicates the source domain The domain-invariant features corresponding to the private graphs of all samples in , Represents the domain-invariant features corresponding to the private graph of all samples in the target domain, and Source domain and the one-hot domain label corresponding to the sample in the target domain, is a downstream classifier network with fully connected layers, is the cross entropy loss function.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] (1) This invention solves the negative transfer problem caused by the integration and distribution differences of multi-source domain information during the migration process, and realizes the effective migration of knowledge from the source domain to the target domain;

[0055] (2) This paper proposes for the first time a concept of invariant relation alignment for graph neural network learning, which can effectively transfer transferable information to the target domain and fully transfer knowledge from multiple source domains;

[0056] (3) The present invention designs two core modules, a public graph information alignment module and a private graph information filtering module, to realize the extraction of transferable relations. The public graph information alignment module uses topological similarity constraints and parameter sharing GNN to realize the maximum similarity across domains of graphs, and the private graph information filtering module uses feature orthogonality constraints to ensure the exclusion of non-transferable components.

[0057] (4) This paper defines a new invariant relationship extraction mechanism, forming a generalizable framework that can realize the generalization and migration of multi-source knowledge and is suitable for various downstream tasks such as speech and industrial sensor parameter soft measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a structural schematic diagram of the dual-graph neural network model of the present invention;

[0059] Figure 2 It is a training flow chart of the dual-graph neural network model of the present invention;

[0060] Figure 3 A schematic diagram of the structure of obtaining a public graph and a private graph through a public-private graph construction module in the present invention;

[0061] Figure 4 A schematic diagram of topological structure migration and feature sharing mechanism for calculating topological similarity in the process of public graph invariant relation alignment in the present invention;

[0062] Figure 5 The figure is a schematic diagram showing the comparison results of prediction errors of soft measurement tasks performed by the method of the present invention and other methods; wherein: Figure 5 (a) is a comparison result of the predicted value and the true value of the method of the present invention and a schematic diagram of the prediction error, Figure 5 (b) is a comparison of the predicted value of the MDAN method with the true value and a schematic diagram of the prediction error. Figure 5 (c) in the figure is the comparison result between the predicted value of ADDA method and the true value and the prediction error diagram. Figure 5 (d) in the figure is the comparison result between the predicted value of the MSDA method and the true value, as well as a schematic diagram of the prediction error. DETAILED DESCRIPTION

[0063] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0064] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0065] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0066] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0067] The following takes the coal mill data set of a 1000 MW supercritical thermoelectric process as an example to explain in detail the dual-graph driven supervised modeling method for multi-source domain migration of the present invention.

[0068] The method specifically comprises the following steps:

[0069] (1) Obtain multiple source domain datasets and target domain datasets.

[0070] Specifically, the coal mill measurement point data under different working conditions, i.e., different coal grinding rates, are used as different data domains, where the source domain contains two industrial control sub-datasets AB, each with 800 samples, and the target domain uses the third working condition data set with a coal grinding rate of 50-65, with a total of 600 samples. The multi-source domain coal mill data set and the target domain coal mill data set constructed based on the multi-source domain coal mill data collected above and the target domain coal mill data are shown in Table 1, where is the number of training samples in the target domain, is the number of test samples.

[0071] Table 1: Multi-source domain coal mill dataset and target domain coal mill dataset

[0072]

[0073] In this embodiment, the multi-source domain dataset includes K source domain datasets in total, and the kth source domain dataset is represented as ,in , and They represent the i-th sample and the corresponding label in the k-th source domain dataset, respectively. , Representation sample The dimensional measurement value, d represents the sample There are d-dimensional measurement samples, Representation sample The l-th dimension measurement in Contains continuous Time series data, Represents the kth source domain dataset The target domain dataset is represented as ,in and Respectively represent the i-th sample and the corresponding label in the target domain dataset, Represents the total number of samples in the target domain dataset.

[0074] It should be noted that the data in the above K source domain datasets and target domain datasets are used as input data for the subsequently constructed dual-graph neural network model.

[0075] (2) Based on the invariant relationship alignment, a dual-graph neural network model is constructed, whose structure is as follows: Figure 1 As shown, the dual-graph neural network model includes a public-private graph construction module, a public graph information alignment module and a private graph information filtering module, wherein the public graph information alignment module and the private graph information filtering module are connected in parallel after the public-private graph construction module.

[0076] In this embodiment, the public-private graph construction module includes K public graph learning layers arranged in parallel: and K private graph learning layers , where K represents the number of source domains, and the kth public graph learning layer is represented as , the kth private graph learning layer is represented as Here, K=2, indicating that there are two source domains, source domain A and source domain B. The data in each domain (i.e., source domain or target domain) Input into the public-private graph construction module and pass through K public graph learning layers respectively and K private graph learning layers , the last public graph learning layer Output the public graph corresponding to the domain , the last private graph learning layer Output the private graph corresponding to this domain .

[0077] Furthermore, if Figure 3As shown, the public-private graph construction module specifically obtains the public graph and the private graph through the following sub-steps:

[0078] (2.1) Based on the multi-source domain coal mill dataset and the target domain coal mill dataset and the dual graph neural network model in step (1), the source domain data in the K source domain datasets are and the target domain data in the target domain dataset Input into the public-private graph construction module and enter the K public graph learning layers When , the variable is calculated according to the following formula and variables The quantitative invariant relationship between :

[0079]

[0080]

[0081] in, It uses a public graph learning layer Parameterized Sigmoid function, Indicates a connection operation. represents the sparsification threshold of the public graph learning layer. Only retain variables when they are strong enough and variables The connection relationship between them.

[0082] It should be understood that the quantitative relationship After that, it needs to be compared with the sparse threshold Comparison is only possible when the quantitative relationship Only retain variables when they are strong enough and variables The connection relationship between When As a variable and variables The quantitative invariant relationship between , otherwise the variable and variables The quantitative invariant relationship between Both are 0.

[0083] (2.2) According to the quantized invariant relationship calculated in step (2.1) , the public graph is calculated based on the following formula :

[0084]

[0085] in, represents the identity matrix, It is composed of elements The adjacency matrix.

[0086] (2.3) Based on the multi-source domain coal mill dataset and the target domain coal mill dataset and the dual graph neural network model in step (1), the source domain data in the K source domain datasets are and the target domain data in the target domain dataset Input into the public-private graph construction module and enter the K private graph learning layers When , the variable is calculated according to the following formula and variables Domain-specific quantitative relationships between :

[0087]

[0088]

[0089] in, Is to use a private graph learning layer Parameterized Sigmoid function, Indicates a connection operation. represents the sparsification threshold of the private graph learning layer. Only retain variables when they are strong enough and variables The connection relationship between them.

[0090] It should be understood that the quantitative relationship After that, it needs to be compared with the sparse threshold Comparison is only possible when the quantitative relationship Only retain variables when they are strong enough and variables The connection relationship between When As a variable and variables Domain-specific quantitative relationships between , otherwise the variable and variables Domain-specific quantitative relationships between Both are 0.

[0091] (2.4) According to the quantitative relationship calculated in step (2.3) , the private graph is calculated based on the following formula :

[0092]

[0093] in, represents the identity matrix, It is composed of elements The adjacency matrix.

[0094] In this embodiment, the public graph information alignment module includes K graph neural networks (GNNs) arranged in parallel. , its graph neural network parameters are shared, and the public graph information alignment module extracts domain invariant relations and their corresponding graph representations through graph neural network parameter sharing and topological similarity constraints. The public graph output by the public-private graph construction module Input to K graph neural networks In the final output, the domain invariant features corresponding to the public graph , which is calculated as:

[0095]

[0096] in, Represents input data, which here represents the data input by the public graph information alignment module; represents the activation function; Graph neural networks representing different domains share weight parameters.

[0097] In this embodiment, the private graph information filtering module includes K graph neural networks composed in parallel The parameters of the graph neural network are not shared. The private graph information filtering module is used to learn and model the variable relationship in the private domain in each domain, and obtain a private graph different from the public graph through feature decoupling. The private graph output by the public-private graph construction module Input to K graph neural networks In the final output, the domain invariant features corresponding to the private graph , which is calculated as:

[0098]

[0099] in, Represents input data, which here represents the data input by the private graph information filtering module; represents the activation function; Represents the non-shared weight parameters of graph neural networks from different domains.

[0100] (3) Using the multi-source domain dataset and the target domain dataset obtained in step (1) to train the dual-graph neural network model, with the goal of minimizing the loss function of the dual-graph neural network model, the parameters of the dual-graph neural network model are optimized until a preset number of training rounds is reached to obtain a trained dual-graph neural network model.

[0101] Specifically, Figure 2 As shown in the figure, the source domain data in the K source domain data sets and the target domain data in the target domain data set are input into the dual-graph neural network model, first entering the public-private graph construction module, and then passing through the K public graph learning layers. and K private graph learning layers , get the public graph corresponding to each source domain and target domain and private graph ; Then the public map The domain-invariant features corresponding to the public graph are obtained through the public graph information alignment module , private graph The domain-invariant features corresponding to the private graph are obtained through the private graph information filtering module ; Then according to the quantitative invariant relationship Domain-invariant features corresponding to the public graph Calculate the public graph information alignment loss ; According to the domain invariant features corresponding to the public graph Domain-invariant features corresponding to private graphs Calculating private graph information filtering loss ; Align loss based on public graph information and private graph information filtering loss Calculate the loss function of the dual-graph neural network model; with the goal of minimizing the loss function of the dual-graph neural network model, use the SGD optimizer to adjust and optimize the parameters of the dual-graph neural network model until the preset training rounds are reached to obtain the trained dual-graph neural network model.

[0102] Furthermore, the public graph information alignment loss The calculation formula is:

[0103]

[0104]

[0105]

[0106] in, Indicates that the mth variable is in the source domain and source domain The quantitative similarity between Source Domain The corresponding topological vector of the mth variable in the invariant graph; Indicates the source domain The domain-invariant features corresponding to the public graphs corresponding to all samples in , Indicates the source domain The labels corresponding to all samples in ; Represents the domain-invariant features corresponding to the common graph of all samples in the target domain, Represents the labels corresponding to all samples in the target domain; is the root mean square error loss, It is a downstream task network with fully connected layers.

[0107] Furthermore, the source domain The corresponding topological vector of the mth variable in the invariant graph According to the quantitative invariant relationship Calculated, such as Figure 4 As shown, the calculation formula is:

[0108]

[0109] in, Source Domain Variables in and variables The quantitative invariant relationship between Source Domain Variables in and variables The quantitative invariant relationship between Source Domain Variables in and variables The quantitative invariant relationship between them.

[0110] Furthermore, private graph information filtering loss The calculation formula is:

[0111]

[0112]

[0113]

[0114] in, Indicates the source domain The domain-invariant features corresponding to the private graphs of all samples in , Represents the domain-invariant features corresponding to the private graph of all samples in the target domain, and Source domain and the one-hot domain label corresponding to the sample in the target domain, is a downstream classifier network with fully connected layers, is the cross entropy loss function.

[0115] In summary, the loss function of the dual-graph neural network model is calculated as follows:

[0116]

[0117] in, Represents the loss function of the dual graph neural network model.

[0118] (4) Use the trained dual-graph neural network model to perform soft measurement tasks in different target domains.

[0119] For example, a coal mill data set is used for performance verification, and the method of the present invention and seven other different methods are used to perform soft sensing tasks under different target domain training sample settings, and their performances are compared, and the root mean square error (RMSE) is used as an evaluation index, and the results are shown in Table 2. It can be seen from Table 2 that compared with other methods, the method of the present invention has the smallest RMSE, and compared with the second best method, the RMSE performance is improved by an average of 15%.

[0120] Table 2: Soft sensor performance of different methods under different sample numbers of coal mill dataset (RMS error)

[0121]

[0122] For example, the method of the present invention and three other methods are used to perform soft measurement tasks, and the comparison results between the predicted values ​​and the true values ​​of different methods and the errors between the predicted values ​​and the true values ​​are obtained. The results are as follows: Figure 5 shown. Figure 5 The performance comparison results of the method of the present invention and the other three methods, namely MDAN, ADDA and MSDA, are given when the number of labeled data in the target domain is 50. The comparison results between the predicted value and the true value and the prediction error of the method of the present invention are shown in Figure 2. Figure 5 As shown in (a) in the figure, the comparison results between the predicted value and the true value of the MDAN method and the prediction error are shown in Figure 5 As shown in (b), the comparison results between the predicted value and the true value of the ADDA method and the prediction error are shown in Figure 5 As shown in (c) in the figure, the comparison results between the predicted value and the true value of the MSDA method and the prediction error are shown in Figure 5 As shown in (d), it can be seen that compared with MDAN, ADDA and MSDA methods, the performance of the method of the present invention is still the best. Among them, MDAN, ADDA and MSDA show similar performance, and ADDA is 11.97% and 12.87% higher than MDAN and MSDA respectively. When the number of labeled samples in the target domain is very limited, such as 50, the method proposed in the present invention performs best, and the prediction errors of most test sample points are kept within (-0.1, 0.1).

[0123] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dual-graph-driven supervised modeling approach for multi-source domain migration. It is characterized in that The following steps are involved: (1) obtaining a multi-source domain dataset and a target domain dataset; wherein the multi-source domain dataset and the target domain dataset include speech and industrial sensor parameter soft measurement data; (2) constructing a dual-graph neural network model based on invariant relationship alignment, wherein the dual-graph neural network model includes a public-private graph construction module, a public graph information alignment module, and a private graph information filtering module; wherein the public graph information alignment module and the private graph information filtering module are connected in parallel after the public-private graph construction module; The public-private graph construction module includes K public graph learning layers θ in parallel. c and K private graph learning layers θ P , where K represents the number of source domains, and the kth public graph learning layer is represented as The kth private graph learning layer is represented as The data x in each domain is 1 …, v d ] is input into the public-private graph construction module and passes through K public graph learning layers θ respectively. c and K private graph learning layers θ P , the last public graph learning layer θ c Output the public graph A corresponding to the domain inv , the last private graph learning layer θ P Output the private graph A corresponding to the domain spe ; The public graph information alignment module includes K graph neural networks GNNs arranged in parallel c , the parameters of the graph neural network are shared, and the public graph information alignment module extracts domain invariant relations and their corresponding graph representations through graph neural network parameter sharing and topological similarity constraints; the public graph A output by the public-private graph construction module inv Input to K graph neural networks GNN c In the final output, the domain invariant feature h corresponding to the public graph c , which is calculated as: h c =GNN c (X)=σ(A inv XW shared ) Where X represents the input data, which here represents the input data of the public graph information alignment module; σ represents the activation function; W shared Graph neural networks representing different domains share weight parameters; The private graph information filtering module includes K graph neural networks GNNs arranged in parallel p , the parameters of the graph neural network are not shared, the private graph information filtering module is used to learn and model the variable relationship in the private domain in each domain, and obtain a private graph different from the public graph through feature decoupling; the private graph A output by the public-private graph construction module spe Input to K graph neural networks GNN p In the final output, the domain invariant feature hp corresponding to the private graph is calculated as: h p =GNN p (X)=σ(A ape XW unshared ) Where X represents the input data, which here represents the input data of the private graph information filtering module; σ represents the activation function; W unshared Representing non-shared weight parameters of graph neural networks in different domains; (3) using the multi-source domain dataset and the target domain dataset obtained in step (1) to train the dual-graph neural network model, with the goal of minimizing the loss function of the dual-graph neural network model, optimizing the parameters of the dual-graph neural network model until a preset training round is reached, so as to obtain a trained dual-graph neural network model; Wherein, the step (3) specifically includes: The source domain data in the K source domain data sets and the target domain data in the target domain data sets are input into the dual-graph neural network model, first entering the public-private graph construction module, and then passing through the K public graph learning layers θ c and K private graph learning layers θ P , get the public graph A corresponding to each source domain and target domain inv and private graph A spe ; Then the public graph A inv The domain invariant feature h corresponding to the public graph is obtained through the public graph information alignment module c , private graph A spe The domain invariant feature h corresponding to the private graph is obtained through the private graph information filtering module. p ; Then according to the quantitative invariant relationship The domain-invariant feature h corresponding to the public graph c Calculate the public graph information alignment loss L 1 ; According to the domain invariant feature h corresponding to the public graph c The domain-invariant feature h corresponding to the private graph p Calculate the private graph information filtering loss L 2 ; Alignment loss L based on public graph information 1 and private graph information filtering loss L 2 Calculate the loss function of the dual-graph neural network model; use the SGD optimizer to adjust and optimize the parameters of the dual-graph neural network model with the goal of minimizing the loss function of the dual-graph neural network model until the preset training rounds are reached to obtain the trained dual-graph neural network model; The calculation formula of the loss function of the dual-graph neural network model is: L total =L 1 +L 2 Among them, L total represents the loss function of the dual graph neural network model, L 1 represents the public graph information alignment loss, L 2 represents the private graph information filtering loss; The public graph information alignment loss L 1 The calculation formula is: in, represents the domain invariant features corresponding to all samples in the source domain k, Represents the labels corresponding to the public graphs of all samples in the source domain k; Represents the domain-invariant features corresponding to the common graph of all samples in the target domain, Represents the labels corresponding to all samples in the target domain; is the root mean square error loss, Task() is the downstream task network with fully connected layers; represents the quantitative similarity between the source domain k and the source domain k′ of the mth variable, is the corresponding topological vector of the mth variable in the source domain k in the invariant graph, and its expression is: in, is the variable v in the source domain k 1 and the variable v m The quantitative invariant relationship between is the variable v in the source domain k m-1 and the variable v m The quantitative invariant relationship between is the variable v in the source domain k d and the variable v m The quantitative invariant relationship between The private graph information filtering loss L 2 The calculation formula is: in, represents the domain-invariant features corresponding to the private graph of all samples in the source domain k, h t,p Represents the domain-invariant features corresponding to the private graph of all samples in the target domain, and d t are the one-hot domain labels corresponding to samples in the source domain k and the target domain, respectively. Domain() is the downstream classifier network with a fully connected layer. is the cross entropy loss function; (4) Use the trained dual-graph neural network model to perform soft measurement tasks in different target domains.

2. The dual-graph-driven supervised modeling method for multi-source domain migration according to claim 1, It is characterized in that The multi-source domain dataset includes K source domain datasets in total, and the kth source domain dataset of the multi-source domain dataset is represented as where k = 1, 2, ..., K, and They represent the i-th sample and the corresponding label in the k-th source domain dataset, respectively. Representation sample The lth dimension measurement value in , d represents the sample There are d-dimensional measurement samples, Representation sample The ,th dimension measurement in Contains w consecutive time series data, Represents the kth source domain dataset The total number of samples in ; The target domain dataset is represented as where x t,i and t,i Respectively represent the i-th sample and the corresponding label in the target domain dataset, n t Represents the total number of samples in the target domain dataset.

3. The dual-graph-driven supervised modeling method for multi-source domain migration according to claim 1, It is characterized in that The public-private graph construction module specifically obtains the public graph and the private graph through the following sub-steps: (2.1) The source domain data in K source domain datasets and the target domain data x in the target domain dataset t =[v 1 , ..., v d ] is input into the public-private graph construction module and enters K public graph learning layers θ c When the variable v is calculated according to the following formula l and the variable v m The quantitative invariant relationship between Where f is the layer θ learned using the public graph c Parameterized Sigmoid function, Indicates the connection operation, thr c represents the sparsification threshold of the public graph learning layer. Greater than the sparseness threshold thr c Only then the variable v is retained l and the variable v m The connection relationship between them; (2.2) According to the quantized invariant relationship calculated in step (2.1) The public graph A is calculated based on the following formula inv : A inv ←A inv +(A inv ) T +I Where I represents the identity matrix, A inv It is composed of elements The adjacency matrix composed of (2.3) The source domain data in K source domain datasets and the target domain data x in the target domain dataset t =[v 1 , ..., v d ] is input into the public-private graph construction module and enters K private graph learning layers θ P When the variable v is calculated according to the following formula l and the variable v m Domain-specific quantitative relationships between where f is the private graph learning layer θ p Parameterized Sigmoid function, Indicates the connection operation, thr p represents the sparsification threshold of the private graph learning layer. The variable v is retained only when it is strong enough l and the variable v m The connection relationship between them; (2.4) According to the quantitative relationship calculated in step (2.3) The private graph A is calculated based on the following formula spe : A spe ←A spe +(A spe ) T +I Where I represents the identity matrix, A spe It is composed of elements The adjacency matrix.