Prediction model training method, device, equipment and storage medium

Abstract

The present invention proposes a training method and apparatus, device and storage medium for a prediction model, wherein the method comprises: obtaining multiple source domain data; wherein the multiple source domain data include source labels; inputting the multiple source domain data into a neural network for training, obtaining a first prediction result, adjusting the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model; obtaining target domain data, wherein the target domain data includes a target label; inputting the target domain data into the pre-trained model for training, obtaining a second prediction result, adjusting the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain. Thus, a pre-trained model is obtained by pre-training with data from the source domain, and the parameters of the pre-trained model are fine-tuned with data from the target domain to generate a prediction model to achieve accurate prediction even when the amount of data is small.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a prediction model training method, device, equipment and storage medium. Background Art

[0002] Generally, the prediction of indoor temperature is very important for the predictive control of district heating systems. On the one hand, it can maintain the indoor temperature within a comfortable range for human body, and on the other hand, it can reduce the fluctuation of heating temperature and reduce heat loss, thus achieving the goal of energy saving and emission reduction.

[0003] In the related art, the indoor temperature prediction method relies on sufficient training data for time series prediction. However, in real life, for newly built communities, sensor data is often very scarce, which can lead to the problem of overfitting in training, thereby reducing the prediction performance for the future. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] The present invention proposes a prediction model training method, device, equipment and storage medium, which solves the problem of how to ensure accurate prediction in a newly built community when the amount of data is small. The pre-training model is obtained by pre-training with data from the source domain, and the parameters of the pre-training model are fine-tuned with data from the target domain to generate a prediction model to achieve accurate prediction even when the amount of data is small.

[0006] The first embodiment of the present invention provides a prediction model training method, comprising:

[0007] Acquire multiple source domain data; wherein the multiple source domain data include source labels;

[0008] Inputting the plurality of source domain data into a neural network for training, obtaining a first prediction result, and adjusting a network parameter of the neural network according to the first prediction result and the source label to generate a pre-trained model;

[0009] Acquire target domain data, wherein the target domain data includes a target tag;

[0010] The target domain data is input into the pre-trained model for training to obtain a second prediction result, and the network parameters of the pre-trained model are adjusted according to the second prediction result and the target label to generate a prediction model for the target domain.

[0011] The training method of the prediction model of the embodiment of the present invention is to obtain multiple source domain data; wherein the multiple source domain data include source labels; input the multiple source domain data into a neural network for training, obtain a first prediction result, adjust the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model; obtain target domain data, wherein the target domain data includes a target label; input the target domain data into the pre-trained model for training, obtain a second prediction result, adjust the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain. Thus, the pre-trained model is obtained by pre-training with the source domain data, and the parameters of the pre-trained model are fine-tuned with the target domain data to generate a prediction model to achieve accurate prediction even with a small amount of data.

[0012] The second embodiment of the present invention provides a training device for a prediction model, comprising:

[0013] A first acquisition module is used to acquire a plurality of source domain data; wherein the plurality of source domain data includes a source label;

[0014] A first training module, used for inputting the plurality of source domain data into a neural network for training to obtain a first prediction result;

[0015] A first generating module, configured to adjust a network parameter of the neural network according to the first prediction result and the source label to generate a pre-trained model;

[0016] A second acquisition module is used to acquire target domain data, wherein the target domain data includes a target tag;

[0017] A second training module, used for inputting the target domain data into the pre-training model for training to obtain a second prediction result;

[0018] The second generation module is used to adjust the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model of the target domain.

[0019] The training device of the prediction model of the embodiment of the present invention obtains multiple source domain data, wherein the multiple source domain data include source labels; inputs the multiple source domain data into a neural network for training, obtains a first prediction result, adjusts the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model; obtains target domain data, wherein the target domain data includes a target label; inputs the target domain data into the pre-trained model for training, obtains a second prediction result, adjusts the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain. Thus, the pre-trained model is obtained by pre-training with the source domain data, and the parameters of the pre-trained model are fine-tuned with the target domain data to generate a prediction model that can achieve accurate prediction even with a small amount of data.

[0020] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the training method of the prediction model provided in the first aspect of the present invention is implemented.

[0021] The fourth aspect of the present invention provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the prediction model training method provided in the first aspect of the present invention.

[0022] The fifth aspect of the present invention provides a computer program product. When the instructions in the computer program product are executed by a processor, the training method of the prediction model provided in the first aspect of the present invention is performed.

[0023] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 It is a schematic diagram showing different indoor temperature changes in different cells in an embodiment of the present invention;

[0026] Figure 2 A flowchart of a method for training a prediction model provided in the first embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of a model architecture in an embodiment of the present invention;

[0028] Figure 4 is a schematic diagram of the structure of the joint decomposition in an embodiment of the present invention;

[0029] Figure 5 A schematic diagram of a flow chart of a method for training a prediction model provided in Embodiment 2 of the present invention;

[0030] Figure 6 This is a schematic diagram of knowledge aggregation in an embodiment of the present invention;

[0031] Figure 7 is a schematic diagram of a memory network in an embodiment of the present invention;

[0032] Figure 8 A schematic diagram of the structure of a training device for a prediction model provided in Embodiment 4 of the present invention;

[0033] Fig. 9 A block diagram of an exemplary electronic device or server suitable for implementing embodiments of the present invention is shown. DETAILED DESCRIPTION

[0034] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0035] The following describes the prediction model training method, apparatus, device and storage medium according to the embodiments of the present invention with reference to the accompanying drawings.

[0036] In practical applications, the indoor temperature changes in different communities are different, such as Figure 1 As shown in (a), the room temperature changes in the three communities are different in magnitude and even direction. Indoor temperature is affected by a variety of dynamic factors, such as Figure 1 As shown in (b), various factors can be divided into external factors and internal factors. External factors, such as weather and heating system, mainly reflect changes in the external environment; internal factors, such as building structure and indoor activities, reflect the characteristics of the building itself and are difficult to measure with data. Therefore, directly migrating the model trained in the source domain to the target domain will lead to poor prediction results.

[0037] Therefore, affected by various factors, the data distribution of indoor temperature deviates over time. For newly built communities, there is only a small amount of data, which is not enough to cover the entire data distribution. Therefore, the observed data may have a large distribution difference with future data. Figure 1The distribution of the red solid line (existing data of the new community) in (a) in interval 1 is very different from the distribution of the red dotted line (future data of the new community) in interval 2. Therefore, it is necessary to solve the problem of how to accurately predict the newly built community when the amount of data is very small.

[0038] In response to the above problems, the present application proposes a prediction model training method, which obtains a pre-trained model by pre-training with data from the source domain, and fine-tunes the parameters of the pre-trained model with data from the target domain to generate a prediction model, thereby improving the generalization ability of the model and achieving accurate prediction even with very little data, such as future indoor temperature, future indoor traffic flow, etc.

[0039] Figure 2 A flowchart of a method for training a prediction model provided in Embodiment 1 of the present invention.

[0040] The conversation recognition method of the embodiment of the present invention can be applied to electronic devices. The electronic device can be any device with computing capabilities, such as a PC (Personal Computer), a mobile terminal, etc. The mobile terminal can be a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens, and / or display screens.

[0041] like Figure 2 As shown, the training method of the prediction model may include the following steps:

[0042] Step 101, obtaining a plurality of source domain data; wherein the plurality of source domain data includes source labels.

[0043] In an embodiment of the present invention, different source domains and target domains can be determined for different application scenarios. For example, the source domain may be a community that has been established for many years, and the relevant temperature sensors have a sufficient amount of data such as detecting temperature over time, while the target domain is a newly built community, which only detects a small amount of temperature data. For another example, the source domain may be a shopping mall that has been established for many years, and the relevant cameras have a sufficient amount of data such as detecting pedestrian flow over time, while the target domain is a newly built shopping mall, which only detects a small amount of pedestrian flow data, etc. The specific settings are selected according to the application needs.

[0044] In an embodiment of the present invention, in order to make the pre-trained model more accurate, data from multiple different source domains are obtained as samples for training, that is, multiple source domain data. It can be understood that the source domain data has accurate temperature, pedestrian flow and other data for a period of time in the future relative to the prediction time as source labels.

[0045] Step 102: input multiple source domain data into a neural network for training, obtain a first prediction result, and adjust the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model.

[0046] In an embodiment of the present invention, multiple source domain data are input into a neural network for training, and there are many ways to obtain a first prediction result. As an example of a scenario, multiple source domain data are processed by a feature extraction network of a neural network to generate multiple source domain features; multiple source domain features are processed separately by an independent subnetwork and multiple unique subnetworks of the neural network to generate source domain shared features and multiple source domain unique features; the source domain shared features and each source domain unique feature are spliced ​​and input into a predictor to obtain a first prediction result.

[0047] Among them, as an example, the first prediction result is the temperature prediction result of a period of time in the future relative to the prediction time, and the source label is the accurate temperature data of a period of time in the future relative to the prediction time. The network parameters of the neural network are continuously adjusted according to the difference between the first prediction result and the source label, so that the first prediction result and the source label are consistent or the difference is maintained within a reasonable range. The training can be stopped to generate a pre-trained model.

[0048] Step 103: Acquire target domain data, wherein the target domain data includes a target tag.

[0049] In the embodiment of the present invention, the selection of the target domain refers to the detailed description of the selection of the source domain and the target domain in step 101, which will not be described in detail here.

[0050] In the embodiment of the present invention, in order to obtain the prediction model of the target domain, it is necessary to obtain the target domain data. It can be understood that the target domain data has accurate temperature, human flow and other data for a period of time in the future relative to the prediction time as the target label.

[0051] Step 104: input the target domain data into the pre-trained model for training, obtain a second prediction result, and adjust the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain.

[0052] In an embodiment of the present invention, the target domain data is input into a pre-trained model for training. There are many ways to obtain a second prediction result. As an example scenario, the target domain data is processed by a feature extraction network of a pre-trained model to generate target domain features; the target domain features are processed by a target domain-source domain memory network, an independent sub-network, and a unique sub-network of the pre-trained model respectively to generate target domain-source domain shared features and target domain-specific features; the target domain-source domain related features, the target domain shared features, and each target domain-specific feature are spliced ​​and input into a predictor to obtain a second prediction result.

[0053] Among them, as an example, the second prediction result is the temperature prediction result of a period of time in the future relative to the prediction time, and the target label is the accurate temperature data of a period of time in the future relative to the prediction time. The network parameters of the pre-trained model are continuously adjusted according to the difference between the second prediction result and the target label, so that the second prediction result and the target label are consistent or the difference is maintained within a reasonable range. The training can be stopped to generate a prediction model of the target domain.

[0054] The training method of the prediction model of the embodiment of the present invention is to obtain multiple source domain data; wherein the multiple source domain data include source labels; input the multiple source domain data into a neural network for training, obtain a first prediction result, adjust the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model; obtain target domain data, wherein the target domain data includes a target label; input the target domain data into the pre-trained model for training, obtain a second prediction result, adjust the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain. Thus, the pre-trained model is obtained by pre-training with the source domain data, and the parameters of the pre-trained model are fine-tuned with the target domain data to generate a prediction model to achieve accurate prediction even with a small amount of data.

[0055] The above process is described in detail below in conjunction with Example 2.

[0056] In the embodiment of the present invention, Figure 3 The model architecture example shown in the figure uses data from the source domain for pre-training to learn knowledge in multiple source domains. The network parameters are then fine-tuned using data from the target domain to transfer the knowledge learned in the source domain to the target domain.

[0057] Specifically, the source domain data are sent to the neural network for processing. First, the input source domain data passes through a feature extraction network to learn the original feature expression, and then passes through a source domain knowledge memory module. The source domain knowledge memory module adopts a joint decomposition architecture to learn the shared and unique knowledge of different source domains. Among them, the independent sub-network is used to extract the features common to each source domain, and the unique sub-network is used to learn the unique information of each source domain. Finally, the features common to each source domain and the unique information of each source domain are spliced ​​and sent to the predictor to obtain the first prediction result corresponding to the input source domain. The network parameters of the neural network are adjusted based on the first prediction result and the source label to generate a pre-trained model.

[0058] Among them, the indoor temperature changes of different source domains, such as communities, are different, but they all follow the same physical law of heat exchange. A source domain knowledge memory module based on joint decomposition is proposed to learn the shared and unique knowledge in each source domain community. The schematic diagram of the joint decomposition architecture is shown in the figure below. Figure 4As shown in (a), the data of each source domain is fed into an independent sub-network and a unique sub-network respectively. The parameters of the independent sub-network are shared by all source domains, while there is a branch of a unique sub-network for each source domain to remember the historical information of each source domain. The structure of the independent sub-network is as follows: Figure 4 As shown in (b), it is composed of two layers of fully connected networks, batch normalization (Batch Normalization, BN) and rectified linear unit (Rectified Linear Unit, ReLU).

[0059] Furthermore, the parameters of the feature extraction network and the source domain knowledge memory module in the pre-trained model are fixed, and then a memory-enhanced knowledge fusion module is designed to utilize the knowledge learned in the source domain. Specifically, the target domain features extracted by the feature extraction network are sent to the independent sub-network and the unique sub-network respectively to learn the shared and multiple source domain-specific information respectively. In addition, the target domain source domain memory module is used to memorize the correlation between the target domain and the source domain, and use these correlations to fuse the source domain-specific knowledge. Finally, the domain shared and unique features are spliced, and the second prediction result of the final target domain is obtained through the predictor. The network parameters of the pre-trained model are adjusted based on the second prediction result and the target label to generate a prediction model for the target domain.

[0060] Figure 5 A flowchart of a method for training a prediction model provided in Embodiment 2 of the present invention.

[0061] like Figure 5 As shown, it mainly describes how to obtain the shared features of the target domain and the source domain. The training method of the prediction model can also include the following steps:

[0062] Step 201: The target domain source domain memory network processes the target domain features to obtain historically relevant features of the source domain and the target domain.

[0063] In step 202, the memory network of the unique sub-network retrieves the target domain features to obtain multiple memory expression features.

[0064] Step 203, calculate based on the historical related features of the source domain and the target domain and multiple memory expression features to obtain the shared features of the target domain and the source domain.

[0065] In the embodiment of the present invention, the parameters of the independent sub-network and the unique sub-network of the pre-trained model are frozen, and the input target domain features need to be used to learn domain-shared and independent features through the independent sub-network and the unique sub-network, such as Figure 6 As shown in Figure 1, when the target domain feature passes through the unique sub-network, the memory network of all branches will be retrieved at the same time, and n memory expression features will be generated. At the same time, a target domain source domain memory network is designed to memorize the historical correlation between the source domain and the target domain. When the target domain feature passes through, the source domain target domain historical correlation feature m is generated t .

[0066] Furthermore, the historically relevant features of the source domain and the target domain and multiple memory expression features are calculated to obtain the shared features of the target domain and the source domain. For example, the weights of different source domain information are calculated through a linear transformation, and the historically relevant features of the source domain and the target domain and multiple memory expression features are calculated through weighted fusion to obtain the shared features of the target domain and the source domain. The specific calculation process is as follows: c = softmax(W·m t +b), Among them, W, b are the parameters of the fully connected network, c represents the normalized similarity vector, and o represents the result after the aggregation of source domain knowledge, that is, the shared features of the target domain and the source domain.

[0067] In an embodiment of the present invention, the memory network of the unique sub-network retrieves the target domain features, and there are many ways to obtain multiple memory expression features. As a possible implementation method, the similarity between the target domain features and each memory unit in the memory network is calculated to obtain multiple similarities, and calculations are performed based on each similarity and the stored memory feature matrix to generate multiple memory expression features.

[0068] Specifically, for independent sub-networks, each branch is implemented by a memory network, such as Figure 7 As shown, it mainly contains an embedding matrix, namely the above memory feature matrix Contains V memory expressions, each vector has a dimension of d. The memory retrieval process is similar to the key-value query. Given a query vector k, the similarity p between it and each memory unit is calculated as follows: M i represents the i-th row of matrix M, p i is a probability value ranging from 0 to 1, and the retrieved vector v is calculated, that is, the way to remember the expression feature is:

[0069] Therefore, the joint decomposition architecture is used to learn the shared and unique knowledge of multiple source domains, remember the historical correlation between the source domain and the target domain, and calculate the weights of knowledge from different source domains to transfer the knowledge of the source domain to the target domain, and generate a prediction model to achieve accurate prediction even with very little data.

[0070] With the above Figures 1 to 7 Corresponding to the training method of the prediction model provided in the embodiment, the present invention also provides a training device for the prediction model. Figures 1 to 7The training method of the prediction model provided in the embodiment corresponds to the training method of the prediction model, so the implementation method of the training method of the prediction model is also applicable to the training device of the prediction model provided in the embodiment of the present invention, and will not be described in detail in the embodiment of the present invention.

[0071] Figure 8 This is a schematic diagram of the structure of a training device for a prediction model provided in Embodiment 4 of the present invention.

[0072] like Figure 8 As shown, the training device 800 of the prediction model is applied to an electronic device, and includes: a first acquisition module 801, a first training module 802, a first generation module 803, a second acquisition module 804, a second training module 805 and a second generation module 806.

[0073] The first acquisition module 801 is used to acquire a plurality of source domain data; wherein the plurality of source domain data includes source labels.

[0074] The first training module 802 is used to input the multiple source domain data into the neural network for training to obtain a first prediction result.

[0075] The first generating module 803 is used to adjust the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model.

[0076] The second acquisition module 804 is used to acquire target domain data, wherein the target domain data includes a target tag.

[0077] The second training module 805 is used to input the target domain data into the pre-training model for training to obtain a second prediction result.

[0078] The second generating module 806 is used to adjust the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model of the target domain.

[0079] Furthermore, in a possible implementation manner of an embodiment of the present invention, the first training module 802 is specifically used to: process the multiple source domain data through the feature extraction network of the neural network to generate multiple source domain features; process the multiple source domain features respectively through the independent subnetwork and multiple unique subnetworks of the neural network to generate source domain shared features and multiple source domain unique features; splice the source domain shared features and each of the source domain unique features and input them into the predictor to obtain the first prediction result.

[0080] Furthermore, in a possible implementation of an embodiment of the present invention, the second training module 805 includes: a first processing unit, used to process the target domain data through the feature extraction network of the pre-trained model to generate target domain features; a second processing unit, used to process the target domain features through the target domain-source domain memory network, independent sub-network and unique sub-network of the pre-trained model respectively to generate target domain-source domain shared features and target domain unique features; an acquisition unit, used to splice the target domain-source domain related features, the target domain shared features and each of the target domain unique features and input them into the predictor to obtain the second prediction result.

[0081] Furthermore, in a possible implementation manner of an embodiment of the present invention, the second processing unit includes: a processing subunit, which is used for the target domain-source domain memory network to process the target domain features and obtain source domain-target domain historically related features; a retrieval subunit, which is used for the memory network of the unique subnetwork to retrieve the target domain features and obtain multiple memory expression features; a calculation subunit, which is used to calculate based on the source domain-target domain historically related features and the multiple memory expression features to obtain the target domain-source domain shared features.

[0082] Further, in a possible implementation manner of the embodiment of the present invention, the retrieval subunit is specifically used to: calculate the similarity between the target domain feature and each memory unit in the memory network to obtain multiple similarities;

[0083] The training device of the prediction model of the embodiment of the present invention obtains multiple source domain data, wherein the multiple source domain data include source labels; inputs the multiple source domain data into a neural network for training, obtains a first prediction result, adjusts the network parameters of the neural network according to the first prediction result and the source label to generate a pre-trained model; obtains target domain data, wherein the target domain data includes a target label; inputs the target domain data into the pre-trained model for training, obtains a second prediction result, adjusts the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain. Thus, the pre-trained model is obtained by pre-training with the source domain data, and the parameters of the pre-trained model are fine-tuned with the target domain data to generate a prediction model that can achieve accurate prediction even with a small amount of data.

[0084] In order to implement the above embodiments, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above embodiments of the present invention are implemented. Figure 1-Figure 7 The training method of the prediction model proposed in any embodiment.

[0085] In order to implement the above embodiments, the present invention further proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the training method of the prediction model proposed in any of the above embodiments of the present invention is implemented.

[0086] In order to implement the above embodiments, the present invention further proposes a computer program product. When the instructions in the computer program product are executed by a processor, the training method of the prediction model proposed in any of the above embodiments of the present invention is executed.

[0087] Fig. 9 A block diagram of an exemplary electronic device or server suitable for implementing embodiments of the present invention is shown. Fig. 9 The electronic device or server 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0088] like Fig. 9 As shown, the electronic device or server 12 is in the form of a general-purpose computing device. The components of the electronic device or server 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0089] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0090] The electronic device or server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device or server 12, including volatile and non-volatile media, removable and non-removable media.

[0091] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device or server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Fig. 9 not shown, usually called a "hard drive"). Although Fig. 9 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (CD-ROM), a digital versatile disc read only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present invention.

[0092] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0093] The electronic device or server 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device or server 12, and / or communicate with any device that enables the electronic device or server 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the electronic device or server 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device or server 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device or server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0094] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the methods mentioned in the above embodiments.

[0095] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0096] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0097] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0098] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0099] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0100] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0101] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0102] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A prediction model training method, characterized in that: The following steps are involved: Acquire multiple source domain data; wherein the multiple source domain data include source labels; Inputting the plurality of source domain data into a neural network for training, obtaining a first prediction result, and adjusting a network parameter of the neural network according to the first prediction result and the source label to generate a pre-trained model; Acquire target domain data, wherein the target domain data includes a target tag; Inputting the target domain data into the pre-trained model for training, obtaining a second prediction result, and adjusting the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model for the target domain; The step of inputting the plurality of source domain data into a neural network for training to obtain a first prediction result includes: Processing the plurality of source domain data by a feature extraction network of the neural network to generate a plurality of source domain features; The multiple source domain features are processed respectively by the independent subnetwork and multiple unique subnetworks of the neural network to generate source domain shared features and multiple source domain unique features; wherein the independent subnetwork is used to extract the features common to the multiple source domains, and the unique subnetwork is used to learn the unique information of each source domain; each source domain will have a branch of the unique subnetwork to remember the historical information of each source domain; The source domain shared feature and each of the source domain specific features are concatenated and input into a predictor to obtain the first prediction result.

2. The method according to claim 1, characterized in that The step of inputting the target domain data into the pre-trained model for training to obtain a second prediction result includes: The target domain data is processed by the feature extraction network of the pre-trained model to generate target domain features; the target domain features are processed by the target domain-source domain memory network, the independent sub-network and the unique sub-network of the pre-trained model respectively to generate target domain-source domain shared features, target domain shared features and target domain unique features; Fusing the target domain and source domain shared features with the target domain specific features to obtain fused features; The target domain shared features and the fusion features are concatenated and input into a predictor to obtain the second prediction result.

3. The method according to claim 2, characterized in that The target domain features are processed by the target domain source domain memory network and the unique sub-network of the pre-trained model to generate target domain source domain shared features, including: The target domain source domain memory network processes the target domain features to obtain source domain target domain history related features; The memory network of the unique sub-network retrieves the target domain features to obtain multiple memory expression features; Calculation is performed based on the historically related features of the source domain and the target domain and the multiple memory expression features to obtain the shared features of the target domain and the source domain.

4. The method according to claim 3, characterized in that The memory network of the unique sub-network retrieves the target domain features to obtain multiple memory expression features, including: Calculating similarity between the target domain feature and each memory unit in the memory network to obtain multiple similarities; Calculation is performed based on each of the similarities and the stored memory feature matrix to generate the multiple memory expression features.

5. A training device for a prediction model, characterized in that: include: A first acquisition module is used to acquire a plurality of source domain data; wherein the plurality of source domain data includes a source label; A first training module, used for inputting the plurality of source domain data into a neural network for training to obtain a first prediction result; A first generating module, configured to adjust a network parameter of the neural network according to the first prediction result and the source label to generate a pre-trained model; A second acquisition module is used to acquire target domain data, wherein the target domain data includes a target tag; A second training module, used for inputting the target domain data into the pre-training model for training to obtain a second prediction result; A second generation module, configured to adjust the network parameters of the pre-trained model according to the second prediction result and the target label to generate a prediction model of the target domain; Wherein, the first training module is specifically used for: Processing the plurality of source domain data by a feature extraction network of the neural network to generate a plurality of source domain features; The multiple source domain features are processed respectively by the independent subnetwork and multiple unique subnetworks of the neural network to generate source domain shared features and multiple source domain unique features; wherein the independent subnetwork is used to extract features common to each source domain, and the unique subnetwork is used to learn information unique to each source domain; each source domain will have a branch of the unique subnetwork to remember the historical information of each source domain; The source domain shared feature and each of the source domain specific features are concatenated and input into a predictor to obtain the first prediction result.

6. The device according to claim 5, characterized in that The second training module comprises: A first processing unit, configured to process the target domain data through a feature extraction network of the pre-trained model to generate target domain features; A second processing unit is used to process the target domain features respectively through the target domain source domain memory network, the independent subnetwork and the unique subnetwork of the pre-trained model to generate target domain source domain shared features, target domain shared features and target domain unique features; An acquisition unit is used to fuse the target domain and source domain shared features with the target domain specific features to obtain fused features; the target domain shared features and the fused features are spliced ​​and input into a predictor to obtain the second prediction result.

7. The device according to claim 6, characterized in that The second processing unit comprises: A processing subunit is used for the target domain source domain memory network to process the target domain features and obtain the source domain target domain history related features; A retrieval subunit, configured to use the memory network of the unique subnetwork to retrieve the target domain features and obtain a plurality of memory expression features; The computing subunit is used to perform computing based on the historically relevant features of the source domain and the target domain and the multiple memory expression features to obtain the shared features of the target domain and the source domain.

8. The device according to claim 7, characterized in that The retrieval subunit is specifically used for: Calculating similarity between the target domain feature and each memory unit in the memory network to obtain multiple similarities; Calculation is performed based on each of the similarities and the stored memory feature matrix to generate the multiple memory expression features.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the training method of the prediction model as described in any one of claims 1 to 4 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the training method of the prediction model as described in any one of claims 1 to 4 is implemented.

11. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor, the training method of the prediction model as described in any one of claims 1 to 4 is performed.

Images