Model migration method, device and electronic equipment

Abstract

The present application relates to a model migration method in the field of artificial intelligence, including: obtaining sample data of a target task, wherein the sample data includes multiple image samples; evaluating N pre-trained models based on the sample data to obtain N evaluation values, which are used to characterize the degree of fit between the pre-trained model and the target task, where N≥2; determining K pre-trained models from the N pre-trained models based on the N evaluation values, wherein the K pre-trained models are models corresponding to the first K evaluation values after sorting the N evaluation values from large to small, where 1≤K≤N; processing the sample data based on the K pre-trained models to obtain a target model for processing the target task, wherein the target model includes K pre-trained models. In this way, a pre-trained model suitable for the current task can be quickly found from a model library composed of a large number of pre-trained models, and the capabilities of multiple optimal pre-trained models can be integrated to jointly solve the current task, effectively improving the distribution-outside generalization capability of the pre-trained model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, and electronic device for generating a pseudo-label frame. Background Art

[0002] With the rapid development of deep learning, the industry and academia have trained a large number of deep learning models on various tasks. This makes it easy for users to obtain a variety of pre-trained models (i.e., pre-trained network models), such as target detection models, animal recognition models, and so on. Among them, each pre-trained model is trained using a large amount of data and computing power, and has a variety of different functions and is powerful. However, since the target task is often unknown, no information about the target task can be obtained during the training process of the pre-trained model. Therefore, it is difficult for the pre-trained model to achieve good performance in the target task, making it difficult to migrate the pre-trained model to downstream tasks, that is, it is difficult to directly process downstream tasks through the pre-trained model.

[0003] In addition, in deep learning, it is usually assumed that the training samples and test samples come from the same probability distribution, and then the corresponding model is designed. The corresponding model is processed through the training samples to obtain the required network model. Among them, the network model is expected to have good performance on the test samples. However, in many practical problem scenarios, the probability distribution of training samples and the probability distribution of test samples are different, which makes it difficult to adapt the trained network model to the test samples. For example: in the image processing scenario, due to the difference in components, the original images collected by different models of electronic devices (such as mobile phones, etc.) are different. The network model trained based on the data of one model of electronic equipment is difficult to migrate to other electronic devices.

[0004] Therefore, how to migrate the pre-trained model to downstream tasks to improve the out-of-distribution generalization ability of the pre-trained model is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present application provides a model migration method, device, electronic device, computer storage medium and computer program product, which can screen out a model suitable for a target task from a large number of pre-trained models, and can combine the screened models into a model for processing the target task, and then the target task can be processed by the model, effectively improving the distribution out-of-generalization ability of the pre-trained model.

[0006] In the first aspect, the present application provides a model migration method, comprising: obtaining sample data of a target task, the sample data including multiple image samples; based on the sample data, evaluating N pre-trained models respectively to obtain N evaluation values, the evaluation values being used to characterize the degree of fitness between the pre-trained model and the target task, wherein one pre-trained model corresponds to one evaluation value, N≥2; based on the N evaluation values, determining K pre-trained models from the N pre-trained models, the K pre-trained models being models corresponding to the first K evaluation values after sorting the N evaluation values from large to small, 1≤K≤N; based on the K pre-trained models, processing the sample data to obtain a target model, the target model including the K pre-trained models, and the target model being used to process the target task.

[0007] In this way, after determining the target task, the pre-trained model suitable for the target task can be quickly screened out from the model library composed of a large number of pre-trained models, and based on the screened pre-trained model, a target model suitable for processing the target task and containing the screened pre-trained model can be obtained. The target task can then be processed by the target model, effectively improving the distribution out-generalization ability of the pre-trained model.

[0008] In one possible implementation, based on the sample data of the target task, N pre-trained models are evaluated respectively to obtain N evaluation values, specifically including: dividing the sample data into M parts of data, M≥2; selecting one part of the M parts of data as verification data, and using the remaining data of the M parts of data as training data; based on the training data and the verification data, determining the first validity value of each pre-trained model respectively, and determining the first stability value of each pre-trained model respectively, wherein the first validity value is used to characterize the accuracy of predicting the label of the verification data by the features extracted from the training data by the pre-trained model, and the first stability value is used to characterize the similarity between the features of the training data extracted by the pre-trained model and the features of the verification data extracted; based on the first validity value and the first stability value of each pre-trained model, N evaluation values are determined.

[0009] In one possible implementation, based on the training data and the verification data, the first validity value of each pre-trained model is determined respectively, specifically including: for any first pre-trained model among the N pre-trained models, using the first pre-trained model to extract features of the training data to obtain a first feature set, the first feature set including features corresponding to each sample in the training data; according to the first feature set, a first label set associated with the first feature set, a second feature set and a second label set associated with the second feature set, the first validity value of the first pre-trained model is determined, the first label set including labels corresponding to each sample in the training data, the second feature set including features corresponding to each sample in the verification data extracted by the first pre-trained model, and the second label set including labels corresponding to each sample in the verification data.

[0010] In one possible implementation, determining a first validity value of a first pre-trained model based on a first feature set, a first label set associated with the first feature set, a second feature set, and a second label set associated with the second feature set specifically includes: constructing a first prediction model based on the first feature set, the first label set, and a pre-defined classifier; and processing at least the second feature set and the second label set based on the first prediction model to obtain the first validity value of the first pre-trained model. Exemplarily, the pre-defined classifier may be, but is not limited to, a linear classifier.

[0011] In one possible implementation, determining a first stability value of each pre-trained model based on the training data and the validation data specifically includes: for any first pre-trained model among the N pre-trained models, using the first pre-trained model to extract features from the training data to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; determining a first stability value of the first pre-trained model based on the first feature set and a second feature set, where the second feature set includes features corresponding to each sample in the validation data extracted by the first pre-trained model;

[0012] In one possible implementation, a first stability value of the first pre-trained model is determined based on the first feature set and the second feature set, specifically including: constructing a target distribution based on the first feature set, the target distribution including: one or more of: Gaussian distribution, Bernoulli distribution, Poisson distribution, geometric distribution, or beta distribution; determining a likelihood function value corresponding to the second feature set based on the target distribution; and using the likelihood function value corresponding to the second feature set as the first stability value of the first pre-trained model.

[0013] In one possible implementation, the method also includes: using each of the M pieces of data as verification data once, and determining the first validity value of each pre-trained model once based on the verification data selected each time, and determining the first stability value of each pre-trained model once, wherein each pre-trained model corresponds to M first validity values and M first stability values.

[0014] Based on the first validity value and the first stability value of each pre-trained model, N evaluation values are determined, and the method also includes: based on the M first validity values and the M first stability values corresponding to each pre-trained model, respectively determining the second validity value of each pre-trained model, and respectively determining the second stability value of each pre-trained model; based on the second validity value and the second stability value of each pre-trained model, N evaluation values are determined.

[0015] In this way, by rotating the verification data and training data, the final required validity value can be determined from multiple validity values, and the final required stability value can be determined from multiple stability values, thereby improving the accuracy of the validity value and stability value, and further improving the accuracy of subsequent evaluation values.

[0016] In one possible implementation, the target validity value is a first validity value, and the target stability value is a first stability value; alternatively, the target validity value is a second validity value, and the target stability value is a second stability value.

[0017] Based on the target effectiveness value and target stability value of each pre-trained model, N evaluation values are determined, specifically including: based on the N target effectiveness values and N target stability values, determining the weight value of the target value, the target value is the target effectiveness value or the target stability value; based on the weight value of the target value, processing the target effectiveness value and target stability value corresponding to each pre-trained model respectively to obtain the evaluation value of each pre-trained model.

[0018] In one possible implementation, sample data is processed based on K pre-trained models to obtain a target model, specifically including: using the K pre-trained models to extract features from the sample data respectively to obtain K third feature sets, where a third feature set is a set of features extracted by one of the K pre-trained models; using the K third feature sets to train the network model to be trained to obtain a first network model; and obtaining a target model based on the K pre-trained models and the first network model.

[0019] In one possible implementation, the network model to be trained is trained using K third feature sets, specifically including: performing feature screening on each of the K third feature sets to obtain K fourth feature sets, each fourth feature set corresponding to a third feature set; and training the network model to be trained using the K fourth feature sets. In this way, by screening the features in each third feature set, necessary features are retained and unnecessary features are discarded, thereby improving the accuracy of subsequent model training and, consequently, the prediction accuracy of the resulting model.

[0020] In one possible implementation, feature screening is performed on each of the K third feature sets, specifically including: for any one of the K third feature sets, constructing a probability model of any one of the K third feature sets based on the spike-and-slab prior; determining the marginal probability value of the target label set corresponding to any one of the feature sets based on the probability model and the hierarchical Bayesian modeling method; determining the second marginal likelihood function value of the target label set based on the marginal probability value of the target label set; determining the probability that the indicator variable corresponding to each feature in any one of the feature sets is equal to 1 based on the maximum expectation algorithm of variational inference with the goal of maximizing the second marginal likelihood function value; when the probability that the indicator variable corresponding to the target feature in any one of the feature sets is equal to 1 is greater than or equal to a preset probability value, retaining the target feature; when the probability that the indicator variable corresponding to the target feature is equal to 1 is less than the preset probability value, eliminating the target feature.

[0021] In the second aspect, the present application provides a model migration device, comprising: a communication module and a processing module. The communication module is used to obtain sample data of a target task, and the sample data includes multiple image samples. The processing module is used to evaluate N pre-trained models based on the sample data to obtain N evaluation values, and the evaluation values are used to characterize the degree of fit between the pre-trained model and the target task, wherein one pre-trained model corresponds to one evaluation value, and N≥2. The processing module is also used to determine K pre-trained models from the N pre-trained models based on the N evaluation values, and the K pre-trained models are models corresponding to the first K evaluation values after sorting the N evaluation values from large to small, and 1≤K≤N. The processing module is also used to process the sample data based on the K pre-trained models to obtain a target model, and the target model includes K pre-trained models, and the target model is used to process the target task.

[0022] In one possible implementation, when N pre-trained models are evaluated based on sample data of a target task to obtain N evaluation values, the processing module is specifically used to: divide the sample data into M parts of data, M≥2; select one part of the data from the M parts of data as verification data, and use the remaining data in the M parts of data as training data; determine the first validity value of each pre-trained model based on the training data and the verification data, and determine the first stability value of each pre-trained model, wherein the first validity value is used to characterize the accuracy of predicting the label of the verification data by the features extracted from the training data by the pre-trained model, and the first stability value is used to characterize the similarity between the features of the training data extracted by the pre-trained model and the features of the verification data extracted; based on the first validity value and the first stability value of each pre-trained model, N evaluation values are determined.

[0023] In one possible implementation, when determining the first validity value of each pre-trained model based on the training data and the verification data, the processing module is specifically used to: for any first pre-trained model among the N pre-trained models, use the first pre-trained model to extract features from the training data to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; determine the first validity value of the first pre-trained model based on the first feature set, a first label set associated with the first feature set, a second feature set, and a second label set associated with the second feature set, where the first label set includes labels corresponding to each sample in the training data, the second feature set includes features corresponding to each sample in the verification data extracted by the first pre-trained model, and the second label set includes labels corresponding to each sample in the verification data.

[0024] In one possible implementation, when determining the first validity value of the first pre-trained model based on the first feature set, the first label set associated with the first feature set, the second feature set, and the second label set associated with the second feature set, the processing module is specifically used to: construct a first prediction model based on the first feature set, the first label set, and a pre-set classifier; and process at least the second feature set and the second label set based on the first prediction model to obtain the first validity value of the first pre-trained model.

[0025] In one possible implementation, when determining the first stability value of each pre-trained model based on the training data and the verification data, the processing module is specifically configured to: for any one of the N pre-trained models, use the first pre-trained model to extract features from the training data to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; determine the first stability value of the first pre-trained model based on the first feature set and a second feature set, where the second feature set includes features corresponding to each sample in the verification data extracted by the first pre-trained model;

[0026] In one possible implementation, when determining the first stability value of the first pre-trained model based on the first feature set and the second feature set, the processing module is specifically used to: construct a target distribution based on the first feature set, the target distribution including: one or more of Gaussian distribution, Bernoulli distribution, Poisson distribution, geometric distribution, or beta distribution; determine the likelihood function value corresponding to the second feature set based on the target distribution; and use the likelihood function value corresponding to the second feature set as the first stability value of the first pre-trained model.

[0027] In one possible implementation, the processing module is also used to: use each of the M pieces of data as verification data once, and, based on the verification data selected each time, determine the first validity value of each pre-trained model once, and determine the first stability value of each pre-trained model once, wherein each pre-trained model corresponds to M first validity values and M first stability values.

[0028] In which, when N evaluation values are determined based on the first validity value and the first stability value of each pre-trained model, the processing module is also used to: determine the second validity value of each pre-trained model based on the M first validity values and the M first stability values corresponding to each pre-trained model, and determine the second stability value of each pre-trained model; determine N evaluation values based on the second validity value and the second stability value of each pre-trained model.

[0029] In one possible implementation, the target validity value is a first validity value, and the target stability value is a first stability value; alternatively, the target validity value is a second validity value, and the target stability value is a second stability value.

[0030] When N evaluation values are determined based on the target effectiveness value and target stability value of each pre-trained model, the processing module is specifically used to: determine the weight value of the target value based on the N target effectiveness values and N target stability values, where the target value is the target effectiveness value or the target stability value; based on the weight value of the target value, process the target effectiveness value and target stability value corresponding to each pre-trained model respectively to obtain the evaluation value of each pre-trained model.

[0031] In one possible implementation, when sample data is processed based on K pre-trained models to obtain a target model, the processing module is specifically used to: use the K pre-trained models to extract features from the sample data respectively to obtain K third feature sets, wherein a third feature set is a set of features extracted by one of the K pre-trained models; use the K third feature sets to train the network model to be trained to obtain a first network model; and obtain the target model based on the K pre-trained models and the first network model.

[0032] In one possible implementation, when using K third feature sets to train the network model to be trained, the processing module is specifically used to: perform feature screening on each feature set in the K third feature sets to obtain K fourth feature sets, each fourth feature set corresponding to a third feature set; and use the K fourth feature sets to train the network model to be trained.

[0033] In one possible implementation, when feature screening is performed on each of the K third feature sets, the processing module is specifically used to: for any one of the K third feature sets, construct a probability model of any one of the K third feature sets based on the spike-and-slab prior; determine the marginal probability value of the target label set corresponding to any one of the feature sets based on the probability model and the hierarchical Bayesian modeling method; determine the second marginal likelihood function value of the target label set based on the marginal probability value of the target label set; determine the probability that the indicator variable corresponding to each feature in any one of the feature sets is equal to 1 based on the maximum expectation algorithm of variational inference, with the goal of maximizing the second marginal likelihood function value; when the probability that the indicator variable corresponding to the target feature in any one of the feature sets is equal to 1 is greater than or equal to a preset probability value, retain the target feature; when the probability that the indicator variable corresponding to the target feature is equal to 1 is less than the preset probability value, eliminate the target feature.

[0034] In the third aspect, the present application provides a model migration device, comprising at least one processor and an interface; at least one processor obtains program instructions or data through the interface; and at least one processor is used to execute program line instructions to implement the method described in the first aspect or any possible implementation of the first aspect.

[0035] In a fourth aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory; wherein, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0036] In a fifth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0037] In a sixth aspect, the present application provides a computer program product, characterized in that when the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0038] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0041] Figure 3 This is a schematic diagram of the hardware structure of a server provided in an embodiment of the present application;

[0042] Figure 4 This is a flow chart of a model migration method provided in an embodiment of the present application;

[0043] Figure 5 This is a schematic diagram of the steps for determining the validity value of a pre-trained model provided in an embodiment of the present application;

[0044] Figure 6 This is a process diagram of a model migration method provided in an embodiment of the present application;

[0045] Figure 7 1 is a schematic diagram comparing the results obtained by processing data using ZooD and other evaluation methods provided in an embodiment of the present application;

[0046] Figure 8 1 is a schematic diagram comparing the time required to evaluate 35 models using ZooD and the time required to evaluate 35 models using fine-tuning, provided in an embodiment of the present application;

[0047] Figure 9 This is a schematic diagram comparing the classification accuracy of models obtained by using different numbers of pre-trained models for feature screening and without feature screening, provided in an embodiment of the present application;

[0048] Figure 10 This is a schematic diagram comparing the classification effect after model selection, feature screening and fusion provided by an embodiment of the present application with other methods for improving the generalization effect of the model;

[0049] Figure 11 This is a schematic diagram of the structure of a model migration device provided in an embodiment of the present application;

[0050] Figure 12 It is a structural diagram of another model migration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The term "and / or" as used herein describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The symbol " / " as used herein indicates that the related objects are in an "or" relationship, for example, A / B means either A or B.

[0052] The terms "first" and "second" in this specification and claims are used to distinguish different objects rather than to describe a specific order of objects. For example, "first response message" and "second response message" are used to distinguish different response messages rather than to describe a specific order of response messages.

[0053] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0054] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc.

[0055] Generally speaking, in many practical problem scenarios, the probability distribution of training samples is different from the probability distribution of test samples. To improve the out-of-distribution generalization ability of deep learning models (that is, the ability of the model to generalize tasks in scenarios with distribution changes), one way is to collect as much labeled data as possible. However, collecting a large amount of labeled data is time-consuming and labor-intensive, and in many cases, it is not possible to collect a large amount of training data. On the other hand, it is now easy to obtain a variety of pre-trained models, which have strong out-of-distribution generalization capabilities when migrated to downstream tasks. If the most suitable model for the current task can be quickly selected from a large number of pre-trained models for the downstream target task, and the capabilities of multiple models can be integrated, then the generalization ability on the target task can be further improved.

[0056] In order to solve these problems, this application proposes a model migration method, which can quickly find a pre-trained model suitable for the current task in a model library composed of a large number of pre-trained models, and can integrate the capabilities of multiple optimal pre-trained models to jointly solve the current task, effectively improving the distribution out-of-generalization ability of the pre-trained model.

[0057] For example, Figure 1 An application scenario is shown. Figure 1 As shown, this scenario may include an electronic device 100 and a server 200. The user can access the data provided by the server 200 through the electronic device 100. For example, the user can browse various pre-trained models provided by the server 200 on the electronic device 100. In addition, the user can also upload sample data of the target task to the server 200 through the electronic device 100, so that the server 200 can determine a network model suitable for the target task.

[0058] In some embodiments, the electronic device 100 and the server 200 can be connected via a network such as a wired network or a wireless network. For example, the network can be a local area network (LAN) or a wide area network (WAN) (such as the Internet). The network between the electronic device 100 and the server 200 can be implemented using any known network communication protocol, and the above-mentioned network communication protocol can be various wired or wireless communication protocols, such as Ethernet, universal serial bus (USB), FireWire, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), New Radio (NR), Bluetooth, Wireless Fidelity (Wi-Fi) and other communication protocols.

[0059] For example, Figure 2 The hardware structure of an electronic device 100 is shown. The electronic device 100 may be, but is not limited to, a mobile phone, a tablet computer, a laptop computer, a wearable device, a smart TV, or other electronic device. Exemplary embodiments of electronic devices include, but are not limited to, electronic devices running iOS, Android, Windows, Harmony OS, or other operating systems. The embodiments of this application do not specifically limit the type of electronic device.

[0060] like Figure 2 As shown, the electronic device 100 may include: a processor 110, a memory 120, a display screen 130, a communication module 140, and an input device 150. The processor 110, the memory 120, the display screen 130, the communication module 140, and the input device 150 may be connected via a bus or other means.

[0061] The processor 110 is the computing core and control core of the electronic device 100. The processor 110 may include one or more processing units. For example, the processor 110 may include one or more of an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0062] The memory 120 may store a program that can be executed by the processor 110, so that the processor 110 executes part or all of the methods required to be executed by the electronic device 100 provided in the embodiments of the present application. The memory 120 may also store data. The processor 110 may read the data stored in the memory 120. The memory 120 and the processor 110 may be provided separately. Alternatively, the memory 120 may also be integrated into the processor 110.

[0063] Display screen 130 is used to display images, videos, etc. Display screen 130 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLED, or a quantum dot light-emitting diode (QLED).

[0064] The communication module 140 may include at least one of a mobile communication module and a wireless communication module. When the communication module 140 includes a mobile communication module, the communication module 140 may provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. For example, global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio (NR), etc. When the communication module 140 includes a wireless communication module, the communication module 140 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), nearfield communication technology (NFC), infrared technology (IR), etc., applied to the electronic device 100. Exemplarily, the communication module 140 can be used for the electronic device 100 to communicate with the server 200 to complete data exchange.

[0065] In some embodiments, the electronic device 100 may further include an input device 150. Information may be input and / or control instructions may be issued to the electronic device 100 via the input device 150. For example, the input device 150 may be, but is not limited to, a mouse, a keyboard, or the like.

[0066] It is understandable that the embodiments of this application Figure 2The illustrated structure does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0067] For example, Figure 3 The figure shows a hardware structure of a server 200. The server 200 can be used, but is not limited to, to provide cloud services. It can be a server or a super electronic device that can establish a communication connection with the electronic device 100 and provide data processing, computing, and / or storage functions for the electronic device 100. The server 200 can be a hardware server or embedded in a virtualized environment. For example, the server 200 can be a virtual machine executed on a hardware server that includes one or more other virtual machines.

[0068] like Figure 3 As shown, the server 200 may include: a processor 210, a network interface 220, and a memory 230. The processor 210, the network interface 220, and the memory 230 may be connected via a bus or other means.

[0069] In the embodiment of the present application, the processor 210 (also called a central processing unit (CPU)) is the computing core and control core of the server 200. In some embodiments, the processor 210 can determine a network model suitable for the target task based on sample data of the target task provided by the electronic device 100.

[0070] The network interface 220 may include a standard wired interface, a wireless interface (such as WI-FI, a mobile communication interface, etc.), and is controlled by the processor 210 to send and receive data, for example, receiving sample data of a target task sent by the electronic device 100 from the network.

[0071] The memory 230 (memory) is a memory device of the server 200, which is used to store programs and data, such as pre-trained models. It is understandable that the memory 230 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one storage device located away from the aforementioned processor 210. The memory 230 provides a storage space that stores the server's operating system and executable program code, which may include but is not limited to: Windows system (an operating system), Linux system (an operating system), Hongmeng system (an operating system), etc., which are not limited here.

[0072] It is understandable that the embodiments of this application Figure 3 The illustrated structure does not constitute a specific limitation on the server 200. In other embodiments of the present application, the server 200 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0073] The above is an introduction to the application scenarios involved in the embodiment of the application, the hardware structure of the electronic device 100, and the hardware structure of the server 200. Next, based on the above description, the model migration method provided in the embodiment of the present application is introduced.

[0074] For example, Figure 4 A model migration method is shown. It is understood that the method can be executed by any device, equipment, platform, or equipment cluster with computing and processing capabilities, such as the aforementioned server 200. For ease of description, the following description is based on the server 200 as an example. Figure 4 As shown, the model migration method may include the following steps:

[0075] S401: Obtain sample data of the target task.

[0076] In this embodiment, a user can upload sample data for a target task to server 200 via electronic device 100. In this way, server 200 obtains the sample data for the target task. The target task is a downstream task specified by the user, such as an image recognition task. In some embodiments, the sample data may include multiple samples, each of which may correspond to a label. For example, the samples in the sample data may be, but are not limited to, images.

[0077] S402. Based on the sample data of the target task, N pre-trained models are evaluated respectively to obtain N evaluation values, where one pre-trained model corresponds to one evaluation value, and N ≥ 2.

[0078] In this embodiment, N pre-trained models can be evaluated based on the sample data of the target task to obtain N evaluation values. Each pre-trained model can correspond to an evaluation value. The evaluation value can be used to represent the degree of fit between the pre-trained model and the target task.

[0079] In some embodiments, the validity value and stability value of each pre-trained model can be evaluated separately, and the evaluation value corresponding to each pre-trained model can be determined based on the validity value and stability value corresponding to each pre-trained model. Among them, the validity value can be used to characterize the accuracy of predicting the label of the verification data by the features extracted from the training data by the pre-trained model. The stability value can be used to characterize the similarity between the features corresponding to the training data and the features corresponding to the verification data in the sample data. In other embodiments, the validity value can also be used as the evaluation value, or the stability value can be used as the evaluation value, which is not limited here.

[0080] The following describes the process of obtaining the validity value, stability value, and evaluation value of each pre-training model.

[0081] (a) For determining the effectiveness of the pre-trained model

[0082] In this embodiment, for any pre-training model, the validity value of the pre-training model is determined, such as Figure 5 As shown, the following steps may be included:

[0083] S501. Determine training data and verification data based on sample data.

[0084] In this embodiment, the samples in the sample data can be divided into multiple parts, and one part is selected as the verification data, and the rest are used as the training data.

[0085] As a possible implementation method for segmenting sample data, when the sample data has a distribution label for identifying the distribution of the data, the sample data can be segmented by the distribution label. For example, when the sample data is image data, if the distribution labels include: daytime and sunny, daytime and rainy, nighttime and sunny, and nighttime and rainy, then the image data of daytime and sunny can be divided into one group, the image data of daytime and rainy can be divided into one group, the image data of nighttime and sunny can be divided into one group, and the image data of nighttime and rainy can be divided into one group. Exemplarily, the distribution label can be used to characterize the probability distribution of the sample.

[0086] As another possible implementation, when the sample data does not have a distribution label, the sample data can be segmented based on a pre-set segmentation rule. In some embodiments, the segmentation rule can be to segment the sample data based on a pre-set number of segments. For example, if the pre-set number of segments is 5, the sample data can be segmented into 5. In other embodiments, the segmentation rule can also be to determine the number of segments based on the type of the target task; and then, based on the number of segments, the sample data can be segmented. Exemplarily, based on the type of the target task, a mapping relationship table between a pre-set type and the number of segments can be queried to determine the required number of segments.

[0087] S502: Use the pre-training model to extract features from samples in the training data to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data.

[0088] In this embodiment, each sample in the training data can be input into a pre-training model to extract features from each sample in the training data through the pre-training model, thereby obtaining a first feature set. Exemplarily, the first feature set includes features corresponding to each sample in the training data.

[0089] S503. Determine the validity value of the pre-trained model based on the first feature set, the first label set associated with the first feature set, the second feature set, and the second label set associated with the second feature set, wherein the first label set includes labels corresponding to each sample in the training data, the second feature set includes features corresponding to each sample in the verification data, and the second label set includes labels corresponding to each sample in the verification data.

[0090] In this embodiment, a prediction model can be first obtained by maximizing the Bayesian model evidence based on the first feature set and the first label set. Then, the second feature set and the second label set are processed by the prediction model to obtain the validity value of the pre-trained model. The second feature set can be obtained by extracting features from each sample in the verification data using the pre-trained module to be evaluated (i.e., the model in S502 above).

[0091] As a possible implementation, the first feature set can be used Indicates that the first tag set can be used Indicates that the second feature set can be used Indicates that the second tag set can be used express.

[0092] On the first feature set, a prediction model can be obtained by maximizing the Bayesian model evidence. Specifically, consider the linear classifier Among them, w satisfies the Gaussian distribution N(0,α -1 ), given w, In the case of y i Satisfies Gaussian distribution When w is fixed, the conditional probability value of each label in the first label set can be calculated And the conditional probability values of all labels in the first label set are

[0093] Through each possible w, a marginal probability value corresponding to the first label set can be calculated. From the marginal probability value obtained through each possible w, the marginal probability value of the first label set p(Y|Φ; α, β) = ∫p(w|α)p(Y|w,Φ)dw can be finally obtained. By inputting the features contained in the first feature set and the labels contained in the first label set into the marginal probability value p(Y|Φ; α, β) of the first label set, the marginal likelihood function value log p(Y|Φ; α, β) corresponding to the first label set can be obtained. By optimizing α and β in the marginal likelihood function value log p(Y|Φ; α, β), the marginal likelihood function value can be maximized and the corresponding Then we can get the required prediction model

[0094] Then, the prediction model can be used Process the first feature set, the first label set, the second feature set, and the second label set to obtain the marginal likelihood function values corresponding to all labels in the sample data.

[0095] Finally, the marginal likelihood function values corresponding to all labels in the sample data are And, the marginal likelihood function value corresponding to the first label set The marginal likelihood function value corresponding to the second label set (i.e., the label in the verification data) can be obtained. Among them, the marginal likelihood function value of the second label set (i.e., the label in the verification data) is The marginal likelihood function value corresponding to the second label set is the validity value of the pre-training module to be evaluated (ie, the model in the aforementioned S502).

[0096] In addition, in order to improve the accuracy of the validity value, you can also Figure 5 The validation data and the training data are rotated to determine multiple validity values, and the final validity value is obtained from the determined validity values. For example, the average value or variance value of the multiple validity values can be used as the final validity value.

[0097] For example, suppose the training data is split into two parts. For the convenience of description, one of the two parts is called the first data and the other is called the second data. In this case, the first part of the data can be used as training data and the second part of the data can be used as verification data. Figure 5 Then, the second data is used as training data, the first data is used as verification data, and the above Figure 5 The method described in can obtain a validity value b. Finally, the final validity value c = (a + b) / 2 can be obtained from the validity values a and b.

[0098] For methods of determining the validity values of other pre-trained models, please refer to the above description and will not be repeated here.

[0099] (b) For determining the stability value of the pre-trained model

[0100] In this embodiment, for any pre-trained model, to determine the stability value of the pre-trained model, the distribution of the features of the training data extracted by the pre-trained model can be estimated on the training data, and then the stability value of the pre-trained model can be determined based on the distribution of the features corresponding to the training data.

[0101] As a possible implementation method, Gaussian distribution, Bernoulli distribution, Poisson distribution, geometric distribution, or beta distribution can be used to determine the stability value of the pre-trained model, which is not limited here.

[0102] Taking the Gaussian distribution as an example, the first feature set can be expressed as Indicates that the first tag set can be used Indicates that the second feature set can be used Indicates that the second tag set can be used In this case, the first feature set can be The mean μ of all features in is used as the mean of the Gaussian distribution, and the first feature set The variance ∑ of all features in is used as the variance of the Gaussian distribution, thus obtaining a Gaussian distribution N(μ,∑). Then, the aforementioned second feature set can be determined based on the Gaussian distribution. (i.e., feature extraction of samples in the validation data by the pre-trained model to obtain the feature set) the corresponding likelihood function value Among them, the second feature set The corresponding likelihood function value is the stability value of the pre-training module to be evaluated (ie, the model in the aforementioned S502 ).

[0103] In addition, to improve the accuracy of the stability value, the validation data and training data can be rotated to determine multiple stability values, and the final stability value can be obtained from the determined stability values. For example, the average value or variance value of the multiple stability values can be used as the final stability value.

[0104] For example, suppose the training data is split into two parts. For ease of description, one part is called the first data, and the other is called the second data. In this case, the first part can be used as training data and the second part as validation data to obtain a stability value d. Then, the second part is used as training data and the first part as validation data to obtain a stability value e. Finally, the stability values d and e can be used to obtain the final stability value f = (d + e) / 2.

[0105] For methods of determining the stability values of other pre-trained models, please refer to the above description and will not be repeated here.

[0106] (c) Determining the evaluation value of the pre-trained model

[0107] In this embodiment, after the validity value and the evaluation value of each pre-training model are determined, the evaluation value of each pre-training model can be determined based on the determined validity value and evaluation value.

[0108] As a possible approach, for any pre-trained model, its corresponding validity value and evaluation value can be calculated (such as summed up) according to a pre-set weight value, and the result obtained can be used as the evaluation value of the pre-trained model.

[0109] As another possible implementation method, the weights of the validity value and the stability value can be determined based on the validity value and the evaluation value of each pre-trained model. Then, based on the determined weights, the validity value and the stability value corresponding to each pre-trained model are calculated respectively, and the obtained results are used as the evaluation value of the corresponding pre-trained model. In some embodiments, the standard deviation of all validity values and the standard deviation of all stability values can be calculated, and then the ratio between the standard deviation of the validity value and the standard deviation of the stability value is used as the weight value of the stability value. The weight value for the validity value can be (1-the weight value of the stability value) or other fixed values, which are not limited here. After determining the weight of the validity value and the weight of the stability value, when determining the evaluation value of any pre-trained model, the validity value and the stability value corresponding to the pre-trained model can be weighted and summed, and the obtained result can be used as the evaluation value of the pre-trained model. Exemplarily, if the weight of the stability value determined by summing the validity values and the stability values of all pre-trained models is γ, then for any pre-trained model, if its validity value is The stability value is Its evaluation value can be:

[0110] After obtaining the evaluation values of each pre-trained model, S403 may be executed.

[0111] S403. Based on the N evaluation values, determine K pre-training models from the N pre-training models, where the K pre-training models are models corresponding to the first K evaluation values after sorting the N evaluation values from large to small, 1≤K≤N.

[0112] In this embodiment, K pre-trained models can be determined from N pre-trained models based on the size of the N evaluation values. In some embodiments, the N evaluation values can be sorted from large to small, and the first K evaluation values can be selected according to the sorting results, and the pre-trained models corresponding to the K evaluation values can be used as the K pre-trained models determined from the N pre-trained models. Of course, when the N evaluation values are sorted from small to large, the last K evaluation values can be selected according to the sorting results, and the pre-trained models corresponding to the K evaluation values can be used as the K pre-trained models determined from the N pre-trained models. Among them, the evaluation value corresponding to each model in the K pre-trained models is greater than the evaluation value corresponding to the remaining models in the N pre-trained models.

[0113] S404: Use K pre-trained models to perform feature extraction on sample data of the target task respectively to obtain K third feature sets, where a third feature set is a set of features extracted by one of the K pre-trained models.

[0114] In this embodiment, after determining K pre-training models, the K pre-training models can be used to perform feature extraction on the sample data of the target task respectively to obtain K third feature sets. Among them, one third feature set is a set of features obtained by extracting features from the sample data using one of the K pre-training models. For example, when K=2, two third feature sets can be obtained, one of which is a set of features obtained by extracting features from the sample data set of the target task using one of the pre-training models, and the other third feature set is a set of features obtained by extracting features from the sample data set of the target task using another pre-training model.

[0115] S405 : Using the K third feature sets, train the network model to be trained to obtain a first network model.

[0116] In this embodiment, after obtaining K third feature sets, the K third feature sets can be used to train the network model to be trained to obtain the first network model. For example, the network model to be trained can be, but is not limited to, a classifier. In some embodiments, the features in the K feature sets can be first concatenated into a single feature, and then the concatenated feature can be used to train the network model to be trained.

[0117] In some embodiments, after a third feature set is extracted using a pre-trained model, the features in the third feature set can be screened to retain necessary features and discard some unnecessary features, thereby improving the accuracy of subsequent model training and further improving the prediction accuracy of the final model.

[0118] As a possible implementation method, take the screening of the third feature set extracted by a pre-trained model as an example. The third feature set can be used Indicates that the third tag set can be used The third label set includes labels corresponding to each sample in the sample data of the target task.

[0119] Among them, before performing feature screening, you can first set the third feature Build a probability model. In building a probability model, you can set the third feature Each feature in introduces a "0-1" indicator variable Among them, d is the feature dimension, z i The probability of being 1 is denoted as π i , π i The initial value can be preset, such as 0.5.

[0120] When the indicator variable is 0, the prior distribution of the weight of the corresponding feature is a Spike prior distribution with a density concentrated at 0. When the indicator variable is 1, the prior distribution of the weight of the corresponding feature is a Slab prior distribution with a dispersed density. Specifically, consider the linear classifier Among them, the weight w of the linear classifier satisfies the following distribution:

[0121]

[0122] Perform hierarchical Bayesian processing on the prior distribution of weight w, assuming α i，1 Satisfies the gamma distribution Г(v i,1 ,v i,2 ), α i,2 Satisfies the gamma distribution Г(v i,3 ,v i,4 ). Given w, In the case of i Satisfies Gaussian distribution β satisfies the gamma distribution Г(v 0,1 ,v 0,2 For ease of description, we will use v={v i,j} represents the parameters in the above gamma distribution. Among them, p(w i |z i ) is the constructed probability model.

[0123] Then, based on the hierarchical Bayesian modeling method, the third feature set can be calculated The third tag set for the condition Specifically, when w is fixed, the third label set can be calculated The conditional probability value of each label in And the third tag set The conditional probability values of all labels in

[0124] Through each possible w, the third label set can be calculated A corresponding marginal probability value. By obtaining the marginal probability value through each possible w, we can finally get the third label set The marginal probability value of the marginal probability value is p(Y|Φ;π,v)=∫p(w|π,v)p(Y|w,Φ)dw. By inputting the features included in the third feature set and the labels included in the third label set into the marginal probability value of the third label set, the marginal likelihood function value logp(Y|Φ;π,v) corresponding to the third label set can be obtained.

[0125] After obtaining the marginal likelihood function value logp(Y|Φ;π,v) corresponding to the third label set, the Expectation-Maximization algorithm (EM) based on variational inference can be used to optimize the marginal likelihood function value logp(Y|Φ;π,v), thereby maximizing the marginal likelihood function value and obtaining the corresponding Among them, each feature in the third feature set corresponds to That is the probability that it is equal to 1.

[0126] When the indicator variable z i The probability π is equal to 1 i If the value is greater than a certain value, the feature corresponding to the indicator variable is retained in the third feature set. Otherwise, the feature corresponding to the indicator variable is discarded. This completes the feature screening in the third feature set.

[0127] After feature screening is performed on the third feature set corresponding to each pre-trained model, the features obtained after screening can be spliced, and the network model to be trained can be trained using the spliced features to obtain the first network model.

[0128] S406: Obtain a target model based on the K pre-trained models and the first network model, and the target model is used to process the target task.

[0129] In this embodiment, after obtaining the first network model, a target task can be processed based on the K pre-trained models and the first network model. For example, the K pre-trained models and the first network model can be concatenated to obtain the target model. The outputs of the K pre-trained models can serve as inputs to the first network model.

[0130] In addition, when performing feature screening on each third feature set in S404, a feature screening module can be set between the K pre-trained models and the first network model, thereby screening out the required features through the feature screening module. In this case, the target model mainly consists of the K pre-trained models, the feature screening module and the first network model. Among them, the output of the K pre-trained models can be used as the input of the feature screening module, and the output of the feature screening module can be used as the input of the first network model.

[0131] Therefore, after determining the target task, the pre-trained model suitable for the target task can be quickly screened out from the model library composed of a large number of pre-trained models, and based on the screened pre-trained model, a target model suitable for processing the target task and containing the screened pre-trained model can be obtained. The target task can then be processed by the target model, effectively improving the distribution out-generalization ability of the pre-trained model.

[0132] For ease of understanding, the following Figure 6 The above scheme is described.

[0133] For example, Figure 6 As shown, in Figure 6 It mainly includes: pre-training model evaluation stage and pre-training model screening and fusion stage.

[0134] In the pre-training model evaluation phase, we can first obtain the sample data of the target task (i.e. Figure 6 Then, the sample data is divided into training data and validation data. Figure 6 Each pre-trained model in the ModelZoo in the

[15] extracts features from the training data and the validation data. Finally, the stability value (i.e. Figure 6S in) and validity value (i.e. Figure 6 Finally, the evaluation value of each pre-training model is determined by the stability value and effectiveness value of each pre-training model, so that the ranking of each pre-training model can be obtained (i.e. Figure 6 After obtaining the evaluation values of each pre-trained model, we can enter the pre-trained model screening and fusion stage.

[0135] In the pre-training model screening and fusion stage, K pre-training models (i.e. Figure 6 Then, the sample data of the target task can be extracted by the K pre-trained models to obtain K feature sets. Then, the features in the K feature sets can be filtered (i.e. Figure 6 Finally, the spliced features can be used to treat the trained network model (i.e. Figure 6 Classifier) for training.

[0136] After the pre-training model screening and fusion stage, the K pre-training models can be fused with the model trained in the pre-training model screening and fusion stage to obtain a model suitable for the target task, which can then be used to process the target task.

[0137] Furthermore, to demonstrate the effectiveness of the model transfer method provided in the examples of this application, we used a model library consisting of 35 pre-trained models and an image classification task trained on seven datasets as an example to compare this method with other solutions. For ease of description, the model transfer method in the examples of this application will be referred to as ZooD.

[0138] The experimental data includes: PACS, VLCS, OfficeHome, TerraIncognita, NICO-Animals, NICO-Vehicles, and DomainNet. The PACS dataset contains natural images from 7 categories and 4 distributions, with nearly 10,000 training images. The VLCS dataset contains natural images from 5 categories and 4 distributions, with over 10,000 training images. The OfficeHome dataset contains office scene images from 65 categories and 4 distributions, with 15,000 training images. The TerraIncognita dataset contains wild animal images from 10 categories and 4 distributions, with nearly 25,000 training images. The NICO-Animals dataset contains animal images from 10 categories and 4 distributions, with over 10,000 training images. The NICO-Vehicles dataset contains vehicle images from 9 categories and 4 distributions, with over 10,000 training images. The DomainNet dataset contains natural images from 345 categories and 6 distributions, with nearly 600,000 training images.

[0139] The evaluation criteria are: Pre-trained models are evaluated using Kendall's coefficient, comparing the correlation between the ZooD evaluation results and the results of fine-tuning each pre-trained model on the target task, with higher values being preferred. Additionally, the effectiveness of feature selection can be assessed by using the predicted classification accuracy obtained by retraining the classifier on the target task after selection, with higher values being preferred.

[0140] like Figure 7 As shown in (a) of Figure 1, this figure compares ZooD and feature-based model evaluation methods on seven datasets, and the results of evaluating the models in a fine-tuning manner. Figure 7 (b) shows the results of comparing ZooD and classifier-based model evaluation methods on seven datasets, and evaluating the models in a fine-tuning manner. Figure 7 It can be seen that ZooD is more accurate and stable than other methods. Among them, the results on these data sets show that the method of the embodiment of the present application has achieved good results for different types of training data. By evaluating the effectiveness and stability of feature extraction of the pre-training model on training data with multiple distributions, the method of the embodiment of the present application can more accurately evaluate the out-of-distribution generalization ability of the pre-training model on the target downstream task, and perform more stably on data sets of different types and with different degrees of distribution differences.

[0141] like Figure 8 As shown in the figure, the time required to evaluate 35 models using ZooD is compared with the time required to evaluate 35 models using fine-tuning. Figure 8 It can be seen from the results shown that compared with the previous method of fine-tuning each pre-trained model on the target task, the time required to evaluate the pre-trained model using the method provided in the embodiment of the present application is greatly shortened, which makes it possible to further expand the scale of the pre-trained model library and give full play to the advantages of the model library.

[0142] like Figure 9 As shown in the figure, the classification accuracy of the model obtained by using different numbers of pre-trained models for feature screening and without feature screening is compared. Figure 9 The results show that when feature screening is performed on the models selected by the pre-trained model evaluation module, the models trained after feature screening consistently outperform the models trained without feature screening in terms of classification accuracy, regardless of the number of pre-trained models. The scheme in the embodiment of the present application can effectively select the most informative features for the classification task and filter out those features that have a negative impact on the classification task, thereby improving the classification accuracy.

[0143] like Figure 10 As shown in FIG, the classification effect after model selection, feature screening and fusion by the embodiment of the present application is compared with other methods to improve the generalization effect of the model. Figure 10 The results shown show that after model selection, feature screening and fusion, the model trained based on the target downstream task training data has a significant improvement in classification accuracy compared to the single model trained using a special method, demonstrating the effectiveness of the solution in the embodiment of this application under the model library setting. Figure 10 In the above example, Single refers to a single model, Ensemble refers to multiple models, F.Selection refers to feature screening, and F.Ratio is the number of features obtained after screening.

[0144] Based on the method in the above embodiment, the present application embodiment provides a model migration device. Figure 11 , Figure 11 This is a schematic diagram of the structure of a model migration device provided in an embodiment of the present application. Figure 11As shown, the model migration device 1100 includes: a communication module 1101 and a processing module 1102. The communication module 1101 is used to obtain sample data of the target task, and the sample data includes multiple image samples. The processing module 1102 is used to evaluate N pre-trained models based on the sample data to obtain N evaluation values. The evaluation values are used to characterize the degree of fit between the pre-trained model and the target task, wherein one pre-trained model corresponds to one evaluation value, and N≥2. The processing module 1102 is also used to determine K pre-trained models from the N pre-trained models based on the N evaluation values. The K pre-trained models are models corresponding to the first K evaluation values after sorting the N evaluation values from large to small, and 1≤K≤N. The processing module 1102 is also used to process the sample data based on the K pre-trained models to obtain a target model. The target model includes K pre-trained models, and the target model is used to process the target task. Exemplarily, the communication module 1101 can be, but is not limited to, Figure 3 The network interface 220 shown in FIG. 1 , the processing module 1102 may be, but is not limited to, Figure 3 The processor 210 shown in .

[0145] In some embodiments, when N pre-trained models are evaluated based on sample data of a target task to obtain N evaluation values, the processing module 1102 is specifically used to: divide the sample data into M portions of data, M≥2; select one portion of data from the M portions of data as verification data, and use the remaining data in the M portions of data as training data; determine the first validity value of each pre-trained model based on the training data and the verification data, and determine the first stability value of each pre-trained model, wherein the first validity value is used to characterize the accuracy of predicting the label of the verification data by extracting features from the training data by the pre-trained model, and the first stability value is used to characterize the similarity between the features of the training data extracted by the pre-trained model and the features of the verification data extracted; determine N evaluation values based on the first validity value and the first stability value of each pre-trained model.

[0146] In some embodiments, when determining the first validity value of each pre-trained model based on the training data and the verification data, the processing module 1102 is specifically used to: for any first pre-trained model among the N pre-trained models, use the first pre-trained model to extract features from the training data to obtain a first feature set, the first feature set including features corresponding to each sample in the training data; determine the first validity value of the first pre-trained model based on the first feature set, the first label set associated with the first feature set, the second feature set and the second label set associated with the second feature set, the first label set including labels corresponding to each sample in the training data, the second feature set including features corresponding to each sample in the verification data extracted by the first pre-trained model, and the second label set including labels corresponding to each sample in the verification data.

[0147] In some embodiments, when determining the first validity value of the first pre-trained model based on the first feature set, the first label set associated with the first feature set, the second feature set and the second label set associated with the second feature set, the processing module 1102 is specifically used to: construct a first prediction model based on the first feature set, the first label set and a pre-set classifier; based on the first prediction model, process at least the second feature set and the second label set to obtain the first validity value of the first pre-trained model.

[0148] In some embodiments, when determining the first stability value of each pre-trained model based on the training data and the validation data, the processing module 1102 is specifically configured to: for any one of the N pre-trained models, use the first pre-trained model to extract features from the training data to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; determine the first stability value of the first pre-trained model based on the first feature set and the second feature set, where the second feature set includes features corresponding to each sample in the validation data extracted by the first pre-trained model;

[0149] In some embodiments, when determining the first stability value of the first pre-trained model based on the first feature set and the second feature set, the processing module 1102 is specifically used to: construct a target distribution based on the first feature set, the target distribution including: one or more of Gaussian distribution, Bernoulli distribution, Poisson distribution, geometric distribution, or beta distribution; determine the likelihood function value corresponding to the second feature set based on the target distribution; and use the likelihood function value corresponding to the second feature set as the first stability value of the first pre-trained model.

[0150] In some embodiments, the processing module 1102 is also used to: use each of the M pieces of data as verification data once, and, based on the verification data selected each time, determine the first validity value of each pre-trained model once, and determine the first stability value of each pre-trained model once, wherein each pre-trained model corresponds to M first validity values and M first stability values.

[0151] In which, when N evaluation values are determined based on the first validity value and the first stability value of each pre-trained model, the processing module 1102 is also used to: determine the second validity value of each pre-trained model based on the M first validity values and the M first stability values corresponding to each pre-trained model, and determine the second stability value of each pre-trained model; determine N evaluation values based on the second validity value and the second stability value of each pre-trained model.

[0152] In some embodiments, the target validity value is a first validity value and the target stability value is a first stability value, or the target validity value is a second validity value and the target stability value is a second stability value.

[0153] When N evaluation values are determined based on the target validity values and target stability values of each pre-trained model, the processing module 1102 is specifically used to: determine the weight value of the target value based on the N target validity values and N target stability values, where the target value is the target validity value or the target stability value; based on the weight value of the target value, process the target validity value and target stability value corresponding to each pre-trained model respectively to obtain the evaluation value of each pre-trained model.

[0154] In some embodiments, when the sample data is processed based on K pre-trained models to obtain a target model, the processing module 1102 is specifically used to: use the K pre-trained models to extract features from the sample data respectively to obtain K third feature sets, where a third feature set is a set of features extracted by one of the K pre-trained models; use the K third feature sets to train the network model to be trained to obtain a first network model; and obtain the target model based on the K pre-trained models and the first network model.

[0155] In some embodiments, when using K third feature sets to train the network model to be trained, the processing module 1102 is specifically used to: perform feature screening on each feature set in the K third feature sets to obtain K fourth feature sets, each fourth feature set corresponding to a third feature set; and use the K fourth feature sets to train the network model to be trained.

[0156] In some embodiments, when feature screening is performed on each feature set in the K third feature sets, the processing module 1102 is specifically used to: for any feature set in the K third feature sets, construct a probability model of any feature set based on the spike-and-slab prior; determine the marginal probability value of the target label set corresponding to any feature set based on the probability model and the hierarchical Bayesian modeling method; determine the second marginal likelihood function value of the target label set based on the marginal probability value of the target label set; based on the maximum expectation algorithm of variational inference, with the goal of maximizing the second marginal likelihood function value, determine the probability that the indicator variable corresponding to each feature in any feature set is equal to 1; when the probability that the indicator variable corresponding to the target feature in any feature set is equal to 1 is greater than or equal to the preset probability value, retain the target feature; when the probability that the indicator variable corresponding to the target feature is equal to 1 is less than the preset probability value, eliminate the target feature.

[0157] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0158] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device. The electronic device may include: at least one memory for storing programs; at least one processor for executing the programs stored in the memory; wherein, when the programs stored in the memory are executed, the processor is used to execute the method in the above embodiment. For example, the electronic device may be, but is not limited to, Figure 3 The server 200 shown in FIG.

[0159] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0160] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0161] Based on the method in the above embodiment, the present application embodiment also provides a model migration device. Figure 12 , Figure 12 This is a schematic diagram of the structure of a model migration device provided in an embodiment of the present application. Figure 12As shown, the model migration device 1200 includes one or more processors 1201 and an interface circuit 1202. Optionally, the model migration device 1200 may further include a bus 1203.

[0162] The processor 1201 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 1201 or instructions in the form of software. The above-mentioned processor 1201 can be a general-purpose processor, a digital communicator (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods and steps disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0163] The interface circuit 1202 can be used to send or receive data, instructions or information. The processor 1201 can use the data, instructions or other information received by the interface circuit 1202 to process it, and can send the processing completion information through the interface circuit 1202.

[0164] Optionally, the model migration apparatus 1200 further includes a memory, which may include a read-only memory and a random access memory, and provides operation instructions and data to the processor. Part of the memory may also include a non-volatile random access memory (NVRAM).

[0165] Optionally, the memory stores an executable software module or a data structure, and the processor can perform corresponding operations by calling an operation instruction stored in the memory (the operation instruction may be stored in an operating system).

[0166] Optionally, the interface circuit 1202 may be configured to output the execution result of the processor 1201 .

[0167] It should be noted that the corresponding functions of the processor 1201 and the interface circuit 1202 can be implemented through hardware design, software design, or a combination of hardware and software, and there is no limitation here.

[0168] It should be understood that each step of the above method embodiment can be completed by a hardware-based logic circuit or a software-based instruction in a processor.

[0169] It is understood that the order of execution of the steps in the above embodiments does not necessarily imply a specific order of execution. The order of execution of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in some possible implementations, the steps in the above embodiments can be selectively executed according to actual circumstances, and can be executed partially or completely, which is not limited here.

[0170] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0171] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0173] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

Claims

1. A model migration method, characterized in that: The method comprises: Acquire sample data of a target task, wherein the sample data includes a plurality of image samples; Based on the sample data, N pre-trained models are evaluated respectively to obtain N evaluation values, where the evaluation values are used to characterize the fitness between the pre-trained models and the target task, wherein one pre-trained model corresponds to one evaluation value, and N ≥ 2; Based on the N evaluation values, determine K pre-trained models from the N pre-trained models, where the K pre-trained models are models corresponding to the first K evaluation values after sorting the N evaluation values from large to small, and 1≤K≤N; Based on the K pre-trained models, the sample data is processed to obtain a target model, where the target model includes the K pre-trained models, and the target model is used to process the target task; The processing of the sample data based on the K pre-trained models to obtain a target model specifically includes: Using the K pre-trained models to perform feature extraction on the sample data respectively, to obtain K third feature sets, wherein one of the third feature sets is a set of features extracted by one of the K pre-trained models; Using the K third feature sets, training the network model to be trained to obtain a first network model; The target model is obtained according to the K pre-trained models and the first network model.

2. The method according to claim 1, characterized in that The N pre-trained models are evaluated based on the sample data of the target task to obtain N evaluation values, specifically including: Divide the sample data into M parts of data, where M≥2; Select one piece of data from the M pieces of data as verification data, and use the remaining data in the M pieces of data as training data; Based on the training data and the verification data, respectively determine a first validity value for each of the pre-trained models, and respectively determine a first stability value for each of the pre-trained models, wherein the first validity value is used to characterize the accuracy of predicting the label of the verification data using features extracted from the training data by the pre-trained model, and the first stability value is used to characterize the similarity between the features of the training data extracted by the pre-trained model and the features of the verification data extracted; The N evaluation values are determined based on the first validity value and the first stability value of each of the pre-trained models.

3. The method according to claim 2, characterized in that Determining the first validity value of each of the pre-trained models based on the training data and the verification data specifically includes: For any first pre-trained model among the N pre-trained models, extract features from the training data using the first pre-trained model to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; A first validity value of the first pre-trained model is determined based on the first feature set, a first label set associated with the first feature set, a second feature set, and a second label set associated with the second feature set, wherein the first label set includes labels corresponding to each sample in the training data, the second feature set includes features corresponding to each sample in the verification data extracted by the first pre-trained model, and the second label set includes labels corresponding to each sample in the verification data.

4. The method according to claim 3, characterized in that The determining, based on the first feature set, a first label set associated with the first feature set, a second feature set, and a second label set associated with the second feature set, a first validity value of the first pre-trained model specifically includes: Constructing a first prediction model based on the first feature set, the first label set, and a pre-set classifier; Based on the first prediction model, at least the second feature set and the second label set are processed to obtain a first validity value of the first pre-trained model.

5. The method according to any one of claims 2 to 4, characterized in that: Determining the first stability value of each of the pre-trained models based on the training data and the verification data specifically includes: For any first pre-trained model among the N pre-trained models, extract features from the training data using the first pre-trained model to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; A first stability value of the first pre-trained model is determined based on the first feature set and the second feature set, where the second feature set includes features corresponding to each sample in the verification data extracted by the first pre-trained model.

6. The method according to claim 5, characterized in that The determining, based on the first feature set and the second feature set, a first stability value of the first pre-trained model specifically includes: Constructing a target distribution based on the first feature set, the target distribution comprising: one or more of: Gaussian distribution, Bernoulli distribution, Poisson distribution, geometric distribution, or beta distribution; Determining a likelihood function value corresponding to the second feature set based on the target distribution; The likelihood function value corresponding to the second feature set is used as a first stability value of the first pre-trained model.

7. The method according to any one of claims 2 to 4, characterized in that: The method further comprises: Using each of the M pieces of data as verification data, and determining a first validity value of each of the pre-trained models based on the verification data selected each time, and determining a first stability value of each of the pre-trained models, wherein each pre-trained model corresponds to M first validity values and M first stability values; The determining of the N evaluation values based on the first validity value and the first stability value of each of the pre-trained models further includes: Based on the M first validity values and the M first stability values corresponding to the pre-trained models, respectively determine a second validity value of each pre-trained model, and respectively determine a second stability value of each pre-trained model; The N evaluation values are determined based on the second validity value and the second stability value of each of the pre-trained models.

8. The method according to any one of claims 2 to 4, characterized in that: The target validity value is the first validity value, and the target stability value is the first stability value; or the target validity value is the second validity value, and the target stability value is the second stability value; Based on the target effectiveness value and target stability value of each of the pre-trained models, the N evaluation values are determined, specifically including: Determining a weight value of a target value based on the N target effectiveness values and the N target stability values, the target value being the target effectiveness value or the target stability value; Based on the weight value of the target value, the target validity value and the target stability value corresponding to each of the pre-training models are processed respectively to obtain the evaluation value of each of the pre-training models.

9. The method according to claim 1, characterized in that The training of the network model to be trained by using the K third feature sets specifically includes: Performing feature screening on each of the K third feature sets to obtain K fourth feature sets, each of the fourth feature sets corresponding to one of the third feature sets; The network model to be trained is trained using the K fourth feature sets.

10. The method according to claim 9, characterized in that The feature screening is performed on each of the K third feature sets, specifically including: For any one of the K third feature sets, construct a probability model of the any one feature set based on a spike-and-slab prior; Determine the marginal probability value of the target label set corresponding to any one of the feature sets based on the probability model and the hierarchical Bayesian modeling method; Determining a second marginal likelihood function value of the target label set based on the marginal probability value of the target label set; Based on the maximum expectation algorithm of variational inference, with the goal of maximizing the second marginal likelihood function value, determining the probability that the indicator variable corresponding to each feature in the arbitrary feature set is equal to 1; When the probability that the indicator variable corresponding to the target feature in any one of the feature sets is equal to 1 is greater than or equal to a preset probability value, retain the target feature; When the probability that the indicator variable corresponding to the target feature is equal to 1 is less than the preset probability value, the target feature is eliminated.

11. A model migration device, characterized in that: include: A communication module, configured to obtain sample data of a target task, wherein the sample data includes a plurality of image samples; A processing module is configured to evaluate N pre-trained models based on the sample data to obtain N evaluation values, wherein the evaluation values are used to characterize the fitness between the pre-trained models and the target task, wherein one pre-trained model corresponds to one evaluation value, and N ≥ 2; The processing module is further configured to determine K pre-trained models from the N pre-trained models based on the N evaluation values, where the K pre-trained models are models corresponding to the first K evaluation values after the N evaluation values are sorted from largest to smallest, and 1≤K≤N; The processing module is further configured to process the sample data based on the K pre-trained models to obtain a target model, wherein the target model includes the K pre-trained models, and the target model is configured to process the target task; Wherein, when the sample data is processed based on the K pre-trained models to obtain the target model, the processing module is specifically used to: Using the K pre-trained models to perform feature extraction on the sample data respectively, to obtain K third feature sets, wherein one of the third feature sets is a set of features extracted by one of the K pre-trained models; Using the K third feature sets, training the network model to be trained to obtain a first network model; The target model is obtained according to the K pre-trained models and the first network model.

12. The device according to claim 11, characterized in that When N pre-trained models are evaluated based on sample data of the target task to obtain N evaluation values, the processing module is specifically used to: Divide the sample data into M parts of data, where M≥2; Select one piece of data from the M pieces of data as verification data, and use the remaining data in the M pieces of data as training data; Based on the training data and the verification data, respectively determine a first validity value for each of the pre-trained models, and respectively determine a first stability value for each of the pre-trained models, wherein the first validity value is used to characterize the accuracy of predicting the label of the verification data using features extracted from the training data by the pre-trained model, and the first stability value is used to characterize the similarity between the features of the training data extracted by the pre-trained model and the features of the verification data extracted; The N evaluation values are determined based on the first validity value and the first stability value of each of the pre-trained models.

13. The device according to claim 12, characterized in that When determining the first validity value of each of the pre-trained models based on the training data and the verification data, the processing module is specifically configured to: For any first pre-trained model among the N pre-trained models, extract features from the training data using the first pre-trained model to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; A first validity value of the first pre-trained model is determined based on the first feature set, a first label set associated with the first feature set, a second feature set, and a second label set associated with the second feature set, wherein the first label set includes labels corresponding to each sample in the training data, the second feature set includes features corresponding to each sample in the verification data extracted by the first pre-trained model, and the second label set includes labels corresponding to each sample in the verification data.

14. The device according to claim 13, characterized in that When determining a first validity value of the first pre-trained model based on the first feature set, a first label set associated with the first feature set, a second feature set, and a second label set associated with the second feature set, the processing module is specifically configured to: Constructing a first prediction model based on the first feature set, the first label set, and a pre-set classifier; Based on the first prediction model, at least the second feature set and the second label set are processed to obtain a first validity value of the first pre-trained model.

15. The device according to any one of claims 12 to 14, characterized in that: When determining the first stability value of each of the pre-trained models based on the training data and the verification data, the processing module is specifically configured to: For any first pre-trained model among the N pre-trained models, extract features from the training data using the first pre-trained model to obtain a first feature set, where the first feature set includes features corresponding to each sample in the training data; A first stability value of the first pre-trained model is determined based on the first feature set and the second feature set, where the second feature set includes features corresponding to each sample in the verification data extracted by the first pre-trained model.

16. The device according to claim 15, characterized in that When determining the first stability value of the first pre-trained model according to the first feature set and the second feature set, the processing module is specifically configured to: Constructing a target distribution based on the first feature set, the target distribution comprising: one or more of: Gaussian distribution, Bernoulli distribution, Poisson distribution, geometric distribution, or beta distribution; Determining a likelihood function value corresponding to the second feature set based on the target distribution; The likelihood function value corresponding to the second feature set is used as a first stability value of the first pre-trained model.

17. The device according to any one of claims 12 to 14, characterized in that The processing module is further configured to: Using each of the M pieces of data as verification data, and determining a first validity value of each of the pre-trained models based on the verification data selected each time, and determining a first stability value of each of the pre-trained models, wherein each pre-trained model corresponds to M first validity values and M first stability values; Wherein, when the N evaluation values are determined based on the first validity value and the first stability value of each of the pre-trained models, the processing module is further configured to: Based on the M first validity values and the M first stability values corresponding to the respective pre-trained models, respectively determine a second validity value of each of the pre-trained models, and respectively determine a second stability value of each of the pre-trained models; The N evaluation values are determined based on the second validity value and the second stability value of each of the pre-trained models.

18. The device according to any one of claims 12 to 14, characterized in that The target validity value is the first validity value, and the target stability value is the first stability value; or the target validity value is the second validity value, and the target stability value is the second stability value; When the N evaluation values are determined based on the target effectiveness value and the target stability value of each of the pre-trained models, the processing module is specifically configured to: Determining a weight value of a target value based on the N target effectiveness values and the N target stability values, the target value being the target effectiveness value or the target stability value; Based on the weight value of the target value, the target validity value and the target stability value corresponding to each of the pre-training models are processed respectively to obtain the evaluation value of each of the pre-training models.

19. The device according to claim 11, characterized in that When the network model to be trained is trained using the K third feature sets, the processing module is specifically configured to: Performing feature screening on each of the K third feature sets to obtain K fourth feature sets, each of the fourth feature sets corresponding to one of the third feature sets; The network model to be trained is trained using the K fourth feature sets.

20. The device according to claim 19, characterized in that When feature screening is performed on each of the K third feature sets, the processing module is specifically configured to: For any one of the K third feature sets, construct a probability model of the any one feature set based on a spike-and-slab prior; Determine the marginal probability value of the target label set corresponding to any one of the feature sets based on the probability model and the hierarchical Bayesian modeling method; Determining a second marginal likelihood function value of the target label set based on the marginal probability value of the target label set; Based on the maximum expectation algorithm of variational inference, with the goal of maximizing the second marginal likelihood function value, determining the probability that the indicator variable corresponding to each feature in the arbitrary feature set is equal to 1; When the probability that the indicator variable corresponding to the target feature in any one of the feature sets is equal to 1 is greater than or equal to a preset probability value, retain the target feature; When the probability that the indicator variable corresponding to the target feature is equal to 1 is less than the preset probability value, the target feature is eliminated.

21. A model migration device, characterized in that: comprising at least one processor and an interface; The at least one processor obtains program instructions or data through the interface; The at least one processor is configured to execute the program line instructions to implement the method according to any one of claims 1-10.

22. An electronic device, characterized in that: include: at least one memory for storing a program; at least one processor, configured to execute the program stored in the memory; When the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1 to 10.

23. A computer-readable storage medium storing a computer program, wherein when the computer program is executed on a processor, the processor is caused to execute the method according to any one of claims 1 to 10.

24. A computer program product, characterized in that When the computer program product is run on a processor, the processor is caused to perform the method according to any one of claims 1 to 10.

Images